The Role of Evaluation and Plans for Evaluating the Current Testing Program.

ERIC Educational Resources Information Center

Winters, Lynn

The Palos Verdes Peninsula Unified School District Office of Program Evaluation and Research is responsible for providing information for program development and improvement; providing test information to special programs coordinators; and acting as a clearinghouse for all information concerning tests, evaluation methodology, and educational…

Evaluation Methods Sourcebook.

ERIC Educational Resources Information Center

Love, Arnold J., Ed.

The chapters commissioned for this book describe key aspects of evaluation methodology as they are practiced in a Canadian context, providing representative illustrations of recent developments in evaluation methodology as it is currently applied. The following chapters are included: (1) "Program Evaluation with Limited Fiscal and Human…

Collection Evaluation in Research Libraries: The Search for Quality, Consistency, and System in Collection Development.

ERIC Educational Resources Information Center

Mosher, Paul H.

1979-01-01

Reviews the history, literature, and methodology of collection evaluation or assessment in American research libraries; discusses current problems, tools, and methodology of evaluation; and describes an ongoing collection evaluation program at the Stanford University Libraries. (Author/MBR)

Training Evaluation: An Analysis of the Stakeholders' Evaluation Needs

ERIC Educational Resources Information Center

Guerci, Marco; Vinante, Marco

2011-01-01

Purpose: In recent years, the literature on program evaluation has examined multi-stakeholder evaluation, but training evaluation models and practices have not generally taken this problem into account. The aim of this paper is to fill this gap. Design/methodology/approach: This study identifies intersections between methodologies and approaches…

Methodological Review of Intimate Partner Violence Prevention Research

ERIC Educational Resources Information Center

Murray, Christine E.; Graybeal, Jennifer

2007-01-01

The authors present a methodological review of empirical program evaluation research in the area of intimate partner violence prevention. The authors adapted and utilized criterion-based rating forms to standardize the evaluation of the methodological strengths and weaknesses of each study. The findings indicate that the limited amount of…

Assuring data transparency through design methodologies

NASA Technical Reports Server (NTRS)

Williams, Allen

1990-01-01

This paper addresses the role of design methodologies and practices in the assurance of technology transparency. The development of several subsystems on large, long life cycle government programs was analyzed to glean those characteristics in the design, development, test, and evaluation that precluded or enabled the insertion of new technology. The programs examined were Minuteman, DSP, B1-B, and space shuttle. All these were long life cycle, technology-intensive programs. The design methodologies (or lack thereof) and design practices for each were analyzed in terms of the success or failure in incorporating evolving technology. Common elements contributing to the success or failure were extracted and compared to current methodologies being proposed by the Department of Defense and NASA. The relevance of these practices to the design and deployment of Space Station Freedom were evaluated. In particular, appropriate methodologies now being used on the core development contract were examined.

Five Steps for Improving Evaluation Reports by Using Different Data Analysis Methods.

ERIC Educational Resources Information Center

Thompson, Bruce

Although methodological integrity is not the sole determinant of the value of a program evaluation, decision-makers do have a right, at a minimum, to be able to expect competent work from evaluators. This paper explores five areas where evaluators might improve methodological practices. First, evaluation reports should reflect the limited…

Evaluating Federal Social Programs: An Uncertain Act.

ERIC Educational Resources Information Center

Levitan, Sar A.; Wurzburg, Gregory K.

This study of the federal government's evaluation of social programs indicates that it is virtually impossible to establish a bias-free, valid, and reliable system of inquiry to determine the effects of social programs. Divided into five chapters, the document examines the aspirations and limitations of evaluations, methodology, evaluation in the…

Tracer methodology: an appropriate tool for assessing compliance with accreditation standards?

PubMed

Bouchard, Chantal; Jean, Olivier

2017-10-01

Tracer methodology has been used by Accreditation Canada since 2008 to collect evidence on the quality and safety of care and services, and to assess compliance with accreditation standards. Given the importance of this methodology in the accreditation program, the objective of this study is to assess the quality of the methodology and identify its strengths and weaknesses. A mixed quantitative and qualitative approach was adopted to evaluate consistency, appropriateness, effectiveness and stakeholder synergy in applying the methodology. An online questionnaire was sent to 468 Accreditation Canada surveyors. According to surveyors' perceptions, tracer methodology is an effective tool for collecting useful, credible and reliable information to assess compliance with Qmentum program standards and priority processes. The results show good coherence between methodology components (appropriateness of the priority processes evaluated, activities to evaluate a tracer, etc.). The main weaknesses are the time constraints faced by surveyors and management's lack of cooperation during the evaluation of tracers. The inadequate amount of time allowed for the methodology to be applied properly raises questions about the quality of the information obtained. This study paves the way for a future, more in-depth exploration of the identified weaknesses to help the accreditation organization make more targeted improvements to the methodology. Copyright © 2016 John Wiley & Sons, Ltd. Copyright © 2016 John Wiley & Sons, Ltd.

Evaluation: Review of the Past, Preview of the Future.

ERIC Educational Resources Information Center

Smith, M. F.

1994-01-01

This paper summarized contributors' ideas about evaluation as a field and where it is going. Topics discussed were qualitative versus quantitative debate; evaluation's purpose; professionalization; program failure; program development; evaluators as advocates; evaluation knowledge; evaluation expansion; and methodology and design. (SLD)

77 FR 47077 - Statement of Organization, Functions, and Delegations of Authority; Office of Planning, Research...

Federal Register 2010, 2011, 2012, 2013, 2014

2012-08-07

...; surveys, research and evaluation methodologies; demonstration testing and model development; synthesis and..., policy and program analysis; surveys, research and evaluation methodologies; demonstration testing and... Organization, Functions, and Delegations of Authority; Office of Planning, Research and Evaluation AGENCY...

The US Navy’s Helicopter Integrated Diagnostics System (HIDS) Program: Power Drive Train Crack Detection Diagnostics and Prognostics Life Usage Monitoring and Damage Tolerance; Techniques, Methodologies, and Experiences

DTIC Science & Technology

2000-02-01

HIDS] Program: Power Drive Train Crack Detection Diagnostics and Prognostics ife Usage Monitoring and Damage Tolerance; Techniques, Methodologies, and...and Prognostics , Life Usage Monitoring , and Damage Tolerance; Techniques, Methodologies, and Experiences Andrew Hess Harrison Chin William Hardman...continuing program and deployed engine monitoring systems in fixed to evaluate helicopter diagnostic, prognostic , and wing aircraft, notably on the A

State of the Art Methodology for the Design and Analysis of Future Large Scale Evaluations: A Selective Examination.

ERIC Educational Resources Information Center

Burstein, Leigh

Two specific methods of analysis in large-scale evaluations are considered: structural equation modeling and selection modeling/analysis of non-equivalent control group designs. Their utility in large-scale educational program evaluation is discussed. The examination of these methodological developments indicates how people (evaluators,…

What Have We Learned about the Politics of Program Evaluation?

ERIC Educational Resources Information Center

Chelimsky, Eleanor

The politics of program evaluation are discussed from the personal perspective of the Director of the General Accounting Office's Program Evaluation and Methodology Division, which has produced reports for committees of the United States Congress. It is concluded that successful evaluations must be useful to others and must understand the…

The Logic of Evaluation.

ERIC Educational Resources Information Center

Welty, Gordon A.

The logic of the evaluation of educational and other action programs is discussed from a methodological viewpoint. However, no attempt is made to develop methods of evaluating programs. In Part I, the structure of an educational program is viewed as a system with three components--inputs, transformation of inputs into outputs, and outputs. Part II…

Evaluating Payments for Environmental Services: Methodological Challenges

PubMed Central

2016-01-01

Over the last fifteen years, Payments for Environmental Services (PES) schemes have become very popular environmental policy instruments, but the academic literature has begun to question their additionality. The literature attempts to estimate the causal effect of these programs by applying impact evaluation (IE) techniques. However, PES programs are complex instruments and IE methods cannot be directly applied without adjustments. Based on a systematic review of the literature, this article proposes a framework for the methodological process of designing an IE for PES schemes. It revises and discusses the methodological choices at each step of the process and proposes guidelines for practitioners. PMID:26910850

Methodology for the evaluation of the Stephanie Alexander Kitchen Garden program.

PubMed

Gibbs, L; Staiger, P K; Townsend, M; Macfarlane, S; Gold, L; Block, K; Johnson, B; Kulas, J; Waters, E

2013-04-01

Community and school cooking and gardening programs have recently increased internationally. However, despite promising indications, there is limited evidence of their effectiveness. This paper presents the evaluation framework and methods negotiated and developed to meet the information needs of all stakeholders for the Stephanie Alexander Kitchen Garden (SAKG) program, a combined cooking and gardening program implemented in selectively funded primary schools across Australia. The evaluation used multiple aligned theoretical frameworks and models, including a public health ecological approach, principles of effective health promotion and models of experiential learning. The evaluation is a non-randomised comparison of six schools receiving the program (intervention) and six comparison schools (all government-funded primary schools) in urban and rural areas of Victoria, Australia. A mixed-methods approach was used, relying on qualitative measures to understand changes in school cultures and the experiential impacts on children, families, teachers, parents and volunteers, and quantitative measures at baseline and 1 year follow up to provide supporting information regarding patterns of change. The evaluation study design addressed the limitations of many existing evaluation studies of cooking or garden programs. The multistrand approach to the mixed methodology maintained the rigour of the respective methods and provided an opportunity to explore complexity in the findings. Limited sensitivity of some of the quantitative measures was identified, as well as the potential for bias in the coding of the open-ended questions. The SAKG evaluation methodology will address the need for appropriate evaluation approaches for school-based kitchen garden programs. It demonstrates the feasibility of a meaningful, comprehensive evaluation of school-based programs and also demonstrates the central role qualitative methods can have in a mixed-method evaluation. So what? This paper contributes to debate about appropriate evaluation approaches to meet the information needs of all stakeholders and will support the sharing of measures and potential comparisons between program outcomes for comparable population groups and settings.

Strategic Analysis Overview

NASA Technical Reports Server (NTRS)

Cirillo, William M.; Earle, Kevin D.; Goodliff, Kandyce E.; Reeves, J. D.; Stromgren, Chel; Andraschko, Mark R.; Merrill, R. Gabe

2008-01-01

NASA s Constellation Program employs a strategic analysis methodology in providing an integrated analysis capability of Lunar exploration scenarios and to support strategic decision-making regarding those scenarios. The strategic analysis methodology integrates the assessment of the major contributors to strategic objective satisfaction performance, affordability, and risk and captures the linkages and feedbacks between all three components. Strategic analysis supports strategic decision making by senior management through comparable analysis of alternative strategies, provision of a consistent set of high level value metrics, and the enabling of cost-benefit analysis. The tools developed to implement the strategic analysis methodology are not element design and sizing tools. Rather, these models evaluate strategic performance using predefined elements, imported into a library from expert-driven design/sizing tools or expert analysis. Specific components of the strategic analysis tool set include scenario definition, requirements generation, mission manifesting, scenario lifecycle costing, crew time analysis, objective satisfaction benefit, risk analysis, and probabilistic evaluation. Results from all components of strategic analysis are evaluated a set of pre-defined figures of merit (FOMs). These FOMs capture the high-level strategic characteristics of all scenarios and facilitate direct comparison of options. The strategic analysis methodology that is described in this paper has previously been applied to the Space Shuttle and International Space Station Programs and is now being used to support the development of the baseline Constellation Program lunar architecture. This paper will present an overview of the strategic analysis methodology and will present sample results from the application of the strategic analysis methodology to the Constellation Program lunar architecture.

Reliability Centered Maintenance - Methodologies

NASA Technical Reports Server (NTRS)

Kammerer, Catherine C.

2009-01-01

Journal article about Reliability Centered Maintenance (RCM) methodologies used by United Space Alliance, LLC (USA) in support of the Space Shuttle Program at Kennedy Space Center. The USA Reliability Centered Maintenance program differs from traditional RCM programs because various methodologies are utilized to take advantage of their respective strengths for each application. Based on operational experience, USA has customized the traditional RCM methodology into a streamlined lean logic path and has implemented the use of statistical tools to drive the process. USA RCM has integrated many of the L6S tools into both RCM methodologies. The tools utilized in the Measure, Analyze, and Improve phases of a Lean Six Sigma project lend themselves to application in the RCM process. All USA RCM methodologies meet the requirements defined in SAE JA 1011, Evaluation Criteria for Reliability-Centered Maintenance (RCM) Processes. The proposed article explores these methodologies.

Incorporation of lean methodology into pharmacy residency programs.

PubMed

John, Natalie; Snider, Holly; Edgerton, Lisa; Whalin, Laurie

2017-03-15

The implementation of lean methodology into pharmacy residency programs at a community teaching hospital is described. New Hanover Regional Medical Center, a community teaching hospital in southeastern North Carolina, fully adopted a lean culture in 2010. Given the success of lean strategies organizationally, this methodology was used to assist with the evaluation and development of its pharmacy residency programs in 2014. Lean tools and activities have also been incorporated into residency requirements and rotation learning activities. The majority of lean events correspond to the required competency areas evaluating leadership and management, teaching, and education. These events have included participation in and facilitation of various lean problem-solving and communication tools. The application of the 4 rules of lean has resulted in enhanced management of the programs and provides a set of tools by which continual quality improvement can be ensured. Regular communication and direct involvement of all invested parties have been critical in developing and sustaining new improvements. In addition to program enhancements, lean methodology offers novel methods by which residents may be incorporated into leadership activities. The incorporation of lean methodology into pharmacy residency programs has translated into a variety of realized and potential benefits for the programs, the preceptors and residents, and the health system. Specific areas of growth have included quality-improvement processes, the expansion of leadership opportunities for residents, and improved communication among program directors, preceptors, and residents. Copyright © 2017 by the American Society of Health-System Pharmacists, Inc. All rights reserved.

Cognitive Restructuring through Dreams and Imagery: Descriptive Analysis of a Women's Prison-Based Program

ERIC Educational Resources Information Center

DeHart, Dana

2010-01-01

This report describes process and outcome evaluation of an innovative program based in a women's maximum-security correctional facility. Methodology included review of program materials, unobtrusive observation of group process, participant evaluation forms, focus groups, and individual interviews with current and former program participants.…

Evaluation of radiological dispersion/consequence codes supporting DOE nuclear facility SARs

DOE Office of Scientific and Technical Information (OSTI.GOV)

O`Kula, K.R.; Paik, I.K.; Chung, D.Y.

1996-12-31

Since the early 1990s, the authorization basis documentation of many U.S. Department of Energy (DOE) nuclear facilities has been upgraded to comply with DOE orders and standards. In this process, many safety analyses have been revised. Unfortunately, there has been nonuniform application of software, and the most appropriate computer and engineering methodologies often are not applied. A DOE Accident Phenomenology and Consequence (APAC) Methodology Evaluation Program was originated at the request of DOE Defense Programs to evaluate the safety analysis methodologies used in nuclear facility authorization basis documentation and to define future cost-effective support and development initiatives. Six areas, includingmore » source term development (fire, spills, and explosion analysis), in-facility transport, and dispersion/ consequence analysis (chemical and radiological) are contained in the APAC program. The evaluation process, codes considered, key results, and recommendations for future model and software development of the Radiological Dispersion/Consequence Working Group are summarized in this paper.« less

Implementation of the Career Education Incentive Act. First Interim Report on the Evaluability Assessment.

ERIC Educational Resources Information Center

Jung, Steven M.; And Others

Survey activities are reported which were designed to provide the foundation for a national evaluation of the effectiveness of programs assisted under the Career Education Incentive Act of 1977 (PL 95-207). The methodology described, called "program evaluability assessment," focuses on detailed analysis of program assumptions in order to…

Appendix C: Background and Methodology for Alternative Certification Pilot. [2014 Teacher Prep Review

ERIC Educational Resources Information Center

Greenberg, Julie; Walsh, Kate; McKee, Arthur

2014-01-01

The "NCTQ Teacher Prep Review" evaluates the quality of programs that provide preservice preparation of public school teachers. As part of the "Review," this appendix reports on a pilot study of new standards for assessing the quality of alternative certification programs. Background and methodology for alternative…

Progress Report for the Robotic Intelligence Evaluation. Program Year 1: Developing Test Methodology for Anti-Rollover Systems

DTIC Science & Technology

2006-06-01

Scientific Research. 5PAM-Crash is a trademark of the ESI Group . 6MATLAB and SIMULINK are registered trademarks of the MathWorks. 14 maneuvers...Laboratory (ARL) to develop methodologies to evaluate robotic behavior algorithms that control the actions of individual robots or groups of robots...methodologies to evaluate robotic behavior algorithms that control the actions of individual robots or groups of robots acting as a team to perform a

Quality Assessment of Published Articles in Iranian Journals Related to Economic Evaluation in Health Care Programs Based on Drummond's Checklist: A Narrative Review.

PubMed

Rezapour, Aziz; Jafari, Abdosaleh; Mirmasoudi, Kosha; Talebianpour, Hamid

2017-09-01

Health economic evaluation research plays an important role in selecting cost-effective interventions. The purpose of this study was to assess the quality of published articles in Iranian journals related to economic evaluation in health care programs based on Drummond's checklist in terms of numbers, features, and quality. In the present review study, published articles (Persian and English) in Iranian journals related to economic evaluation in health care programs were searched using electronic databases. In addition, the methodological quality of articles' structure was analyzed by Drummond's standard checklist. Based on the inclusion criteria, the search of databases resulted in 27 articles that fully covered economic evaluation in health care programs. A review of articles in accordance with Drummond's criteria showed that the majority of studies had flaws. The most common methodological weakness in the articles was in terms of cost calculation and valuation. Considering such methodological faults in these studies, it is anticipated that these studies would not provide an appropriate feedback to policy makers to allocate health care resources correctly and select suitable cost-effective interventions. Therefore, researchers are required to comply with the standard guidelines in order to better execute and report on economic evaluation studies.

Quality Assessment of Published Articles in Iranian Journals Related to Economic Evaluation in Health Care Programs Based on Drummond’s Checklist: A Narrative Review

PubMed Central

Rezapour, Aziz; Jafari, Abdosaleh; Mirmasoudi, Kosha; Talebianpour, Hamid

2017-01-01

Health economic evaluation research plays an important role in selecting cost-effective interventions. The purpose of this study was to assess the quality of published articles in Iranian journals related to economic evaluation in health care programs based on Drummond’s checklist in terms of numbers, features, and quality. In the present review study, published articles (Persian and English) in Iranian journals related to economic evaluation in health care programs were searched using electronic databases. In addition, the methodological quality of articles’ structure was analyzed by Drummond’s standard checklist. Based on the inclusion criteria, the search of databases resulted in 27 articles that fully covered economic evaluation in health care programs. A review of articles in accordance with Drummond’s criteria showed that the majority of studies had flaws. The most common methodological weakness in the articles was in terms of cost calculation and valuation. Considering such methodological faults in these studies, it is anticipated that these studies would not provide an appropriate feedback to policy makers to allocate health care resources correctly and select suitable cost-effective interventions. Therefore, researchers are required to comply with the standard guidelines in order to better execute and report on economic evaluation studies. PMID:29234174

The Bronx Computer Literacy and Methodologies of Bilingual Education Program for Vietnamese and Cambodian High School Students: Project CLIMB. Evaluation Section Report. OREA Report.

ERIC Educational Resources Information Center

Berney, Tomi D.; Keyes, Jose L.

The Bronx Computer Literacy and Methodologies of Bilingual Education Program for Vietnamese and Cambodian High School Students (Project CLIMB) served 221 students of limited English proficiency (LEP) at Christopher Columbus and Walton High Schools in the Bronx (New York City). The objectives of the program were to develop the students' academic…

Strengthening a Design for Evaluating a Broad-Aimed Functional Literacy and Family Life Education Program

ERIC Educational Resources Information Center

Farmer, James A., Jr.

1974-01-01

Techniques are illustrated in this article for strengthening evaluative designs and methodologies relevant to educational programs that incorporate within their broad aims assisting in social problem-solving. (AJ)

Evaluating Mentoring Programs. P/PV Brief

ERIC Educational Resources Information Center

Grossman, Jean Baldwin

2009-01-01

This methodological brief is designed to provide both program operators and researchers with practical advice about how to assess a program's implementation and impact. Adapted from an article that first appeared in "The Handbook of Youth Mentoring" (DuBois and Karcher, ed. 2005), the brief focuses on the evaluation of mentoring programs, but the…

Evaluating Federal Education Programs.

ERIC Educational Resources Information Center

Baker, Eva L., Ed.

A series of papers was developed by the authors as part of their deliberations as members of the National Research Council's Committee of Program Evaluation in Education. The papers provide a broad range of present evaluative thinking. The conflict between preferences in evaluation methodology comes through in these papers. The selections include:…

Structural Optimization Methodology for Rotating Disks of Aircraft Engines

NASA Technical Reports Server (NTRS)

Armand, Sasan C.

1995-01-01

In support of the preliminary evaluation of various engine technologies, a methodology has been developed for structurally designing the rotating disks of an aircraft engine. The structural design methodology, along with a previously derived methodology for predicting low-cycle fatigue life, was implemented in a computer program. An interface computer program was also developed that gathers the required data from a flowpath analysis program (WATE) being used at NASA Lewis. The computer program developed for this study requires minimum interaction with the user, thus allowing engineers with varying backgrounds in aeropropulsion to successfully execute it. The stress analysis portion of the methodology and the computer program were verified by employing the finite element analysis method. The 10th- stage, high-pressure-compressor disk of the Energy Efficient Engine Program (E3) engine was used to verify the stress analysis; the differences between the stresses and displacements obtained from the computer program developed for this study and from the finite element analysis were all below 3 percent for the problem solved. The computer program developed for this study was employed to structurally optimize the rotating disks of the E3 high-pressure compressor. The rotating disks designed by the computer program in this study were approximately 26 percent lighter than calculated from the E3 drawings. The methodology is presented herein.

A Big Data Analytics Methodology Program in the Health Sector

ERIC Educational Resources Information Center

Lawler, James; Joseph, Anthony; Howell-Barber, H.

2016-01-01

The benefits of Big Data Analytics are cited frequently in the literature. However, the difficulties of implementing Big Data Analytics can limit the number of organizational projects. In this study, the authors evaluate business, procedural and technical factors in the implementation of Big Data Analytics, applying a methodology program. Focusing…

The Effect of Business and Economics Education Programs on Students' Entrepreneurial Intention

ERIC Educational Resources Information Center

De Jorge-Moreno, Justo; Castillo, Leopoldo Laborda; Triguero, Maria Sanz

2012-01-01

Purpose: This paper aims to evaluate the effect of participation in business and economics education programs on the student's entrepreneurial intention in terms of perceptions of the desirability and personal feasibility of starting a business. Design/methodology/approach: The methodology used to measure the student's entrepreneurial intention is…

76 FR 6795 - Statement of Organization, Functions, and Delegations of Authority; Office of the National...

Federal Register 2010, 2011, 2012, 2013, 2014

2011-02-08

... Coordinator; (2) applies research methodologies to perform evaluation studies of health information technology grant programs; and, (3) applies advanced mathematical or quantitative modeling to the U.S. health care... remaining items in the paragraph accordingly: ``(1) Applying research methodologies to perform evaluation...

Situated Research Design and Methodological Choices in Formative Program Evaluation

ERIC Educational Resources Information Center

Supovitz, Jonathan

2013-01-01

Design-based implementation research offers the opportunity to rethink the relationships between intervention, research, and situation to better attune research and evaluation to the program development process. Using a heuristic called the intervention development curve, I describe the rough trajectory that programs typically follow as they…

Training the Poor. A Benefit-Cost Analysis of Manpower Programs in the U.S. Antipoverty Program.

ERIC Educational Resources Information Center

Sewell, D. O.

This report critically reviews past benefit-cost studies of manpower programs and offers an alternative methodology for program evaluation. Because of selective admission criteria and a tendency to attribute all income changes to training, previous studies have had an upward bias in measuring the effectiveness of training. This study evaluates a…

Using a Non-Equivalent Groups Quasi Experimental Design to Reduce Internal Validity Threats to Claims Made by Math and Science K-12 Teacher Recruitment Programs

NASA Astrophysics Data System (ADS)

Moin, Laura

2009-10-01

The American Recovery and Reinvestment Act national policy established in 2009 calls for ``meaningful data'' that demonstrate educational improvements, including the recruitment of high-quality teachers. The scant data available and the low credibility of many K-12 math/science teacher recruitment program evaluations remain the major barriers for the identification of effective recruitment strategies. Our study presents a methodology to better evaluate the impact of recruitment programs on increasing participants' interest in teaching careers. The research capitalizes on the use of several control groups and presents a non-equivalent groups quasi-experimental evaluation design that produces program effect claims with higher internal validity than claims generated by current program evaluations. With this method that compares responses to a teaching career interest question from undergraduates all along a continuum from just attending an information session to participating (or not) in the recruitment program, we were able to compare the effect of the program in increasing participants' interest in teaching careers versus the evolution of the same interest but in the absence of the program. We were also able to make suggestions for program improvement and further research. While our findings may not apply to other K-12 math/science teacher recruitment programs, we believe that our evaluation methodology does and will contribute to conduct stronger program evaluations. In so doing, our evaluation procedure may inform recruitment program designers and policy makers.

Workplace Literacy Program (WPL) at Chinatown Manpower Project, Inc. Final Evaluation.

ERIC Educational Resources Information Center

Friedenberg, Joan E.

This document describes the procedures for and results of the external evaluation of the workplace literacy program for underemployed garment industry workers with low English skills at Chinatown Manpower Project, Inc. in Chinatown in New York City. The document describes the evaluation design and methodology as well as the evaluation results,…

New Briefing Methodology for the Brazilian Study and Monitoring of Space Weather (Embrace) Program.

NASA Astrophysics Data System (ADS)

Dal Lago, A.; Cecatto, J. R.; Costa, J. E. R.; Da Silva, L. A.; Rockenbach, M.; Braga, C. R.; Mendonca, R. R. S.; Mendes, O., Jr.; Koga, D.; Alves, L. R.; Becker-Guedes, F.; Wrasse, C. M.; Takahashi, H.; Resende, L.; Banik de Padua, M.; De Nardin, C. M.

2016-12-01

The Brazilian Study and Monitoring of Space Weather (Embrace) Program is being conducted by the National Institute for Space Research (INPE, Brazil) since 2008. Among several activities of the EMBRACE program, there are weekly briefings, held since 2012, where an evaluation is made of all space weather events occurred in the past week. At the beginning, an intuitive methodology was used, in which scientists were invited to present their reports on their subjects of expertise: solar, interplanetary space, geomagnetism, ionosphere and upper atmosphere. Latter on, an additional subject was introduced, with the inclusion of a separate report on the earth's magnetosphere, with special attention to the dynamics of the earth's radiation belts. Since late 2015, the need for a more efficient methodology was felt by the EMBRACE program, inspired by practices long used in forecasting of metheorological weather and climate. In that sense, an atempt to develop scales of disturbances was made. The aim is to be able to faster represent the level of space weather activity in all reported subjects. A huge effort was put together to produce sound indices, based on statistical significance of occurrence of distinct levels. This methodology is partially under practical evaluation since early 2016. In this work we present a report on the progress of the new methodology for EMBRACE program briefing meetings.

Comparing the Medicaid Retrospective Drug Utilization Review Program Cost-Savings Methods Used by State Agencies.

PubMed

Prada, Sergio I

2017-12-01

The Medicaid Drug Utilization Review (DUR) program is a 2-phase process conducted by Medicaid state agencies. The first phase is a prospective DUR and involves electronically monitoring prescription drug claims to identify prescription-related problems, such as therapeutic duplication, contraindications, incorrect dosage, or duration of treatment. The second phase is a retrospective DUR and involves ongoing and periodic examinations of claims data to identify patterns of fraud, abuse, underutilization, drug-drug interaction, or medically unnecessary care, implementing corrective actions when needed. The Centers for Medicare & Medicaid Services requires each state to measure prescription drug cost-savings generated from its DUR programs on an annual basis, but it provides no guidance or unified methodology for doing so. To describe and synthesize the methodologies used by states to measure cost-savings using their Medicaid retrospective DUR program in federal fiscal years 2014 and 2015. For each state, the cost-savings methodologies included in the Medicaid DUR 2014 and 2015 reports were downloaded from Medicaid's website. The reports were then reviewed and synthesized. Methods described by the states were classified according to research designs often described in evaluation textbooks. In 2014, the most often used prescription drugs cost-savings estimation methodology for the Medicaid retrospective DUR program was a simple pre-post intervention method, without a comparison group (ie, 12 states). In 2015, the most common methodology used was a pre-post intervention method, with a comparison group (ie, 14 states). Comparisons of savings attributed to the program among states are still unreliable, because of a lack of a common methodology available for measuring cost-savings. There is great variation among states in the methods used to measure prescription drug utilization cost-savings. This analysis suggests that there is still room for improvement in terms of methodology transparency, which is important, because lack of transparency hinders states from learning from each other. Ultimately, the federal government needs to evaluate and improve its DUR program.

Adapting Job Analysis Methodology to Improve Evaluation Practice

ERIC Educational Resources Information Center

Jenkins, Susan M.; Curtin, Patrick

2006-01-01

This article describes how job analysis, a method commonly used in personnel research and organizational psychology, provides a systematic method for documenting program staffing and service delivery that can improve evaluators' knowledge about program operations. Job analysis data can be used to increase evaluators' insight into how staffs…

In Vitro Evaluation of a Program for Machine-Aided Indexing.

ERIC Educational Resources Information Center

Jacquemin, Christian; Daille, Beatrice; Royaute, Jean; Polanco, Xavier

2002-01-01

Presents the human evaluation of ILIAD, a program for machine-aided indexing that was designed to assist expert librarians in computer-aided indexing and document analysis. Topics include controlled indexing and free indexing; natural language and concept-based information retrieval; evaluation methodology; syntactic variations; and a comparison…

75 FR 42760 - Statement of Organization, Functions, and Delegations of Authority

Federal Register 2010, 2011, 2012, 2013, 2014

2010-07-22

... accounting reports and invoices, and monitoring all spending. The Team develops, defends and executes the... results; performance measurement; research and evaluation methodologies; demonstration testing and model... ACF programs; strategic planning; performance measurement; program and policy evaluation; research and...

Methodology for Evaluating an Adaptation of Evidence-Based Drug Abuse Prevention in Alternative Schools

ERIC Educational Resources Information Center

Hopson, Laura M.; Steiker, Lori K. H.

2008-01-01

The purpose of this article is to set forth an innovative methodological protocol for culturally grounding interventions with high-risk youths in alternative schools. This study used mixed methods to evaluate original and adapted versions of a culturally grounded substance abuse prevention program. The qualitative and quantitative methods…

Economic Evaluation of Adolescent Addiction Programs: Methodological Challenges and Recommendations

PubMed Central

Homer, Jenny F.; Drummond, Michael F.; French, Michael T.

2008-01-01

This paper identifies and describes several methodological challenges encountered in economic evaluations of substance abuse interventions for adolescents. Topics include study design, the choice of perspective, the estimation of costs and outcomes, and the generalizability of results. Recommendations are offered for confronting these challenges using examples from adolescent addiction research. PMID:19027640

Innovations in Doctoral Education: Distance Education Methodology Applied

ERIC Educational Resources Information Center

Bettmann, Joanna; Thompson, Kimberly; Padykula, Nora; Berzoff, Joan

2009-01-01

This study evaluated the impact of a distance education program to meet the practice learning needs of first-year doctoral students. The program, a six-session case-based telephonic seminar, was taught to 19 first-year doctoral students. Evaluation of the program included self-report quantitative and qualitative data gathered pre- and postseminar,…

Title I Institutionalized Facilities Program. O.E.E. Evaluation Report, 1981-82.

ERIC Educational Resources Information Center

New York City Board of Education, Brooklyn, NY. Office of Educational Evaluation.

This report documents the 1981-82 evaluation of the Elementary and Secondary Education Act, Title I, Part B, Institutional Facilities Program carried out through the Division of Special Education of the New York City public schools. Using a diagnostic-prescriptive methodology, the program provided after-school and pull-out individualized remedial…

An Evaluation of a Program to Help Dual-Earner Couples Share the Second Shift.

ERIC Educational Resources Information Center

Hawkins, Alan J.; And Others

1994-01-01

Used both traditional scientific and feminist methodologies to evaluate effectiveness of family life education program designed to help dual-earner couples (n=14 couples) share domestic labor. Both quantitative and qualitative data suggest that program produced small increases in husbands' involvement in both housework and child care and large…

Investigating the Effectiveness of Special Education: An Analysis of Methodology.

ERIC Educational Resources Information Center

Tindal, Gerald

1985-01-01

The review examines evaluations of the efficacy of special education programs for mildly disabled children. The author suggests that serious methodological flaws make our present knowledge in this area very weak and proposes a methodology to address and overcome many of the limitations of previous research. (Author)

An Integrated Nutrition Education Program for Dental Students.

ERIC Educational Resources Information Center

DePaola, Dominick P.; And Others

1978-01-01

Presents a diagonal system of nutrition education in a dental program that integrates didactic, clinical work, and community sites. Discusses the three phases including educational methodology, evaluation strategy, and unique program features. (MA)

Identfying the Needs of Pre-Service Classroom Teachers about Science Teaching Methodology Course in Terms of Parlett's Illuminative Program Evaluation Model

ERIC Educational Resources Information Center

Çaliskan, Ilke

2014-01-01

The aim of this study was to identify the needs of third grade classroom teaching students about science teaching course in terms of Parlett's Illuminative program evaluation model. Phenomographic research design was used in this study. Illuminative program evaluation model was chosen for this study in terms of its eclectic and process-based…

Identfying the Needs of Pre-Service Classroom Teachers about Science Teaching Methodology Courses in Terms of Parlett's Illuminative Program Evaluation Model

ERIC Educational Resources Information Center

Çaliskan, Ilke

2014-01-01

The aim of this study was to identify the needs of third grade classroom teaching students about science teaching course in terms of Parlett's Illuminative program evaluation model. Phenomographic research design was used in this study. Illuminative program evaluation model was chosen for this study in terms of its eclectic and process-based…

Social Return on Investment: A New Approach to Understanding and Advocating for Value in Healthcare.

PubMed

Laing, Catherine M; Moules, Nancy J

2017-12-01

To determine whether the methodology of social return on investment (SROI) could be a way in which the value of a healthcare-related program (children's cancer camp) could be captured, evaluated, and communicated. The value of healthcare goes beyond what can be captured in financial terms; however, this is the most common type of value that is measured. The SROI methodology accounts for a broader concept of value by measuring social, environmental, and economic outcomes and uses monetary values to represent them. The steps/stages of an SROI analysis were applied to the context of a children's camp for this article. Applying the SROI methodology to this healthcare-related program was feasible and provided insight and understanding related to the impacts of this program. Because of SROI's flexibility, it is a tool that has great potential in a healthcare environment and for leaders to evaluate programmatic return on investment.

Buffering Negative Impacts of Divorce on Children: Evaluating Impact of Divorce Education

ERIC Educational Resources Information Center

Crawford, Jennifer K.; Riffe, Jane; Trevisan, Dominic A.; Adesope, Olusola O.

2014-01-01

Following the call for more stringent evaluation methodology and recently documented national Extension presence in the field of divorce education for parents and children, the study reported here describes a local multi-level evaluation to capture program impact of a stakeholder-accepted divorce education program. Using a post-then-pre…

Measuring the Impact of Employment-Related Social Programs. A Primer on the Evaluation of Employment and Training, Vocational Education, Vocational Rehabilitation, and Other Job-Oriented Programs.

ERIC Educational Resources Information Center

Borus, Michael E.

An approach and methodology for the systematic measurement of the impact of employment-related social programs is presented in this primer. Chapter 1 focuses on evaluation as the third step (the first two being planning and operation) in the process of program implementation. Chapter 2 examines the impacts of social programs. Topics include…

Effective Evaluation of the FRIENDS Anxiety Prevention Program in School Settings: A Response to Maggin and Johnson

ERIC Educational Resources Information Center

Barrett, Paula M.; Cooper, Marita; Stallard, Paul; Zeggio, Larissa; Gallegos- Guajardo, Julia

2017-01-01

This response aims to critically evaluate the methodology and aims of the meta-analytic review written by Maggin and Johnson (2014). The present authors systematically provide responses for each of the original criticisms and highlight concerns regarding Maggin and Johnson's methodology, while objectively describing the current state of evidence…

An Art Program Evaluation of Daily Life Therapy for Children with Autism

ERIC Educational Resources Information Center

Talusan-Dunn, Rowena

2012-01-01

The author evaluated a private school's art program in 2009-2010 that used Daily Life Therapy (DLT) for students diagnosed with autism spectrum disorders (ASD). Significant increases in numbers of persons diagnosed with ASD have been noted in the last two decades. Several methodologies claim success in programming for children with ASD, but lack…

Collaborative Evaluation of a High School Prevention Curriculum: How Methods of Collaborative Evaluation Enhanced a Randomized Control Trial to Inform Program Improvement

ERIC Educational Resources Information Center

Orsini, Muhsin Michael; Wyrick, David L.; Milroy, Jeffrey J.

2012-01-01

Blending high-quality and rigorous research with pure evaluation practice can often be best accomplished through thoughtful collaboration. The evaluation of a high school drug prevention program (All Stars Senior) is an example of how perceived competing purposes and methodologies can coexist to investigate formative and summative outcome…

Realistic evaluation of an emergency department-based mental health nurse practitioner outpatient service in Australia.

PubMed

Wand, Timothy; White, Kathryn; Patching, Joanna

2011-06-01

Evaluation of new models of care requires consideration of the complexity inherent within health care programs and their sensitivity to local contextual factors as well as broader community, social and political influences. Evaluation frameworks that are flexible and responsive while maintaining research rigor are therefore required. Realistic evaluation was adopted as the methodology for the implementation and evaluation of an emergency department-based mental health nurse practitioner outpatient service in Sydney, Australia. The aim of realistic evaluation is to generate, test and refine theories of how programs work within a given context. This paper represents the final methodological step from the completed evaluation. A summary of quantitative and qualitative findings from the mixed-methods evaluation is presented, which is transformed into a set of overarching statements or "middle range theories". Middle range theory statements seek to explain the success of a program and provide transferable lessons for practitioners wishing to implement similar programs elsewhere. For example, the research team consider that early consultation with key local stakeholders and emergency department ownership of the project was pivotal to the implementation process. © 2011 Blackwell Publishing Asia Pty Ltd.

Systematic iteration between model and methodology: A proposed approach to evaluating unintended consequences.

PubMed

Morell, Jonathan A

2018-06-01

This article argues that evaluators could better deal with unintended consequences if they improved their methods of systematically and methodically combining empirical data collection and model building over the life cycle of an evaluation. This process would be helpful because it can increase the timespan from when the need for a change in methodology is first suspected to the time when the new element of the methodology is operational. The article begins with an explanation of why logic models are so important in evaluation, and why the utility of models is limited if they are not continually revised based on empirical evaluation data. It sets the argument within the larger context of the value and limitations of models in the scientific enterprise. Following will be a discussion of various issues that are relevant to model development and revision. What is the relevance of complex system behavior for understanding predictable and unpredictable unintended consequences, and the methods needed to deal with them? How might understanding of unintended consequences be improved with an appreciation of generic patterns of change that are independent of any particular program or change effort? What are the social and organizational dynamics that make it rational and adaptive to design programs around single-outcome solutions to multi-dimensional problems? How does cognitive bias affect our ability to identify likely program outcomes? Why is it hard to discern change as a result of programs being embedded in multi-component, continually fluctuating, settings? The last part of the paper outlines a process for actualizing systematic iteration between model and methodology, and concludes with a set of research questions that speak to how the model/data process can be made efficient and effective. Copyright © 2017 Elsevier Ltd. All rights reserved.

Research methodology workshops evaluation using the Kirkpatrick's model: translating theory into practice.

PubMed

Abdulghani, Hamza Mohammad; Shaik, Shaffi Ahamed; Khamis, Nehal; Al-Drees, Abdulmajeed Abdulrahman; Irshad, Mohammad; Khalil, Mahmoud Salah; Alhaqwi, Ali Ibrahim; Isnani, Arthur

2014-04-01

Qualitative and quantitative evaluation of academic programs can enhance the development, effectiveness, and dissemination of comparative quality reports as well as quality improvement efforts. To evaluate the five research methodology workshops through assessing participants' satisfaction, knowledge and skills gain and impact on practices by the Kirkpatrick's evaluation model. The four level Kirkpatrick's model was applied for the evaluation. Training feedback questionnaires, pre and post tests, learner development plan reports and behavioral surveys were used to evaluate the effectiveness of the workshop programs. Of the 116 participants, 28 (24.1%) liked with appreciation, 62 (53.4%) liked with suggestions and 26 (22.4%) disliked the programs. Pre and post MCQs tests mean scores showed significant improvement of relevant basic knowledge and cognitive skills by 17.67% (p ≤ 0.005). Pre-and-post tests scores on workshops sub-topics also significantly improved for the manuscripts (p ≤ 0.031) and proposal writing (p ≤ 0.834). As for the impact, 56.9% of participants started research, and 6.9% published their studies. The results from participants' performance revealed an overall positive feedback and 79% of participant reported transfer of training skills at their workplace. The course outcomes achievement and suggestions given for improvements offer insight into the program which were encouraging and very useful. Encouraging "research culture" and work-based learning are probably the most powerful determinants for research promotion. These findings therefore encourage faculty development unit to continue its training and development in the research methodology aspects.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Brigantic, Robert T.; Papatyi, Anthony F.; Perkins, Casey J.

This report summarizes a study and corresponding model development conducted in support of the United States Pacific Command (USPACOM) as part of the Federal Energy Management Program (FEMP) American Reinvestment and Recovery Act (ARRA). This research was aimed at developing a mathematical programming framework and accompanying optimization methodology in order to simultaneously evaluate energy efficiency (EE) and renewable energy (RE) opportunities. Once developed, this research then demonstrated this methodology at a USPACOM installation - Camp H.M. Smith, Hawaii. We believe this is the first time such an integrated, joint EE and RE optimization methodology has been constructed and demonstrated.

Evaluating Writing Programs: Paradigms, Problems, Possibilities.

ERIC Educational Resources Information Center

McLeod, Susan H.

1992-01-01

Describes two methodological approaches (qualitative and quantitative) that grow out of two different research examples. Suggests the problems these methods present. Discusses the ways in which an awareness of these problems can help teachers to understand how to work with researchers in designing useful evaluations of writing programs. (PRA)

Formative Evaluation of the ACSC Distance Learning Program: A Status Study.

ERIC Educational Resources Information Center

MacCuish, Donald A.

2001-01-01

Describes a formative evaluation of the Air Command and Staff College (ACSC) distance learning program, which offers professional military education (PME). Highlights include use of the status study methodology; curriculum development; course design; learning theories; instructional systems design; best practices; and student assessment. (LRW)

Evaluation of freeway motorist assist program : final report, September 30, 2009.

DOT National Transportation Integrated Search

2010-02-01

This evaluation of the Motorist Assist (MA) program in St. Louis estimated that MA has an annual benefit-cost ratio (B/C) of 38.25:1 using 2009 dollars. This estimate was based on nationally accepted AASHTO methodology and was based on 1082 secondary...

Care management program evaluation: constituents, conflicts, and moves toward standardization.

PubMed

Long, D Adam; Perry, Theodore L; Pelletier, Kenneth R; Lehman, Gregg O

2006-06-01

Care management program evaluations bring together constituents from finance, medicine, and social sciences. The differing assumptions and scientific philosophies that these constituents bring to the task often lead to frustrations and even contentions. Given the forms and variations of care management programs, the difficulty associated with program outcomes measurement should not be surprising. It is no wonder then that methods for clinical and economic evaluations of program efficacy continue to be debated and have yet to be standardized. We describe these somewhat hidden processes, examine where the industry stands, and provide recommendations for steps to standardize evaluation methodology.

Planning and Selecting Evaluation Designs for Leadership Training: A Toolkit for Nurse Managers and Educators.

PubMed

Dunne, Simon; Lunn, Cora; Kirwan, Marcia; Matthews, Anne; Condell, Sarah

2015-01-01

Leadership development training and education for nurses is a priority in modern health care systems. Consequently, effective evaluation of nurse leadership development programs is essential for managers and educators in health care organizations to determine the impact of such programs on staff behaviors and patient outcomes. Our team has identified a framework for the evaluation of the design and implementation of such programs. Following this, we provide practical tools for the selection of evaluation methodologies for leadership development programs for use by health care educators and program commissioners. Copyright © 2015 Elsevier Inc. All rights reserved.

Evaluating Educational Programs. ETS R&D Scientific and Policy Contributions Series. ETS SPC-11-01. ETS Research Report No. RR-11-15

ERIC Educational Resources Information Center

Ball, Samuel

2011-01-01

Since its founding in 1947, ETS has conducted a significant and wide-ranging research program that has focused on, among other things, psychometric and statistical methodology; educational evaluation; performance assessment and scoring; large-scale assessment and evaluation; cognitive, developmental, personality, and social psychology; and…

Treatment effects model for assessing disease management: measuring outcomes and strengthening program management.

PubMed

Wendel, Jeanne; Dumitras, Diana

2005-06-01

This paper describes an analytical methodology for obtaining statistically unbiased outcomes estimates for programs in which participation decisions may be correlated with variables that impact outcomes. This methodology is particularly useful for intraorganizational program evaluations conducted for business purposes. In this situation, data is likely to be available for a population of managed care members who are eligible to participate in a disease management (DM) program, with some electing to participate while others eschew the opportunity. The most pragmatic analytical strategy for in-house evaluation of such programs is likely to be the pre-intervention/post-intervention design in which the control group consists of people who were invited to participate in the DM program, but declined the invitation. Regression estimates of program impacts may be statistically biased if factors that impact participation decisions are correlated with outcomes measures. This paper describes an econometric procedure, the Treatment Effects model, developed to produce statistically unbiased estimates of program impacts in this type of situation. Two equations are estimated to (a) estimate the impacts of patient characteristics on decisions to participate in the program, and then (b) use this information to produce a statistically unbiased estimate of the impact of program participation on outcomes. This methodology is well-established in economics and econometrics, but has not been widely applied in the DM outcomes measurement literature; hence, this paper focuses on one illustrative application.

Ohio's Abandoned Mine Lands Reclamation Program: a Study of Data Collection and Evaluation Techniques

NASA Technical Reports Server (NTRS)

Sperry, S. L.

1982-01-01

The planning process for a statewide reclamation plan of Ohio abandoned minelands in response to the Federal Surface Mining Control and Reclamation Act of 1977 included: (1) the development of a screening and ranking methodology; (2) the establishment of a statewide review of major watersheds affected by mining; (3) the development of an immediate action process; and (4) a prototypical study of a priority watershed demonstrating the data collection, analysis, display and evaluation to be used for the remaining state watersheds. Historical methods for satisfying map information analysis and evaluation, as well as current methodologies being used were discussed. Various computer mapping and analysis programs were examined for their usability in evaluating the priority reclamation sites. Hand methods were chosen over automated procedures; intuitive evaluation was the primary reason.

How Do US Pediatric Residency Programs Teach and Evaluate Community Pediatrics and Advocacy Training?

PubMed

Lichtenstein, Cara; Hoffman, Benjamin D; Moon, Rachel Y

2017-07-01

In 2013, the Accreditation Council for Graduate Medical Education updated requirements for training in community pediatrics and advocacy in pediatric residency programs. In light of this update, the aim of this study was to better understand how community pediatrics is being taught and evaluated in pediatric residency programs in the United States. Cross-sectional exploratory study using a Web-based survey of pediatric residency program directors in September 2014. Questions focused on teaching and evaluation of 10 community pediatrics competencies. Of 85 programs (43% response rate), 30% offered a separate training track and/or 6-block individualized curriculum in community pediatrics or advocacy. More than 75% required all residents to learn 7 of 10 competencies queried. Respondents in urban settings were more likely to teach care of special populations (P = .02) and public speaking (P < .01). Larger programs were more likely to teach (P = .04) and evaluate (P = .02) community-based research. Experiential learning and classroom-based didactics were the most frequent teaching methodologies. Many programs used multiple teaching methodologies for all competencies. Observation was the most frequent evaluation technique used; portfolio review and written reflection were also commonly reported. Our findings show a strong emphasis on community pediatrics and advocacy teaching among responding US pediatric residency programs. Although respondents reported a variety of teaching and evaluation methods, there were few statistically significant differences between programs. Copyright © 2017 Academic Pediatric Association. Published by Elsevier Inc. All rights reserved.

Novel horizontal and vertical integrated bioethics curriculum for medical courses.

PubMed

D'Souza, Russell F; Mathew, Mary; D'Souza, Derek S J; Palatty, Princy

2018-02-28

Studies conducted by the University of Haifa, Israel in 2001, evaluating the effectiveness of bioethics being taught in medical colleges, suggested that there was a significant lack of translation in clinical care. Analysis also revealed, ineffectiveness with the teaching methodology used, lack of longitudinal integration of bioethics into the undergraduate medical curriculum, and the limited exposure to the technology in decision making when confronting ethical dilemmas. A modern novel bioethics curriculum and innovative methodology for teaching bioethics for the medical course was developed by the UNESCO Chair in Bioethics, Haifa. The horizontal (subject-wise) curriculum was vertically integrated seamlessly through the entire course. An innovative bioethics teaching methodology was employed to implement the curriculum. This new curriculum was piloted in a few medical colleges in India from 2011 to 2015 and the outcomes were evaluated. The evaluation confirmed gains over the earlier identified translation gap with added high student acceptability and satisfaction. This integrated curriculum is now formally implemented in the Indian program's Health Science Universities which is affiliated with over 200 medical schools in India. This article offers insights from the evaluated novel integrated bioethics curriculum and the innovative bioethics teaching methodology that was used in the pilot program.

Estimating Impacts on Students, Teachers, and Schools of Programs Targeted for Preschool through Transition to College--Lessons for Methodologists from the Designs of 117 Evaluations Being Conducted in the Investing in Innovation Fund (i3) Grant Program.

ERIC Educational Resources Information Center

Price, Cristofer; Wolf, Anne; Goodson, Barbara; Boulay, Beth

2015-01-01

When presenting results from rigorous, field-based evaluations of educational interventions, researchers often reveal methodological barriers they face in designing studies to assessing program effectiveness. The evaluations funded by the Investing in Innovation Fund (i3) present a timely opportunity for identifying relevant and pressing issues…

Information System Design Methodology Based on PERT/CPM Networking and Optimization Techniques.

ERIC Educational Resources Information Center

Bose, Anindya

The dissertation attempts to demonstrate that the program evaluation and review technique (PERT)/Critical Path Method (CPM) or some modified version thereof can be developed into an information system design methodology. The methodology utilizes PERT/CPM which isolates the basic functional units of a system and sets them in a dynamic time/cost…

Comparing the Medicaid Retrospective Drug Utilization Review Program Cost-Savings Methods Used by State Agencies

PubMed Central

Prada, Sergio I.

2017-01-01

Background The Medicaid Drug Utilization Review (DUR) program is a 2-phase process conducted by Medicaid state agencies. The first phase is a prospective DUR and involves electronically monitoring prescription drug claims to identify prescription-related problems, such as therapeutic duplication, contraindications, incorrect dosage, or duration of treatment. The second phase is a retrospective DUR and involves ongoing and periodic examinations of claims data to identify patterns of fraud, abuse, underutilization, drug–drug interaction, or medically unnecessary care, implementing corrective actions when needed. The Centers for Medicare & Medicaid Services requires each state to measure prescription drug cost-savings generated from its DUR programs on an annual basis, but it provides no guidance or unified methodology for doing so. Objectives To describe and synthesize the methodologies used by states to measure cost-savings using their Medicaid retrospective DUR program in federal fiscal years 2014 and 2015. Method For each state, the cost-savings methodologies included in the Medicaid DUR 2014 and 2015 reports were downloaded from Medicaid's website. The reports were then reviewed and synthesized. Methods described by the states were classified according to research designs often described in evaluation textbooks. Discussion In 2014, the most often used prescription drugs cost-savings estimation methodology for the Medicaid retrospective DUR program was a simple pre-post intervention method, without a comparison group (ie, 12 states). In 2015, the most common methodology used was a pre-post intervention method, with a comparison group (ie, 14 states). Comparisons of savings attributed to the program among states are still unreliable, because of a lack of a common methodology available for measuring cost-savings. Conclusion There is great variation among states in the methods used to measure prescription drug utilization cost-savings. This analysis suggests that there is still room for improvement in terms of methodology transparency, which is important, because lack of transparency hinders states from learning from each other. Ultimately, the federal government needs to evaluate and improve its DUR program. PMID:29403573

Peace Works: A Program Evaluation Model

ERIC Educational Resources Information Center

LeBlanc, Patrice; Lacey, Candace

2002-01-01

The purpose of this paper is to describe the model used to evaluate the two-year Allegany Grant between the Peace Education Foundation and the Miami-Dade County Public Schools that focused on reducing school violence through implementing the "Peace Works" program. A mixed method design using simultaneous methodologies was employed. For…

How Methodological Features Affect Effect Sizes in Education

ERIC Educational Resources Information Center

Cheung, Alan C. K.; Slavin, Robert E.

2016-01-01

As evidence becomes increasingly important in educational policy, it is essential to understand how research design might contribute to reported effect sizes in experiments evaluating educational programs. A total of 645 studies from 12 recent reviews of evaluations of preschool, reading, mathematics, and science programs were studied. Effect…

Military Design Insights for Online Education Program Evaluation: A Revised Theoretical Construct

ERIC Educational Resources Information Center

Culkin, David T.

2017-01-01

This theoretical development article examines how design methodology currently applied in United States military doctrine can offer insights into the increasingly complex arena of program evaluations of online modes for adult distance education. The article presents key themes that emerge from a multidisciplinary literature review. These themes…

Evaluating Intervention Programs with a Pretest-Posttest Design: A Structural Equation Modeling Approach

PubMed Central

Alessandri, Guido; Zuffianò, Antonio; Perinelli, Enrico

2017-01-01

A common situation in the evaluation of intervention programs is the researcher's possibility to rely on two waves of data only (i.e., pretest and posttest), which profoundly impacts on his/her choice about the possible statistical analyses to be conducted. Indeed, the evaluation of intervention programs based on a pretest-posttest design has been usually carried out by using classic statistical tests, such as family-wise ANOVA analyses, which are strongly limited by exclusively analyzing the intervention effects at the group level. In this article, we showed how second order multiple group latent curve modeling (SO-MG-LCM) could represent a useful methodological tool to have a more realistic and informative assessment of intervention programs with two waves of data. We offered a practical step-by-step guide to properly implement this methodology, and we outlined the advantages of the LCM approach over classic ANOVA analyses. Furthermore, we also provided a real-data example by re-analyzing the implementation of the Young Prosocial Animation, a universal intervention program aimed at promoting prosociality among youth. In conclusion, albeit there are previous studies that pointed to the usefulness of MG-LCM to evaluate intervention programs (Muthén and Curran, 1997; Curran and Muthén, 1999), no previous study showed that it is possible to use this approach even in pretest-posttest (i.e., with only two time points) designs. Given the advantages of latent variable analyses in examining differences in interindividual and intraindividual changes (McArdle, 2009), the methodological and substantive implications of our proposed approach are discussed. PMID:28303110

Evaluating Intervention Programs with a Pretest-Posttest Design: A Structural Equation Modeling Approach.

PubMed

Alessandri, Guido; Zuffianò, Antonio; Perinelli, Enrico

2017-01-01

A common situation in the evaluation of intervention programs is the researcher's possibility to rely on two waves of data only (i.e., pretest and posttest), which profoundly impacts on his/her choice about the possible statistical analyses to be conducted. Indeed, the evaluation of intervention programs based on a pretest-posttest design has been usually carried out by using classic statistical tests, such as family-wise ANOVA analyses, which are strongly limited by exclusively analyzing the intervention effects at the group level. In this article, we showed how second order multiple group latent curve modeling (SO-MG-LCM) could represent a useful methodological tool to have a more realistic and informative assessment of intervention programs with two waves of data. We offered a practical step-by-step guide to properly implement this methodology, and we outlined the advantages of the LCM approach over classic ANOVA analyses. Furthermore, we also provided a real-data example by re-analyzing the implementation of the Young Prosocial Animation, a universal intervention program aimed at promoting prosociality among youth. In conclusion, albeit there are previous studies that pointed to the usefulness of MG-LCM to evaluate intervention programs (Muthén and Curran, 1997; Curran and Muthén, 1999), no previous study showed that it is possible to use this approach even in pretest-posttest (i.e., with only two time points) designs. Given the advantages of latent variable analyses in examining differences in interindividual and intraindividual changes (McArdle, 2009), the methodological and substantive implications of our proposed approach are discussed.

Indigenous health program evaluation design and methods in Australia: a systematic review of the evidence.

PubMed

Lokuge, Kamalini; Thurber, Katherine; Calabria, Bianca; Davis, Meg; McMahon, Kathryn; Sartor, Lauren; Lovett, Raymond; Guthrie, Jill; Banks, Emily

2017-10-01

Indigenous Australians experience a disproportionately higher burden of disease compared to non-Indigenous Australians. High-quality evaluation of Indigenous health programs is required to inform health and health services improvement. We aimed to quantify methodological and other characteristics of Australian Indigenous health program evaluations published in the peer-reviewed literature. Systematic review of peer-reviewed literature (November 2009-2014) on Indigenous health program evaluation. We identified 118 papers describing evaluations of 109 interventions; 72.0% were university/research institution-led. 82.2% of evaluations included a quantitative component; 49.2% utilised quantitative data only and 33.1% used both quantitative and qualitative data. The most common design was a before/after comparison (30.5%, n=36/118). 7.6% of studies (n=9/118) used an experimental design: six individual-level and three cluster-randomised controlled trials. 56.8% (67/118) reported on service delivery/process outcomes (versus health or health risk factor outcomes) only. Given the number of Indigenous health programs that are implemented, few evaluations overall are published in the peer-reviewed literature and, of these, few use optimal methodologies such as mixed methods and experimental design. Implications for public health: Multiple strategies are required to increase high-quality, accessible evaluation in Indigenous health, including supporting stronger research-policy-practice partnerships and capacity building for evaluation by health services and government. © 2017 The Authors.

Methodology used for Calculating VMT

EPA Pesticide Factsheets

The report provides the estimation of the annual VMT in California to help evaluate the effectiveness of the California s inspection and maintenance (I/M) program, also known as the Smog Check program.

Economic evaluation of drug abuse treatment and HIV prevention programs in pregnant women: a systematic review.

PubMed

Ruger, Jennifer Prah; Lazar, Christina M

2012-01-01

Drug abuse and transmission of HIV during pregnancy are public health problems that adversely affect pregnant women, their children and surrounding communities. Programs that address this vulnerable population have the ability to be cost-effective due to resulting cost savings for mother, child and society. Economic evaluations of programs that address these issues are an important tool to better understand the costs of services and create sustainable healthcare systems. This study critically examined economic evaluations of drug abuse treatment and HIV prevention programs in pregnant women. A systematic review was conducted using the criteria recommended by the Panel on Cost-Effectiveness in Health and Medicine and the British Medical Journal (BMJ) checklist for economic evaluations. The search identified 6 economic studies assessing drug abuse treatment for pregnant women, and 12 economic studies assessing programs that focus on prevention of mother-to-child transmission (PMTCT) of HIV. Results show that many programs for drug abuse treatment and PMTCT among pregnant women are cost-effective or even cost-saving. This study identified several shortcomings in methodology and lack of standardization of current economic evaluations. Efforts to address methodological challenges will help make future studies more comparable and have more influence on policy makers, clinicians and the public. Copyright © 2011 Elsevier Ltd. All rights reserved.

ITS impacts assessment for Seattle MMDI evaluation : modeling methodology and results

DOT National Transportation Integrated Search

1999-01-01

This document presents a modeling analysis of Intelligent Transportation Systems (ITS) impacts from the SmartTrek program in Seattle, Washington. This report describes the methodology of the study and presents the finding for a mixed freeway/arterial...

ITS impacts assessment for Seattle MMDI evaluation : modeling methodology and results

DOT National Transportation Integrated Search

1999-09-01

This document presents an executive summary of a modeling analysis of Intelligent Transportation Systems (ITS) impacts from the SmartTrek program in Seattle, Washington. This report describes the methodology of the study and presents the finding for ...

Guidelines for reporting evaluations based on observational methodology.

PubMed

Portell, Mariona; Anguera, M Teresa; Chacón-Moscoso, Salvador; Sanduvete-Chaves, Susana

2015-01-01

Observational methodology is one of the most suitable research designs for evaluating fidelity of implementation, especially in complex interventions. However, the conduct and reporting of observational studies is hampered by the absence of specific guidelines, such as those that exist for other evaluation designs. This lack of specific guidance poses a threat to the quality and transparency of these studies and also constitutes a considerable publication hurdle. The aim of this study thus was to draw up a set of proposed guidelines for reporting evaluations based on observational methodology. The guidelines were developed by triangulating three sources of information: observational studies performed in different fields by experts in observational methodology, reporting guidelines for general studies and studies with similar designs to observational studies, and proposals from experts in observational methodology at scientific meetings. We produced a list of guidelines grouped into three domains: intervention and expected outcomes, methods, and results. The result is a useful, carefully crafted set of simple guidelines for conducting and reporting observational studies in the field of program evaluation.

Evaluating department of transportation's research program : a methodology and case study.

DOT National Transportation Integrated Search

2012-06-01

An effective research program within a transportation organization can be a valuable asset to accomplish the goals of the overall : mission. Determining whether a research program is pursuing relevant research projects and obtaining results for the s...

A Critical Meta-Analysis of All Evaluations of State-Funded Preschool from 1977 to 1998: Implications for Policy, Service Delivery and Program Evaluation.

ERIC Educational Resources Information Center

Gilliam, Walter S.; Zigler, Edward F.

2000-01-01

Presents a meta-analytic review of evaluations of state-funded preschool programs over 20 years. Identifies several methodological flaws but also suggests that pattern of findings offers modest support for positive impact in improving children's developmental competence, improving later school attendance and performance, and reducing subsequent…

REQUIREMENT VERIFICATION AND SYSTEMS ENGINEERING TECHNICAL REVIEW (SETR) ON A COMMERCIAL DERIVATIVE AIRCRAFT (CDA) PROGRAM

DTIC Science & Technology

2017-09-01

the SETR entrance criteria of these events. Out of 30 evaluated SETR entrance criteria, 22 map to FAA elements. A case study of a military CDA...evaluated SETR entrance criteria, 22 map to FAA elements. A case study of a military CDA program, the Presidential Helicopter Replacement Program...3 C. SCOPE AND METHODOLOGY .................................................. 4 D. ORGANIZATION OF THESIS

Parts on Demand: Evaluation of Approaches to Achieve Flexible Manufacturing Systems for Navy Parts on Demand. Volume 2. Appendices

DTIC Science & Technology

1984-02-01

of Cost Analysis Worksheets * POD Program-Economic Analysis & Methodology - Economic Evaluation Procedures for POD Investment Program - System...Approval Considerations - POD Investment Program - Potential Improvement Areas for POD - Example Cost Categories and Determinants Appendix E Long Range R&D...Funding Profiles * Investment Strategy for Integrated Circuits Diminishing Sources of Supply - Problem Scope - Approach - Alternatives - Proposed

[Use of evidence in heath policies and programs contributions of the instituto nacional de salud].

PubMed

Gutierrez, Ericson L; Piazza, Marina; Gutierrez-Aguado, Alfonso; Hijar, Gisely; Carmona, Gloria; Caballero, Patricia; Reyes, Nora; Canelo, Carlos; Aparco, Juan Pablo; Tejada, Romina A; Bolaños-Díaz, Rafael; Saravia, Silvia; Gozzer, Ernesto

2016-01-01

This article analyzes some examples about how the Ministry of Health of Peru has used evidence for policy and program formulation, implementation and evaluation. It describes the process by which health budget programs are based and strengthened with scientific evidence. Provides an overview about how the development of clinical guidelines methodology is facilitating the generation of high quality evidence based clinical guidelines.It presents some examples of specific information needs of the Ministry of Health to which the Instituto Nacional de Salud has responded, and the impact of that collaboration. Finally, the article proposes future directions for the use of research methodology especially relevant for the development and evaluation of policy and programs, as well as the development of networks of health technology assessment at the national and international level.

Procedures for Trade and Industrial Program Development.

ERIC Educational Resources Information Center

Campbell, Clifton P.

The instructional systems development (ISD) approach for the development and accomplishment of vocational training programs provides a methodology for gathering and analyzing job information, developing instructional materials in a variety of media, conducting instruction, and evaluating and improving the effectiveness of training programs. This…

Interactive multi-mode blade impact analysis

NASA Technical Reports Server (NTRS)

Alexander, A.; Cornell, R. W.

1978-01-01

The theoretical methodology used in developing an analysis for the response of turbine engine fan blades subjected to soft-body (bird) impacts is reported, and the computer program developed using this methodology as its basis is described. This computer program is an outgrowth of two programs that were previously developed for the purpose of studying problems of a similar nature (a 3-mode beam impact analysis and a multi-mode beam impact analysis). The present program utilizes an improved missile model that is interactively coupled with blade motion which is more consistent with actual observations. It takes into account local deformation at the impact area, blade camber effects, and the spreading of the impacted missile mass on the blade surface. In addition, it accommodates plate-type mode shapes. The analysis capability in this computer program represents a significant improvement in the development of the methodology for evaluating potential fan blade materials and designs with regard to foreign object impact resistance.

List randomization for soliciting experience of intimate partner violence: Application to the evaluation of Zambia's unconditional child grant program.

PubMed

Peterman, Amber; Palermo, Tia M; Handa, Sudhanshu; Seidenfeld, David

2018-03-01

Social scientists have increasingly invested in understanding how to improve data quality and measurement of sensitive topics in household surveys. We utilize the technique of list randomization to collect measures of physical intimate partner violence in an experimental impact evaluation of the Government of Zambia's Child Grant Program. The Child Grant Program is an unconditional cash transfer, which targeted female caregivers of children under the age of 5 in rural areas to receive the equivalent of US $24 as a bimonthly stipend. The implementation results show that the list randomization methodology functioned as planned, with approximately 15% of the sample identifying 12-month prevalence of physical intimate partner violence. According to this measure, after 4 years, the program had no measurable effect on partner violence. List randomization is a promising approach to incorporate sensitive measures into multitopic evaluations; however, more research is needed to improve upon methodology for application to measurement of violence. Copyright © 2017 John Wiley & Sons, Ltd.

Class Size Reduction in a Large Urban School District: A Mixed Methodology Evaluation Research Study.

ERIC Educational Resources Information Center

Munoz, Marco A.

This study evaluated the Class Size Reduction (CSR) program in 34 elementary schools in Kentucky's Jefferson County Public Schools. The CSR program is a federal initiative to help elementary schools improve student learning by hiring additional teachers. Qualitative data were collected using unstructured interviews, site observations, and document…

Evaluating Quality in Associate Degree Culinary Arts Programs

ERIC Educational Resources Information Center

Hertzman, Jean; Ackerman, Robert

2010-01-01

Purpose: The purpose of this study is to determine which categories and indicators of quality are best suited to evaluating associate degree culinary arts programs (ADCAP). Design/methodology/approach: The researchers surveyed a national sample of culinary educators and industry chefs in the USA. The instrument asked the participants to rate the…

Evaluation and communication: using a communication audit to evaluate organizational communication.

PubMed

Hogard, Elaine; Ellis, Roger

2006-04-01

This article identifies a surprising dearth of studies that explicitly link communication and evaluation at substantive, theoretical, and methodological levels. A three-fold typology of evaluation studies referring to communication is proposed and examples given. The importance of organizational communication in program delivery is stressed and illustrative studies reviewed. It is proposed that organizational communication should be considered in all program evaluations and that this should be approached through communication audit. Communication audits are described with particular reference to established survey questionnaire instruments. Two case studies exemplify the use of such instruments in the evaluation of educational and social programs.

Effects of surface chemistry on hot corrosion life

NASA Technical Reports Server (NTRS)

Fryxell, R. E.; Leese, G. E.

1985-01-01

This program has its primary objective: the development of hot corrosion life prediction methodology based on a combination of laboratory test data and evaluation of field service turbine components which show evidence of hot corrosion. The laboratory program comprises burner rig testing by TRW. A summary of results is given for two series of burner rig tests. The life prediction methodology parameters to be appraised in a final campaign of burner rig tests are outlined.

Empowerment Evaluation: A Form of Self-Evaluation.

ERIC Educational Resources Information Center

Fetterman, David M.

Empowerment evaluation is an innovative approach that uses evaluation concepts and techniques to foster improvement and self-determination. Empowerment evaluation employs qualitative and quantitative methodologies. Although it can be applied to individuals and organizations, the usual focus is on programs. The value orientation of empowerment…

Appendix B: Methodology. [2014 Teacher Prep Review

ERIC Educational Resources Information Center

Greenberg, Julie; Walsh, Kate; McKee, Arthur

2014-01-01

The "NCTQ Teacher Prep Review" evaluates the quality of programs that provide preservice preparation of public school teachers. This appendix describes the scope, methodology, timeline, staff, and standards involved in the production of "Teacher Prep Review 2014." Data collection, validation, and analysis for the report are…

Studying the Education of Educators: Methodology.

ERIC Educational Resources Information Center

Sirotnik, Kenneth A.

1988-01-01

Describes the methodology and research design of SEE, the study of the Education of Educators. The approach is multimethodological, exploratory, descriptive, and evaluative. The research design permits examination of working assumptions and concentration on the individual site--the college, the education departments, and specific programs within…

A Modeling Methodology to Support Evaluation Public Health Impacts on Air Pollution Reduction Programs

EPA Science Inventory

Environmental public health protection requires a good understanding of types and locations of pollutant emissions of health concern and their relationship to environmental public health indicators. Therefore, it is necessary to develop the methodologies, data sources, and tools...

2009 DOE Vehicle Technologies Program Annual Merit Review

DOE Office of Scientific and Technical Information (OSTI.GOV)

none,

2009-10-01

Annual Merit Review and Peer Evaluation Meeting to review the FY2008 accomplishments and FY2009 plans for the Vehicle Technologies Program, and provide an opportunity for industry, government, and academic to give inputs to DOE on the Program with a structured and formal methodology.

Culturally Competent Research with American Indians and Alaska Natives: Findings and Recommendations of the First Symposium of the Work Group on American Indian Research and Program Evaluation Methodology

ERIC Educational Resources Information Center

Caldwell, Joyce Y.; Davis, Jamie D.; Du Bois, Barbara; Echo-Hawk, Holly; Erickson, Jill Shepard; Goins, R. Turner; Hill, Calvin; Hillabrant, Walter; Johnson, Sharon R.; Kendall, Elizabeth; Keemer, Kelly; Manson, Spero M.; Marshall, Catherine A.; Running Wolf, Paulette; Santiago, Rolando L.; Schacht, Robert; Stone, Joseph B.

2005-01-01

This article describes the collective experience of a multidisciplinary network of researchers, practitioners, and program evaluators who support appropriate research and evaluation methods in working with Native peoples. Our experience underlines the critical importance of culture in understanding and conducting research with the diverse…

Identifying Potential Biasing Variables in Student Evaluation of Teaching in a Newly Accredited Business Program in the UAE

ERIC Educational Resources Information Center

Badri, Masood A.; Abdulla, Mohamed; Kamali, Mohammed A.; Dodeen, Hamzeh

2006-01-01

Purpose: The purpose of this paper is to investigate the effect of many factors on student evaluation of teaching. Design/methodology/approach: The study analyzed 3,185 student evaluations of faculty from a newly accredited business program at the United Arab Emirates University using univariate and multi-analysis of variance (ANOVA and MANOVA).…

The development of a streamlined, coordinated and sustainable evaluation methodology for a diverse chronic disease management program.

PubMed

Berlowitz, David J; Graco, Marnie

2010-05-01

The Northern Alliance Hospital Admission Risk Program-Chronic Disease Management comprises 13 services delivering care to those with chronic disease and older people with complex care needs, who are frequent hospital users. To develop and implement a system-wide approach to the evaluation of this existing program. The Northern Clinical Research Centre audited all existing, routinely collected administrative data within the program and then met with each service to develop service specific outcome measures. The evaluators then developed and implemented a system-wide evaluation approach to measure performance in terms of: client profile; access and entry; service efficiency; client outcomes; and hospital demand. Data are collected electronically and more than 80% are derived from existing, administrative datasets, minimising staff and client burden. Additional data include client outcomes and a health related quality of life measure. The preliminary twelve month data suggest that clients have the equivalent of 'fair' or 'poor' self-reported health status (n = 862) and the average health utility scores are significantly (P < 0.05) worse than population control data. These analyses reveal, for the first time, that the Northern Alliance Hospital Admission Risk Program-Chronic Disease Management program is targeting appropriate clients. This methodology will enable many prospective assessments to be performed including; client outcome evaluation, service model comparisons, and cost-utility analyses. This evaluation approach demonstrates the feasibility of a highly coordinated 'whole of system' evaluation. Such an approach may ultimately contribute to the development of evidence-based policy.

Development of Testing Methodologies to Evaluate Postflight Locomotor Performance

NASA Technical Reports Server (NTRS)

Mulavara, A. P.; Peters, B. T.; Cohen, H. S.; Richards, J. T.; Miller, C. A.; Brady, R.; Warren, L. E.; Bloomberg, J. J.

2006-01-01

Crewmembers experience locomotor and postural instabilities during ambulation on Earth following their return from space flight. Gait training programs designed to facilitate recovery of locomotor function following a transition to a gravitational environment need to be accompanied by relevant assessment methodologies to evaluate their efficacy. The goal of this paper is to demonstrate the operational validity of two tests of locomotor function that were used to evaluate performance after long duration space flight missions on the International Space Station (ISS).

Impact Evaluation of the U.S. Department of Energy's Solar Decathlon Program

DOE Office of Scientific and Technical Information (OSTI.GOV)

Barnes, Harley

2012-12-01

This report includes the methodology and findings in evaluating DOE’s Solar Decathlon event. The primary purpose of this evaluation is to learn how effectively the Solar Decathlon event is in meeting its objectives.

Feasibility of "Standardized Clinician" Methodology for Patient Training on Hospital-to-Home Transitions.

PubMed

Wehbe-Janek, Hania; Hochhalter, Angela K; Castilla, Theresa; Jo, Chanhee

2015-02-01

Patient engagement in health care is increasingly recognized as essential for promoting the health of individuals and populations. This study pilot tested the standardized clinician (SC) methodology, a novel adaptation of standardized patient methodology, for teaching patient engagement skills for the complex health care situation of transitioning from a hospital back to home. Sixty-seven participants at heightened risk for hospitalization were randomly assigned to either simulation exposure-only or full-intervention group. Both groups participated in simulation scenarios with "standardized clinicians" around tasks related to hospital discharge and follow-up. The full-intervention group was also debriefed after scenario sets and learned about tools for actively participating in hospital-to-home transitions. Measures included changes in observed behaviors at baseline and follow-up and an overall program evaluation. The full-intervention group showed increases in observed tool possession (P = 0.014) and expression of their preferences and values (P = 0.043). The simulation exposure-only group showed improvement in worksheet scores (P = 0.002) and fewer engagement skills (P = 0.021). Both groups showed a decrease in telling an SC about their hospital admission (P < 0.05). Open-ended comments from the program evaluation were largely positive. Both groups benefited from exposure to the SC intervention. Program evaluation data suggest that simulation training is feasible and may provide a useful methodology for teaching patient skills for active engagement in health care. Future studies are warranted to determine if this methodology can be used to assess overall patient engagement and whether new patient learning transfers to health care encounters.

Evaluating employee assistance programs. A review of methods, outcomes, and future directions.

PubMed

Jerrell, J M; Rightmyer, J F

1982-01-01

Renewed interest in assisting troubled employees has led to an upsurge in the development of employee assistance programs, coupled with demands for demonstrable effectiveness. This review examines the nature and scope of these programs, their administrative and methodological context, and the types and outcomes of evaluation studies conducted thus far. Proposals for improving future investigations through a number of different approaches and strategies are then made.

Manned space station environmental control and life support system computer-aided technology assessment program

NASA Technical Reports Server (NTRS)

Hall, J. B., Jr.; Pickett, S. J.; Sage, K. H.

1984-01-01

A computer program for assessing manned space station environmental control and life support systems technology is described. The methodology, mission model parameters, evaluation criteria, and data base for 17 candidate technologies for providing metabolic oxygen and water to the crew are discussed. Examples are presented which demonstrate the capability of the program to evaluate candidate technology options for evolving space station requirements.

Benefit-cost analysis of addiction treatment: methodological guidelines and empirical application using the DATCAP and ASI.

PubMed

French, Michael T; Salomé, Helena J; Sindelar, Jody L; McLellan, A Thomas

2002-04-01

To provide detailed methodological guidelines for using the Drug Abuse Treatment Cost Analysis Program (DATCAP) and Addiction Severity Index (ASI) in a benefit-cost analysis of addiction treatment. A representative benefit-cost analysis of three outpatient programs was conducted to demonstrate the feasibility and value of the methodological guidelines. Procedures are outlined for using resource use and cost data collected with the DATCAP. Techniques are described for converting outcome measures from the ASI to economic (dollar) benefits of treatment. Finally, principles are advanced for conducting a benefit-cost analysis and a sensitivity analysis of the estimates. The DATCAP was administered at three outpatient drug-free programs in Philadelphia, PA, for 2 consecutive fiscal years (1996 and 1997). The ASI was administered to a sample of 178 treatment clients at treatment entry and at 7-months postadmission. The DATCAP and ASI appear to have significant potential for contributing to an economic evaluation of addiction treatment. The benefit-cost analysis and subsequent sensitivity analysis all showed that total economic benefit was greater than total economic cost at the three outpatient programs, but this representative application is meant to stimulate future economic research rather than justifying treatment per se. This study used previously validated, research-proven instruments and methods to perform a practical benefit-cost analysis of real-world treatment programs. The study demonstrates one way to combine economic and clinical data and offers a methodological foundation for future economic evaluations of addiction treatment.

Advanced Public Transportation Systems. Technical Assistance Brief 2

DOT National Transportation Integrated Search

1993-01-01

The Advanced Public Transportation Systems (APTS) Program requires evaluation : of pilot projects. This technical assistance brief discusses the guidelines set : for developing evaluation framework and methodology for local projects. 4p.

NASA airframe structural integrity program

NASA Technical Reports Server (NTRS)

Harris, Charles E.

1991-01-01

NASA has initiated a research program with the long-term objective of supporting the aerospace industry in addressing issues related to the aging commercial transport fleet. The interdisciplinary program combines advanced fatigue crack growth prediction methodology with innovative nondestructive examination technology with the focus on multi-site damage (MSD) at riveted connections. A fracture mechanics evaluation of the concept of pressure proof testing the fuselage to screen for MSD has been completed. Also, a successful laboratory demonstration of the ability of the thermal flux method to detect disbonds at riveted lap splice joints has been conducted. All long-term program elements have been initiated and the plans for the methodology verification program are being coordinated with the airframe manufacturers.

NASA airframe structural integrity program

NASA Technical Reports Server (NTRS)

Harris, Charles E.

1990-01-01

NASA initiated a research program with the long-term objective of supporting the aerospace industry in addressing issues related to the aging of the commercial transport fleet. The program combines advanced fatigue crack growth prediction methodology with innovative nondestructive examination technology with the focus on multi-stage damage (MSD) at rivited connections. A fracture mechanics evaluation of the concept of pressure proof testing the fuselage to screen for MSD was completed. A successful laboratory demonstration of the ability of the thermal flux method to detect disbonds at rivited lap splice joints was conducted. All long-term program elements were initiated, and the plans for the methodology verification program are being coordinated with the airframe manufacturers.

Facility Energy Performance Benchmarking in a Data-Scarce Environment

DTIC Science & Technology

2017-08-01

environment, and analyze occupant-, system-, and component-level faults contributing to energy in- efficiency. A methodology for developing DoD-specific...Research, Development, Test, and Evaluation (RDTE) Program to develop an intelligent framework, encompassing methodology and model- ing, that...energy performers by installation, climate zone, and other criteria. A methodology for creating the DoD-specific EUIs would be an important part of a

United States of America Ocean Dumping Report for Calendar Year 1981. Dredged Material.

DTIC Science & Technology

1982-06-01

to monitoring studies under the Disposal Area Monitoring System ( DAHOS ) program. The program is designed to identify and evaluate impacts resulting...sites. The DAHOS program continually contributes to the development of new monitoring methodologies that reflect on the efficiency of field observations

Assessing Conformity to Standards for Treatment Foster Care.

ERIC Educational Resources Information Center

Farmer, Elizabeth M. Z.; Burns, Barbara J.; Dubs, Melanie S.; Thompson, Shealy

2002-01-01

This study examined conformity to the Program Standards for Treatment Foster Care among 42 statewide programs. Findings suggest fair to good overall conformity, with considerable variation among programs. A discussion of methodological and substantive considerations for future research and evaluation using this approach is included. (Contains…

Toward a Capacity Framework for Useful Student Learning Outcomes Assessment in College Foreign Language Programs

ERIC Educational Resources Information Center

Davis, John McEwan

2016-01-01

The educational impacts of mandated assessment in U.S. colleges is part of a growing research agenda focused on how methodologies of program evaluation best enable educators to improve teaching and learning. Accordingly, research has tried to identify the key aspects of evaluation/assessment "capacity" in college language departments…

76 FR 51056 - Notice of Random Assignment Study To Evaluate the YouthBuild Program; Request for Comment

Federal Register 2010, 2011, 2012, 2013, 2014

2011-08-17

... assigned to the control group can re-apply to the YouthBuild program. The Department is soliciting comments... ``utilize appropriate methodology and research designs, including the use of control groups chosen by.... Accordingly, ETA is conducting a multisite control group evaluation to provide rigorous estimates [[Page 51058...

Development, Implementation, and Evaluation of Leadership/Management Training Within Army Battalions: Volume I: Summary of Findings.

ERIC Educational Resources Information Center

Fry, John P.; Cliborn, Robert E.

The report describes the development and evaluation of an in-unit, leadership/management training program (based on experimental training methodology for providing Army leaders with behavioral skills and techniques) implemented within three battalion-sized combat units at Fort Bliss, Texas, in 1974. The program was organized around workshops…

The Evaluator's Perspective: Evaluating the State Capacity Building Program.

ERIC Educational Resources Information Center

Madey, Doren L.

A historical antagonism between the advocates of quantitative evaluation methods and the proponents of qualitative evaluation methods has stymied the recognition of the value to be gained by utilizing both methodologies in the same study. The integration of quantitative and qualitative methods within a single evaluation has synergistic effects in…

Methodological Considerations for Use of Routine Health Information System Data to Evaluate Malaria Program Impact in an Era of Declining Malaria Transmission

PubMed Central

Ashton, Ruth A.; Bennett, Adam; Yukich, Joshua; Bhattarai, Achuyt; Keating, Joseph; Eisele, Thomas P.

2017-01-01

Abstract. Coverage of malaria control interventions is increasing dramatically across endemic countries. Evaluating the impact of malaria control programs and specific interventions on health indicators is essential to enable countries to select the most effective and appropriate combination of tools to accelerate progress or proceed toward malaria elimination. When key malaria interventions have been proven effective under controlled settings, further evaluations of the impact of the intervention using randomized approaches may not be appropriate or ethical. Alternatives to randomized controlled trials are therefore required for rigorous evaluation under conditions of routine program delivery. Routine health management information system (HMIS) data are a potentially rich source of data for impact evaluation, but have been underused in impact evaluation due to concerns over internal validity, completeness, and potential bias in estimates of program or intervention impact. A range of methodologies were identified that have been used for impact evaluations with malaria outcome indicators generated from HMIS data. Methods used to maximize internal validity of HMIS data are presented, together with recommendations on reducing bias in impact estimates. Interrupted time series and dose-response analyses are proposed as the strongest quasi-experimental impact evaluation designs for analysis of malaria outcome indicators from routine HMIS data. Interrupted time series analysis compares the outcome trend and level before and after the introduction of an intervention, set of interventions or program. The dose-response national platform approach explores associations between intervention coverage or program intensity and the outcome at a subnational (district or health facility catchment) level. PMID:28990915

Teaching Environmental Education through PBL: Evaluation of a Teaching Intervention Program

NASA Astrophysics Data System (ADS)

Vasconcelos, Clara

2012-04-01

If our chosen aim in science education is to be inclusive and to improve students' learning achievements, then we must identify teaching methodologies that are appropriate for teaching and learning specific knowledge. Karagiorgi and Symeo 2005) remind us that instructional designers are thus challenged to translate the philosophy of constructivism into current practice. Thus, research in science education must focus on evaluating intervention programs which ensure the effective construction of knowledge and development of competencies. The present study reports the elaboration, application and evaluation of a problem-based learning (PBL) program with the aim of examining its effectiveness with students learning Environmental Education. Prior research on both PBL and Environmental Education (EE) was conducted within the context of science education so as to elaborate and construct the intervention program. Findings from these studies indicated both the PBL methodology and EE as helpful for teachers and students. PBL methodology has been adopted in this study since it is logically incorporated in a constructivism philosophy application (Hendry et al. 1999) and it was expected that this approach would assist students towards achieving a specific set of competencies (Engel 1997). On the other hand, EE has evolved at a rapid pace within many countries in the new millennium (Hart 2007), unlike any other educational area. However, many authors still appear to believe that schools are failing to prepare students adequately in EE (Walsche 2008; Winter 2007). The following section describes the research that was conducted in both areas so as to devise the intervention program.

[Educative programs based on self-management: an integrative review].

PubMed

Nascimento, Luciana da Silva; de Gutierrez, Maria Gaby Rivero; De Domenico, Edvane Birelo Lopes

2010-06-01

The objective was to identify definitions and/or explanations of the term self-management in educative programs that aim its development. The authors also aimed to describe the educative plans and results of the educative programs analyzed. As a methodology we used integrative review, with 15 published articles (2002 the 2007). The inclusion criteria was: the occurrence of the term self-management; the existence of an educative program for the development of self-management; to be related to the area of the health of the adult. Self-management means the improvement or acquisition of abilities to solve problems in biological, social and affective scopes. The review pointed to different educational methodologies. However, it also showed the predominance of traditional methods, with conceptual contents and of physiopathological nature. The learning was evaluated as favorable, with warns in relation to the application in different populations and contexts and to the increase of costs of the educative intervention. It was concluded that research has evidenced the importance of the education for self-management, but lacked in strength for not relating the biopsychosocial demands of the chronic patient and for not describing in detail the teaching and evaluation methodologies employed.

Residency Programs' Evaluations of the Competencies: Data Provided to the ACGME About Types of Assessments Used by Programs

PubMed Central

Holt, Kathleen D.; Miller, Rebecca S.; Nasca, Thomas J.

2010-01-01

Background In 1999, the Accreditation Council for Graduate Medical Education (ACGME) Outcome Project began to focus on resident performance in the 6 competencies of patient care, medical knowledge, professionalism, practice-based learning and improvement, interpersonal communication skills, and professionalism. Beginning in 2007, the ACGME began collecting information on how programs assess these competencies. This report provides information on the nature and extent of those assessments. Methods Using data collected by the ACGME for site visits, we use descriptive statistics and percentages to describe the number and type of methods and assessors accredited programs (n  =  4417) report using to assess the competencies. Observed differences among specialties, methodologies, and assessors are tested with analysis of variance procedures. Results Almost all (>97%) of programs report assessing all of the competencies and using multiple methods and multiple assessors. Similar assessment methods and evaluator types were consistently used across the 6 competencies. However, there were some differences in the use of patient and family as assessors: Primary care and ambulatory specialties used these to a greater extent than other specialties. Conclusion Residency programs are emphasizing the competencies in their evaluation of residents. Understanding the scope of evaluation methodologies that programs use in resident assessment is important for both the profession and the public, so that together we may monitor continuing improvement in US graduate medical education. PMID:22132294

Activities for Engaging Schools in Health Promotion

ERIC Educational Resources Information Center

Bardi, Mohammad; Burbank, Andrea; Choi, Wayne; Chow, Lawrence; Jang, Wesley; Roccamatisi, Dawn; Timberley-Berg, Tonia; Sanghera, Mandeep; Zhang, Margaret; Macnab, Andrew J.

2014-01-01

Purpose: The purpose of this paper is to describe activities used to initiate health promotion in the school setting. Design/Methodology/Approach: Description of successful pilot Health Promoting School (HPS) initiatives in Canada and Uganda and the validated measures central to each program. Evaluation methodologies: quantitative data from the…

Methodological Foundations for the Empirical Evaluation of Non-Experimental Methods in Field Settings

ERIC Educational Resources Information Center

Wong, Vivian C.; Steiner, Peter M.

2015-01-01

Across the disciplines of economics, political science, public policy, and now, education, the randomized controlled trial (RCT) is the preferred methodology for establishing causal inference about program impacts. But randomized experiments are not always feasible because of ethical, political, and/or practical considerations, so non-experimental…

Oral Health Promotion in Schools: Rationale and Evaluation

ERIC Educational Resources Information Center

Kizito, Alex; Caitlin, Meredith; Wang, Yili; Kasangaki, Arabat; Macnab, Andrew J.

2014-01-01

Purpose: The purpose of this paper is to explain the rationale and potential for the WHO health promoting schools (HPS) to improve children's oral health, and describe validated quantitative methodologies and qualitative approaches to measure program impact. Design/Methodology/Approach: Critical discussion of the impact of poor oral health and…

How Methodological Features Affect Effect Sizes in Education

ERIC Educational Resources Information Center

Cheung, Alan; Slavin, Robert

2016-01-01

As evidence-based reform becomes increasingly important in educational policy, it is becoming essential to understand how research design might contribute to reported effect sizes in experiments evaluating educational programs. The purpose of this study was to examine how methodological features such as types of publication, sample sizes, and…

Clinical and cost outcomes of multifactorial, cardiovascular risk management interventions in worksites: a comprehensive review and analysis.

PubMed

Pelletier, K R

1997-12-01

This paper is a critical review of the clinical and cost outcome evaluation studies of multifactorial, comprehensive, cardiovascular risk management programs in worksites. A comprehensive international literature search conducted under the auspices of the National Heart, Lung and Blood Institute identified 17 articles based on 12 studies that examined the clinical outcomes of multifactorial, comprehensive programs. These articles were identified through MEDLINE, manual searches of recent journals, and through direct inquiries to worksite health promotion researchers. All studies were conducted between 1978 and 1995, with 1978 being the date of the first citation of a methodologically rigorous evaluation. Of the 12 research studies, only 8 utilized the worksite as both the unit of assignment and as the unit of analysis. None of the studies analyzed adequately for cost effectiveness. Given this limitation, this review briefly considers the relevant worksite research that has demonstrated cost outcomes. Worksite-based, multifactorial cardiovascular intervention programs reviewed for this article varied widely in the comprehensiveness, intensity, and duration of both the interventions and evaluations. Results from randomized trials suggest that providing opportunities for individualized, cardiovascular risk reduction counseling for high-risk employees within the context of comprehensive programming may be the critical component of an effective worksite intervention. Despite the many limitations of the current methodologies of the 12 studies, the majority of the research to date indicates the following: (1) favorable clinical and cost outcomes; (2) that more recent and more rigorously designed research tends to support rather than refute earlier and less rigorously designed studies; and (3) that rather than interpreting the methodological flaws and diversity as inherently negative, one may consider it as indicative of a robust phenomena evident in many types of worksites, with diverse employees, differing interventions, and varying degrees of methodological sophistication. Results of these studies reviewed provide both cautious optimism about the effectiveness of these worksite programs and insights regarding the essential components and characteristics of successful programs.

Culturally Sensitive Risk Behavior Prevention Programs for African American Adolescents: A Systematic Analysis

ERIC Educational Resources Information Center

Metzger, Isha; Cooper, Shauna M.; Zarrett, Nicole; Flory, Kate

2013-01-01

The current review conducted a systematic assessment of culturally sensitive risk prevention programs for African American adolescents. Prevention programs meeting the inclusion and exclusion criteria were evaluated across several domains: (1) theoretical orientation and foundation; (2) methodological rigor; (3) level of cultural integration; (4)…

Evaluating a Health Risk Reduction Program.

ERIC Educational Resources Information Center

Nagelberg, Daniel B.

1981-01-01

A health risk reduction program at Bowling Green State University (Ohio) tested the efficacy of peer education against the efficacy of returning (by mail) health questionnaire results. A peer health education program did not appear to be effective in changing student attitudes or lifestyles; however, the research methodology may not have been…

Title I, Part B, Institutionalized Facilities Program, Summer 1982. Annual Evaluation Report.

ERIC Educational Resources Information Center

New York City Board of Education, Brooklyn, NY. Office of Educational Evaluation.

This 1982 summer Title I, Part B, Institutionalized Facilities Program provided supplementary career instruction to 188 students residing in facilities for neglected and delinquent children and youth. This report briefly describes the program, providing information on methodology, physical setting, equipment, and supplies; population and class…

Exploratory Evaluation of a School-Based Child Sexual Abuse Prevention Program

ERIC Educational Resources Information Center

Barron, Ian G.; Topping, Keith J.

2013-01-01

Internationally, efficacy studies of school-based child sexual abuse prevention programs display a series of methodological shortcomings. Few studies include adolescent participants, recording of disclosures has been inconsistent, and no studies to date have assessed presenter adherence to program protocols or summated the costs of program…

The relationship between return on investment and quality of study methodology in workplace health promotion programs.

PubMed

Baxter, Siyan; Sanderson, Kristy; Venn, Alison J; Blizzard, C Leigh; Palmer, Andrew J

2014-01-01

To determine the relationship between return on investment (ROI) and quality of study methodology in workplace health promotion programs. Data were obtained through a systematic literature search of National Health Service Economic Evaluation Database (NHS EED), Database of Abstracts of Reviews of Effects (DARE), Health Technology Database (HTA), Cost Effectiveness Analysis (CEA) Registry, EconLit, PubMed, Embase, Wiley, and Scopus. Included were articles written in English or German reporting cost(s) and benefit(s) and single or multicomponent health promotion programs on working adults. Return-to-work and workplace injury prevention studies were excluded. Methodological quality was graded using British Medical Journal Economic Evaluation Working Party checklist. Economic outcomes were presented as ROI. ROI was calculated as ROI = (benefits - costs of program)/costs of program. Results were weighted by study size and combined using meta-analysis techniques. Sensitivity analysis was performed using two additional methodological quality checklists. The influences of quality score and important study characteristics on ROI were explored. Fifty-one studies (61 intervention arms) published between 1984 and 2012 included 261,901 participants and 122,242 controls from nine industry types across 12 countries. Methodological quality scores were highly correlated between checklists (r = .84-.93). Methodological quality improved over time. Overall weighted ROI [mean ± standard deviation (confidence interval)] was 1.38 ± 1.97 (1.38-1.39), which indicated a 138% return on investment. When accounting for methodological quality, an inverse relationship to ROI was found. High-quality studies (n = 18) had a smaller mean ROI, 0.26 ± 1.74 (.23-.30), compared to moderate (n = 16) 0.90 ± 1.25 (.90-.91) and low-quality (n = 27) 2.32 ± 2.14 (2.30-2.33) studies. Randomized control trials (RCTs) (n = 12) exhibited negative ROI, -0.22 ± 2.41(-.27 to -.16). Financial returns become increasingly positive across quasi-experimental, nonexperimental, and modeled studies: 1.12 ± 2.16 (1.11-1.14), 1.61 ± 0.91 (1.56-1.65), and 2.05 ± 0.88 (2.04-2.06), respectively. Overall, mean weighted ROI in workplace health promotion demonstrated a positive ROI. Higher methodological quality studies provided evidence of smaller financial returns. Methodological quality and study design are important determinants.

Methodology of ETV Formative Evaluation.

ERIC Educational Resources Information Center

Duby, Aliza

Formative evaluation is discussed as it relates to educational television (ETV). Formative evaluation, used to improve a project during its development, is especially valuable in the production of television programs. Methods useful in the formative evaluation of ETV are: (1) try-out studies; (2) learner verification and revision; (3)…

Power processing methodology. [computerized design of spacecraft electric power systems

NASA Technical Reports Server (NTRS)

Fegley, K. A.; Hansen, I. G.; Hayden, J. H.

1974-01-01

Discussion of the interim results of a program to investigate the feasibility of formulating a methodology for the modeling and analysis of aerospace electrical power processing systems. The object of the total program is to develop a flexible engineering tool which will allow the power processor designer to effectively and rapidly assess and analyze the tradeoffs available by providing, in one comprehensive program, a mathematical model, an analysis of expected performance, simulation, and a comparative evaluation with alternative designs. This requires an understanding of electrical power source characteristics and the effects of load control, protection, and total system interaction.

Community Based Competitive Employment Preparation of Developmentally Disabled Persons: A Program Description and Evaluation.

ERIC Educational Resources Information Center

Stodden, Robert A.; Browder, Phyllis Meighen

1986-01-01

Success in competitive employment placement of a demonstration project serving 53 persons with developmental disabilities is associated with three factors: (1) training approach and methodology; (2) program management; and (3) trainee characteristics. (CL)

Computer program for discounted cash flow/rate of return evaluations

NASA Technical Reports Server (NTRS)

Robson, W. D.

1971-01-01

Technique, incorporated into set of three computer programs, provides economic methodology for reducing all parameters to financially sound common denominator of present worth, and calculates resultant rate of return on new equipment, processes, or systems investments.

Methodology to evaluate teen driver training programs : [brief].

DOT National Transportation Integrated Search

2014-03-01

In the United States, teenage drivers are more at risk of being involved in crashes than : any other age group. Statistics reveal a clear need for improving teenagers driving : skills, judgment and behavior. Driver education programs are a crucial...

ENVIRONMENTAL HELATH INDICATORS-DIVERSE PROGRAMS, COMMON METHODOLOGICAL CHALLENGES

EPA Science Inventory

Abstract: Environmental Health Indicators are quantitative measures of important environmental health factors that can be used to identify specific areas and populations for intervention and prevention efforts and to evaluate the outcomes of implemented policies or programs for ...

Evaluation of an Alcohol Risk Reduction Program (PRIME for Life) in Young Swedish Military Conscripts

ERIC Educational Resources Information Center

Hallgren, Mats A.; Kallmen, Hakan; Leifman, Hakan; Sjolund, Torbjorn; Andreasson, Sven

2009-01-01

Purpose: The purpose of this paper is to evaluate the effectiveness of the PRIME for Life risk reduction program in reducing alcohol consumption and improving knowledge and attitudes towards alcohol use in male Swedish military conscripts, aged 18 to 22 years. Design/methodology/approach: A quasi-experimental design was used in which 1,371…

Evaluation Methodologies: A Study to Evaluate the Functioning and Impact of the President's Council on Physical Fitness and Sports. Report No. 3.

ERIC Educational Resources Information Center

Bruce, Robert; And Others

This report presents an overview of research objectives, sampling approaches, data collection procedures, and instruments and plans for analysis used in assessing the impact of the President's Council on Physical Fitness and Sports on different types of fitness programs. Surveys were conducted of: (1) community fitness programs; (2) employee…

Economic value evaluation in disease management programs.

PubMed

Magnezi, Racheli; Reicher, Sima; Shani, Mordechai

2008-05-01

Chronic disease management has been a rapidly growing entity in the 21st century as a strategy for managing chronic illnesses in large populations. However, experience has shown that disease management programs have not been able to demonstrate their financial value. The objectives of disease management programs are to create quality benchmarks, such as principles and guidelines, and to establish a uniform set of metrics and a standardized methodology for evaluating them. In order to illuminate the essence of disease management and its components, as well as the complexity and the problematic nature of performing economic calculations of their profitability and value, we collected data from several reports that dealt with the economic intervention of disease management programs. The disease management economic evaluation is composed of a series of steps, including the following major categories: data/information technology, information generation, assessment/recommendations, actionable customer plans, and program assessment/reassessment. We demonstrate the elements necessary for economic analysis. Disease management is one of the most innovative tools in the managed care environment and is still in the process of being defined. Therefore, objectives should include the creation of quality measures, such as principles and guidelines, and the establishment of a uniform set of metrics and a standardized methodology for evaluating them.

Evaluation and Communication: Using a Communication Audit to Evaluate Organizational Communication

ERIC Educational Resources Information Center

Hogard, Elaine; Ellis, Roger

2006-01-01

This article identifies a surprising dearth of studies that explicitly link communication and evaluation at substantive, theoretical, and methodological levels. A three-fold typology of evaluation studies referring to communication is proposed and examples given. The importance of organizational communication in program delivery is stressed and…

[Comparative analysis of the efficacy of a playful-narrative program to teach mathematics at pre-school level].

PubMed

Gil Llario, M D; Vicent Catalá, Consuelo

2009-02-01

Comparative analysis of the efficacy of a playful-narrative program to teach mathematics at pre-school level. In this paper, the effectiveness of a programme comprising several components that are meant to consolidate mathematical concepts and abilities at the pre-school level is analyzed. The instructional methodology of this programme is compared to other methodologies. One-hundred 5-6 year-old children made up the sample that was distributed in the following conditions: (1) traditional methodology; (2) methodology with perceptual and manipulative components, and (3) methodology with language and playful components. Mathematical competence was assessed with the Mathematical Criterial Pre-school Test and the subtest of quantitative-numeric concepts of BADyG. Participants were evaluated before and after the academic course during which they followed one of these methodologies. The results show that the programme with language and playful components is more effective than the traditional methodology (p<.000) and also more effective than the perceptual and manipulative methodology (p<.000). Implications of the results for instructional practices are analyzed.

Implementation and evaluation of a multisite drug usage evaluation program across Australian hospitals - a quality improvement initiative

PubMed Central

2011-01-01

Background With the use of medicines being a broad and extensive part of health management, mechanisms to ensure quality use of medicines are essential. Drug usage evaluation (DUE) is an evidence-based quality improvement methodology, designed to improve the quality, safety and cost-effectiveness of drug use. The purpose of this paper is to describe a national DUE methodology used to improve health care delivery across the continuum through multi-faceted intervention involving audit and feedback, academic detailing and system change, and a qualitative assessment of the methodology, as illustrated by the Acute Postoperative Pain Management (APOP) project. Methods An established methodology, consisting of a baseline audit of inpatient medical records, structured patient interviews and general practitioner surveys, followed by an educational intervention and follow-up audit, is used. Australian hospitals, including private, public, metropolitan and regional, are invited to participate on a voluntary basis. De-identified data collected by hospitals are collated and evaluated nationally to provide descriptive comparative analyses. Hospitals benchmark their practices against state and national results to facilitate change. The educational intervention consists of academic detailing, group education, audit and feedback, point-of-prescribing prompts and system changes. A repeat data collection is undertaken to assess changes in practice. An online qualitative survey was undertaken to evaluate the APOP program. Qualitative assessment of hospitals' perceptions of the effectiveness of the overall DUE methodology and changes in procedure/prescribing/policy/clinical practice which resulted from participation were elicited. Results 62 hospitals participated in the APOP project. Among 23 respondents to the evaluation survey, 18 (78%) reported improvements in the documentation of pain scores at their hospital. 15 (65%) strongly agreed or agreed that participation in APOP directly resulted in increased prescribing of multimodal analgesia for pain relief in postoperative patients. Conclusions This national DUE program has facilitated the engagement and participation of a number of acute health care facilities to address issues relating to quality use of medicine. This approach has been perceived to be effective in helping them achieve improvements in patient care. PMID:21871132

Implementation and evaluation of a multisite drug usage evaluation program across Australian hospitals - a quality improvement initiative.

PubMed

Pulver, Lisa K; Wai, Angela; Maxwell, David J; Robertson, Marion B; Riddell, Steven

2011-08-29

With the use of medicines being a broad and extensive part of health management, mechanisms to ensure quality use of medicines are essential. Drug usage evaluation (DUE) is an evidence-based quality improvement methodology, designed to improve the quality, safety and cost-effectiveness of drug use. The purpose of this paper is to describe a national DUE methodology used to improve health care delivery across the continuum through multi-faceted intervention involving audit and feedback, academic detailing and system change, and a qualitative assessment of the methodology, as illustrated by the Acute Postoperative Pain Management (APOP) project. An established methodology, consisting of a baseline audit of inpatient medical records, structured patient interviews and general practitioner surveys, followed by an educational intervention and follow-up audit, is used. Australian hospitals, including private, public, metropolitan and regional, are invited to participate on a voluntary basis. De-identified data collected by hospitals are collated and evaluated nationally to provide descriptive comparative analyses. Hospitals benchmark their practices against state and national results to facilitate change. The educational intervention consists of academic detailing, group education, audit and feedback, point-of-prescribing prompts and system changes. A repeat data collection is undertaken to assess changes in practice.An online qualitative survey was undertaken to evaluate the APOP program. Qualitative assessment of hospitals' perceptions of the effectiveness of the overall DUE methodology and changes in procedure/prescribing/policy/clinical practice which resulted from participation were elicited. 62 hospitals participated in the APOP project. Among 23 respondents to the evaluation survey, 18 (78%) reported improvements in the documentation of pain scores at their hospital. 15 (65%) strongly agreed or agreed that participation in APOP directly resulted in increased prescribing of multimodal analgesia for pain relief in postoperative patients. This national DUE program has facilitated the engagement and participation of a number of acute health care facilities to address issues relating to quality use of medicine. This approach has been perceived to be effective in helping them achieve improvements in patient care.

Testing the DMAA's recommendations for disease management program evaluation.

PubMed

Serxner, Seth; Mattke, Soeren; Zakowski, Sarah; Gold, Daniel

2008-10-01

The objective of this study was to compare and contrast findings regarding the financial savings projections of the disease management (DM) programs of 2 large employers based on different evaluation methods. In particular, this research tests the impact of differences in assumptions on the underlying growth rate of group health costs, exclusions of high-cost conditions and claims, and the length of the baseline period for determined health care costs. A pre-post study design was used. The data for this research came from 2 large employers in the consumer goods industry with comprehensive Health and Productivity Management programs. It contained medical and prescription drug claims and health plan enrollment data as well as program activity data from 2001 to 2005, covering an average yearly sample size of 201,037 members with 12 consecutive months of enrollment. Analyses were done on group-level averages using nominal cost data and were run to reflect the impact of a DM-only intervention. While the trend estimate and length of baseline had the largest effects on estimated program impact, the use of exclusions had an important effect as well. These findings demonstrate the importance of developing and instituting a standardized evaluation methodology. Without increasing consistency in the way evaluators develop their methodologies, it will remain difficult to be able to compare one evaluation to another, or to have faith in the results at hand.

Evaluating Training Programs for Primary Care Providers in Child/Adolescent Mental Health in Canada: A Systematic Review.

PubMed

Gotovac, Sandra; Espinet, Stacey; Naqvi, Reza; Lingard, Lorelei; Steele, Margaret

2018-04-01

The need for child/adolescent mental health care in Canada is growing. Primary care can play a key role in filling this gap, yet most providers feel they do not have adequate training. This paper reviews the Canadian literature on capacity building programs in child and adolescent psychiatry for primary care providers, to examine how these programs are being implemented and evaluated to contribute to evidence-based initiatives. A systematic literature review of peer-reviewed published articles of capacity building initiatives in child/adolescent mental health care for primary care practitioners that have been implemented in Canada. Sixteen articles were identified that met inclusion criteria. Analysis revealed that capacity building initiatives in Canada are varied but rigorous evaluation methodology is lacking. Primary care providers welcome efforts to increase mental health care capacity and were satisfied with the implementation of most programs. Objective conclusions regarding the effectiveness of these programs to increase mental health care capacity is challenging given the evaluation methodology of these studies. Rigorous evaluation methods are needed to make evidence-based decisions on ways forward to be able to build child/adolescent mental health care capacity in primary care. Outcome measures need to move beyond self-report to more objective measures, and should expand the measurement of patient outcomes to ensure that these initiative are indeed leading to improved care for families.

Use of an Information Retrieval Service in an Obstetrics/Gynecology Residency Program.

ERIC Educational Resources Information Center

And Others; Gunning, John E.

1980-01-01

A program that uses the clinical librarian as a member of the patient care team has been developed by an obstetrics and gynecology department of a university medical center to keep faculty and hospital house staff knowledgeable about current developments and research. Program objectives, methodology, costs, evaluation, and information utilization…

Comparative Costs of Manpower Education: A Methodological Study.

ERIC Educational Resources Information Center

Lyman, Jay Rich

The objective of this study was to establish a criteria and model for comparative evaluation of manpower educational programs. The criteria developed deals with resource allocation in manpower education programs and how well those programs meet the needs of industry. In the proposed model, an occupation is reduced to its basic skills, which are…

A Conceptual Three-Dimensional Model for Evaluating Community-Based Substance Abuse Prevention Programs.

ERIC Educational Resources Information Center

Albers, Eric C.; Santangelo, Linda K.; McKinlay, George; Cavote, Steve; Rock, Stephen L.; Evans, William

2002-01-01

Presents a three-dimensional model for conceptualizing existing prevention programs, defining and measuring effects of prevention programs, and making a connection between those programmatic effects, and the interests of the funder. This paper describes the methodology and its use for promoting the efficiency and effectiveness of substance abuse…

Nuclear medicine and quantitative imaging research (instrumentation and quantitative methods of evaluation)

DOE Office of Scientific and Technical Information (OSTI.GOV)

Beck, R.N.; Cooper, M.D.

1990-09-01

This report summarizes goals and accomplishments of the research program supported under DOE Grant No. FG02-86ER60418 entitled Instrumentation and Quantitative Methods of Evaluation, with R. Beck, P. I. and M. Cooper, Co-P.I. during the period January 15, 1990 through September 1, 1990. This program addresses the problems involving the basic science and technology underlying the physical and conceptual tools of radioactive tracer methodology as they relate to the measurement of structural and functional parameters of physiologic importance in health and disease. The principal tool is quantitative radionuclide imaging. The overall objective of this program is to further the development andmore » transfer of radiotracer methodology from basic theory to routine clinical practice in order that individual patients and society as a whole will receive the maximum net benefit from the new knowledge gained. The focus of the research is on the development of new instruments and radiopharmaceuticals, and the evaluation of these through the phase of clinical feasibility. 7 figs.« less

Design and Implementation of a Pilot Obesity Prevention Program in a Low-Resource School: Lessons Learned and Research Recommendations

ERIC Educational Resources Information Center

Baskin, Monica L.; Zunker, Christie; Worley, Courtney B.; Dial, Brenda; Kimbrough, Linda

2009-01-01

Purpose: This paper seeks to describe the design, implementation, and lessons learned from an obesity prevention pilot program delivered in a low resource school in the USA. Design/methodology/approach: A planned program evaluation was conducted to: document explicitly the process of designing and implementing the program; and assess the…

Methodologic European external quality assurance for DNA sequencing: the EQUALseq program.

PubMed

Ahmad-Nejad, Parviz; Dorn-Beineke, Alexandra; Pfeiffer, Ulrike; Brade, Joachim; Geilenkeuser, Wolf-Jochen; Ramsden, Simon; Pazzagli, Mario; Neumaier, Michael

2006-04-01

DNA sequencing is a key technique in molecular diagnostics, but to date no comprehensive methodologic external quality assessment (EQA) programs have been instituted. Between 2003 and 2005, the European Union funded, as specific support actions, the EQUAL initiative to develop methodologic EQA schemes for genotyping (EQUALqual), quantitative PCR (EQUALquant), and sequencing (EQUALseq). Here we report on the results of the EQUALseq program. The participating laboratories received a 4-sample set comprising 2 DNA plasmids, a PCR product, and a finished sequencing reaction to be analyzed. Data and information from detailed questionnaires were uploaded online and evaluated by use of a scoring system for technical skills and proficiency of data interpretation. Sixty laboratories from 21 European countries registered, and 43 participants (72%) returned data and samples. Capillary electrophoresis was the predominant platform (n = 39; 91%). The median contiguous correct sequence stretch was 527 nucleotides with considerable variation in quality of both primary data and data evaluation. The association between laboratory performance and the number of sequencing assays/year was statistically significant (P <0.05). Interestingly, more than 30% of participants neither added comments to their data nor made efforts to identify the gene sequences or mutational positions. Considerable variations exist even in a highly standardized methodology such as DNA sequencing. Methodologic EQAs are appropriate tools to uncover strengths and weaknesses in both technique and proficiency, and our results emphasize the need for mandatory EQAs. The results of EQUALseq should help improve the overall quality of molecular genetics findings obtained by DNA sequencing.

Did JCLP Really Work?

ERIC Educational Resources Information Center

Hencey, Robert

1981-01-01

Considers the methodology, findings, and conclusions of three studies conducted by Stephen Epler, Louis Bender, and James Perkins, which evaluated the Junior College Leadership Program. The project was funded from 1959 through 1974 to establish university programs for the preparation of two-year college administrators. (DML)

Suicide Risk Assessment Training for Psychology Doctoral Programs: Core Competencies and a Framework for Training

PubMed Central

Cramer, Robert J.; Johnson, Shara M.; McLaughlin, Jennifer; Rausch, Emilie M.; Conroy, Mary Alice

2014-01-01

Clinical and counseling psychology programs currently lack adequate evidence-based competency goals and training in suicide risk assessment. To begin to address this problem, this article proposes core competencies and an integrated training framework that can form the basis for training and research in this area. First, we evaluate the extent to which current training is effective in preparing trainees for suicide risk assessment. Within this discussion, sample and methodological issues are reviewed. Second, as an extension of these methodological training issues, we integrate empirically- and expert-derived suicide risk assessment competencies from several sources with the goal of streamlining core competencies for training purposes. Finally, a framework for suicide risk assessment training is outlined. The approach employs Objective Structured Clinical Examination (OSCE) methodology, an approach commonly utilized in medical competency training. The training modality also proposes the Suicide Competency Assessment Form (SCAF), a training tool evaluating self- and observer-ratings of trainee core competencies. The training framework and SCAF are ripe for empirical evaluation and potential training implementation. PMID:24672588

Suicide Risk Assessment Training for Psychology Doctoral Programs: Core Competencies and a Framework for Training.

PubMed

Cramer, Robert J; Johnson, Shara M; McLaughlin, Jennifer; Rausch, Emilie M; Conroy, Mary Alice

2013-02-01

Clinical and counseling psychology programs currently lack adequate evidence-based competency goals and training in suicide risk assessment. To begin to address this problem, this article proposes core competencies and an integrated training framework that can form the basis for training and research in this area. First, we evaluate the extent to which current training is effective in preparing trainees for suicide risk assessment. Within this discussion, sample and methodological issues are reviewed. Second, as an extension of these methodological training issues, we integrate empirically- and expert-derived suicide risk assessment competencies from several sources with the goal of streamlining core competencies for training purposes. Finally, a framework for suicide risk assessment training is outlined. The approach employs Objective Structured Clinical Examination (OSCE) methodology, an approach commonly utilized in medical competency training. The training modality also proposes the Suicide Competency Assessment Form (SCAF), a training tool evaluating self- and observer-ratings of trainee core competencies. The training framework and SCAF are ripe for empirical evaluation and potential training implementation.

Development and Current Status of Skull-Image Superimposition - Methodology and Instrumentation.

PubMed

Lan, Y

1992-12-01

This article presents a review of the literature and an evaluation on the development and application of skull-image superimposition technology - both instrumentation and methodology - contributed by a number of scholars since 1935. Along with a comparison of the methodologies involved in the two superimposition techniques - photographic and video - the author characterized the techniques in action and the recent advances in computer image superimposition processing technology. The major disadvantage of conventional approaches is its relying on subjective interpretation. Through painstaking comparison and analysis, computer image processing technology can make more conclusive identifications by direct testing and evaluating the various programmed indices. Copyright © 1992 Central Police University.

A Programming Environment Evaluation Methodology for Object-Oriented Systems. Ph.D Thesis Final Report, 1 Jul. 1985 - 31 Dec. 1987

NASA Technical Reports Server (NTRS)

Dominick, Wayne D. (Editor); Moreau, Dennis R.

1987-01-01

The object-oriented design strategy as both a problem decomposition and system development paradigm has made impressive inroads into the various areas of the computing sciences. Substantial development productivity improvements have been demonstrated in areas ranging from artificial intelligence to user interface design. However, there has been very little progress in the formal characterization of these productivity improvements and in the identification of the underlying cognitive mechanisms. The development and validation of models and metrics of this sort require large amounts of systematically-gathered structural and productivity data. There has, however, been a notable lack of systematically-gathered information on these development environments. A large part of this problem is attributable to the lack of a systematic programming environment evaluation methodology that is appropriate to the evaluation of object-oriented systems.

Team-Teaching in Physical Education for Promoting Coordinative Motor Skills in Children: The More You Invest the More You Get

ERIC Educational Resources Information Center

Bardaglio, Giulia; Marasso, Danilo; Magno, Francesca; Rabaglietti, Emanuela; Ciairano, Silvia

2015-01-01

Background: Standard physical education (PE) programs and the team-teaching methodology have rarely been evaluated to investigate their real efficacy in changing children's motor skills. Aims: The aims of this study are two-fold: The first aim is to evaluate the effectiveness of a PE program for improving coordinative motor skills in the team…

Performance evaluation in full-mission simulation - Methodological advances and research challenges. [in air transport operations

NASA Technical Reports Server (NTRS)

Chidester, Thomas R.; Kanki, Barbara G.; Helmreich, Robert L.

1989-01-01

The crew-factors research program at NASA Ames has developed a methodology for studying the impact of a variety of variables on the effectiveness of crews flying realistic but high workload simulated trips. The validity of investigations using the methodology is enhanced by careful design of full-mission scenarios, performance assessment using converging sources of data, and recruitment of representative subjects. Recently, portions of this methodology have been adapted for use in assessing the effectiveness of crew coordination among participants in line-oriented flight training.

Causal inferences on the effectiveness of complex social programs: Navigating assumptions, sources of complexity and evaluation design challenges.

PubMed

Chatterji, Madhabi

2016-12-01

This paper explores avenues for navigating evaluation design challenges posed by complex social programs (CSPs) and their environments when conducting studies that call for generalizable, causal inferences on the intervention's effectiveness. A definition is provided of a CSP drawing on examples from different fields, and an evaluation case is analyzed in depth to derive seven (7) major sources of complexity that typify CSPs, threatening assumptions of textbook-recommended experimental designs for performing impact evaluations. Theoretically-supported, alternative methodological strategies are discussed to navigate assumptions and counter the design challenges posed by the complex configurations and ecology of CSPs. Specific recommendations include: sequential refinement of the evaluation design through systems thinking, systems-informed logic modeling; and use of extended term, mixed methods (ETMM) approaches with exploratory and confirmatory phases of the evaluation. In the proposed approach, logic models are refined through direct induction and interactions with stakeholders. To better guide assumption evaluation, question-framing, and selection of appropriate methodological strategies, a multiphase evaluation design is recommended. Copyright © 2016 Elsevier Ltd. All rights reserved.

Integrative Discovery Doing Science.

ERIC Educational Resources Information Center

Harry, Vickie; Belzer, William

1990-01-01

The article details a program in which gifted upper elementary grade students used videomicroscopy in a study of microscopic life in pond water. Each child produced a narrated videotape of a specific species studied. Program evaluation confirmed the motivational benefits of early opportunities with scientific instrumentation and methodology. (DB)

Collaborative evaluation of a high school prevention curriculum: How methods of collaborative evaluation enhanced a randomized control trial to inform program improvement.

PubMed

Orsini, Muhsin Michael; Wyrick, David L; Milroy, Jeffrey J

2012-11-01

Blending high-quality and rigorous research with pure evaluation practice can often be best accomplished through thoughtful collaboration. The evaluation of a high school drug prevention program (All Stars Senior) is an example of how perceived competing purposes and methodologies can coexist to investigate formative and summative outcome variables that can be used for program improvement. Throughout this project there were many examples of client learning from evaluator and evaluator learning from client. This article presents convincing evidence that collaborative evaluation can improve the design, implementation, and findings of the randomized control trial. Throughout this paper, we discuss many examples of good science, good evaluation, and other practical benefits of practicing collaborative evaluation. Ultimately, the authors created the term pre-formative evaluation to describe the period prior to data collection and before program implementation, when collaborative evaluation can inform program improvement. Copyright © 2012 Elsevier Ltd. All rights reserved.

Earth Sciences Data and Information System (ESDIS) program planning and evaluation methodology development

NASA Technical Reports Server (NTRS)

Dickinson, William B.

1995-01-01

An Earth Sciences Data and Information System (ESDIS) Project Management Plan (PMP) is prepared. An ESDIS Project Systems Engineering Management Plan (SEMP) consistent with the developed PMP is also prepared. ESDIS and related EOS program requirements developments, management and analysis processes are evaluated. Opportunities to improve the effectiveness of these processes and program/project responsiveness to requirements are identified. Overall ESDIS cost estimation processes are evaluated, and recommendations to improve cost estimating and modeling techniques are developed. ESDIS schedules and scheduling tools are evaluated. Risk assessment, risk mitigation strategies and approaches, and use of risk information in management decision-making are addressed.

Usability and Instructional Design Heuristics for E-Learning Evaluation.

ERIC Educational Resources Information Center

Reeves, Thomas C.; Benson, Lisa; Elliott, Dean; Grant, Michael; Holschuh, Doug; Kim, Beaumie; Kim, Hyeonjin; Lauber, Erick; Loh, Sebastian

Heuristic evaluation is a methodology for investigating the usability of software originally developed by Nielsen (1993, 2000). Nielsen's protocol was modified and refined for evaluating e-learning programs by participants in a doctoral seminar held at the University of Georgia in 2001. The modifications primarily involved expanding Nielsen's…

Systematic, Cooperative Evaluation.

ERIC Educational Resources Information Center

Nassif, Paula M.

Evaluation procedures based on a systematic evaluation methodology, decision-maker validity, new measurement and design techniques, low cost, and a high level of cooperation on the part of the school staff were used in the assessment of a public school mathematics program for grades 3-8. The mathematics curriculum was organized into Spirals which…

Use of Electronic Surveys in Course Evaluation.

ERIC Educational Resources Information Center

Moss, Jaclyn; Hendry, Graham

2002-01-01

Reviews evidence for the effectiveness of electronic surveys as evaluation methods for electronic courses and discusses issues in email and Web survey methodologies. Describes the successful use of Web-based surveys in the evaluation system of the University of Sydney Medical Program and includes recommendations for online survey use in course…

Nuclear medicine and imaging research (Instrumentation and quantitative methods of evaluation)

DOE Office of Scientific and Technical Information (OSTI.GOV)

Beck, R.N.; Cooper, M.D.

1989-09-01

This program addresses the problems involving the basic science and technology underlying the physical and conceptual tools of radioactive tracer methodology as they relate to the measurement of structural and functional parameters of physiologic importance in health and disease. The principal tool is quantitative radionuclide imaging. The overall objective of this program is to further the development and transfer of radiotracer methodology from basic theory to routine clinical practice in order that individual patients and society as a whole will receive the maximum net benefit from the new knowledge gained. The focus of the research is on the development ofmore » new instruments and radiopharmaceuticals, and the evaluation of these through the phase of clinical feasibility.« less

Alternative Learning Methodologies through Academics (Project ALMA). Final Evaluation Report, 1993-94. OER Report.

ERIC Educational Resources Information Center

Roman, Elliott M.

The Alternative Learning Methodologies through Academics Project (Project ALMA) was an Elementary and Secondary Education Act Title VII-funded project in its fourth year of operation in two high schools in Queens and the Bronx (New York). The program served 436 Spanish-speaking students, most of whom were of limited English proficiency.…

Hypermedia and Research Methodology: An Associative Research Model. Final Report.

ERIC Educational Resources Information Center

City Univ. of New York, NY. Bernard Baruch Coll.

The purpose of the project described in this three-part report was to test the value of hypermedia for library instruction. The project included the development of a hypermedia program built around a marketing topic; implementation and testing of the program with one group of students assigned to use the new program; evaluation of the project by…

Evaluating Mission Drift in Microfinance: Lessons for Programs with Social Mission

ERIC Educational Resources Information Center

Hishigsuren, Gaamaa

2007-01-01

The article contributes to a better understanding of implications of scaling up on the social mission of microfinance programs. It proposes a methodology to measure the extent, if any, to which a microfinance program with a poverty alleviation mission drifts away from its mission during rapid scaling up and presents findings from a field research…

An Evaluation of the Right to Read Inexpensive Book Distribution Program. Executive Summary.

ERIC Educational Resources Information Center

General Research Corp., McLean, VA.

This document is an executive summary of a study examining the Inexpensive Book Distribution Program (IBDP), a federally funded and sponsored program operated by Reading is Fundamental (RIF). Sections of the report provide the following information: (1) methodology and procedures for conducting the study, (2) a brief description of RIF national…

Using Concept Mapping as as Tool for Program Theory Development

ERIC Educational Resources Information Center

Orsi, Rebecca

2011-01-01

The purpose of this methodological study is to explore how well a process called "concept mapping" (Trochim, 1989) can articulate the theory which underlies a social program. Articulation of a program's theory is a key step in completing a sound theory based evaluation (Weiss, 1997a). In this study, concept mapping is used to…

An Analysis of Legal Hearings and Cases Related to Individualized Education Programs for Children with Autism.

ERIC Educational Resources Information Center

Etscheidt, Susan

2003-01-01

This article reviews 68 hearings and cases concerned with disputes regarding individualized education programs (IEPs) for students with disabilities. It concludes that IEP goals must be matched to evaluation data, team members must be qualified to develop programs, and the methodology selected must be able to assist the students in achieving…

Assessing the effects of employee assistance programs: a review of employee assistance program evaluations.

PubMed

Colantonio, A

1989-01-01

Employee assistance programs have grown at a dramatic rate, yet the effectiveness of these programs has been called into question. The purpose of this paper was to assess the effectiveness of employee assistance programs (EAPs) by reviewing recently published EAP evaluations. All studies evaluating EAPs published since 1975 from peer-reviewed journals in the English language were included in this analysis. Each of the articles was assessed in the following areas: (a) program description (subjects, setting, type of intervention, format), (b) evaluation design (research design, variables measured, operational methods), and (c) program outcomes. Results indicate numerous methodological and conceptual weaknesses and issues. These weaknesses included lack of controlled research designs and short time lags between pre- and post-test measures. Other problems identified are missing information regarding subjects, type of intervention, how variables are measured (operational methods), and reliability and validity of evaluation instruments. Due to the aforementioned weaknesses, positive outcomes could not be supported. Recommendations are made for future EAP evaluations.

Assessing the effects of employee assistance programs: a review of employee assistance program evaluations.

PubMed Central

Colantonio, A.

1989-01-01

Employee assistance programs have grown at a dramatic rate, yet the effectiveness of these programs has been called into question. The purpose of this paper was to assess the effectiveness of employee assistance programs (EAPs) by reviewing recently published EAP evaluations. All studies evaluating EAPs published since 1975 from peer-reviewed journals in the English language were included in this analysis. Each of the articles was assessed in the following areas: (a) program description (subjects, setting, type of intervention, format), (b) evaluation design (research design, variables measured, operational methods), and (c) program outcomes. Results indicate numerous methodological and conceptual weaknesses and issues. These weaknesses included lack of controlled research designs and short time lags between pre- and post-test measures. Other problems identified are missing information regarding subjects, type of intervention, how variables are measured (operational methods), and reliability and validity of evaluation instruments. Due to the aforementioned weaknesses, positive outcomes could not be supported. Recommendations are made for future EAP evaluations. PMID:2728498

Evaluation of Reading Achievement of the Program School 2.0 in Spain Using PISA 2012

ERIC Educational Resources Information Center

Vilaplana Prieto, Cristina

2016-01-01

In 2009, some Spanish regions implemented the Program School 2.0 with the purpose of introducing digital methodologies at schools. The aim of this paper is to analyse which part of the variation in reading scores is due to this program. For this purpose, we use data from the Program for International Student Assessment (PISA 2009 and 2012) for…

An effective CPR home learning system. A program evaluation.

PubMed

Starr, L M

1998-06-01

1. Program evaluation using a self instruction CPR home learning system indicated that people could learn CPR without attending a formal class. Reported training time for adult CPR averaged 1 hour, and ease of understanding was rated 4.9 on a 5 point scale. 2. Use of a home learning system can benefit a workplace by reducing reliance on a CPR instructor, reducing total training time, and exposing families and others in the community to the learning experience. 3. This program evaluation suggests that this training methodology supports the mission of AHA, and also can meet corporate objectives.

Technical Feasibility Aspects of the Geothermal Resource Reporting Methodology (GRRM)

DOE Office of Scientific and Technical Information (OSTI.GOV)

Badgett, Alex; Young, Katherine R; Dobson, Patrick F.

This paper reviews the technical assessment of the Geothermal Research Reporting Methodology (GRRM, http://en.openei.org/wiki/GRRM) being developed for reporting geothermal resources and project progress. The goal of the methodology is to provide the U.S. Department of Energy's Geothermal Technologies Office (GTO) with a consistent and comprehensible means of evaluating the impacts of its funding programs. The GRRM is designed to provide uniform assessment criteria for geothermal resource grades and developmental phases of geothermal resource exploration and development. This resource grade system provides information on twelve attributes of geothermal resource locations (e.g., temperature, permeability, land access) to indicate potential for geothermal development.more » The GTO plans to use these Protocols to help quantitatively identify the greatest barriers to geothermal development, develop measureable program goals that will have the greatest impact to geothermal deployment, objectively evaluate proposals based (in part) on a project's ability to contribute to program goals, monitor project progress, and report on GTO portfolio performance. The GRRM assesses three areas of geothermal potential: geological, socio-economic, and technical. Previous work and publications have discussed the work done on the geological aspects of this methodology (Young et al. 2015c); this paper details the development of the technical assessment of the GRRM. Technical development attributes considered include: reservoir management, drilling, logistics, and power conversion.« less

Illuminating the Black Box of Entrepreneurship Education Programs

ERIC Educational Resources Information Center

Maritz, Alex; Brown, Christopher R.

2013-01-01

Purpose: The aim of this paper is to explore the components of entrepreneurship education programs (EEPs) and their interrelationships to develop a conceptual framework through which entrepreneurship education may be contextually evaluated and developed. Design/methodology/approach: The paper presents an extensive literature review of the…

A situational analysis methodology to inform comprehensive HIV prevention and treatment programming, applied in rural South Africa.

PubMed

Treves-Kagan, Sarah; Naidoo, Evasen; Gilvydis, Jennifer M; Raphela, Elsie; Barnhart, Scott; Lippman, Sheri A

2017-09-01

Successful HIV prevention programming requires engaging communities in the planning process and responding to the social environmental factors that shape health and behaviour in a specific local context. We conducted two community-based situational analyses to inform a large, comprehensive HIV prevention programme in two rural districts of North West Province South Africa in 2012. The methodology includes: initial partnership building, goal setting and background research; 1 week of field work; in-field and subsequent data analysis; and community dissemination and programmatic incorporation of results. We describe the methodology and a case study of the approach in rural South Africa; assess if the methodology generated data with sufficient saturation, breadth and utility for programming purposes; and evaluate if this process successfully engaged the community. Between the two sites, 87 men and 105 women consented to in-depth interviews; 17 focus groups were conducted; and 13 health facilities and 7 NGOs were assessed. The methodology succeeded in quickly collecting high-quality data relevant to tailoring a comprehensive HIV programme and created a strong foundation for community engagement and integration with local health services. This methodology can be an accessible tool in guiding community engagement and tailoring future combination HIV prevention and care programmes.

Integrated mass transportation system study/definition/implementation program definition

NASA Technical Reports Server (NTRS)

Ransone, R. K.; Deptula, D. A.; Yorke, G. G.

1975-01-01

Specific actions needed to plan and effect transportation system improvements are identified within the constraints of limited financial, energy and land use resources, and diverse community requirements. A specific program is described which would develop the necessary generalized methodology for devising improved transportation systems and evaluate them against specific criteria for intermodal and intramodal optimization. A consistent, generalized method is provided for study and evaluation of transportation system improvements.

Systematic Review of Empirically Evaluated School-Based Gambling Education Programs.

PubMed

Keen, Brittany; Blaszczynski, Alex; Anjoul, Fadi

2017-03-01

Adolescent problem gambling prevalence rates are reportedly five times higher than in the adult population. Several school-based gambling education programs have been developed in an attempt to reduce problem gambling among adolescents; however few have been empirically evaluated. The aim of this review was to report the outcome of studies empirically evaluating gambling education programs across international jurisdictions. A systematic review following guidelines outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement searching five academic databases: PubMed, Scopus, Medline, PsycINFO, and ERIC, was conducted. A total of 20 papers and 19 studies were included after screening and exclusion criteria were applied. All studies reported intervention effects on cognitive outcomes such as knowledge, perceptions, and beliefs. Only nine of the studies attempted to measure intervention effects on behavioural outcomes, and only five of those reported significant changes in gambling behaviour. Of these five, methodological inadequacies were commonly found including brief follow-up periods, lack of control comparison in post hoc analyses, and inconsistencies and misclassifications in the measurement of gambling behaviour, including problem gambling. Based on this review, recommendations are offered for the future development and evaluation of school-based gambling education programs relating to both methodological and content design and delivery considerations.

Mixed method evaluation of a community-based physical activity program using the RE-AIM framework: practical application in a real-world setting.

PubMed

Koorts, Harriet; Gillison, Fiona

2015-11-06

Communities are a pivotal setting in which to promote increases in child and adolescent physical activity behaviours. Interventions implemented in these settings require effective evaluation to facilitate translation of findings to wider settings. The aims of this paper are to i) present findings from a RE-AIM evaluation of a community-based physical activity program, and ii) review the methodological challenges faced when applying RE-AIM in practice. A single mixed-methods case study was conducted based on a concurrent triangulation design. Five sources of data were collected via interviews, questionnaires, archival records, documentation and field notes. Evidence was triangulated within RE-AIM to assess individual and organisational-level program outcomes. Inconsistent availability of data and a lack of robust reporting challenged assessment of all five dimensions. Reach, Implementation and setting-level Adoption were less successful, Effectiveness and Maintenance at an individual and organisational level were moderately successful. Only community-level Adoption was highly successful, reflecting the key program goal to provide community-wide participation in sport and physical activity. This research highlighted important methodological constraints associated with the use of RE-AIM in practice settings. Future evaluators wishing to use RE-AIM may benefit from a mixed-method triangulation approach to offset challenges with data availability and reliability.

Evaluating Training Programs for Primary Care Providers in Child/Adolescent Mental Health in Canada: A Systematic Review

PubMed Central

Espinet, Stacey; Naqvi, Reza; Lingard, Lorelei; Steele, Margaret

2018-01-01

Introduction The need for child/adolescent mental health care in Canada is growing. Primary care can play a key role in filling this gap, yet most providers feel they do not have adequate training. This paper reviews the Canadian literature on capacity building programs in child and adolescent psychiatry for primary care providers, to examine how these programs are being implemented and evaluated to contribute to evidence-based initiatives. Methods A systematic literature review of peer-reviewed published articles of capacity building initiatives in child/adolescent mental health care for primary care practitioners that have been implemented in Canada. Results Sixteen articles were identified that met inclusion criteria. Analysis revealed that capacity building initiatives in Canada are varied but rigorous evaluation methodology is lacking. Primary care providers welcome efforts to increase mental health care capacity and were satisfied with the implementation of most programs. Discussion Objective conclusions regarding the effectiveness of these programs to increase mental health care capacity is challenging given the evaluation methodology of these studies. Conclusion Rigorous evaluation methods are needed to make evidence-based decisions on ways forward to be able to build child/adolescent mental health care capacity in primary care. Outcome measures need to move beyond self-report to more objective measures, and should expand the measurement of patient outcomes to ensure that these initiative are indeed leading to improved care for families. PMID:29662521

Management of the aging of critical safety-related concrete structures in light-water reactor plants

DOE Office of Scientific and Technical Information (OSTI.GOV)

Naus, D.J.; Oland, C.B.; Arndt, E.G.

1990-01-01

The Structural Aging Program has the overall objective of providing the USNRC with an improved basis for evaluating nuclear power plant safety-related structures for continued service. The program consists of a management task and three technical tasks: materials property data base, structural component assessment/repair technology, and quantitative methodology for continued-service determinations. Objectives, accomplishments, and planned activities under each of these tasks are presented. Major program accomplishments include development of a materials property data base for structural materials as well as an aging assessment methodology for concrete structures in nuclear power plants. Furthermore, a review and assessment of inservice inspection techniquesmore » for concrete materials and structures has been complete, and work on development of a methodology which can be used for performing current as well as reliability-based future condition assessment of concrete structures is well under way. 43 refs., 3 tabs.« less

Integrating research, policy, and practice in juvenile justice education.

PubMed

Blomberg, Thomas G; Waldo, Gordon P

2002-06-01

This article provides an overview of the history and context leading to Florida's efforts to implement an evaluation-driven research and associated quality assurance system for its juvenile justice education policies and practices. The Juvenile Justice Educational Enhancement Program began implementing Florida's evaluation research and quality assurance system to juvenile justice education in 1998. The article includes a brief summary of articles comprising this special issue of Evaluation Review that address the Juvenile Justice Educational Enhancement Program's various functions, methodological components, data, preliminary findings, continuing evaluation research efforts, and impediments.

Operations Research as a Metaphor for Evaluation. Research on Evaluation Program Paper and Report Series.

ERIC Educational Resources Information Center

Page, Ellis B.

One of a series of research reports examining objective principles successfully used in other fields which can lend integrity and legitimacy to evaluation, this report presents an overview of operations research (OR) as a potential source of evaluation methodology. The nature of the methods common to this discipline are summarized and the…

An Evaluation of Research Ethics in Undergraduate Health Science Research Methodology Programs at a South African University.

PubMed

Coetzee, Tanya; Hoffmann, Willem A; de Roubaix, Malcolm

2015-10-01

The amended research ethics policy at a South African University required the ethics review of undergraduate research projects, prompting the need to explore the content and teaching approach of research ethics education in health science undergraduate programs. Two qualitative data collection strategies were used: document analysis (syllabi and study guides) and semi-structured interviews with research methodology coordinators. Five main themes emerged: (a) timing of research ethics courses, (b) research ethics course content, (c) sub-optimal use of creative classroom activities to facilitate research ethics lectures, (d) understanding the need for undergraduate project research ethics review, and (e) research ethics capacity training for research methodology lecturers and undergraduate project supervisors. © The Author(s) 2015.

The Development of a Checklist to Enhance Methodological Quality in Intervention Programs.

PubMed

Chacón-Moscoso, Salvador; Sanduvete-Chaves, Susana; Sánchez-Martín, Milagrosa

2016-01-01

The methodological quality of primary studies is an important issue when performing meta-analyses or systematic reviews. Nevertheless, there are no clear criteria for how methodological quality should be analyzed. Controversies emerge when considering the various theoretical and empirical definitions, especially in relation to three interrelated problems: the lack of representativeness, utility, and feasibility. In this article, we (a) systematize and summarize the available literature about methodological quality in primary studies; (b) propose a specific, parsimonious, 12-items checklist to empirically define the methodological quality of primary studies based on a content validity study; and (c) present an inter-coder reliability study for the resulting 12-items. This paper provides a precise and rigorous description of the development of this checklist, highlighting the clearly specified criteria for the inclusion of items and a substantial inter-coder agreement in the different items. Rather than simply proposing another checklist, however, it then argues that the list constitutes an assessment tool with respect to the representativeness, utility, and feasibility of the most frequent methodological quality items in the literature, one that provides practitioners and researchers with clear criteria for choosing items that may be adequate to their needs. We propose individual methodological features as indicators of quality, arguing that these need to be taken into account when designing, implementing, or evaluating an intervention program. This enhances methodological quality of intervention programs and fosters the cumulative knowledge based on meta-analyses of these interventions. Future development of the checklist is discussed.

The Development of a Checklist to Enhance Methodological Quality in Intervention Programs

PubMed Central

Chacón-Moscoso, Salvador; Sanduvete-Chaves, Susana; Sánchez-Martín, Milagrosa

2016-01-01

The methodological quality of primary studies is an important issue when performing meta-analyses or systematic reviews. Nevertheless, there are no clear criteria for how methodological quality should be analyzed. Controversies emerge when considering the various theoretical and empirical definitions, especially in relation to three interrelated problems: the lack of representativeness, utility, and feasibility. In this article, we (a) systematize and summarize the available literature about methodological quality in primary studies; (b) propose a specific, parsimonious, 12-items checklist to empirically define the methodological quality of primary studies based on a content validity study; and (c) present an inter-coder reliability study for the resulting 12-items. This paper provides a precise and rigorous description of the development of this checklist, highlighting the clearly specified criteria for the inclusion of items and a substantial inter-coder agreement in the different items. Rather than simply proposing another checklist, however, it then argues that the list constitutes an assessment tool with respect to the representativeness, utility, and feasibility of the most frequent methodological quality items in the literature, one that provides practitioners and researchers with clear criteria for choosing items that may be adequate to their needs. We propose individual methodological features as indicators of quality, arguing that these need to be taken into account when designing, implementing, or evaluating an intervention program. This enhances methodological quality of intervention programs and fosters the cumulative knowledge based on meta-analyses of these interventions. Future development of the checklist is discussed. PMID:27917143

Methodological and Design Considerations in Evaluating the Impact of Prevention Programs on Violence and Related Health Outcomes.

PubMed

Massetti, Greta M; Simon, Thomas R; Smith, Deborah Gorman

2016-10-01

Drawing on research that has identified specific predictors and trajectories of risk for violence and related negative outcomes, a multitude of small- and large-scale preventive interventions for specific risk behaviors have been developed, implemented, and evaluated. One of the principal challenges of these approaches is that a number of separate problem-specific programs targeting different risk areas have emerged. However, as many negative health behaviors such as substance abuse and violence share a multitude of risk factors, many programs target identical risk factors. There are opportunities to understand whether evidence-based programs can be leveraged for potential effects across a spectrum of outcomes and over time. Some recent work has documented longitudinal effects of evidence-based interventions on generalized outcomes. This work has potential for advancing our understanding of the effectiveness of promising and evidence-based prevention strategies. However, conducting longitudinal follow-up of established interventions presents a number of methodological and design challenges. To answer some of these questions, the Centers for Disease Control and Prevention convened a panel of multidisciplinary experts to discuss opportunities to take advantage of evaluations of early prevention programs and evaluating multiple long-term outcomes. This special section of the journal Prevention Science includes a series of papers that begin to address the relevant considerations for conducting longitudinal follow-up evaluation research. This collection of papers is intended to inform our understanding of the challenges and strategies for conducting longitudinal follow-up evaluation research that could be used to drive future research endeavors.

Rigorous Measures of Implementation: A Methodological Framework for Evaluating Innovative STEM Programs

ERIC Educational Resources Information Center

Cassata-Widera, Amy; Century, Jeanne; Kim, Dae Y.

2011-01-01

The practical need for multidimensional measures of fidelity of implementation (FOI) of reform-based science, technology, engineering, and mathematics (STEM) instructional materials, combined with a theoretical need in the field for a shared conceptual framework that could support accumulating knowledge on specific enacted program elements across…

Identifying Best Practices for Multicultural Education in a Psychology Graduate Program

ERIC Educational Resources Information Center

Hunnicutt, Adrienne D.

2010-01-01

This study was designed to examine teaching practices at Fuller Theological Seminary's graduate psychology program, using secondary analysis of existing data. Lee, Shields, and Oh (2008) collected data from approximately 300 students who evaluated the helpfulness of 18 different instructional methodologies used at Fuller, and answered questions…

The Application of Cost-Benefit Analysis in Manpower Area.

ERIC Educational Resources Information Center

Barsby, Steven L.

The relative efficiency of various manpower programs as seen through cost-benefit analysis is assessed, and the contribution that cost-benefit analysis has made in evaluating manpower programs is discussed, taking into account a variety of methodologies presented in different studies. Vocational rehabilitation appears to yield the highest…

The Metropolitan Advanced Technical Magnet High School: 1990-1991. Formative Evaluation.

ERIC Educational Resources Information Center

Seever, Mark L.

This report evaluates the implementation of the "Long-Range Magnet School Plan" at Metropolitan Advanced Technical Magnet High School in Kansas City, Missouri for academic year 1990-91. Introductory sections describe the program, and the evaluation design and methodology. Results on enrollment indicated that only ninth and tenth grades…

Interviewer as Instrument: Accounting for Human Factors in Evaluation Research

ERIC Educational Resources Information Center

Brown, Joel H.

2006-01-01

This methodological study examines an original data collection model designed to incorporate human factors and enhance data richness in qualitative and evaluation research. Evidence supporting this model is drawn from in-depth youth and adult interviews in one of the largest policy/program evaluations undertaken in the United States, the Drug,…

Automated Corrosion Detection Program

DTIC Science & Technology

2001-10-01

More detailed explanations of the methodology development can be found in Hidden Corrosion Detection Technology Assessment, a paper presented at...Detection Program, a paper presented at the Fourth Joint DoD/FAA/NASA Conference on Aging Aircraft, 2000. AS&M PULSE. The PULSE system, developed...selection can be found in The Evaluation of Hidden Corrosion Detection Technologies on the Automated Corrosion Detection Program, a paper presented

Impact of "Grassroots on Work" (GROW) Extension Program to the Bachelor of Arts in Political Science Students' Sense of Civic Responsibility

ERIC Educational Resources Information Center

Paga, Mark Leo Huit

2015-01-01

Purpose: The purpose of this study was to determine the medium term effect of service-learning program or "Grassroots on Work" extension program to civic responsibility of AB Political Science students. Methodology: This study employed an impact evaluation research design and both qualitative and quantitative. The data on goals and…

Biomedical application in space, pilot program in the southern California region

NASA Technical Reports Server (NTRS)

Kelton, A. A.

1979-01-01

A pilot program is presented which was to promote utilization of the Shuttle/Spacelab for medical and biological research applied to terrestrial needs. The program was limited to the Southern California region and consisted of the following five tasks: (1) preparation of educational materials; (2) identification of principal investigators; (3) initial contact and visit; (4)development of promising applications; and (5) evaluation of regional program methodology.

Oregon regional intelligent transportation systems (ITS) integration program. Final phase III report, transit tracker information displays

DOT National Transportation Integrated Search

2003-11-14

Transit Tracker uses global positioning system (GPS) technology to track how far a bus is along its scheduled route. This document presents the evaluation strategies and objectives, the data collection methodologies, and the results of the evaluation...

EVALUATING THE WATER QUALITY EFFECTIVENESS OF WATERSHED-SCALE SOURCE WATER PROTECTION PROGRAMS

EPA Science Inventory

The US EPA Office of Research and Development, the Ohio River Valley Water Sanitation Commission (ORSANCO) and the Upper Big Walnut Creek Quality Partnership created a collaborative team of eleven agencies and universities to develop a methodology for evaluating the effectiveness...

Experiments in evaluation capacity building: Enhancing brain disorders research impact in Ontario.

PubMed

Nylen, Kirk; Sridharan, Sanjeev

2017-05-08

This paper is the introductory paper on a forum on evaluation capacity building for enhancing impacts of research on brain disorders. It describes challenges and opportunities of building evaluation capacity among community-based organizations in Ontario involved in enhancing brain health and supporting people living with a brain disorder. Using an example of a capacity building program called the "Evaluation Support Program", which is run by the Ontario Brain Institute, this forum discusses multiple themes including evaluation capacity building, evaluation culture and evaluation methodologies appropriate for evaluating complex community interventions. The goal of the Evaluation Support Program is to help community-based organizations build the capacity to demonstrate the value that they offer in order to improve, sustain, and spread their programs and activities. One of the features of this forum is that perspectives on the Evaluation Support Program are provided by multiple stakeholders, including the community-based organizations, evaluation team members involved in capacity building, thought leaders in the fields of evaluation capacity building and evaluation culture, and the funders. Copyright © 2017. Published by Elsevier Ltd.

Developing a Methodology for Eliciting Subjective Probability Estimates During Expert Evaluations of Safety Interventions: Application for Bayesian Belief Networks

NASA Technical Reports Server (NTRS)

Wiegmann, Douglas A.a

2005-01-01

The NASA Aviation Safety Program (AvSP) has defined several products that will potentially modify airline and/or ATC operations, enhance aircraft systems, and improve the identification of potential hazardous situations within the National Airspace System (NAS). Consequently, there is a need to develop methods for evaluating the potential safety benefit of each of these intervention products so that resources can be effectively invested to produce the judgments to develop Bayesian Belief Networks (BBN's) that model the potential impact that specific interventions may have. Specifically, the present report summarizes methodologies for improving the elicitation of probability estimates during expert evaluations of AvSP products for use in BBN's. The work involved joint efforts between Professor James Luxhoj from Rutgers University and researchers at the University of Illinois. The Rutgers' project to develop BBN's received funding by NASA entitled "Probabilistic Decision Support for Evaluating Technology Insertion and Assessing Aviation Safety System Risk." The proposed project was funded separately but supported the existing Rutgers' program.

Impact of the Surgical Research Methodology Program on surgical residents' research profiles.

PubMed

Farrokhyar, Forough; Amin, Nalin; Dath, Deepak; Bhandari, Mohit; Kelly, Stephan; Kolkin, Ann M; Gill-Pottruff, Catherine; Skot, Martina; Reid, Susan

2014-01-01

To evaluate whether implementing the formal Surgical Research Methodology (SRM) Program in the surgical residency curriculum improved research productivity compared with the preceding informal Research Seminar Series (RSS). The SRM Program replaced the RSS in July 2009. In the SRM Program, the curriculum in Year-1 consisted of 12 teaching sessions on the principles of clinical epidemiology and biostatistics, whereas the focus in Year-2 was on the design, conduct, and presentation of a research project. The RSS consisted of 8 research methodology sessions repeated annually for 2 years along with the design, conduct, and presentation of a research project. Research productivity was measured as the number of peer-reviewed publications and the generation of studies with higher levels of evidence. Outcome measures were independently assessed by 2 authors to avoid bias. Student t test and chi-square test were used for the analysis. Frequencies, mean differences with 95% CI, and effect sizes have been reported. In this study, 81 SRM residents were compared with 126 RSS residents. The performance of the SRM residents was superior on all metrics in our evaluation. They were significantly more productive and published more articles than the RSS residents (mean difference = 1.0 [95% CI: 0.5-1.5], p < 0.001) with an effect size of 0.26. The SRM residents presented significantly more projects that were of higher levels of evidence (systematic reviews/meta-analyses, randomized controlled trials, and prospective cohorts) than the RSS residents (52.5% vs 29%, p = 0.005). In addition, the research performance improved 11.0 grades (95% CI: 8.5%-13.5%, p < 0.001) with an effect size of 0.51 in favor of the SRM Program. Although not all surgeons opt for a career as surgeon-scientist, knowledge of research methodology is crucial to appropriately apply evidence-based findings in clinical practice. The SRM Program has significantly improved the research productivity and performance of the surgical residents from all disciplines. The implementation of a similar research methodology program is highly recommended for the benefit of residents' future careers and ultimately, evidence-based patient care. Copyright © 2014 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Evaluative methodology for prioritizing transportation energy conservation strategies

DOE Office of Scientific and Technical Information (OSTI.GOV)

Pang, L.M.G.

An analytical methodology was developed for the purpose of prioritizing a set of transportation energy conservation (TEC) strategies within an urban environment. Steps involved in applying the methodology consist of 1) defining the goals, objectives and constraints of the given urban community, 2) identifying potential TEC strategies, 3) assessing the impact of the strategies, 4) applying the TEC evaluation model, and 5) utilizing a selection process to determine the optimal set of strategies for implementation. This research provides an overview of 21 TEC strategies, a quick-response technique for estimating energy savings, a multiattribute utility theory approach for assessing subjective impacts,more » and a computer program for making the strategy evaluations, all of which assist in expediting the execution of the entire methodology procedure. The critical element of the methodology is the strategy evaluation model which incorporates a number of desirable concepts including 1) a comprehensive accounting of all relevant impacts, 2) the application of multiobjective decision-making techniques, 3) an approach to assure compatibilty among quantitative and qualitative impact measures, 4) the inclusion of the decision maker's preferences in the evaluation procedure, and 5) the cost-effectiveness concept. Application of the methodolgy to Salt Lake City, Utah demonstrated its utility, ease of use and favorability by decision makers.« less

The State of Evaluation in Internal Medicine Residency

PubMed Central

Holmboe, Eric; Beasley, Brent W.

2008-01-01

Background There are no nationwide data on the methods residency programs are using to assess trainee competence. The Accreditation Council for Graduate Medical Education (ACGME) has recommended tools that programs can use to evaluate their trainees. It is unknown if programs are adhering to these recommendations. Objective To describe evaluation methods used by our nation’s internal medicine residency programs and assess adherence to ACGME methodological recommendations for evaluation. Design Nationwide survey. Participants All internal medicine programs registered with the Association of Program Directors of Internal Medicine (APDIM). Measurements Descriptive statistics of programs and tools used to evaluate competence; compliance with ACGME recommended evaluative methods. Results The response rate was 70%. Programs were using an average of 4.2–6.0 tools to evaluate their trainees with heavy reliance on rating forms. Direct observation and practice and data-based tools were used much less frequently. Most programs were using at least 1 of the Accreditation Council for Graduate Medical Education (ACGME)’s “most desirable” methods of evaluation for all 6 measures of trainee competence. These programs had higher support staff to resident ratios than programs using less desirable evaluative methods. Conclusions Residency programs are using a large number and variety of tools for evaluating the competence of their trainees. Most are complying with ACGME recommended methods of evaluation especially if the support staff to resident ratio is high. PMID:18612734

Evaluation of a federally funded workforce development program: The Centers for Public Health Preparedness☆

PubMed Central

Sobelson, Robyn K.; Young, Andrea C.

2017-01-01

The Centers for Public Health Preparedness (CPHP) program was a five-year cooperative agreement funded by the Centers for Disease Control and Prevention (CDC). The program was initiated in 2004 to strengthen terrorism and emergency preparedness by linking academic expertise to state and local health agency needs. The purposes of the evaluation study were to identify the results achieved by the Centers and inform program planning for future programs. The evaluation was summative and retrospective in its design and focused on the aggregate outcomes of the CPHP program. The evaluation results indicated progress was achieved on program goals related to development of new training products, training members of the public health workforce, and expansion of partnerships between accredited schools of public health and state and local public health departments. Evaluation results, as well as methodological insights gleaned during the planning and conduct of the CPHP evaluation, were used to inform the design of the next iteration of the CPHP Program, the Preparedness and Emergency Response Learning Centers (PERLC). PMID:23380597

Methodological approach to a multidimensional evaluation of food and nutrition policies.

PubMed

Santos, Leonor Maria Pacheco; dos Santos, Sandra Maria Chaves

2003-01-01

Recommendations arising from global conferences and summits, expressed the need to formulate and implement public policies to improve household food security. In the context of maximizing benefits given available resources, comprehensive evaluations of the nutrition policies and programs are needed. One obstacle to overcome was a clear definition of terminology; the words efficiency, effectiveness, efficacy and impact, widely used in the context of program evaluation, are sometimes interchanged and there is no consensus about their precise definition. Another approach to health evaluation is based in the paradigm structure-process-outcome. The level structure contemplated the installations, resources, instruments (physical and discursive), as well as the technical bureaucratic organizational structures. Process encompassed the whole set of intervention activities developed, whereas the dimension of outcome comprised the effects of interventions on the health and nutrition of beneficiaries. Each of the three dimensions constitutes a continuum in the evaluation procedure: the structure only fulfills its purposes if the processes are adequate and conversely, processes cannot alone supersede structural limitations. Moreover, all the three dimensions only reach their ultimate objectives through the completion of outcomes. The methodology proposed here has been tested in the case of Bahia, Northeast Brazil, and it was found to be adequate for this type of analysis. We hope this approach of a comprehensive multidimensional evaluation, constitutes an effective contribution for program planners and program managers, in particular with regard to the obstacles detected, some of which can surely be overcome.

Toward a Formal Evaluation of Refactorings

NASA Technical Reports Server (NTRS)

Paul, John; Kuzmina, Nadya; Gamboa, Ruben; Caldwell, James

2008-01-01

Refactoring is a software development strategy that characteristically alters the syntactic structure of a program without changing its external behavior [2]. In this talk we present a methodology for extracting formal models from programs in order to evaluate how incremental refactorings affect the verifiability of their structural specifications. We envision that this same technique may be applicable to other types of properties such as those that concern the design and maintenance of safety-critical systems.

CARES/Life Ceramics Durability Evaluation Software Enhanced for Cyclic Fatigue

NASA Technical Reports Server (NTRS)

Nemeth, Noel N.; Powers, Lynn M.; Janosik, Lesley A.

1999-01-01

The CARES/Life computer program predicts the probability of a monolithic ceramic component's failure as a function of time in service. The program has many features and options for materials evaluation and component design. It couples commercial finite element programs--which resolve a component's temperature and stress distribution--to reliability evaluation and fracture mechanics routines for modeling strength-limiting defects. The capability, flexibility, and uniqueness of CARES/Life have attracted many users representing a broad range of interests and has resulted in numerous awards for technological achievements and technology transfer. Recent work with CARES/Life was directed at enhancing the program s capabilities with regards to cyclic fatigue. Only in the last few years have ceramics been recognized to be susceptible to enhanced degradation from cyclic loading. To account for cyclic loads, researchers at the NASA Lewis Research Center developed a crack growth model that combines the Power Law (time-dependent) and the Walker Law (cycle-dependent) crack growth models. This combined model has the characteristics of Power Law behavior (decreased damage) at high R ratios (minimum load/maximum load) and of Walker law behavior (increased damage) at low R ratios. In addition, a parameter estimation methodology for constant-amplitude, steady-state cyclic fatigue experiments was developed using nonlinear least squares and a modified Levenberg-Marquardt algorithm. This methodology is used to give best estimates of parameter values from cyclic fatigue specimen rupture data (usually tensile or flexure bar specimens) for a relatively small number of specimens. Methodology to account for runout data (unfailed specimens over the duration of the experiment) was also included.

Evaluation methodologies for an advanced information processing system

NASA Technical Reports Server (NTRS)

Schabowsky, R. S., Jr.; Gai, E.; Walker, B. K.; Lala, J. H.; Motyka, P.

1984-01-01

The system concept and requirements for an Advanced Information Processing System (AIPS) are briefly described, but the emphasis of this paper is on the evaluation methodologies being developed and utilized in the AIPS program. The evaluation tasks include hardware reliability, maintainability and availability, software reliability, performance, and performability. Hardware RMA and software reliability are addressed with Markov modeling techniques. The performance analysis for AIPS is based on queueing theory. Performability is a measure of merit which combines system reliability and performance measures. The probability laws of the performance measures are obtained from the Markov reliability models. Scalar functions of this law such as the mean and variance provide measures of merit in the AIPS performability evaluations.

Parametric evaluation of the cost effectiveness of Shuttle payload vibroacoustic test plans

NASA Technical Reports Server (NTRS)

Stahle, C. V.; Gongloff, H. R.; Keegan, W. B.; Young, J. P.

1978-01-01

Consideration is given to alternate vibroacoustic test plans for sortie and free flyer Shuttle payloads. Statistical decision models for nine test plans provide a viable method of evaluating the cost effectiveness of alternate vibroacoustic test plans and the associated test levels. The methodology is a major step toward the development of a useful tool for the quantitative tailoring of vibroacoustic test programs to sortie and free flyer payloads. A broader application of the methodology is now possible by the use of the OCTAVE computer code.

Identification and evaluation of educational uses and users for the STS. Educational planning for utilization of space shuttle ED-PLUSS

NASA Technical Reports Server (NTRS)

Engle, H. A.; Christensen, D. L.

1974-01-01

A planning and feasibility study to identify and document a methodology needed to incorporate educational programs into future missions and operations of the space transportation system was conducted. Six tasks were identified and accomplished during the study. The task statements are as follows: (1) potential user identification, (2) a review of space education programs, (3) development of methodology for user involvement, (4) methods to encourage user awareness, (5) compilation of follow-on ideas, and (6) response to NASA questions. Specific recommendations for improving the educational coverage of space activities are provided.

Adult day health care evaluation study: methodology and implementation. Adult Day Health Care Evaluation Development Group.

PubMed Central

Hedrick, S C; Rothman, M L; Chapko, M; Inui, T S; Kelly, J R; Ehreth, J

1991-01-01

The Adult Day Health Care Evaluation Study was developed in response to a congressional mandate to study the medical efficacy and cost effectiveness of the Adult Day Health Care (ADHC) effort in the Department of Veterans Affairs (VA). Four sites providing ADHC in VA facilities are participating in an ongoing randomized controlled trial. Three years of developmental work prior to the study addressed methodological issues that were problematic in previous studies. This developmental work resulted in the methodological approaches described here: (1) a patient recruitment process that actively recruits and screens all potential candidates using empirically developed admission criteria based on predictors of nursing home placement in VA; (2) the selection and development of measures of medical efficacy that assess a wide range of patient and caregiver outcomes with sufficient sensitivity to detect small but clinically important changes; and (3) methods for detailed, accurate, and efficient measurement of utilization and costs of health care within and outside VA. These approaches may be helpful to other researchers and may advance the methodological sophistication of long-term care program evaluation. PMID:1991678

Payload training methodology study

NASA Technical Reports Server (NTRS)

1990-01-01

The results of the Payload Training Methodology Study (PTMS) are documented. Methods and procedures are defined for the development of payload training programs to be conducted at the Marshall Space Flight Center Payload Training Complex (PCT) for the Space Station Freedom program. The study outlines the overall training program concept as well as the six methodologies associated with the program implementation. The program concept outlines the entire payload training program from initial identification of training requirements to the development of detailed design specifications for simulators and instructional material. The following six methodologies are defined: (1) The Training and Simulation Needs Assessment Methodology; (2) The Simulation Approach Methodology; (3) The Simulation Definition Analysis Methodology; (4) The Simulator Requirements Standardization Methodology; (5) The Simulator Development Verification Methodology; and (6) The Simulator Validation Methodology.

An Elusive Policy Imperative: Data and Methodological Challenges When Using Growth in Student Achievement to Evaluate Teacher Education Programs' "Value-Added"

ERIC Educational Resources Information Center

Amrein-Beardsley, Audrey; Lawton, Kerry; Ronan, Katherine

2017-01-01

In this study researchers examined the effectiveness of one of the largest teacher education programs located within the largest research-intensive universities within the US. They did this using a value-added model as per current federal educational policy imperatives to assess the measurable effects of teacher education programs on their teacher…

A Maritime Phase Zero Force for the Year 2020

DTIC Science & Technology

2009-06-01

mind, the team constructed maritime forces and then evaluated them against the same scenarios to determine which ones performed better. The...120  1.  Project Methodology and Choice of Missions ...............................120  2.  Missions and Scenarios construction methodology... projects are designed to build tools that students in the Systems Engineering Analysis curriculum have learned over the 18 month enrollment in the program

Using Q Methodology to Assess Learning Outcomes Following the Implementation of a Concept-Based Curriculum.

PubMed

Hensel, Desiree

The use of a concept-based curriculum in nursing education is increasing, but assessing its impact remains challenging. This project discusses how Q methodology was used to evaluate our prelicensure program's outcome of creating practitioners who were ready to practice in diverse environments before and after a concept-based curricular revision. The successes and challenges of the revision are discussed.

The substantiation of methodical instrumentation to increase the tempo of high-rise construction in region

NASA Astrophysics Data System (ADS)

Belyaeva, Svetlana; Makeeva, Tatyana; Chugunov, Andrei; Andreeva, Peraskovya

2018-03-01

One of the important conditions of effective renovation of accommodation in region on the base of realization of high-rise construction projects is attraction of investments by forming favorable investment climate, as well as reduction if administrative barriers in construction and update of main funds of housing and communal services. The article proposes methodological bases for assessing the state of the investment climate in the region, as well as the methodology for the formation and evaluation of the investment program of the housing and communal services enterprise. The proposed methodologies are tested on the example of the Voronezh region. Authors also showed the necessity and expediency of using the consulting mechanism in the development of state and non-state investment projects and programs.

What Do All These Numbers Mean? Data Visualization as an Innovative Methodology to Make Program Decisions

ERIC Educational Resources Information Center

Porter, Susan G.; Koch, Steven P.; Henderson, Andrew

2010-01-01

Background: There is a lack of consistent, comprehensible data collection and analysis methods for evaluating teacher preparation program's coverage of required standards for accreditation. Of particular concern is the adequate coverage of standards and competencies that address the teaching of English learners and teachers of students from…

Using Photovoice to Initiate Improvement in a PETE Program

ERIC Educational Resources Information Center

Walker, Ashley; Langdon, Jody L.; Colquitt, Gavin; McCollum, Starla

2017-01-01

There is limited research that includes democratic practices to evaluate the PETE program in its ability to prepare preservice teachers (PTs). In other areas such as community health, methodologies have been used to provide a voice to individuals living the experience. The purpose of this study was to examine PTs' perceptions of a teacher…

An Evaluation of In-School Suspension Programs.

ERIC Educational Resources Information Center

Siskind, Theresa G.; And Others

Findings of a study that determined the effectiveness of the Berkeley County (South Carolina) in-school suspension (ISS) program are presented in this paper. Methodology involved personal interviews conducted with the ISS director and ISS teacher in 8 middle and 8 high schools in the county, a total of 16 principals and 16 teachers. Findings…

A Methodological Review of Research on Leadership Development and Social Capital: Is There a Cause and Effect Relationship?

ERIC Educational Resources Information Center

Van De Valk, Lawrence J.; Constas, Mark A.

2011-01-01

Recent interest in studying social aspects of leadership has brought attention to the relationship between leadership and social capital. There is also growing interest among stakeholders (researchers, practitioners, funders, and program participants) to improve evaluation methods for leadership development programs (LDPs). The purpose of the…

Learning "about" and "from" Variation in Program Impacts Using Multisite Trials. MDRC Working Papers on Research Methodology

ERIC Educational Resources Information Center

Raudenbush, Stephen W.; Bloom, Howard S.

2015-01-01

The present paper, which is intended for a diverse audience of evaluation researchers, applied social scientists, and research funders, provides a broad overview of the conceptual and statistical issues involved in using multisite randomized trials to learn "about" and "from" variation in program effects across…

The Use of Outcome Mapping in the Educational Context

ERIC Educational Resources Information Center

Lewis, Anna

2014-01-01

Outcome Mapping is intended to measure the process by which change occurs, it shifts away from the products of the program to focus on changes in behaviors, relationships, actions, and/or activities of the people involved in the treatment program. This process-oriented methodology, most often used in designing and evaluating community development…

Institutional Commitment to Sustainability: An Evaluation of Natural Resource Extension Programs in Universities in Alabama and Oregon

ERIC Educational Resources Information Center

Broussard, Shorna R.; Bliss, John C.

2007-01-01

Purpose: The purpose of this research is to determine institutional commitment to sustainability by examining Natural Resource Extension program inputs, activities, and participation. Design/methodology/approach: A document analysis of Natural Resource Extension planning and reporting documents was conducted to provide contextual and historical…

Assessment of Alternative Student Aid Delivery Systems: Assessment of the Current Delivery System.

ERIC Educational Resources Information Center

Advanced Technology, Inc., Reston, VA.

The effects of the current system for delivering federal financial assistance to students under the Pell Grant, Guaranteed Student Loan (GSL), and campus-based programs are analyzed. Information is included on the use of the assessment model, which combines program evaluation, systems research, and policy analysis methodologies.…

A Program Evaluation Using Client Records and Census Data.

ERIC Educational Resources Information Center

Bachrach, Kenneth M.; Zautra, Alex

Use of client records and census data as a research methodology can provide mental health planners with information on community needs as well as the adequacy of existing programs. Three ways of analyzing client records in conjunction with census data are: (1) tract by tract, comparing client geographic distribution with census characteristics;…

Assessment of Critical Business Skill Development by MBA Alumni

ERIC Educational Resources Information Center

Glynn, Joseph G.; Wood, Gregory R.

2008-01-01

Six years of survey data were analyzed to assess, among other things, the degree to which an AACSB accredited graduate business program successfully developed student skills in a variety of areas deemed important for career success. The study illustrates a methodology institutions can use to respond to increasing demands for program evaluation and…

Geothermal Resource Reporting Metric (GRRM) Developed for the U.S. Department of Energy's Geothermal Technologies Office

DOE Office of Scientific and Technical Information (OSTI.GOV)

Young, Katherine R.; Wall, Anna M.; Dobson, Patrick F.

This paper reviews a methodology being developed for reporting geothermal resources and project progress. The goal is to provide the U.S. Department of Energy's (DOE) Geothermal Technologies Office (GTO) with a consistent and comprehensible means of evaluating the impacts of its funding programs. This framework will allow the GTO to assess the effectiveness of research, development, and deployment (RD&D) funding, prioritize funding requests, and demonstrate the value of RD&D programs to the U.S. Congress and the public. Standards and reporting codes used in other countries and energy sectors provide guidance to develop the relevant geothermal methodology, but industry feedback andmore » our analysis suggest that the existing models have drawbacks that should be addressed. In order to formulate a comprehensive metric for use by the GTO, we analyzed existing resource assessments and reporting methodologies for the geothermal, mining, and oil and gas industries, and sought input from industry, investors, academia, national labs, and other government agencies. Using this background research as a guide, we describe a methodology for evaluating and reporting on GTO funding according to resource grade (geological, technical and socio-economic) and project progress. This methodology would allow GTO to target funding, measure impact by monitoring the progression of projects, or assess geological potential of targeted areas for development.« less

Risk Assessment Methodology for Software Supportability (RAMSS): guidelines for Adapting Software Supportability Evaluations

DTIC Science & Technology

1986-04-14

CONCIPT DIFINITION OIVILOPMINTITIST I OPERATION ANO ■ MAINTENANCE ■ TRACK MOifCTIO PROGRAMS • «VIIW CRITICAL ISSUIS . Mt PARI INPUTS TO PMO...development and beyond, evaluation criteria must Include quantitative goals (the desired value) and thresholds (the value beyond which the charac

Organizational Learning for Library Enhancements: A Collaborative, Research-Driven Analysis of Academic Department Needs

ERIC Educational Resources Information Center

Loo, Jeffery L.; Dupuis, Elizabeth A.

2015-01-01

This article presents a qualitative evaluation methodology of academic departments for library organizational learning and library enhancement planning. This evaluation used campus units' academic program review reports as a data source and employed collaborative content analysis by library liaisons to extract departmental strengths, weaknesses,…

Exploring Stakeholder Values and Interests in Evaluation

ERIC Educational Resources Information Center

Orr, Shannon K.

2010-01-01

One of the challenges facing program evaluation education is how to bridge the need to train students in theoretical and methodological foundations, and also prepare them for the unpredictability and complex environment outside the classroom. This issue is particularly challenging in terms of understanding stakeholder values and interests. The…

Socioeconomic impact assessment in ex ante evaluations: a case study on the rural development programs of the European Union.

PubMed

Vidueira, Pablo; Díaz-Puente, José M; Rivera, María

2014-08-01

Ex ante impact assessment has become a fundamental tool for effective program management, and thus, a compulsory task when establishing a new program in the European Union (EU). This article aims to analyze benefits from ex ante impact assessment, methodologies followed, and difficulties encountered. This is done through the case study on the rural development programs (RDPs) in the EU. Results regarding methodologies are then contrasted with the international context in order to provide solid insights to evaluators and program managing authorities facing ex ante impact assessment. All European RDPs from the period 2007 through 2013 (a total of 88) and their corresponding available ex ante evaluations (a total of 70) were analyzed focusing on the socioeconomic impact assessment. Only 46.6% of the regions provide quantified impact estimations on socioeconomic impacts in spite of it being a compulsory task demanded by the European Commission (EC). Recommended methods by the EC are mostly used, but there is a lack of mixed method approaches since qualitative methods are used in substitution of quantitative ones. Two main difficulties argued were the complexity of program impacts and the lack of needed program information. Qualitative approaches on their own have been found as not suitable for ex ante impact assessment, while quantitative approaches-such as microsimulation models-provide a good approximation to actual impacts. However, time and budgetary constraints make that quantitative and mixed methods should be mainly applied on the most relevant impacts for the program success. © The Author(s) 2014.

Sustainability of a long term professional development program

NASA Astrophysics Data System (ADS)

Ries, Christine E.

Currently, in most school districts, the main form of teacher education comes from professional development (PD) that claims to improve teaching and student achievement. School districts and teachers spend time and money trying to make sure that they are providing the best quality education for their students. Yet, educators are looking for what the most effective form of PD should look like. Utilizing the methodology of a descriptive case study a long-term PD grant, called Science Alliance was evaluated to add to the research on PD and grant program efficacy. Twelve teachers that participated in the Science Alliance grant were interviewed, observed, and given a survey to see how and to what degree they were implementing the inquiry methodology three years after the grant ended. The results were compared with previously existing data that were collected by a company that Science Alliance hired to complete external research on the effects of the PD. The findings suggest that the teachers that participated have sustained the utilization and implementation of the methodology learned during the training. School administrators and/or staff developers could utilize the findings from this study to see what effective PD may entail. Future researchers may use findings from this study when reporting about grant program evaluations and/or PD.

78 FR 70577 - Agency Information Collection Activities; Proposed Collection, Comments Requested, New Collection...

Federal Register 2010, 2011, 2012, 2013, 2014

2013-11-26

... Program the ability to conduct pretests which evaluate the validity and reliability of information... the proposed collection of information, including the validity of the methodology and assumptions used...

KSC management training system project

NASA Technical Reports Server (NTRS)

Sepulveda, Jose A.

1993-01-01

The stated objectives for the summer of 1993 were: to review the Individual Development Plan Surveys for 1994 in order to automate the analysis of the Needs Assessment effort; and to develop and implement evaluation methodologies to perform ongoing program-wide course-to-course assessment. This includes the following: to propose a methodology to develop and implement objective, performance-based assessment instruments for each training effort; to mechanize course evaluation forms and develop software to facilitate the data gathering, analysis, and reporting processes; and to implement the methodology, forms, and software in at lease one training course or seminar selected among those normally offered in the summer at KSC. Section two of this report addresses the work done in regard to the Individual Development Plan Surveys for 1994. Section three presents the methodology proposed to develop and implement objective, performance-based assessment instruments for each training course offered at KSC.

Using Mixed Methods to Evaluate a Community Intervention for Sexual Assault Survivors: A Methodological Tale.

PubMed

Campbell, Rebecca; Patterson, Debra; Bybee, Deborah

2011-03-01

This article reviews current epistemological and design issues in the mixed methods literature and then examines the application of one specific design, a sequential explanatory mixed methods design, in an evaluation of a community-based intervention to improve postassault care for sexual assault survivors. Guided by a pragmatist epistemological framework, this study collected quantitative and qualitative data to understand how the implementation of a Sexual Assault Nurse Examiner (SANE) program affected prosecution rates of adult sexual assault cases in a large midwestern community. Quantitative results indicated that the program was successful in affecting legal systems change and the qualitative data revealed the mediating mechanisms of the intervention's effectiveness. Challenges of implementing this design are discussed, including epistemological and practical difficulties that developed from blending methodologies into a single project. © The Author(s) 2011.

Nuclear medicine and imaging research (instrumentation and quantitative methods of evaluation): Comprehensive 3-year progress report for the period January 15, 1986-January 14, 1989

DOE Office of Scientific and Technical Information (OSTI.GOV)

Beck, R.N.; Cooper, M.D.

1988-06-01

This program addresses the problems involving the basic science and technology underlying the physical and conceptual tools of radioactive tracer methodology as they relate to the measurement of structural and functional parameters of physiologic importance in health and disease. The principal tool is quantitative radionuclide imaging. The overall objective of this program is to further the development and transfer of radiotracer methodology from basic theory to routine clinical practice in order that individual patients and society as a whole will receive the maximum net benefit from the new knowledge gained. The focus of the research is on the development ofmore » new instruments and radiopharmaceuticals, and the evaluation of these through the phase of clinical feasibility. 58 refs., 15 figs., 4 tabs.« less

Sexual assault prevention programs: current issues, future directions, and the potential efficacy of interventions with women.

PubMed

Yeater, E A; O'Donohue, W

1999-11-01

Current problems facing the primary prevention of sexual assault are reviewed. Effective sexual assault prevention programs for both males and females have been slow to develop due to the fact that the etiologies of sexual assault have not been identified. Although dissemination of prevention programs has become increasingly popular in recent years, few programs have evaluated the extent to which the constructs identified in the interventions are effective at decreasing rates of sexual assault. This article discusses previous studies in sexual assault prevention programs, methodological and conceptual problems that currently exist in the field, pragmatic difficulties regarding program implementation and evaluation, and recommendations for future research with an emphasis on interventions with female participants.

Adaptation of Mesoscale Weather Models to Local Forecasting

NASA Technical Reports Server (NTRS)

Manobianco, John T.; Taylor, Gregory E.; Case, Jonathan L.; Dianic, Allan V.; Wheeler, Mark W.; Zack, John W.; Nutter, Paul A.

2003-01-01

Methodologies have been developed for (1) configuring mesoscale numerical weather-prediction models for execution on high-performance computer workstations to make short-range weather forecasts for the vicinity of the Kennedy Space Center (KSC) and the Cape Canaveral Air Force Station (CCAFS) and (2) evaluating the performances of the models as configured. These methodologies have been implemented as part of a continuing effort to improve weather forecasting in support of operations of the U.S. space program. The models, methodologies, and results of the evaluations also have potential value for commercial users who could benefit from tailoring their operations and/or marketing strategies based on accurate predictions of local weather. More specifically, the purpose of developing the methodologies for configuring the models to run on computers at KSC and CCAFS is to provide accurate forecasts of winds, temperature, and such specific thunderstorm-related phenomena as lightning and precipitation. The purpose of developing the evaluation methodologies is to maximize the utility of the models by providing users with assessments of the capabilities and limitations of the models. The models used in this effort thus far include the Mesoscale Atmospheric Simulation System (MASS), the Regional Atmospheric Modeling System (RAMS), and the National Centers for Environmental Prediction Eta Model ( Eta for short). The configuration of the MASS and RAMS is designed to run the models at very high spatial resolution and incorporate local data to resolve fine-scale weather features. Model preprocessors were modified to incorporate surface, ship, buoy, and rawinsonde data as well as data from local wind towers, wind profilers, and conventional or Doppler radars. The overall evaluation of the MASS, Eta, and RAMS was designed to assess the utility of these mesoscale models for satisfying the weather-forecasting needs of the U.S. space program. The evaluation methodology includes objective and subjective verification methodologies. Objective (e.g., statistical) verification of point forecasts is a stringent measure of model performance, but when used alone, it is not usually sufficient for quantifying the value of the overall contribution of the model to the weather-forecasting process. This is especially true for mesoscale models with enhanced spatial and temporal resolution that may be capable of predicting meteorologically consistent, though not necessarily accurate, fine-scale weather phenomena. Therefore, subjective (phenomenological) evaluation, focusing on selected case studies and specific weather features, such as sea breezes and precipitation, has been performed to help quantify the added value that cannot be inferred solely from objective evaluation.

Display/control requirements for automated VTOL aircraft

NASA Technical Reports Server (NTRS)

Hoffman, W. C.; Kleinman, D. L.; Young, L. R.

1976-01-01

A systematic design methodology for pilot displays in advanced commercial VTOL aircraft was developed and refined. The analyst is provided with a step-by-step procedure for conducting conceptual display/control configurations evaluations for simultaneous monitoring and control pilot tasks. The approach consists of three phases: formulation of information requirements, configuration evaluation, and system selection. Both the monitoring and control performance models are based upon the optimal control model of the human operator. Extensions to the conventional optimal control model required in the display design methodology include explicit optimization of control/monitoring attention; simultaneous monitoring and control performance predictions; and indifference threshold effects. The methodology was applied to NASA's experimental CH-47 helicopter in support of the VALT program. The CH-47 application examined the system performance of six flight conditions. Four candidate configurations are suggested for evaluation in pilot-in-the-loop simulations and eventual flight tests.

A Process Evaluation of Project Developmental Continuity. Interim Report IV, Volume I: Pilot Year Impact Study--Instrument Characteristics and Attrition Trends.

ERIC Educational Resources Information Center

Granville, Arthur C.; And Others

This interim report of a pilot year impact study on evaluation methodology is part of a series of documents on the evaluation of Project Developmental Continuity, a Head Start demonstration program aimed at promoting educational and developmental continuity between children's Head Start and primary school experiences. This report deals with…

Evaluating TBLT: The Case of a Task-Based Spanish Program

ERIC Educational Resources Information Center

González-Lloret, Marta; Nielson, Katharine B.

2015-01-01

The need for foreign language education in the US has increased in recent years, and teaching methods based on traditional textbooks are unlikely to meet the real-world needs of current learners. As a response, interest in Language for Specific Purposes programs has grown and so has Task-Based Language Teaching (TBLT) methodology. This article…

A Literacy Exercise: An Extracurricular Reading Program as an Intervention to Enrich Student Reading Habits in Qatar

ERIC Educational Resources Information Center

Nasser, Ramzi

2013-01-01

This paper evaluates an extracurricular reading activity in Qatari schools. The paper presents a two-month extracurricular reading program designed for fourth grade students. The methodology used triangulation of the data to analyze, in detail, the students' primary perceptions towards reading, teacher self-efficacy, teacher instructional behavior…

Utilizing Benchmarking to Study the Effectiveness of Parent-Child Interaction Therapy Implemented in a Community Setting

ERIC Educational Resources Information Center

Self-Brown, Shannon; Valente, Jessica R.; Wild, Robert C.; Whitaker, Daniel J.; Galanter, Rachel; Dorsey, Shannon; Stanley, Jenelle

2012-01-01

Benchmarking is a program evaluation approach that can be used to study whether the outcomes of parents/children who participate in an evidence-based program in the community approximate the outcomes found in randomized trials. This paper presents a case illustration using benchmarking methodology to examine a community implementation of…

Managing Transitions, Building Bridges: An Evaluation of a Summer Bridge Program for African American Scientists and Engineers

ERIC Educational Resources Information Center

Johnson, Jennifer Michelle

2016-01-01

Purpose: This paper aims to highlight the potential of science, technology, engineering and mathematics (STEM) summer bridge programs to promote college persistence by fostering a positive science identity among participants that is culturally consistent with the values and experiences of African American students. Design/methodology/approach:…

Post Landsat-D advanced concept evaluation /PLACE/

NASA Technical Reports Server (NTRS)

Alexander, L. D.; Alvarado, U. R.; Flatow, F. S.

1979-01-01

The aim of the Post Landsat-D Advanced Concept Evaluation (PLACE) program was to identify the key technology requirements of earth resources satellite systems for the 1985-2000 period. The program involved four efforts: (1) examination of future needs in the earth resources area, (2) creation of a space systems technology model capable of satisfying these needs, (3) identification of key technology requirements posed by this model, and (4) development of a methodology (PRISM) to assist in the priority structuring of the resulting technologies.

Evaluating a Policing Strategy Intended to Disrupt an Illicit Street-Level Drug Market

ERIC Educational Resources Information Center

Corsaro, Nicholas; Brunson, Rod K.; McGarrell, Edmund F.

2010-01-01

The authors examined a strategic policing initiative that was implemented in a high crime Nashville, Tennessee neighborhood by utilizing a mixed-methodological evaluation approach in order to provide (a) a descriptive process assessment of program fidelity; (b) an interrupted time-series analysis relying upon generalized linear models; (c)…

Evaluation Issues in the Comprehensive Employment and Training Act (CETA) Legislation.

ERIC Educational Resources Information Center

Spirer, Janet E.

Underutilization of evaluation findings relative to the Comprehensive Employment and Training Act (CETA) legislation may not stem primarily from factors usually identified in the literature (e.g., methodological reasons) but may be superseded by a more potent factor such as the prominence of the policy or program on the national agenda. Viewed…

Assessing the Impact of Planned Social Change. Occasional Paper Series, #8.

ERIC Educational Resources Information Center

Campbell, Donald T.

Program impact methodology--usually referred to as evaluation research--is described as it is developing in the United States. Several problems face the field of evaluation research. First, those issues grouped as "meta-scientific" include: (1) the distinction between qualitative and quantitative studies; (2) the separation of implementation and…

A Gender Wellbeing Composite Indicator: The Best-Worst Global Evaluation Approach

ERIC Educational Resources Information Center

Dominguez-Serrano, Monica; Blancas, Francisco Javier

2011-01-01

In this paper we present a new global evaluation methodology for constructing composite indicators. Given the limitations in the construction of wellbeing indicators that include a gender perspective, this paper proposes separate measures for men and women, using new linear programming models to improve the traditional Data Envelopment Analysis…

Credentials versus Performance: Review of the Teacher Performance Pay Research

ERIC Educational Resources Information Center

Podgursky, Michael; Springer, Matthew G.

2007-01-01

In this article we examine the economic case for merit or performance-based pay for K-12 teachers. We review several areas of germane research. The direct evaluation literature on these incentive plans is slender; highly diverse in terms of methodology, targeted populations, and programs evaluated; and primarily focused on short-run motivational…

Indexing NASA programs for technology transfer methods development and feasibility

NASA Technical Reports Server (NTRS)

Clingman, W. H.

1972-01-01

This project was undertaken to evaluate the application of a previously developed indexing methodology to ongoing NASA programs. These programs are comprehended by the NASA Program Approval Documents (PADS). Each PAD contains a technical plan for the area it covers. It was proposed that these could be used to generate an index to the complete NASA program. To test this hypothesis two PADS were selected by the NASA Technology Utilization Office for trial indexing. Twenty-five individuals indexed the two PADS using NASA Thesaurus terms. The results demonstrated the feasibility of indexing ongoing NASA programs using PADS as the source of information. The same indexing methodology could be applied to other documents containing a brief description of the technical plan. Results of this project showed that over 85% of the concepts in the technology should be covered by the indexing. Also over 85% of the descriptors chosen would be accurate. This completeness and accuracy for the indexing is considered satisfactory for application in technology transfer.

Studying the teaching of kindness: A conceptual model for evaluating kindness education programs in schools.

PubMed

Kaplan, Deanna M; deBlois, Madeleine; Dominguez, Violeta; Walsh, Michele E

2016-10-01

Recent research suggests that school-based kindness education programs may benefit the learning and social-emotional development of youth and may improve school climate and school safety outcomes. However, how and to what extent kindness education programming influences positive outcomes in schools is poorly understood, and such programs are difficult to evaluate in the absence of a conceptual model for studying their effectiveness. In partnership with Kind Campus, a widely adopted school-based kindness education program that uses a bottom-up program framework, a methodology called concept mapping was used to develop a conceptual model for evaluating school-based kindness education programs from the input of 123 middle school students and approximately 150 educators, school professionals, and academic scholars. From the basis of this model, recommendations for processes and outcomes that would be useful to assess in evaluations of kindness education programs are made, and areas where additional instrument development may be necessary are highlighted. The utility of the concept mapping method as an initial step in evaluating other grassroots or non-traditional educational programming is also discussed. Copyright © 2016 Elsevier Ltd. All rights reserved.

Some Contributions of General Systems Theory, Cybernetics Theory and Management Control Theory to Evaluation Theory and Practice. Research on Evaluation Program Paper and Report Series. Interim Draft.

ERIC Educational Resources Information Center

Cook, Desmond L.

This document, one of a series of reports examining the possible contribution of other disciplines to evaluation methodology, describes the major elements of general systems theory (GST), cybernetics theory (CT) and management control theory (MCT). The author suggests that MCT encapsulates major concerns of evaluation since it reveals that…

Three computer codes to read, plot and tabulate operational test-site recorded solar data

NASA Technical Reports Server (NTRS)

Stewart, S. D.; Sampson, R. S., Jr.; Stonemetz, R. E.; Rouse, S. L.

1980-01-01

Computer programs used to process data that will be used in the evaluation of collector efficiency and solar system performance are described. The program, TAPFIL, reads data from an IBM 360 tape containing information (insolation, flowrates, temperatures, etc.) from 48 operational solar heating and cooling test sites. Two other programs, CHPLOT and WRTCNL, plot and tabulate the data from the direct access, unformatted TAPFIL file. The methodology of the programs, their inputs, and their outputs are described.

Adaptation of Lean Six Sigma Methodologies for the Evaluation of Veterans Choice Program at 3 Urban Veterans Affairs Medical Centers.

PubMed

Ball, Sherry L; Stevenson, Lauren D; Ladebue, Amy C; McCreight, Marina S; Lawrence, Emily C; Oestreich, Taryn; Lambert-Kerzner, Anne C

2017-07-01

The Veterans Health Administration (VHA) is adapting to meet the changing needs of our Veterans. VHA leaders are promoting quality improvement strategies including Lean Six Sigma (LSS). This study used LSS tools to evaluate the Veterans Choice Program (VCP), a program that aims to improve access to health care services for eligible Veterans by expanding health care options to non-VHA providers. LSS was utilized to assess the current process and efficiency patterns of the VCP at 3 VHA Medical Centers. LSS techniques were used to assess data obtained through semistructured interviews with Veterans, staff, and providers to describe and evaluate the VCP process by identifying wastes and defects. The LSS methodology facilitated the process of targeting priorities for improvement and constructing suggestions to close identified gaps and inefficiencies. Identified key process wastes included inefficient exchange of clinical information between stakeholders in and outside of the VHA; poor dissemination of VCP programmatic information; shortages of VCP-participating providers; duplication of appointments; declines in care coordination; and lack of program adaptability to local processes. Recommendations for improvement were formulated using LSS. This evaluation illustrates how LSS can be utilized to assess a nationally mandated health care program. By focusing on stakeholder, staff, and Veteran perspectives, process defects in the VCP were identified and improvement recommendations were made. However, the current LSS language used is not intuitive in health care and similar applications of LSS may consider using new language and goals adapted specifically for health care.

Maximum likelihood estimation of signal detection model parameters for the assessment of two-stage diagnostic strategies.

PubMed

Lirio, R B; Dondériz, I C; Pérez Abalo, M C

1992-08-01

The methodology of Receiver Operating Characteristic curves based on the signal detection model is extended to evaluate the accuracy of two-stage diagnostic strategies. A computer program is developed for the maximum likelihood estimation of parameters that characterize the sensitivity and specificity of two-stage classifiers according to this extended methodology. Its use is briefly illustrated with data collected in a two-stage screening for auditory defects.

Seasat-A ASVT: Commercial demonstration experiments. Results analysis methodology for the Seasat-A case studies

NASA Technical Reports Server (NTRS)

1979-01-01

The SEASAT-A commercial demonstration program ASVT is described. The program consists of a set of experiments involving the evaluation of a real time data distributions system, the SEASAT-A user data distribution system, that provides the capability for near real time dissemination of ocean conditions and weather data products from the U.S. Navy Fleet Numerical Weather Central to a selected set of commercial and industrial users and case studies, performed by commercial and industrial users, using the data gathered by SEASAT-A during its operational life. The impact of the SEASAT-A data on business operations is evaluated by the commercial and industrial users. The approach followed in the performance of the case studies, and the methodology used in the analysis and integration of the case study results to estimate the actual and potential economic benefits of improved ocean condition and weather forecast data are described.

METHODOLOGICAL PLURALISM AND MIXED METHODOLOGY TO STRENGTHEN COMMUNITY PSYCHOLOGY RESEARCH: AN EXAMPLE FROM OXFORD HOUSE

PubMed Central

May, Emily M.; Hunter, Bronwyn A.; Jason, Leonard A.

2017-01-01

This article evaluates how a plurality of research methods has served a research program that has functioned in a much-needed area of research: the role of housing and recovery residences in addiction recovery. The review focuses on one mutually supportive recovery residence model, called Oxford House, which represents more than 1,700 democratic, self-governing residences. To date, there has been no comprehensive evaluation of the research methods used with Oxford House or any other recovery residence. In this article, research methods, including study designs and data analyses, are summarized for 114 peer-reviewed empirical studies that included data on Oxford Houses or Oxford House residents. This review of a pluralistic research program can inform community researchers about the value of recovery residences, the many ways in which recovery residences may be assessed, and the benefits of using multiple methods. Implications for future recovery residence research are discussed. PMID:28839344

METHODOLOGICAL PLURALISM AND MIXED METHODOLOGY TO STRENGTHEN COMMUNITY PSYCHOLOGY RESEARCH: AN EXAMPLE FROM OXFORD HOUSE.

PubMed

May, Emily M; Hunter, Bronwyn A; Jason, Leonard A

2017-01-01

This article evaluates how a plurality of research methods has served a research program that has functioned in a much-needed area of research: the role of housing and recovery residences in addiction recovery. The review focuses on one mutually supportive recovery residence model, called Oxford House, which represents more than 1,700 democratic, self-governing residences. To date, there has been no comprehensive evaluation of the research methods used with Oxford House or any other recovery residence. In this article, research methods, including study designs and data analyses, are summarized for 114 peer-reviewed empirical studies that included data on Oxford Houses or Oxford House residents. This review of a pluralistic research program can inform community researchers about the value of recovery residences, the many ways in which recovery residences may be assessed, and the benefits of using multiple methods. Implications for future recovery residence research are discussed.

Curriculum evaluation and revision in a nascent field: the utility of the retrospective pretest--posttest model in a homeland security program of study.

PubMed

Pelfrey, William V; Pelfrey, William V

2009-02-01

Although most academic disciplines evolve at a measured pace, the emerging field of homeland security must, for reasons of safety and security, evolve rapidly. The Department of Homeland Security sponsored the establishment of a graduate educational program for key officials holding homeland security roles. Because homeland security is a nascent field, the establishment of a program curriculum was forced to draw from a variety of disciplines. Curriculum evaluation was complicated by the rapid changes occurring in the emerging discipline, producing response shift bias, and interfering with the pre-post assessments. To compensate for the validity threat associated with response shift bias, a retrospective pretest-posttest evaluative methodology was used. Data indicate the program has evolved in a significant and orderly fashion and these data support the use of this innovative evaluation approach in the development of any discipline.

Measuring Success in Obesity Prevention: A Synthesis of Health Promotion Switzerland's Long-Term Monitoring and Evaluation Strategy

PubMed Central

Ackermann, Günter; Kirschner, Michael; Guggenbühl, Lisa; Abel, Bettina; Klohn, Axel; Mattig, Thomas

2015-01-01

Aims Since 2007, Health Promotion Switzerland has implemented a national priority program for a healthy body weight. This article provides insight into the methodological challenges and results of the program evaluation. Methods Evaluation of the long-term program required targeted monitoring and evaluation projects addressing different outcome levels. The evaluation was carried out according to the Swiss Model for Outcome Classification (SMOC), a model designed to classify the effects of health promotion and prevention efforts. Results The results presented in this article emphasize both content and methods. The national program successfully achieved outcomes on many different levels within complex societal structures. The evaluation system built around the SMOC enabled assessment of program progress and the development of key indicators. However, it is not possible to determine definitively to what extent the national program helped stabilize the prevalence of obesity in Switzerland. Conclusion The model has shown its utility in providing a basis for evaluation and monitoring of the national program. Continuous analysis of data from evaluation and monitoring has made it possible to check the plausibility of suspected causal relationships as well as to establish an overall perspective and assessment of effectiveness supported by a growing body of evidence. PMID:25765161

A systematic review on how to conduct evaluations in community-based rehabilitation.

PubMed

Grandisson, Marie; Hébert, Michèle; Thibeault, Rachel

2014-01-01

Community-based rehabilitation (CBR) must prove that it is making a significant difference for people with disabilities in low- and middle-income countries. Yet, evaluation is not a common practice and the evidence for its effectiveness is fragmented and largely insufficient. The objective of this article was to review the literature on best practices in program evaluation in CBR in relation to the evaluative process, the frameworks, and the methods of data collection. A systematic search was conducted on five rehabilitation databases and the World Health Organization website with keywords associated with CBR and program evaluation. Two independent researchers selected the articles. Twenty-two documents were included. The results suggest that (1) the evaluative process needs to be conducted in close collaboration with the local community, including people with disabilities, and to be followed by sharing the findings and taking actions, (2) many frameworks have been proposed to evaluate CBR but no agreement has been reached, and (3) qualitative methodologies have dominated the scene in CBR so far, but their combination with quantitative methods has a lot of potential to better capture the effectiveness of this strategy. In order to facilitate and improve evaluations in CBR, there is an urgent need to agree on a common framework, such as the CBR matrix, and to develop best practice guidelines based on the literature available and consensus among a group of experts. These will need to demonstrate a good balance between community development and standards for effective evaluations. Implications for Rehabilitation In the quest for evidence of the effectiveness of community-based rehabilitation (CBR), a shared program evaluation framework would better enable the combination of findings from different studies. The evaluation of CBR programs should always include sharing findings and taking action for the sake of the local community. Although qualitative methodologies have dominated the scene in CBR and remain highly relevant, there is also a call for the inclusion of quantitative indicators in order to capture the progress made by people participating in CBR programs. The production of best practice guidelines for evaluation in CBR could foster accountable and empowering program evaluations that are congruent with the principles at the heart of CBR and the standards for effective evaluations.

A systematic review on how to conduct evaluations in community-based rehabilitation

PubMed Central

Hébert, Michèle; Thibeault, Rachel

2014-01-01

Purpose Community-based rehabilitation (CBR) must prove that it is making a significant difference for people with disabilities in low- and middle-income countries. Yet, evaluation is not a common practice and the evidence for its effectiveness is fragmented and largely insufficient. The objective of this article was to review the literature on best practices in program evaluation in CBR in relation to the evaluative process, the frameworks, and the methods of data collection. Method A systematic search was conducted on five rehabilitation databases and the World Health Organization website with keywords associated with CBR and program evaluation. Two independent researchers selected the articles. Results Twenty-two documents were included. The results suggest that (1) the evaluative process needs to be conducted in close collaboration with the local community, including people with disabilities, and to be followed by sharing the findings and taking actions, (2) many frameworks have been proposed to evaluate CBR but no agreement has been reached, and (3) qualitative methodologies have dominated the scene in CBR so far, but their combination with quantitative methods has a lot of potential to better capture the effectiveness of this strategy. Conclusions In order to facilitate and improve evaluations in CBR, there is an urgent need to agree on a common framework, such as the CBR matrix, and to develop best practice guidelines based on the literature available and consensus among a group of experts. These will need to demonstrate a good balance between community development and standards for effective evaluations. Implications for Rehabilitation In the quest for evidence of the effectiveness of community-based rehabilitation (CBR), a shared program evaluation framework would better enable the combination of findings from different studies. The evaluation of CBR programs should always include sharing findings and taking action for the sake of the local community. Although qualitative methodologies have dominated the scene in CBR and remain highly relevant, there is also a call for the inclusion of quantitative indicators in order to capture the progress made by people participating in CBR programs. The production of best practice guidelines for evaluation in CBR could foster accountable and empowering program evaluations that are congruent with the principles at the heart of CBR and the standards for effective evaluations. PMID:23614357

Evaluation of Cumulative Ecosystem Response to Restoration Projects in the Lower Columbia River and Estuary, 2010

DOE Office of Scientific and Technical Information (OSTI.GOV)

Johnson, Gary E.; Diefenderfer, Heida L.; Thom, Ronald M.

This is the seventh and final annual report of a project (2004–2010) addressing evaluation of the cumulative effects of habitat restoration actions in the 235-km-long lower Columbia River and estuary. The project, called the Cumulative Effects (CE) study, was conducted for the U.S. Army Corps of Engineers Portland District by a collaboration of research agencies led by the Pacific Northwest National Laboratory. We achieved the primary goal of the CE study to develop a methodology to evaluate the cumulative effects of habitat actions in the Columbia Estuary Ecosystem Restoration Program. We delivered 1) standard monitoring protocols and methods to prioritizemore » monitoring activities; 2) the theoretical and empirical basis for a CE methodology using levels-of-evidence; 3) evaluations of cumulative effects using ecological relationships, geo-referenced data, hydrodynamic modeling, and meta-analyses; and 4) an adaptive management process to coordinate and coalesce restoration efforts in the LCRE. A solid foundation has been laid for future comprehensive evaluations of progress made by the Columbia Estuary Ecosystem Restoration Program to understand, conserve, and restore ecosystems in the lower Columbia River and estuary.« less

Minimum Hamiltonian Ascent Trajectory Evaluation (MASTRE) program (update to automatic flight trajectory design, performance prediction, and vehicle sizing for support of Shuttle and Shuttle derived vehicles) engineering manual

NASA Technical Reports Server (NTRS)

Lyons, J. T.

1993-01-01

The Minimum Hamiltonian Ascent Trajectory Evaluation (MASTRE) program and its predecessors, the ROBOT and the RAGMOP programs, have had a long history of supporting MSFC in the simulation of space boosters for the purpose of performance evaluation. The ROBOT program was used in the simulation of the Saturn 1B and Saturn 5 vehicles in the 1960's and provided the first utilization of the minimum Hamiltonian (or min-H) methodology and the steepest ascent technique to solve the optimum trajectory problem. The advent of the Space Shuttle in the 1970's and its complex airplane design required a redesign of the trajectory simulation code since aerodynamic flight and controllability were required for proper simulation. The RAGMOP program was the first attempt to incorporate the complex equations of the Space Shuttle into an optimization tool by using an optimization method based on steepest ascent techniques (but without the min-H methodology). Development of the complex partial derivatives associated with the Space Shuttle configuration and using techniques from the RAGMOP program, the ROBOT program was redesigned to incorporate these additional complexities. This redesign created the MASTRE program, which was referred to as the Minimum Hamiltonian Ascent Shuttle TRajectory Evaluation program at that time. Unique to this program were first-stage (or booster) nonlinear aerodynamics, upper-stage linear aerodynamics, engine control via moment balance, liquid and solid thrust forces, variable liquid throttling to maintain constant acceleration limits, and a total upgrade of the equations used in the forward and backward integration segments of the program. This modification of the MASTRE code has been used to simulate the new space vehicles associated with the National Launch Systems (NLS). Although not as complicated as the Space Shuttle, the simulation and analysis of the NLS vehicles required additional modifications to the MASTRE program in the areas of providing additional flexibility in the use of the program, allowing additional optimization options, and providing special options for the NLS configuration.

The Use of Engineering Design Concept for Computer Programming Course: A Model of Blended Learning Environment

ERIC Educational Resources Information Center

Tritrakan, Kasame; Kidrakarn, Pachoen; Asanok, Manit

2016-01-01

The aim of this research is to develop a learning model which blends factors from learning environment and engineering design concept for learning in computer programming course. The usage of the model was also analyzed. This study presents the design, implementation, and evaluation of the model. The research methodology is divided into three…

Is a Web Survey as Effective as a Mail Survey? A Field Experiment Among Computer Users

ERIC Educational Resources Information Center

Kiernan, Nancy; Kiernan, Michaela; Oyler, Mary; Gilles, Carolyn

2005-01-01

With the exponential increase in Web access, program evaluators need to understand the methodological benefits and barriers of using the Web to collect survey data from program participants. In this experimental study, the authors examined whether a Web survey can be as effective as the more established mail survey on three measures of survey…

Department of Defense Dependents Schools Comprehensive Social Studies Program Evaluation, Spring, 1980. Summary Report.

ERIC Educational Resources Information Center

National Evaluation Systems, Inc., Amherst, MA.

The methodology and results of an assessment designed to determine the status of educational quality in grades 4, 8, and 11 social studies programs of the Department of Defense Dependents Schools (DODDS) is summarized. The assessment was part of the DODDS five-year curriculum development cycle comprised of the following phases: (1) development of…

An ex post facto evaluation framework for place-based police interventions.

PubMed

Braga, Anthony A; Hureau, David M; Papachristos, Andrew V

2011-12-01

A small but growing body of research evidence suggests that place-based police interventions generate significant crime control gains. While place-based policing strategies have been adopted by a majority of U.S. police departments, very few agencies make a priori commitments to rigorous evaluations. Recent methodological developments were applied to conduct a rigorous ex post facto evaluation of the Boston Police Department's Safe Street Team (SST) hot spots policing program. A nonrandomized quasi-experimental design was used to evaluate the violent crime control benefits of the SST program at treated street segments and intersections relative to untreated street segments and intersections. Propensity score matching techniques were used to identify comparison places in Boston. Growth curve regression models were used to analyze violent crime trends at treatment places relative to control places. UNITS OF ANALYSIS: Using computerized mapping and database software, a micro-level place database of violent index crimes at all street segments and intersections in Boston was created. Yearly counts of violent index crimes between 2000 and 2009 at the treatment and comparison street segments and intersections served as the key outcome measure. The SST program was associated with a statistically significant reduction in violent index crimes at the treatment places relative to the comparison places without displacing crime into proximate areas. To overcome the challenges of evaluation in real-world settings, evaluators need to continuously develop innovative approaches that take advantage of new theoretical and methodological approaches.

Meta-analytic Evaluation of a Virtual Field Trip to Connect Middle School Students with University Scientists

NASA Astrophysics Data System (ADS)

Adedokun, Omolola A.; Liu, Jia; Parker, Loran Carleton; Burgess, Wilella

2015-02-01

Although virtual field trips are becoming popular, there are few empirical studies of their impacts on student outcomes. This study reports on a meta-analytic evaluation of the impact of a virtual field trip on student perceptions of scientists. Specifically, the study examined the summary effect of zipTrips broadcasts on evaluation participants' perceptions of scientists, as well as the moderating effect of program type on program impact. The results showed statistically significant effect of each broadcast, as well as statistically significant summary (combined) effect of zipTrips on evaluation participants' perceptions of scientists. Results of the moderation analysis showed that the effect was greater for the students that participated in the evaluation of the 8th grade broadcasts, providing additional insight into the role of program variation in predicting differential program impact. This study illustrates how meta-analysis, a methodology that should be of interest to STEM education researchers and evaluation practitioners, can be used to summarize the effects of multiple offerings of the same program. Other implications for STEM educators are discussed.

[Evaluative designs in public health: methodological considerations].

PubMed

López, Ma José; Marí-Dell'Olmo, Marc; Pérez-Giménez, Anna; Nebot, Manel

2011-06-01

Evaluation of public health interventions poses numerous methodological challenges. Randomization of individuals is not always feasible and interventions are usually composed of multiple factors. To face these challenges, certain elements, such as the selection of the most appropriate design and the use of a statistical analysis that includes potential confounders, are essential. The objective of this article was to describe the most frequently used designs in the evaluation of public health interventions (policies, programs or campaigns). The characteristics, strengths and weaknesses of each of these evaluative designs are described. Additionally, a brief explanation of the most commonly used statistical analysis in each of these designs is provided. Copyright © 2011 Sociedad Española de Salud Pública y Administración Sanitaria. Published by Elsevier Espana. All rights reserved.

Can a workbook work? Examining whether a practitioner evaluation toolkit can promote instrumental use.

PubMed

Campbell, Rebecca; Townsend, Stephanie M; Shaw, Jessica; Karim, Nidal; Markowitz, Jenifer

2015-10-01

In large-scale, multi-site contexts, developing and disseminating practitioner-oriented evaluation toolkits are an increasingly common strategy for building evaluation capacity. Toolkits explain the evaluation process, present evaluation design choices, and offer step-by-step guidance to practitioners. To date, there has been limited research on whether such resources truly foster the successful design, implementation, and use of evaluation findings. In this paper, we describe a multi-site project in which we developed a practitioner evaluation toolkit and then studied the extent to which the toolkit and accompanying technical assistance was effective in promoting successful completion of local-level evaluations and fostering instrumental use of the findings (i.e., whether programs directly used their findings to improve practice, see Patton, 2008). Forensic nurse practitioners from six geographically dispersed service programs completed methodologically rigorous evaluations; furthermore, all six programs used the findings to create programmatic and community-level changes to improve local practice. Implications for evaluation capacity building are discussed. Copyright © 2015 Elsevier Ltd. All rights reserved.

An economic evaluation of public programs for internationalization: the case of the Diagnostic Program in Spain.

PubMed

Cansino, José M; Lopez-Melendo, Jaime; Pablo-Romero, María del P; Sánchez-Braza, Antonio

2013-12-01

This paper evaluates the Diagnostic Program in Spain which is a publicly funded program to promote internationalization of companies located in Andalusia (south of Spain). The methodology used is the propensity score-matching. The treatment group consists of companies which participated in the Program until 2008. The control group has companies which planned to participate in the Program but had not done so up to that date. The response variable measures the ratio of export to total sales for each company. Four covariates have been taken into account: activity, location, sales and number of employees. The analysis leads to the conclusion that the companies that participated in the Program improved their ratio of exports to total sales by about 10 percentage points. Copyright © 2013 Elsevier Ltd. All rights reserved.

An improved approach for flight readiness certification: Methodology for failure risk assessment and application examples. Volume 3: Structure and listing of programs

NASA Technical Reports Server (NTRS)

Moore, N. R.; Ebbeler, D. H.; Newlin, L. E.; Sutharshana, S.; Creager, M.

1992-01-01

An improved methodology for quantitatively evaluating failure risk of spaceflight systems to assess flight readiness and identify risk control measures is presented. This methodology, called Probabilistic Failure Assessment (PFA), combines operating experience from tests and flights with engineering analysis to estimate failure risk. The PFA methodology is of particular value when information on which to base an assessment of failure risk, including test experience and knowledge of parameters used in engineering analyses of failure phenomena, is expensive or difficult to acquire. The PFA methodology is a prescribed statistical structure in which engineering analysis models that characterize failure phenomena are used conjointly with uncertainties about analysis parameters and/or modeling accuracy to estimate failure probability distributions for specific failure modes. These distributions can then be modified, by means of statistical procedures of the PFA methodology, to reflect any test or flight experience. Conventional engineering analysis models currently employed for design of failure prediction are used in this methodology. The PFA methodology is described and examples of its application are presented. Conventional approaches to failure risk evaluation for spaceflight systems are discussed, and the rationale for the approach taken in the PFA methodology is presented. The statistical methods, engineering models, and computer software used in fatigue failure mode applications are thoroughly documented.

A Methodological Evaluation of an Environmental Education Survey: Is There a Technological Advantage

ERIC Educational Resources Information Center

Sharp, Ryan L.; Bradley, Michael J.; Maples, James N.

2017-01-01

Environmental education represents a conceivable way to counter the effects of youth's lack of exposure to the natural environment. However, the effectiveness of these programs is often not evaluated, and when they are, the methods for doing so are not consistent. Without proper and reliable methods of data collection, the results may be…

A Quantitative Assessment of an Outsourced Agricultural Extension Service in the Umzimkhulu District of KwaZulu-Natal, South Africa

ERIC Educational Resources Information Center

Lyne, Michael C.; Jonas, Nomonde; Ortmann, Gerald F.

2018-01-01

Purpose: This study evaluates the impact of an outsourced extension service delivered by Lima Rural Development Foundation (Lima) in the Umzimkhulu district of South Africa. The evaluation is conducted at both the household and program levels. Design/methodology/approach: Household impacts were estimated using two-stage regression with…

A Regression-Based Strategy for Defining Subgroups in a Social Experiment. MDRC Working Papers on Research Methodology.

ERIC Educational Resources Information Center

Kemple, James J.; Snipes, Jason C.

Two prominent themes often emerge from evaluations of education and social program evaluations: (1) the interventions being studied serve diverse populations, even if they are intended to target groups with particular characteristics; and (2) the interventions' impacts vary across groups within the population being served. Thus, most evaluations…

Granting Teachers the "Benefit of the Doubt" in Performance Evaluations

ERIC Educational Resources Information Center

Rogge, Nicky

2011-01-01

Purpose: This paper proposes a benefit of the doubt (BoD) approach to construct and analyse teacher effectiveness scores (i.e. SET scores). Design/methodology/approach: The BoD approach is related to data envelopment analysis (DEA), a linear programming tool for evaluating the relative efficiency performance of a set of similar units (e.g. firms,…

Formative Evaluation of an Experimental BE/E [Basic Electricity and Electronics] Program. Report No. 9-75.

ERIC Educational Resources Information Center

Fishburne, R. P., Jr.; Mims, Diane M.

An experimental Basic Electricity and Electronics course (BE/E) utilizing a lock-step, instructor presentation methodology was developed and evaluated at the Service School Command, Great Lakes. The study, directed toward the training of lower mental group, school nonqualified personnel, investigated comparative data on test performance, attitude,…

Methodological Considerations in Evaluating School-Based Programs to Promote Social Competence and Reduce Problem Behavior

ERIC Educational Resources Information Center

Massetti, Greta M.; Crean, Hugh; Johnson, Deborah; DuBois, David; Ji, Peter

2009-01-01

Interventions that aim to promote social competence, reduce problem behavior, and improve school climate are common at all levels of schooling. This whole-school focus, coupled with researchers' concerns about contamination or spillover effects in evaluations that randomly assign classrooms or students to conditions, as well as advances in…

Sandia National Laboratories performance assessment methodology for long-term environmental programs : the history of nuclear waste management.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Marietta, Melvin Gary; Anderson, D. Richard; Bonano, Evaristo J.

2011-11-01

Sandia National Laboratories (SNL) is the world leader in the development of the detailed science underpinning the application of a probabilistic risk assessment methodology, referred to in this report as performance assessment (PA), for (1) understanding and forecasting the long-term behavior of a radioactive waste disposal system, (2) estimating the ability of the disposal system and its various components to isolate the waste, (3) developing regulations, (4) implementing programs to estimate the safety that the system can afford to individuals and to the environment, and (5) demonstrating compliance with the attendant regulatory requirements. This report documents the evolution of themore » SNL PA methodology from inception in the mid-1970s, summarizing major SNL PA applications including: the Subseabed Disposal Project PAs for high-level radioactive waste; the Waste Isolation Pilot Plant PAs for disposal of defense transuranic waste; the Yucca Mountain Project total system PAs for deep geologic disposal of spent nuclear fuel and high-level radioactive waste; PAs for the Greater Confinement Borehole Disposal boreholes at the Nevada National Security Site; and PA evaluations for disposal of high-level wastes and Department of Energy spent nuclear fuels stored at Idaho National Laboratory. In addition, the report summarizes smaller PA programs for long-term cover systems implemented for the Monticello, Utah, mill-tailings repository; a PA for the SNL Mixed Waste Landfill in support of environmental restoration; PA support for radioactive waste management efforts in Egypt, Iraq, and Taiwan; and, most recently, PAs for analysis of alternative high-level radioactive waste disposal strategies including repositories deep borehole disposal and geologic repositories in shale and granite. Finally, this report summarizes the extension of the PA methodology for radioactive waste disposal toward development of an enhanced PA system for carbon sequestration and storage systems. These efforts have produced a generic PA methodology for the evaluation of waste management systems that has gained wide acceptance within the international community. This report documents how this methodology has been used as an effective management tool to evaluate different disposal designs and sites; inform development of regulatory requirements; identify, prioritize, and guide research aimed at reducing uncertainties for objective estimations of risk; and support safety assessments.« less

Understanding parenting in Manitoba First nations: implications for program development.

PubMed

Eni, Rachel; Rowe, Gladys

2011-01-01

This qualitative study introduced the "Manitoba First Nation Strengthening Families Maternal Child Health Pilot Project" program and evaluation methodologies. The study provided a knowledge base for programmers, evaluators, and communities to develop relevant health promotion, prevention, and intervention programming to assist in meeting health needs of pregnant women and young families. Sixty-five open-ended, semistructured interviews were completed in 13 communities. Data analysis was through grounded theory. Three major themes emerged from the data: interpersonal support and relationships; socioeconomic factors; and community initiatives. Complex structural, historical events compromise parenting; capacity and resilience are supported through informal and formal health and social supports.

Evaluating scholarship productivity in COAMFTE-accredited PhD programs.

PubMed

Jared DuPree, W; White, Mark B; Meredith, William H; Ruddick, Lindsay; Anderson, Michael P

2009-04-01

Due to an increasing trend among states to cut higher education funds, many universities are relying more on private donations and federal funding to keep programs afloat. Scholarship productivity in general has become an integral factor in terms of universities granting tenure to faculty, allocating resources, and supporting program goals due to the fact that more research in a particular area tends to increase the likelihood that one will obtain funding from federal, state, and private sources. In the past, ranking systems have also been used to evaluate programs. However, most ranking systems use methodologies that do not quantify research productivity or evaluate factors that match current university trends. The purpose of this article is to explore current scholarship productivity trends among COAMFTE-accredited doctoral programs through the use of several evaluation methods. Specifically, productivity was examined in regard to the following areas: (a) family therapy journal publications; (b) family science journal publications; (c) historic journal publication trends; and (d) recent journal publication trends.

IMPAC: An Integrated Methodology for Propulsion and Airframe Control

NASA Technical Reports Server (NTRS)

Garg, Sanjay; Ouzts, Peter J.; Lorenzo, Carl F.; Mattern, Duane L.

1991-01-01

The National Aeronautics and Space Administration is actively involved in the development of enabling technologies that will lead towards aircraft with new/enhanced maneuver capabilities such as Short Take-Off Vertical Landing (STOVL) and high angle of attack performance. Because of the high degree of dynamic coupling between the airframe and propulsion systems of these types of aircraft, one key technology is the integration of the flight and propulsion control. The NASA Lewis Research Center approach to developing Integrated Flight Propulsion Control (IFPC) technologies is an in-house research program referred to as IMPAC (Integrated Methodology for Propulsion and Airframe Control). The goals of IMPAC are to develop a viable alternative to the existing integrated control design methodologies that will allow for improved system performance and simplicity of control law synthesis and implementation, and to demonstrate the applicability of the methodology to a supersonic STOVL fighter aircraft. Based on some preliminary control design studies that included evaluation of the existing methodologies, the IFPC design methodology that is emerging at the Lewis Research Center consists of considering the airframe and propulsion system as one integrated system for an initial centralized controller design and then partitioning the centralized controller into separate airframe and propulsion system subcontrollers to ease implementation and to set meaningful design requirements for detailed subsystem control design and evaluation. An overview of IMPAC is provided and detailed discussion of the various important design and evaluation steps in the methodology are included.

Centers of excellence: an assessment tool for cardiovascular and orthopedic programs.

PubMed

Ronning, P L; Meyer, J W

1996-10-01

As payers place more weight on contracting with hospital/health system programs that can differentiate themselves in the market as a "true" center of excellence (COE), it becomes imperative that hospitals/health systems understand the payer perspective about those programmatic attributes that can truly differentiate them from other programs. This report describes an evaluation and rating methodology for hospital/health system subspecialty programs, particularly cardiovascular and orthopedic programs, that can be used as a self-assessment tool. Using as its core a Rating Scale and Ranking Taxonomy, the evaluation and rating methodology presented here allow cardiovascular and orthopedic programs to do the following: Understand the differentiating characteristic of COE. Rate itself against detailed criteria that are being used by payers. Compare aspects of its program to premier or benchmark programs. Interpret the results to assist with strategic and operational direction. Allocate scarce resources to implement a subspecialty program that will attract payers. The Rating Scale and Ranking Taxonomy has 20 criteria for assessing cardiovascular programs and 18 criteria for orthopedic programs. The assessment process is designed to produce two important results: dialogue and action. The underpinnings of any action is a solid business plan that clarifies the program's vision, values, and mission. They are important because most programs will ultimately pursue very similar strategies and tactics; however, the most successful subspecialty programs and practices will be the ones that can execute the strategies and tactics quickly and effectively. In addition, the changes that are engendered by this targeted yet comprehensive assessment process can lead to improved clinical and functional outcomes for patients, as well as systemic improvements in the delivery of care.

Evaluation of Fast-Time Wake Vortex Models using Wake Encounter Flight Test Data

NASA Technical Reports Server (NTRS)

Ahmad, Nashat N.; VanValkenburg, Randal L.; Bowles, Roland L.; Limon Duparcmeur, Fanny M.; Gloudesman, Thijs; van Lochem, Sander; Ras, Eelco

2014-01-01

This paper describes a methodology for the integration and evaluation of fast-time wake models with flight data. The National Aeronautics and Space Administration conducted detailed flight tests in 1995 and 1997 under the Aircraft Vortex Spacing System Program to characterize wake vortex decay and wake encounter dynamics. In this study, data collected during Flight 705 were used to evaluate NASA's fast-time wake transport and decay models. Deterministic and Monte-Carlo simulations were conducted to define wake hazard bounds behind the wake generator. The methodology described in this paper can be used for further validation of fast-time wake models using en-route flight data, and for determining wake turbulence constraints in the design of air traffic management concepts.

Probabilistic structural analysis methods of hot engine structures

NASA Technical Reports Server (NTRS)

Chamis, C. C.; Hopkins, D. A.

1989-01-01

Development of probabilistic structural analysis methods for hot engine structures at Lewis Research Center is presented. Three elements of the research program are: (1) composite load spectra methodology; (2) probabilistic structural analysis methodology; and (3) probabilistic structural analysis application. Recent progress includes: (1) quantification of the effects of uncertainties for several variables on high pressure fuel turbopump (HPFT) turbine blade temperature, pressure, and torque of the space shuttle main engine (SSME); (2) the evaluation of the cumulative distribution function for various structural response variables based on assumed uncertainties in primitive structural variables; and (3) evaluation of the failure probability. Collectively, the results demonstrate that the structural durability of hot engine structural components can be effectively evaluated in a formal probabilistic/reliability framework.

Development of a flight software testing methodology

NASA Technical Reports Server (NTRS)

Mccluskey, E. J.; Andrews, D. M.

1985-01-01

The research to develop a testing methodology for flight software is described. An experiment was conducted in using assertions to dynamically test digital flight control software. The experiment showed that 87% of typical errors introduced into the program would be detected by assertions. Detailed analysis of the test data showed that the number of assertions needed to detect those errors could be reduced to a minimal set. The analysis also revealed that the most effective assertions tested program parameters that provided greater indirect (collateral) testing of other parameters. In addition, a prototype watchdog task system was built to evaluate the effectiveness of executing assertions in parallel by using the multitasking features of Ada.

An evaluation of NASA's program in human factors research: Aircrew-vehicle system interaction

NASA Technical Reports Server (NTRS)

1982-01-01

Research in human factors in the aircraft cockpit and a proposed program augmentation were reviewed. The dramatic growth of microprocessor technology makes it entirely feasible to automate increasingly more functions in the aircraft cockpit; the promise of improved vehicle performance, efficiency, and safety through automation makes highly automated flight inevitable. An organized data base and validated methodology for predicting the effects of automation on human performance and thus on safety are lacking and without such a data base and validated methodology for analyzing human performance, increased automation may introduce new risks. Efforts should be concentrated on developing methods and techniques for analyzing man machine interactions, including human workload and prediction of performance.

Advanced stimulation technology deployment program: Sonat Exploration Company, Haynesville Formation, North Shongaloo, Red Rock Field, Columbia County, Arkansas and Webster Parish, Louisiana. Topical report, August 1995-January 1997

DOE Office of Scientific and Technical Information (OSTI.GOV)

Skees, J.L.; Middlebrook, M.L.; Anthony, W.L.

1997-01-01

The objective of this program is to transfer the core GRI advanced stimulation technologies to Sonat Exploration Company for their continued use upon completion of the program. The methodology to accomplish the objective included seminars and training schools, offset well baseline analysis, fracture treatment design optimization, real-time and post-fracture treatment evaluation and documentation of economic benefits.

NASA's In-Space Technology Experiments Program

NASA Technical Reports Server (NTRS)

Levine, J.; Prusha, S. L.

1992-01-01

The objective of the In-Space Technology Experiments Program is to evaluate and validate innovative space technologies and to provide better knowledge of the effects of microgravity and the space environment. The history, organization, methodology, and current program characteristics are presented. Results of the tank pressure control experiment and the middeck zero-gravity dynamics experiment are described to demonstrate the types of technologies that have flown and the experimental results obtained from these low-cost space flight experiments.

Qualitative evaluation: A critical and interpretative complementary approach to improve health programs and services

PubMed Central

Tayabas, Luz María Tejada; León, Teresita Castillo; ESPINO, JOEL MONARREZ

2014-01-01

This short essay aims at commenting on the origin, development, rationale, and main characteristics of qualitative evaluation (QE), emphasizing the value of this methodological tool to evaluate health programs and services. During the past decades, different approaches have come to light proposing complementary alternatives to appraise the performance of public health programs, mainly focusing on the implementation process involved rather than on measuring the impact of such actions. QE is an alternative tool that can be used to illustrate and understand the process faced when executing health programs. It can also lead to useful suggestions to modify its implementation from the stakeholders’ perspectives, as it uses a qualitative approach that considers participants as reflective subjects, generators of meanings. This implies that beneficiaries become involved in an active manner in the evaluated phenomena with the aim of improving the health programs or services that they receive. With this work we want to encourage evaluators in the field of public health to consider the use of QE as a complementary tool for program evaluation to be able to identify areas of opportunity to improve programs’ implementation processes from the perspective of intended beneficiaries. PMID:25152220

Evaluation of the Capillary Blood Glucose Self-monitoring Program

PubMed Central

Augusto, Mariana Cristina; Nitsche, Maria José Trevizani; Parada, Cristina Maria Garcia de Lima; Zanetti, Maria Lúcia; Carvalhaes, Maria Antonieta de Barros Leite

2014-01-01

OBJECTIVE: to evaluate the structure, process and results of the Capillary Blood Glucose Self-monitoring Program in a Brazilian city. METHOD: epidemiological, cross-sectional study. The methodological framework of Donabedian was used to construct indicators of structure, process and outcome. A random sample (n = 288) of users enrolled and 96 health professionals who worked in the program was studied. Two questionnaires were used that were constructed for this study, one for professionals and one for users, both containing data for the evaluation of structure, process and outcome. Anthropometric measures and laboratory results were collected by consulting the patients' health records. The analysis involved descriptive statistics. RESULTS: most of the professionals were not qualified to work in the program and were not knowledgeable about the set of criteria for patient registration. None of the patients received complete and correct orientations about the program and the percentage with skills to perform conducts autonomously was 10%. As regards the result indicators, 86.4% of the patients and 81.3% of the professionals evaluated the program positively. CONCLUSION: the evaluation indicators designed revealed that one of the main objectives of the program, self-care skills, has not been achieved. PMID:25493676

Health economics and outcomes research fellowship practices reviewed.

PubMed

Suh, Kangho; Gabriel, Susan; Adams, Michelle A; Arcona, Steve

2015-01-01

The guidelines for health economics and outcomes research (HEOR) fellowship training programs devised by the American College of Clinical Pharmacy (ACCP) and the International Society of Pharmacoeconomics and Outcomes Research (ISPOR) suggest that continuous improvements are made to ensure that postgraduate training through didactic and professional experiences prepare fellows for HEOR research careers. The HEOR Fellowship Program at Novartis Pharmaceuticals Corporation was standardized to enhance the fellows' HEOR research understanding and align professional skill sets with the ACCP-ISPOR Fellowship Program Guidelines. Based on feedback from an internal task force comprised of HEOR employees and current and former fellows, the HEOR Fellowship Program was normatively and qualitatively assessed to evaluate the current curricular program. Fellowship program activities were instituted to ensure that the suggested minimum level requirements established by the guidelines were being met. Research opportunities enabling fellows to work hand-in-hand with other fellows and HEOR professionals were emphasized. Curricular enhancements in research methodology and professional training and development, and materials for a structured journal club focusing on specific methodological and HEOR research topics were developed. A seminar series (e.g., creating SMART Goals, StrengthsFinder 2.0) and professional courses (e.g., ISPOR short courses, statistics.com) were included to enhance the fellows' short- and long-term professional experience. Additional program attributes include an online reference library developed to enrich the current research facilities and a Statistical Analysis Software training program. Continuously assessing and updating HEOR fellowship programs keeps programs up-to-date in the latest HEOR concepts and approaches used to evaluate health care, both professionally and educationally. Copyright © 2015 Elsevier Inc. All rights reserved.

A Methodology and a Web Platform for the Collaborative Development of Context-Aware Systems

PubMed Central

Martín, David; López-de-Ipiña, Diego; Alzua-Sorzabal, Aurkene; Lamsfus, Carlos; Torres-Manzanera, Emilio

2013-01-01

Information and services personalization is essential for an optimal user experience. Systems have to be able to acquire data about the user's context, process them in order to identify the user's situation and finally, adapt the functionality of the system to that situation, but the development of context-aware systems is complex. Data coming from distributed and heterogeneous sources have to be acquired, processed and managed. Several programming frameworks have been proposed in order to simplify the development of context-aware systems. These frameworks offer high-level application programming interfaces for programmers that complicate the involvement of domain experts in the development life-cycle. The participation of users that do not have programming skills but are experts in the application domain can speed up and improve the development process of these kinds of systems. Apart from that, there is a lack of methodologies to guide the development process. This article presents as main contributions, the implementation and evaluation of a web platform and a methodology to collaboratively develop context-aware systems by programmers and domain experts. PMID:23666131

The Effectiveness of the Instructional Programs Based on Self-Management Strategies in Acquisition of Social Skills by the Children with Intellectual Disabilities

ERIC Educational Resources Information Center

Avcioglu, Hasan

2012-01-01

The purpose of this study is to evaluate the effectiveness of self-management skills training program, based on self-control strategies, on students with intellectual disabilities. A multiple-probe design across subjects single-subject research methodology was used in this study. Nine students with intellectual disabilities, whose ages are between…

ITS impacts assessment for Seattle MMDI evaluation : modeling methodology and results

DOT National Transportation Integrated Search

1999-09-01

At the request of the Joint Program Office (JPO) for Intelligent Transportation Systems (ITS) of the Federal Highway Administration (FHWA), Mitretek Systems has conducted a modeling : analysis of ITS impacts in support of the Metropolitan Model Deplo...

A Procedure for Measuring the Effectiveness of Training and Development Programs.

ERIC Educational Resources Information Center

Helliwell, Tanis

1978-01-01

The article presents an outline of the methodology used in a study designed to evaluate the effectiveness of staff development courses for Ontario's Civil Service Commission. The procedure included a literature search, instructor interviews, and questionnaire development. (MF)

The Cost Effectivenes of the Pomona Plan.

ERIC Educational Resources Information Center

Metzler, Howard C.

1979-01-01

The cost effectiveness of the Pomona Plan, a well-established annuity and trust program, is examined. The historical development of the deferred-giving plan, its ability to elicit support, and methodologies for evaluating gift values and related costs are discussed. (Author/SF)

NASA software specification and evaluation system design, part 1

NASA Technical Reports Server (NTRS)

1976-01-01

The research to develop methods for reducing the effort expended in software and verification is reported. The development of a formal software requirements methodology, a formal specifications language, a programming language, a language preprocessor, and code analysis tools are discussed.

Paper and Symposia Abstracts, 1972 Annual Meeting.

ERIC Educational Resources Information Center

Cunningham, Donald J., Ed.

This compendium contains abstracts of approximately 700 papers which are classified under administration, curriculum and objectives, instruction and learning, measurement and research methodology, counseling and human development, history and historiography, social context of education, school evaluation and program development, or special…

Improving Adherence to Mediterranean-Style Diet With a Community Culinary Coaching Program: Methodology Development and Process Evaluation.

PubMed

Polak, Rani; Pober, David; Morris, Avigail; Arieli, Rakefet; Moore, Margaret; Berry, Elliot; Ziv, Mati

The Community Culinary Coaching Program is a community-based participatory program aimed at improving communal settlement residents' nutrition. The residents, central kitchens, preschools, and communal dining rooms were identified as areas for intervention. Evaluation included goals accomplishment assessed by food purchases by the central kitchens, and residents' feedback through focus groups. Purchasing included more vegetables (mean (standard error) percent change), (+7% (4); P = .32), fish (+115% (11); P < .001), whole grains, and legumes (+77% (9); P < .001); and less soup powders (-40% (9); P < .05), processed beef (-55% (8); P < .001), and margarine (-100% (4); P < .001). Residents recommended continuing the program beyond the project duration. This model might be useful in organizations with communal dining facilities.

Global Health Values of a Multidirectional Near Peer Training Program in Surgery, Pathology, Anatomy, Research Methodology, and Medical Education for Haitian, Rwandan, and Canadian Medical Students.

PubMed

Elharram, Malik; Dinh, Trish; Lalande, Annie; Ge, Susan; Gao, Sophie; Noël, Geoffroy

As health care delivery increasingly requires providers to cross international borders, medical students at McGill University, Canada, developed a multidirectional exchange program with Haiti and Rwanda. The program integrates surgery, pathology, anatomy, research methodology, and medical education. The aim of the present study was to explore the global health value of this international training program to improve medical education within the environment of developing countries, such as Haiti and Rwanda, while improving sociocultural learning of Canadian students. Students from the University of Kigali, Rwanda and Université Quisqueya, Haiti, participated in a 3-week program at McGill University. The students spanned from the first to sixth year of their respective medical training. The program consisted of anatomy dissections, surgical simulations, clinical pathology shadowing, and interactive sessions in research methodology and medical education. To evaluate the program, a survey was administered to students using a mixed methodology approach. Common benefits pointed out by the participants included personal and professional growth. The exchange improved career development, sense of responsibility toward one's own community, teaching skills, and sociocultural awareness. The participants all agreed that the anatomy dissections improved their knowledge of anatomy and would make them more comfortable teaching the material when the returned to their university. The clinical simulation activities and shadowing experiences allowed them to integrate the different disciplines. However, the students all felt the research component had too little time devoted to it and that the knowledge presented was beyond their educational level. The development of an integrated international program in surgery, pathology, anatomy, research methodology, and medical education provided medical students with an opportunity to learn about differences in health care and medical education between the 3 countries. This exchange demonstrated that a crosscultural near-peer teaching environment can be an effective and sustainable method of medical student-centered development in global health. Copyright © 2017 Icahn School of Medicine at Mount Sinai. Published by Elsevier Inc. All rights reserved.

A Methodology for Evaluating the Hygroscopic Behavior of Wood in Adaptive Building Skins using Motion Grammar

NASA Astrophysics Data System (ADS)

El-Dabaa, Rana; Abdelmohsen, Sherif

2018-05-01

The challenge in designing kinetic architecture lies in the lack of applying computational design and human computer interaction to successfully design intelligent and interactive interfaces. The use of ‘programmable materials’ as specifically fabricated composite materials that afford motion upon stimulation is promising for low-cost low-tech systems for kinetic facades in buildings. Despite efforts to develop working prototypes, there has been no clear methodological framework for understanding and controlling the behavior of programmable materials or for using them for such purposes. This paper introduces a methodology for evaluating the motion acquired from programmed material – resulting from the hygroscopic behavior of wood – through ‘motion grammar’. Motion grammar typically allows for the explanation of desired motion control in a computationally tractable method. The paper analyzed and evaluated motion parameters related to the hygroscopic properties and behavior of wood, and introduce a framework for tracking and controlling wood as a programmable material for kinetic architecture.

Certify for success: A methodology for human-centered certification of advanced aviation systems

NASA Technical Reports Server (NTRS)

Small, Ronald L.; Rouse, William B.

1994-01-01

This position paper uses the methodology in Design for Success as a basis for a human factors certification program. The Design for Success (DFS) methodology espouses a multi-step process to designing and developing systems in a human-centered fashion. These steps are as follows: (1) naturalizing - understand stakeholders and their concerns; (2) marketing - understand market-oriented alternatives to meeting stakeholder concerns; (3) engineering - detailed design and development of the system considering tradeoffs between technology, cost, schedule, certification requirements, etc.; (4) system evaluation - determining if the system meets its goal(s); and (5) sales and service - delivering and maintaining the system. Because the main topic of this paper is certification, we will focus our attention on step 4, System Evaluation, since it is the natural precursor to certification. Evaluation involves testing the system and its parts for their correct behaviors. Certification focuses not only on ensuring that the system exhibits the correct behaviors, but ONLY the correct behaviors.

Program of policy studies in science and technology

NASA Technical Reports Server (NTRS)

Mayo, L. H.

1973-01-01

The application of an interdisciplinary, problem-oriented capability to the performance of total social impact evaluations is discussed. The consequences of introducing new configurations, technological or otherwise into future social environments are presented. The primary characteristics of the program are summarized: (1) emphasis on interdisciplinary, problem-oriented analysis; (2) development of intra- and inter-institutional arrangements for the purpose of analyzing social problems, evaluating existing programs, and assessing the social impacts of prospective policies, programs, and other public actions; (3) focus on methodological approaches to the projection of alternative future social environments, the identification of the effects of the introduction of new policies, programs, or other actions into the social system, and the evaluation of the social impacts of such effects; (4) availability of analytical resources for advisory and research tasks, and provision for use of program facilities as a neutral forum for the discussion of public issues involving involving the impact of advancing technology on social value-institutional processes.

Evaluating the Healthy Start program. Design development to evaluative assessment.

PubMed

Raykovich, K S; McCormick, M C; Howell, E M; Devaney, B L

1996-09-01

The national evaluation of the federally funded Healthy Start program involved translating a design for a process and outcomes evaluation and standard maternal and infant data set, both developed prior to the national evaluation contract award, into an evaluation design and client data collection protocol that could be used to evaluate 15 diverse grantees. This article discusses the experience of creating a process and outcomes evaluation design that was both substantively and methodologically appropriate given such issues as the diversity of grantees and their community-based intervention strategies; the process of accessing secondary data sources, including vital records; the quality of client level data submissions; and the need to incorporate both qualitative and quantitative approaches into the evaluation design. The relevance of this experience for the conduct of other field studies of public health interventions is discussed.

A Methodology for Evaluating Artifacts Produced by a Formal Verification Process

NASA Technical Reports Server (NTRS)

Siminiceanu, Radu I.; Miner, Paul S.; Person, Suzette

2011-01-01

The goal of this study is to produce a methodology for evaluating the claims and arguments employed in, and the evidence produced by formal verification activities. To illustrate the process, we conduct a full assessment of a representative case study for the Enabling Technology Development and Demonstration (ETDD) program. We assess the model checking and satisfiabilty solving techniques as applied to a suite of abstract models of fault tolerant algorithms which were selected to be deployed in Orion, namely the TTEthernet startup services specified and verified in the Symbolic Analysis Laboratory (SAL) by TTTech. To this end, we introduce the Modeling and Verification Evaluation Score (MVES), a metric that is intended to estimate the amount of trust that can be placed on the evidence that is obtained. The results of the evaluation process and the MVES can then be used by non-experts and evaluators in assessing the credibility of the verification results.

Adapting a Multiple-Baseline Design Rationale for Evaluating Instructional Interventions: Implications for the Adoption of Direct Instruction Reading Curricula for Evidence-Based Reform

ERIC Educational Resources Information Center

Vitale, Michael R.; Kaniuka, Theodore S.

2012-01-01

Present national methodological standards for evaluating the credibility of the design of individual research studies have resulted in findings reporting the pre-­post effectiveness of Direct Instruction programs to be eliminated from consideration by educational leaders involved in making curricular decisions intended to advance local school…

Effects of the Dutch Skills for Life Program on the Health Behavior, Bullying, and Suicidal Ideation of Secondary School Students

ERIC Educational Resources Information Center

Fekkes, M.; van de Sande, M. C. E.; Gravesteijn, J. C.; Pannebakker, F. D.; Buijs, G. J.; Diekstra, R. F. W.; Kocken, P. L.

2016-01-01

Purpose: The purpose of this paper is to evaluate the effects of the Dutch "Skills for Life" programme on students' health behaviours, bullying behaviour and suicidal ideation. Design/methodology/approach: The effectiveness of the "Skills for Life" programme on health behaviour outcomes was evaluated at three points in time in…

The Comprehensive Longitudinal Evaluation of the Milwaukee Parental Choice Program: Summary of Final Reports. SCDP Milwaukee Evaluation Report #36

ERIC Educational Resources Information Center

Wolf, Patrick J.

2012-01-01

This report contains a summary of the findings from the various topical reports that comprise the author's comprehensive longitudinal study. As a summary, it does not include extensive details regarding the study samples and scientific methodologies employed in those topical studies. The research revealed a pattern of school choice results that…

Comparison of Expert-Based and Empirical Evaluation Methodologies in the Case of a CBL Environment: The ''Orestis'' Experience

ERIC Educational Resources Information Center

Karoulis, Athanasis; Demetriadis, Stavros; Pombortsis, Andreas

2006-01-01

This paper compares several interface evaluation methods applied in the case of a computer based learning (CBL) environment, during a longitudinal study performed in three European countries, Greece, Germany, and Holland, and within the framework of an EC funded Leonardo da Vinci program. The paper firstly considers the particularities of the CBL…

Systems design study of the Pioneer Venus spacecraft. Volume 2. Preliminary program development plan

NASA Technical Reports Server (NTRS)

1973-01-01

The preliminary development plan for the Pioneer Venus program is presented. This preliminary plan treats only developmental aspects that would have a significant effect on program cost. These significant development areas were: master program schedule planning; test planning - both unit and system testing for probes/orbiter/ probe bus; ground support equipment; performance assurance; and science integration Various test planning options and test method techniques were evaluated in terms of achieving a low-cost program without degrading mission performance or system reliability. The approaches studied and the methodology of the selected approach are defined.

Methodological Innovations in Public Health Education: Transdisciplinary Problem Solving

PubMed Central

Lawlor, Edward F.; Sebert-Kuhlmann, Anne K.; McBride, Timothy D.

2015-01-01

In 2008, the faculty of the Brown School at Washington University in St. Louis designed a Master of Public Health program centered on transdisciplinary problem solving in public health. We have described the rationale for our approach, guiding principles and pedagogy for the program, and specific transdisciplinary competencies students acquire. We have explained how transdisciplinary content has been organized and delivered, how the program is being evaluated, and how we have demonstrated the feasibility of this approach for a Master of Public Health degree. PMID:25706031

Engine environmental effects on composite behavior

NASA Technical Reports Server (NTRS)

Chamis, C. C.; Smith, G. T.

1980-01-01

A series of programs were conducted to investigate and develop the application of composite materials to turbojet engines. A significant part of that effort was directed to establishing the impact resistance and defect growth chracteristics of composite materials over the wide range of environmental conditions found in commercial turbojet engine operations. Both analytical and empirical efforts were involved. The experimental programs and the analytical methodology development as well as an evaluation program for the use of composite materials as fan exit guide vanes are summarized.

A Pilot Cost-Effectiveness Analysis of the Economic and Educational Impact of the Mountain-Plains Education and Economic Development Program, Inc. A Research Study for Department of Health, Education and Welfare, National Institute of Education. Revised May 1976.

ERIC Educational Resources Information Center

Stromsdorfer, Ernst W.; Moayed-Dadkhah, Kamran

Presenting a cost-benefit analysis of the Mountain-Plains Career Education Program (a family based program for the economically deprived in the mountain plains states operating out of Glasgow Air Force Base in Montana) and the methodological basis for a full and more detailed study, this evaluation includes: (1) discussion of theoretical issues…

Methodological innovations in public health education: transdisciplinary problem solving.

PubMed

Lawlor, Edward F; Kreuter, Matthew W; Sebert-Kuhlmann, Anne K; McBride, Timothy D

2015-03-01

In 2008, the faculty of the Brown School at Washington University in St. Louis designed a Master of Public Health program centered on transdisciplinary problem solving in public health. We have described the rationale for our approach, guiding principles and pedagogy for the program, and specific transdisciplinary competencies students acquire. We have explained how transdisciplinary content has been organized and delivered, how the program is being evaluated, and how we have demonstrated the feasibility of this approach for a Master of Public Health degree.

Electric Propulsion Test & Evaluation Methodologies for Plasma in the Environments of Space and Testing (EP TEMPEST) (Briefing Charts)

DTIC Science & Technology

2015-04-01

in the Environments of Space and Testing (EP TEMPEST ) - Program Review (Briefing Charts) 5a. CONTRACT NUMBER In-House 5b. GRANT NUMBER 5c...of Space and Testing (EP TEMPEST ) AFOSR T&E Program Review 13-17 April 2015 Dr. Daniel L. Brown In-Space Propulsion Branch (RQRS) Aerospace Systems...Statement A: Approved for public release; distribution is unlimited. EP TEMPEST (Lab Task, FY14-FY16) Program Goals and Objectives Title: Electric

Evaluating a physician leadership development program - a mixed methods approach.

PubMed

Throgmorton, Cheryl; Mitchell, Trey; Morley, Tom; Snyder, Marijo

2016-05-16

Purpose - With the extent of change in healthcare today, organizations need strong physician leaders. To compensate for the lack of physician leadership education, many organizations are sending physicians to external leadership programs or developing in-house leadership programs targeted specifically to physicians. The purpose of this paper is to outline the evaluation strategy and outcomes of the inaugural year of a Physician Leadership Academy (PLA) developed and implemented at a Michigan-based regional healthcare system. Design/methodology/approach - The authors applied the theoretical framework of Kirkpatrick's four levels of evaluation and used surveys, observations, activity tracking, and interviews to evaluate the program outcomes. The authors applied grounded theory techniques to the interview data. Findings - The program met targeted outcomes across all four levels of evaluation. Interview themes focused on the significance of increasing self-awareness, building relationships, applying new skills, and building confidence. Research limitations/implications - While only one example, this study illustrates the importance of developing the evaluation strategy as part of the program design. Qualitative research methods, often lacking from learning evaluation design, uncover rich themes of impact. The study supports how a PLA program can enhance physician learning, engagement, and relationship building throughout and after the program. Physician leaders' partnership with organization development and learning professionals yield results with impact to individuals, groups, and the organization. Originality/value - Few studies provide an in-depth review of evaluation methods and outcomes of physician leadership development programs. Healthcare organizations seeking to develop similar in-house programs may benefit applying the evaluation strategy outlined in this study.

Descriptive Summaries of the Research Development Test & Evaluation. Army Appropriation FY 1984. Supporting Data FY 1984 Budget Estimate Submitted to Congress--February 1983. Volume I.

DTIC Science & Technology

1983-02-01

s.,ccesstully modeled to enhance future computer design simulations; (2) a new methodology for conduc*n dynamic analysis of vehicle mechanics was...to prelminary design methodology for tilt rotors, advancing blade concepts configuration helicopters, and compound helicopters in conjunction with...feasibility of low-level personnel parachutes has been demon- strated. A study was begun to design a free-fall water contalner. An experimental program to

Intern evaluation strategies in family medicine residency education: what is-and is not-being done.

PubMed

Yates, Jennifer E

2013-06-01

Family medicine interns often have deficiencies that are not initially appreciated. By recognizing those growth opportunities early, programs may be able to better meet their interns' training needs. This study provides a needs assessment to ascertain what evaluation tools are being utilized by residency programs to assess their incoming interns. A questionnaire was sent to all US family medicine residency program coordinators (439 programs) via Survey Monkey© inquiring about whether intern evaluation is performed and, if so, what strategies are used. A mixed mode methodology was used: mailing with incentive, email prompts, and telephone calls. Of 439 programs, 220 (50%) responded to the survey. Most respondents (145, 66%) think intern evaluation is needed. However, only 79 (36%) programs are actually doing intern evaluations-only 14 (6.4%) extensively. Most programs are performing simulations (81, 45%) and assessing knowledge/comfort levels (79, 36%); less than one third are considering personality/learning styles, and almost no programs are evaluating skills such as typing (three, 1.4%) and math (one, 0.5%). Many programs use evaluations to guide future planning, help with early identification of challenging learners, and to match training to the residents' needs. Several programs expressed concern about how they would use the information once obtained. The majority of respondents agreed that a baseline intern evaluation is useful; few are actually doing it. This area is not well-described in the literature; residency programs could benefit from information sharing. The next step is to encourage interest in and implementation of such strategies.

Lakatos Revisited.

ERIC Educational Resources Information Center

Court, Deborah

1999-01-01

Revisits and reviews Imre Lakatos' ideas on "Falsification and the Methodology of Scientific Research Programmes." Suggests that Lakatos' framework offers an insightful way of looking at the relationship between theory and research that is relevant not only for evaluating research programs in theoretical physics, but in the social…

Factors affecting evaluation culture within a non-formal educational organization.

PubMed

Vengrin, Courtney; Westfall-Rudd, Donna; Archibald, Thomas; Rudd, Rick; Singh, Kusum

2018-08-01

While research has been done on many aspects of evaluation within a variety of contexts and organizations, there is a lack of research surrounding the culture of evaluation. This study set out to examine this evaluative culture in one of the world's largest non-formal educational organizations through the use of an online survey and quantitative methodology. A path model was developed to examine the factors affecting evaluation culture. Results show perception regarding evaluation, program area, college major, location, training in evaluation, degree level, and years of experience explained 28% of the variance within evaluation culture. Results also found that the culture of evaluation is greatly impacted by leadership. By taking a closer look at the evaluation culture of a large non-formal educational organization, much can be learned about how to better develop and support evaluative work in other similar organizations and programs. Copyright © 2018 Elsevier Ltd. All rights reserved.

How effective is drug abuse resistance education? A meta-analysis of Project DARE outcome evaluations.

PubMed Central

Ennett, S T; Tobler, N S; Ringwalt, C L; Flewelling, R L

1994-01-01

OBJECTIVES. Project DARE (Drug Abuse Resistance Education) is the most widely used school-based drug use prevention program in the United States, but the findings of rigorous evaluations of its effectiveness have not been considered collectively. METHODS. We used meta-analytic techniques to review eight methodologically rigorous DARE evaluations. Weighted effect size means for several short-term outcomes also were compared with means reported for other drug use prevention programs. RESULTS. The DARE effect size for drug use behavior ranged from .00 to .11 across the eight studies; the weighted mean for drug use across studies was .06. For all outcomes considered, the DARE effect size means were substantially smaller than those of programs emphasizing social and general competencies and using interactive teaching strategies. CONCLUSIONS. DARE's short-term effectiveness for reducing or preventing drug use behavior is small and is less than for interactive prevention programs. PMID:8092361

Efficacy of exercise therapy in workers with rotator cuff tendinopathy: a systematic review

PubMed Central

Desmeules, François; Boudreault, Jennifer; Dionne, Clermont E.; Frémont, Pierre; Lowry, Véronique; MacDermid, Joy C.; Roy, Jean-Sébastien

2016-01-01

Objective: To perform a systematic review of randomized controlled trials (RCTs) on the efficacy of therapeutic exercises for workers suffering from rotator cuff (RC) tendinopathy. Methods: A literature search in four bibliographical databases (Pubmed, CINAHL, EMBASE, and PEDro) was conducted from inception up to February 2015. RCTs were included if participants were workers suffering from RC tendinopathy, the outcome measures included work-related outcomes, and at least one of the interventions under study included exercises. The methodological quality of the studies was evaluated with the Cochrane Risk of Bias Assessment tool. Results: The mean methodological score of the ten included studies was 54.4%±17.2%. Types of workers included were often not defined, and work-related outcome measures were heterogeneous and often not validated. Three RCTs of moderate methodological quality concluded that exercises were superior to a placebo or no intervention in terms of function and return-to-work outcomes. No significant difference was found between surgery and exercises based on the results of two studies of low to moderate methodological quality. One study of low methodological quality, comparing a workplace-based exercise program focusing on the participants' work demands to an exercise program delivered in a clinical setting, concluded that the work-based intervention was superior in terms of function and return-to-work outcomes. Conclusion: There is low to moderate-grade evidence that therapeutic exercises provided in a clinical setting are an effective modality to treat workers suffering from RC tendinopathy and to promote return-to-work. Further high quality studies comparing different rehabilitation programs including exercises in different settings with defined workers populations are needed to draw firm conclusions on the optimal program to treat workers. PMID:27488037

Efficacy of exercise therapy in workers with rotator cuff tendinopathy: a systematic review.

PubMed

Desmeules, François; Boudreault, Jennifer; Dionne, Clermont E; Frémont, Pierre; Lowry, Véronique; MacDermid, Joy C; Roy, Jean-Sébastien

2016-09-30

To perform a systematic review of randomized controlled trials (RCTs) on the efficacy of therapeutic exercises for workers suffering from rotator cuff (RC) tendinopathy. A literature search in four bibliographical databases (Pubmed, CINAHL, EMBASE, and PEDro) was conducted from inception up to February 2015. RCTs were included if participants were workers suffering from RC tendinopathy, the outcome measures included work-related outcomes, and at least one of the interventions under study included exercises. The methodological quality of the studies was evaluated with the Cochrane Risk of Bias Assessment tool. The mean methodological score of the ten included studies was 54.4%±17.2%. Types of workers included were often not defined, and work-related outcome measures were heterogeneous and often not validated. Three RCTs of moderate methodological quality concluded that exercises were superior to a placebo or no intervention in terms of function and return-to-work outcomes. No significant difference was found between surgery and exercises based on the results of two studies of low to moderate methodological quality. One study of low methodological quality, comparing a workplace-based exercise program focusing on the participants' work demands to an exercise program delivered in a clinical setting, concluded that the work-based intervention was superior in terms of function and return-to-work outcomes. There is low to moderate-grade evidence that therapeutic exercises provided in a clinical setting are an effective modality to treat workers suffering from RC tendinopathy and to promote return-to-work. Further high quality studies comparing different rehabilitation programs including exercises in different settings with defined workers populations are needed to draw firm conclusions on the optimal program to treat workers.

Outcome evaluation of a 3-year senior health and wellness initiative.

PubMed

Kuczmarksi, Marie Fanelli; Cotugna, Nancy

2009-02-01

The Wilmington Senior Center developed a 3-year multidisciplinary health and wellness initiative entitled The Time of Your Life to promote healthy aging for participants attending countywide senior centers. The purpose of this paper is to describe the development and implementation of the evaluation methodology and outcomes of this senior health and wellness project that consisted of 11 programs in total. Long-term programming can present several challenges in both implementation and evaluation, but assessing outcomes is critical to documenting program value and accountability. Participant-completed surveys and post-series focus groups were utilized as the evaluation tools. Our findings indicate that the program's objectives of increasing knowledge and motivating adoption or maintenance of a healthy lifestyle were met to varying degrees, while a third objective of assessing the usefulness of resources to inform health-related decisions was unable to be evaluated due to the cross-sectional nature of the data collection. Post-series focus groups indicated that topics of most interest to seniors were money management, medications and staying mentally and physically fit. The evaluations showed that long-term programming, while challenging, can be a successful and sustainable format for senior health education.

78 FR 71624 - Submission for OMB Review; 30-Day Comment Request; Data Collection To Understand How NIH Programs...

Federal Register 2010, 2011, 2012, 2013, 2014

2013-11-29

...: Data collection to understand how NIH programs apply methodologies to improve their research programs... research programs apply methodologies to improve their organizational effectiveness. The degree of an...; 30-Day Comment Request; Data Collection To Understand How NIH Programs Apply Methodologies To Improve...

Human Factors Engineering Program Review Model (NUREG-0711)Revision 3: Update Methodology and Key Revisions

DOE Office of Scientific and Technical Information (OSTI.GOV)

OHara J. M.; Higgins, J.; Fleger, S.

The U.S. Nuclear Regulatory Commission (NRC) reviews the human factors engineering (HFE) programs of applicants for nuclear power plant construction permits, operating licenses, standard design certifications, and combined operating licenses. The purpose of these safety reviews is to help ensure that personnel performance and reliability are appropriately supported. Detailed design review procedures and guidance for the evaluations is provided in three key documents: the Standard Review Plan (NUREG-0800), the HFE Program Review Model (NUREG-0711), and the Human-System Interface Design Review Guidelines (NUREG-0700). These documents were last revised in 2007, 2004 and 2002, respectively. The NRC is committed to the periodicmore » update and improvement of the guidance to ensure that it remains a state-of-the-art design evaluation tool. To this end, the NRC is updating its guidance to stay current with recent research on human performance, advances in HFE methods and tools, and new technology being employed in plant and control room design. NUREG-0711 is the first document to be addressed. We present the methodology used to update NUREG-0711 and summarize the main changes made. Finally, we discuss the current status of the update program and the future plans.« less

Drug utilization review: mechanisms to improve its effectiveness and broaden its scope. The U.S. Pharmacopeia Drug Utilization Review Advisory Panel.

PubMed

2000-01-01

To address important problems and needed changes in online and retrospective drug utilization review (DUR) programs. Emphasis is placed on reliability of DUR criteria and the shift of traditional retrospective DUR programs toward disease management and health care outcomes. Published literature evaluating the role of online and retrospective DUR programs. Particular attention was given to studies assessing DUR criteria reliability and new interventions with retrospective DUR programs. A literature review was conducted along with an expert summary from the U.S. Pharmacopeia Drug Utilization Review Advisory Panel. Studies have revealed variations in DUR criteria that could be affecting clinical practice and patient care. Appropriate formal methodologies and use of consistent procedures in developing online prospective DUR programs and systems could help resolve these problems. Traditional retrospective DUR is also shifting to incorporate disease management and methodologies from health outcomes and pharmacoeconomics studies. Refinements are needed to improve the reliability and validity of online DUR criteria and to minimize false positive messages. Databases created as a result of DUR efforts have been used in new and innovative ways to incorporate health outcomes data and disease management interventions. Additional outcomes data, combined with quality assurance efforts, should increase the utility of DUR/disease management efforts in evaluating health systems while improving the effectiveness and efficiency of pharmacists' health care interventions.

["StigMa" - Evaluation of a Psychological Therapy Program for Stigma-Management].

PubMed

Schenner, Manuela; Kohlbauer, Daniela; Meise, Ullrich; Haller, Christina; Pixner-Huber, Martina; Stürz, Kristina; Günther, Verena

2018-01-01

The project "Stigma Management - StigMa" aims on the evaluation of an adaptive therapy program for patients with psychiatric illness to help them in managing internalized stigma and self-stigmatization. The patients for this pilot-study were recruited in day-hospitals of pro mente tirol . 26 patients participated in 11 group sessions, following 6 modules: "Education", "Activation of Resources", "Social Network", "Self-Esteem", "Social competence in public places" and "My personal stigma management". The control group consisted of 20 patients who did not participate in StigMa. Pre-post-evaluation was done by the Internalized Stigma of Mental Illness-Scale 1. No significant interaction effects could be observed, although in the treatment group, the burden of perceived discrimination was significantly less pronounced after training than before it. The program, however, was evaluated as being extremely positive by the participants. The program StigMa will be adapted in accordance with the suggestions of the participants and reevaluated taking into consideration methodological optimization. © Georg Thieme Verlag KG Stuttgart · New York.

Operational Impacts of Wind Energy Resources in the Bonneville Power Administration Control Area - Phase I Report

DOE Office of Scientific and Technical Information (OSTI.GOV)

Makarov, Yuri V.; Lu, Shuai

2008-07-15

This report presents a methodology developed to study the future impact of wind on BPA power system load following and regulation requirements. The methodology uses historical data and stochastic processes to simulate the load balancing processes in the BPA power system, by mimicking the actual power system operations. Therefore, the results are close to reality, yet the study based on this methodology is convenient to conduct. Compared with the proposed methodology, existing methodologies for doing similar analysis include dispatch model simulation and standard deviation evaluation on load and wind data. Dispatch model simulation is constrained by the design of themore » dispatch program, and standard deviation evaluation is artificial in separating the load following and regulation requirements, both of which usually do not reflect actual operational practice. The methodology used in this study provides not only capacity requirement information, it also analyzes the ramp rate requirements for system load following and regulation processes. The ramp rate data can be used to evaluate generator response/maneuverability requirements, which is another necessary capability of the generation fleet for the smooth integration of wind energy. The study results are presented in an innovative way such that the increased generation capacity or ramp requirements are compared for two different years, across 24 hours a day. Therefore, the impact of different levels of wind energy on generation requirements at different times can be easily visualized.« less

Chile: Acceptability of a Training Program for Depression Management in Primary Care.

PubMed

Marín, Rigoberto; Martínez, Pablo; Cornejo, Juan P; Díaz, Berta; Peralta, José; Tala, Álvaro; Rojas, Graciela

2016-01-01

In Chile, there are inconsistencies in the management of depression in primary care settings, and the National Depression Program, currently in effect, was implemented without a standardized training program. The objective of this study is to evaluate the acceptability of a training program on the management of depression for primary care health teams. The study was a randomized controlled trial, and two primary centers from the Metropolitan Region of Santiago were randomly selected to carry out the intervention training program. Pre-post surveys were applied, to evaluate expectations and satisfaction with the intervention, respectively. Descriptive and content analysis was carried out. The sample consisted of 41 health professionals, 56.1% of who reported that their expectations for the intervention were met. All of the training activities were evaluated with scores higher than 6.4 (on a 1-7 scale). The trainers, the methodology, and the learning environment were considered strengths and facilitators of the program, while the limited duration of the training, the logistical problems faced during part of the program, and the lack of educational material were viewed as weaknesses. The intervention was well accepted by primary health care teams. However, the clinical impact in patients still has to be evaluated.

Implementing a Sleep Health Education and Sleep Disorders Screening Program in Fire Departments: A Comparison of Methodology.

PubMed

Barger, Laura K; O'Brien, Conor S; Rajaratnam, Shantha M W; Qadri, Salim; Sullivan, Jason P; Wang, Wei; Czeisler, Charles A; Lockley, Steven W

2016-06-01

The objective of this study is to compare three methods of administering a sleep health program (SHP) in fire departments. An SHP, comprising sleep health education and screening for common sleep disorders, was implemented in eight fire departments using three approaches: expert-led, train-the-trainer, and online. Participation rates, knowledge assessments, surveys, and focus group interviews were analyzed to assess the reach and effectiveness of the methodologies. The Expert-led SHP had the highest participation rate, greatest improvement in knowledge scores, and prompted more firefighters to seek clinical sleep disorder evaluations (41%) than the other approaches (20 to 25%). Forty-two percent of focus group participants reported changing their sleep behaviors. All approaches yielded reasonable participation rates, but expert-led programs had the greatest reach and effectiveness in educating and screening firefighters for sleep disorders.

ASC-AD penetration modeling FY05 status report.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kistler, Bruce L.; Ostien, Jakob T.; Chiesa, Michael L.

2006-04-01

Sandia currently lacks a high fidelity method for predicting loads on and subsequent structural response of earth penetrating weapons. This project seeks to test, debug, improve and validate methodologies for modeling earth penetration. Results of this project will allow us to optimize and certify designs for the B61-11, Robust Nuclear Earth Penetrator (RNEP), PEN-X and future nuclear and conventional penetrator systems. Since this is an ASC Advanced Deployment project the primary goal of the work is to test, debug, verify and validate new Sierra (and Nevada) tools. Also, since this project is part of the V&V program within ASC, uncertaintymore » quantification (UQ), optimization using DAKOTA [1] and sensitivity analysis are an integral part of the work. This project evaluates, verifies and validates new constitutive models, penetration methodologies and Sierra/Nevada codes. In FY05 the project focused mostly on PRESTO [2] using the Spherical Cavity Expansion (SCE) [3,4] and PRESTO Lagrangian analysis with a preformed hole (Pen-X) methodologies. Modeling penetration tests using PRESTO with a pilot hole was also attempted to evaluate constitutive models. Future years work would include the Alegra/SHISM [5] and AlegrdEP (Earth Penetration) methodologies when they are ready for validation testing. Constitutive models such as Soil-and-Foam, the Sandia Geomodel [6], and the K&C Concrete model [7] were also tested and evaluated. This report is submitted to satisfy annual documentation requirements for the ASC Advanced Deployment program. This report summarizes FY05 work performed in the Penetration Mechanical Response (ASC-APPS) and Penetration Mechanics (ASC-V&V) projects. A single report is written to document the two projects because of the significant amount of technical overlap.« less

Evaluating the benefits of government funded R & D aimed at the private sector

NASA Technical Reports Server (NTRS)

Greenberg, J. S.

1977-01-01

Federal funding of technological research and development is discussed with regard to the procedures for an economic analysis with the goals of (1) determining when the public sector should invest in a research and development program, (2) evaluating the likelihood of private sector participation in terms of public sector participation, and (3) considering the major factors in the formulation of a research and development program in terms of defining initiatives. Public sector investments are evaluated, noting procedures for determining whether benefits exceed costs. The role of the public sector research and development planning is described, considering the procedure for private sector implementation decisions and a methodology for evaluating the possibility of private sector commercialization. The economic value of the public sector research and development program is presented with attention given to a specific case of NASA-sponsored research and development aimed toward the commercialization of new public communications services.

Energy use in the marine transportation industry. Task III. Efficiency improvements. Draft report

DOE Office of Scientific and Technical Information (OSTI.GOV)

Not Available

1977-06-02

Research and development areas that hold promise for maritime energy conservation are identified and evaluated. The methodology used is discussed in Chapter II. The technology base of the commercial marine transportation industry relating to energy usage is made up of: main propulsion plants, propulsors, hydrodynamics, vessel operations, and fuels. Fifteen specific program areas in the first four generic technologies are identified and are evaluated. An economic and energy impact analysis and technological risk assessment was performed on the specific program areas and the results are summarized in Chapter III. The first five appendices address the generic technologies. The sixth appendixmore » contains the baseline operating and cost parameters against which the 15 program areas were evaluated, and the last appendix contains sample printouts of the MTEM model used to evaluate the energy consumption and economic impacts associated with the candidate technology areas. (MCW)« less

Research Management--Of What Nature Is the Concept?

ERIC Educational Resources Information Center

Cook, Desmond L.

Research management is defined as the application of both management and management science to a particular field of research and development activities. Seven components of research management include theory and methodology; the planning, implementation, and evaluation of research programs; communications; utilization; and special applications.…

METHODOLOGY FOR EVALUATING THE EFFECTIVENESS OF NONPOINT SOURCE POLLUTION ABATEMENT PROGRAMS

EPA Science Inventory

The Upper Big Walnut Creek watershed encompasses 190 square miles of predominantly agricultural cropland (65%) in Central Ohio (USA) where agronomic fertilizers and herbicides are used in row crop production. Runoff from the watershed drain to Hoover Reservoir which is Central Oh...

Development and Evaluation of an ADME-informed High Throughput Exposure Estimation Tool

EPA Science Inventory

EPA’s Chemical Safety for Sustainability (CSS) research program has been developing new ways to prioritize chemicals used in consumer products and articles. Using a risk-based methodology to account for both toxicity and exposure offers a comprehensive and systematic approa...

Research Out of the Shadows: A Reply to Rist.

ERIC Educational Resources Information Center

Barone, Thomas E.

1987-01-01

Replies to Rist's unfavorable evaluation of Barone's own article critiquing the arts program of a black elementary school. Argues that quantitative social science research has unsuccessfully modeled itself on scientific methodology, but camouflages its subjectivity and fictionalizes the entire research undertaking. Crafting an educational…

42 CFR 86.13 - Project requirements.

Code of Federal Regulations, 2012 CFR

2012-10-01

... the applicant has established good cause for its omission. (1) Provision of a method for development... method for implementation of the needed training; (3) Provision of an evaluation methodology, including... the training program; and (4) Provision of a method by which trainees will be selected. (b) In...

42 CFR 86.13 - Project requirements.

Code of Federal Regulations, 2011 CFR

2011-10-01

... the applicant has established good cause for its omission. (1) Provision of a method for development... method for implementation of the needed training; (3) Provision of an evaluation methodology, including... the training program; and (4) Provision of a method by which trainees will be selected. (b) In...

42 CFR 86.13 - Project requirements.

Code of Federal Regulations, 2014 CFR

2014-10-01

... the applicant has established good cause for its omission. (1) Provision of a method for development... method for implementation of the needed training; (3) Provision of an evaluation methodology, including... the training program; and (4) Provision of a method by which trainees will be selected. (b) In...

42 CFR 86.13 - Project requirements.

Code of Federal Regulations, 2013 CFR

2013-10-01

... the applicant has established good cause for its omission. (1) Provision of a method for development... method for implementation of the needed training; (3) Provision of an evaluation methodology, including... the training program; and (4) Provision of a method by which trainees will be selected. (b) In...

Evaluation of Analytical and Experimental Methodologies for the Characterization of Wheel/Rail Loads

DOT National Transportation Integrated Search

1976-11-01

This report has been prepared as part of the Improved Track Structures Research Program sponsored by the Office of Rail Safety Research of the Federal Railroad Administration. The major modes of track degradation have been reviewed to identify the si...

Alternative Schools. Research Report 1976-3.

ERIC Educational Resources Information Center

National School Boards Association, Washington, DC.

During the 1970's, school districts throughout the country have created alternative educational programs allowing students and parents to choose the appropriate educational structure and methodology that suit their individual needs. School boards who establish alternative schools must assess, plan, design, effect, and evaluate educational programs…

Development of Probabilistic Life Prediction Methodologies and Testing Strategies for MEMS and CMC's

NASA Technical Reports Server (NTRS)

Jadaan, Osama

2003-01-01

This effort is to investigate probabilistic life prediction methodologies for ceramic matrix composites and MicroElectroMechanical Systems (MEMS) and to analyze designs that determine stochastic properties of MEMS. For CMC's this includes a brief literature survey regarding lifing methodologies. Also of interest for MEMS is the design of a proper test for the Weibull size effect in thin film (bulge test) specimens. The Weibull size effect is a consequence of a stochastic strength response predicted from the Weibull distribution. Confirming that MEMS strength is controlled by the Weibull distribution will enable the development of a probabilistic design methodology for MEMS - similar to the GRC developed CARES/Life program for bulk ceramics. A main objective of this effort is to further develop and verify the ability of the Ceramics Analysis and Reliability Evaluation of Structures/Life (CARES/Life) code to predict the time-dependent reliability of MEMS structures subjected to multiple transient loads. A second set of objectives is to determine the applicability/suitability of the CARES/Life methodology for CMC analysis, what changes would be needed to the methodology and software, and if feasible, run a demonstration problem. Also important is an evaluation of CARES/Life coupled to the ANSYS Probabilistic Design System (PDS) and the potential of coupling transient reliability analysis to the ANSYS PDS.

Cost benefit analysis of space communications technology: Volume 1: Executive summary

NASA Technical Reports Server (NTRS)

Holland, L. D.; Sassone, P. G.; Gallagher, J. J.; Robinette, S. L.; Vogler, F. H.; Zimmer, R. P.

1976-01-01

The questions of (1) whether or not NASA should support the further development of space communications technology, and, if so, (2) which technology's support should be given the highest priority are addressed. Insofar as the issues deal principally with resource allocation, an economics perspective is adopted. The resultant cost benefit methodology utilizes the net present value concept in three distinct analysis stages to evaluate and rank those technologies which pass a qualification test based upon probable (private sector) market failure. User-preference and technology state-of-the-art surveys were conducted (in 1975) to form a data base for the technology evaluation. The program encompassed near-future technologies in space communications earth stations and satellites, including the noncommunication subsystems of the satellite (station keeping, electrical power system, etc.). Results of the research program include confirmation of the applicability of the methodology as well as a list of space communications technologies ranked according to the estimated net present value of their support (development) by NASA.

Emergency radiobioassay preparedness exercises through the NIST radiochemistry intercomparison program.

PubMed

Nour, Svetlana; LaRosa, Jerry; Inn, Kenneth G W

2011-08-01

The present challenge for the international emergency radiobioassay community is to analyze contaminated samples rapidly while maintaining high quality results. The National Institute of Standards and Technology (NIST) runs a radiobioassay measurement traceability testing program to evaluate the radioanalytical capabilities of participating laboratories. The NIST Radiochemistry Intercomparison Program (NRIP) started more than 10 years ago, and emergency performance testing was added to the program seven years ago. Radiobioassay turnaround times under the NRIP program for routine production and under emergency response scenarios are 60 d and 8 h, respectively. Because measurement accuracy and sample turnaround time are very critical in a radiological emergency, response laboratories' analytical systems are best evaluated and improved through traceable Performance Testing (PT) programs. The NRIP provides participant laboratories with metrology tools to evaluate their performance and to improve it. The program motivates the laboratories to optimize their methodologies and minimize the turnaround time of their results. Likewise, NIST has to make adjustments and periodical changes in the bioassay test samples in order to challenge the participating laboratories continually. With practice, radioanalytical measurements turnaround time can be reduced to 3-4 h.

Three models of community mental health services In low-income countries

PubMed Central

2011-01-01

Objective To compare and contrast three models of community mental health services in low-income settings. Data Sources/Study Setting Primary and secondary data collected before, during, and after site visits to mental health programs in Nigeria, the Philippines, and India. Study Design Qualitative case study methodology. Data Collection Data were collected through interviews and observations during site visits to the programs, as well as from reviews of documentary evidence. Principal Findings A set of narrative topics and program indicators were used to compare and contrast three community mental health programs in low-income countries. This allowed us to identify a diversity of service delivery models, common challenges, and the strengths and weaknesses of each program. More definitive evaluations will require the establishment of data collection methods and information systems that provide data about the clinical and social outcomes of clients, as well as their use of services. Conclusions Community mental health programs in low-income countries face a number of challenges. Using a case study methodology developed for this purpose, it is possible to compare programs and begin to assess the effectiveness of diverse service delivery models. PMID:21266051

Taking Stock of Parent Education in the Family Courts: Envisioning a Public Health Model

PubMed Central

Salem, Peter; Sandler, Irwin; Wolchik, Sharlene

2012-01-01

The paper reviewed the development and current status of the parent education movement in the Family Courts. Parent education programs are now being implemented in courts throughout the United States and have a high level of public acceptance; however, a stronger research methodology to evaluate the effects and continued work to align the goals with the content and teaching strategies of these programs are needed. A new conceptual framework is proposed for parent education, which views divorce as a public health problem for children as well as a legal issue. The three-level framework uses concepts from public health to align the goals, content and format of parent education programs and to enable rigorous evaluations of the outcomes achieved by these programs. PMID:23641191

Adding a Time-Series Design Element to the Success Case Method to Improve Methodological Rigor: An Application for Nonprofit Program Evaluation

ERIC Educational Resources Information Center

Coryn, Chris L. S.; Schroter, Daniela C.; Hanssen, Carl E.

2009-01-01

Brinkerhoff's Success Case Method (SCM) was developed with the specific purpose of assessing the impact of organizational interventions (e.g., training and coaching) on business goals by analyzing extreme groups using case study techniques and storytelling. As an efficient and cost-effective method of evaluative inquiry, SCM is attractive in other…

A Methodological Study Evaluating a Pretutorial Computer-Compiled Instructional Program in High School Physics Instruction Initiated from Student-Teacher Selected Instructional Objectives. Final Report.

ERIC Educational Resources Information Center

Leonard, B. Charles; Denton, Jon J.

A study sought to develop and evaluate an instructional model which utilized the computer to produce individually prescribed instructional guides to account for the idiosyncratic variations among students in physics classes at the secondary school level. The students in the treatment groups were oriented toward the practices of selecting…

Meeting the Challenge of Doing an RCT Evaluation of Youth Mentoring in Ireland: A Journey in Mixed Methods

ERIC Educational Resources Information Center

Brady, Bernadine; O'Regan, Connie

2009-01-01

The youth mentoring program Big Brothers Big Sisters is one of the first social interventions involving youth in Ireland to be evaluated using a randomized controlled trial methodology. This article sets out the design process undertaken, describing how the research team came to adopt a concurrent embedded mixed methods design as a means of…

NASA electronic message experiment and study: Detailed test plans

NASA Technical Reports Server (NTRS)

1979-01-01

A methodology for evaluating the utility of high speed digital facsimile as a component of the projected NASA-wide electronic message network is presented. Equipment checkout, operator familiarization, pretest calibration, and the development of procedures are addressed. An experimental test program of the facsimile message service which will carry duplicates of the actual messages sent by other means is highlighted. Also, an operational test program during which messages will be sent on a regular basis in order to accumulate the information that will be used to evaluate system performance and project future growth is described.

Using multiple methods to assess learning and outcomes in an online degree-granting dental hygiene program.

PubMed

Springfield, Emily; Gwozdek, Anne E; Peet, Melissa; Kerschbaum, Wendy E

2012-04-01

Program evaluation is a necessary component of curricular change and innovation. It ascertains whether an innovation has met benchmarks and contributes to the body of knowledge about educational methodologies and supports the use of evidence-based practice in teaching. Education researchers argue that rigorous program evaluation should utilize a mixed-method approach, triangulating both qualitative and quantitative methods to understand program effectiveness. This approach was used to evaluate the University of Michigan Dental Hygiene Degree Completion E-Learning (online) Program. Quantitative data included time spent on coursework, grades, publications, course evaluation results, and survey responses. Qualitative data included student and faculty responses in focus groups and on surveys as well as students' portfolio reflections. The results showed the program was academically rigorous, fostering students' ability to connect theory with practice and apply evidence-based practice principles. These results also demonstrated that the students had learned to critically reflect on their practice and develop expanded professional identities; going beyond the role of clinician, they began to see themselves as educators, advocates, and researchers. This evaluation model is easily adaptable and is applicable to any health science or other professional degree program. This study also raised important questions regarding the effect of meta-reflection on student confidence and professional behavior.

The cost-effectiveness of exercise-based cardiac rehabilitation: a systematic review of the characteristics and methodological quality of published literature.

PubMed

Edwards, Katherine; Jones, Natasha; Newton, Julia; Foster, Charlie; Judge, Andrew; Jackson, Kate; Arden, Nigel K; Pinedo-Villanueva, Rafael

2017-10-19

This descriptive review aimed to assess the characteristics and methodological quality of economic evaluations of cardiac rehabilitation (CR) programs according to updated economic guidelines for healthcare interventions. Recommendations will be made to inform future research addressing the impact of a physical exercise component on cost-effectiveness. Electronic databases were searched for economic evaluations of exercise-based CR programs published in English between 2000 and 2014. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement was used to review the methodological quality of included economic evaluations. Fifteen economic evaluations met the review inclusion criteria. Assessed study characteristics exhibited wide variability, particularly in their economic perspective, time horizon, setting, comparators and included costs, with significant heterogeneity in exercise dose across interventions. Ten evaluations were based on randomised controlled trials (RCTs) spanning 6-24 months but often with weak or inconclusive results; two were modelling studies; and the final three utilised longer time horizons of 3.5-5 years from which findings suggest that long-term exercise-based CR results in lower costs, reduced hospitalisations and a longer cumulative patient lifetime. None of the 15 articles met all the CHEERS quality criteria, with the majority either fully or partially meeting a selection of the assessed variables. Evidence exists supporting the cost-effectiveness of exercise-based CR for cardiovascular disease patients. However, variability in CR program delivery and weak consistency between study perspective and design limits study comparability and therefore the accumulation of evidence in support of a particular exercise regime. The generalisability of study findings was limited due to the exclusion of patients with comorbidities as would typically be found in a real-world setting. The use of longer time-horizons would be more comparable with a chronic condition and enable economic assessments of the long-term effects of CR. As none of the articles met recent reporting standards for the economic assessment of healthcare interventions, it is recommended that future studies adhere to such guidelines.

Remedial Action Assessment System: A computer-based methodology for conducting feasibility studies

DOE Office of Scientific and Technical Information (OSTI.GOV)

White, M.K.; Buelt, J.L.; Stottlemyre, J.A.

1991-02-01

Because of the complexity and number of potential waste sites facing the US Department of Energy (DOE) for potential cleanup, DOE is supporting the development of a computer-based methodology to streamline the remedial investigation/feasibility study process. The Remedial Action Assessment System (RAAS), can be used for screening, linking, and evaluating established technology processes in support of conducting feasibility studies. It is also intended to do the same in support of corrective measures studies. The user interface employs menus, windows, help features, and graphical information while RAAS is in operation. Object-oriented programming is used to link unit processes into sets ofmore » compatible processes that form appropriate remedial alternatives. Once the remedial alternatives are formed, the RAAS methodology can evaluate them in terms of effectiveness, implementability, and cost. RAAS will access a user-selected risk assessment code to determine the reduction of risk after remedial action by each recommended alternative. The methodology will also help determine the implementability of the remedial alternatives at a site and access cost estimating tools to provide estimates of capital, operating, and maintenance costs. This paper presents the characteristics of two RAAS prototypes currently being developed. These include the RAAS Technology Information System, which accesses graphical, tabular and textual information about technologies, and the main RAAS methodology, which screens, links, and evaluates remedial technologies. 4 refs., 3 figs., 1 tab.« less

Methods for Evaluating Natural Experiments in Obesity: A Systematic Review.

PubMed

Bennett, Wendy L; Wilson, Renee F; Zhang, Allen; Tseng, Eva; Knapp, Emily A; Kharrazi, Hadi; Stuart, Elizabeth A; Shogbesan, Oluwaseun; Bass, Eric B; Cheskin, Lawrence J

2018-06-05

Given the obesity pandemic, rigorous methodological approaches, including natural experiments, are needed. To identify studies that report effects of programs, policies, or built environment changes on obesity prevention and control and to describe their methods. PubMed, CINAHL, PsycINFO, and EconLit (January 2000 to August 2017). Natural experiments and experimental studies evaluating a program, policy, or built environment change in U.S. or non-U.S. populations by using measures of obesity or obesity-related health behaviors. 2 reviewers serially extracted data on study design, population characteristics, data sources and linkages, measures, and analytic methods and independently evaluated risk of bias. 294 studies (188 U.S., 106 non-U.S.) were identified, including 156 natural experiments (53%), 118 experimental studies (40%), and 20 (7%) with unclear study design. Studies used 106 (71 U.S., 35 non-U.S.) data systems; 37% of the U.S. data systems were linked to another data source. For outcomes, 112 studies reported childhood weight and 32 adult weight; 152 had physical activity and 148 had dietary measures. For analysis, natural experiments most commonly used cross-sectional comparisons of exposed and unexposed groups (n = 55 [35%]). Most natural experiments had a high risk of bias, and 63% had weak handling of withdrawals and dropouts. Outcomes restricted to obesity measures and health behaviors; inconsistent or unclear descriptions of natural experiment designs; and imperfect methods for assessing risk of bias in natural experiments. Many methodologically diverse natural experiments and experimental studies were identified that reported effects of U.S. and non-U.S. programs, policies, or built environment changes on obesity prevention and control. The findings reinforce the need for methodological and analytic advances that would strengthen evaluations of obesity prevention and control initiatives. National Institutes of Health, Office of Disease Prevention, and Agency for Healthcare Research and Quality. (PROSPERO: CRD42017055750).

Life support technology investment strategies for flight programs: An application of decision analysis

NASA Technical Reports Server (NTRS)

Schlater, Nelson J.; Simonds, Charles H.; Ballin, Mark G.

1993-01-01

Applied research and technology development (R&TD) is often characterized by uncertainty, risk, and significant delays before tangible returns are obtained. Given the increased awareness of limitations in resources, effective R&TD today needs a method for up-front assessment of competing technologies to help guide technology investment decisions. Such an assessment approach must account for uncertainties in system performance parameters, mission requirements and architectures, and internal and external events influencing a development program. The methodology known as decision analysis has the potential to address these issues. It was evaluated by performing a case study assessment of alternative carbon dioxide removal technologies for NASA"s proposed First Lunar Outpost program. An approach was developed that accounts for the uncertainties in each technology's cost and performance parameters as well as programmatic uncertainties such as mission architecture. Life cycle cost savings relative to a baseline, adjusted for the cost of money, was used as a figure of merit to evaluate each of the alternative carbon dioxide removal technology candidates. The methodology was found to provide a consistent decision-making strategy for the develpoment of new life support technology. The case study results provided insight that was not possible from more traditional analysis approaches.

Life support technology investment strategies for flight programs: An application of decision analysis

NASA Technical Reports Server (NTRS)

Schlater, Nelson J.; Simonds, Charles H.; Ballin, Mark G.

1993-01-01

Applied research and technology development (R&TD) is often characterized by uncertainty, risk, and significant delays before tangible returns are obtained. Given the increased awareness of limitations in resources, effective R&TD today needs a method for up-front assessment of competing technologies to help guide technology investment decisions. Such an assessment approach must account for uncertainties in system performance parameters, mission requirements and architectures, and internal and external events influencing a development program. The methodology known as decision analysis has the potential to address these issues. It was evaluated by performing a case study assessment of alternative carbon dioxide removal technologies for NASA's proposed First Lunar Outpost program. An approach was developed that accounts for the uncertainties in each technology's cost and performance parameters as well as programmatic uncertainties such as mission architecture. Life cycle cost savings relative to a baseline, adjusted for the cost of money, was used as a figure of merit to evaluate each of the alternative carbon dioxide removal technology candidates. The methodology was found to provide a consistent decision-making strategy for development of new life support technology. The case study results provided insight that was not possible from more traditional analysis approaches.

A systematic review of health economic evaluations of vaccines in Brazil.

PubMed

Sartori, Ana Marli Christovam; Rozman, Luciana Martins; Decimoni, Tassia Cristina; Leandro, Roseli; Novaes, Hillegonda Maria Dutilh; de Soárez, Patrícia Coelho

2017-06-03

In Brazil, since 2005, the Ministry of Health requires Health Economic Evaluation (HEE) of vaccines for introduction into the National Immunization Program. To describe and analyze the full HEE on vaccines conducted in Brazil from 1980 to 2013. Systematic review of the literature. We searched multiple databases. Two researchers independently selected the studies and extracted the data. The methodological quality of individual studies was evaluated using CHEERS items. Twenty studies were reviewed. The most evaluated vaccines were pneumococcal (25%) and HPV (15%). The most used types of HEE were cost-effectiveness analysis (45%) and cost-utility analysis (20%). The research question and compared strategies were stated in all 20 studies and the target population was clear in 95%. Nevertheless, many studies did not inform the perspective of analysis or data sources. HEE of vaccines in Brazil has increased since 2008. However, the studies still have methodological deficiencies.

A systematic review of health economic evaluations of vaccines in Brazil

PubMed Central

2017-01-01

ABSTRACT Background: In Brazil, since 2005, the Ministry of Health requires Health Economic Evaluation (HEE) of vaccines for introduction into the National Immunization Program. Objectives: To describe and analyze the full HEE on vaccines conducted in Brazil from 1980 to 2013. Methods: Systematic review of the literature. We searched multiple databases. Two researchers independently selected the studies and extracted the data. The methodological quality of individual studies was evaluated using CHEERS items. Results: Twenty studies were reviewed. The most evaluated vaccines were pneumococcal (25%) and HPV (15%). The most used types of HEE were cost-effectiveness analysis (45%) and cost-utility analysis (20%). The research question and compared strategies were stated in all 20 studies and the target population was clear in 95%. Nevertheless, many studies did not inform the perspective of analysis or data sources. Conclusions: HEE of vaccines in Brazil has increased since 2008. However, the studies still have methodological deficiencies. PMID:28129026

Effectiveness of vocational rehabilitation intervention on the return to work and employment of persons with multiple sclerosis.

PubMed

Khan, Fary; Ng, Louisa; Turner-Stokes, Lynne

2009-01-21

Multiple sclerosis is a neurological disease that frequently affects adults of working age, resulting in a range of physical, cognitive and psychosocial deficits that impact on workforce participation. Although, the literature supports vocational rehabilitation (VR) approaches in persons with multiple sclerosis (pwMS), the evidence for its effectiveness is yet to be established. To evaluate the effectiveness of VR programs compared to alternative programs or care as usual on return to work, workability and employment in pwMS; to evaluate the cost effectiveness of these programs. Electronic searches included: Cochrane Central Register of Controlled Trials "CENTRAL" 2008 issue 3, MEDLINE (PubMed) (1966- 2008), EMBASE (1988- 2008), CINAHL (1982- 2008), PEDro (1990- 2008), the Cochrane Rehabilitation and Related Therapies Field trials Register and the National Health Service National Research Register. Randomized and controlled clinical trials, including before - after controlled trials, that compare VR rehabilitation with alternative intervention such as standard or a lesser form of intervention or waitlist controls. Two reviewers selected trials and rated their methodological quality independently. A 'best evidence' synthesis was performed, based on methodological quality. Trials were grouped in terms of type and setting of VR programs. Two trials (one RCT and one CCT) (total 80 participants) met the review criteria. Both trials scored poorly on the methodological quality assessment. There was 'insufficient evidence' for VR programs for (a)'competitive employment', in altering rates of job retention, changes in employment, improvement in rates of re-entry into the labour force; (b) for altering 'work ability' by improving participants' confidence in the accommodation request process, or employability maturity or job seeking activity. No evidence could be assimilated for changes in proportions of persons in supported employment or on disability pensions, nor for cost-effectiveness. There was inconclusive evidence to support VR for pwMS. However, the review highlights some of the challenges in providing VR for pwMS. Clinicians need to be aware of vocational issues, and to understand and manage barriers for maintaining employment. Proactive and timely VR programs should incorporate practical solutions to deal with work disability, workplace accommodation and educate employers, and the wider community. Liaison with policy makers is imperative for government initiatives that encourage work focused VR programs. Future research in VR should focus on improving methodological and scientific rigour of clinical trials; on the development of appropriate and valid outcome measures; and on cost effectiveness of VR programs.

Multi-rater feedback with gap analysis: an innovative means to assess communication skill and self-insight.

PubMed

Calhoun, Aaron W; Rider, Elizabeth A; Peterson, Eleanor; Meyer, Elaine C

2010-09-01

Multi-rater assessment with gap analysis is a powerful method for assessing communication skills and self-insight, and enhancing self-reflection. We demonstrate the use of this methodology. The Program for the Approach to Complex Encounters (PACE) is an interdisciplinary simulation-based communication skills program. Encounters are assessed using an expanded Kalamazoo Consensus Statement Essential Elements Checklist adapted for multi-rater feedback and gap analysis. Data from a representative conversation were analyzed. Likert and forced-choice data with gap analysis are used to assess performance. Participants were strong in Demonstrating Empathy and Providing Closure, and needed to improve Relationship Building, Gathering Information, and understanding the Patient's/Family's Perspective. Participants under-appraised their abilities in Relationship Building, Providing Closure, and Demonstrating Empathy, as well as their overall performance. The conversion of these results into verbal feedback is discussed. We describe an evaluation methodology using multi-rater assessment with gap analysis to assess communication skills and self-insight. This methodology enables faculty to identify undervalued skills and perceptual blind spots, provide comprehensive, data driven, feedback, and encourage reflection. Implementation of graphical feedback forms coupled with one-on-one discussion using the above methodology has the potential to enhance trainee self-awareness and reflection, improving the impact of educational programs. Copyright (c) 2010 Elsevier Ireland Ltd. All rights reserved.

Institutionalizing the evaluation of health programs and policies in France: "cuisine internationale" over fast food and "sur mesure" over ready-made.

PubMed

Hartz, Z M

1999-01-01

The purpose of this article is to describe several chronological milestones in institutionalizing the evaluation of public programs and policies in France from a governmental perspective and in the health sector, situating such references in the international context. The institutional nature of evaluation implies integrating it into an action-oriented model, linking analytical activities to management, thus constituting the formulation of an evaluation policy for policy evaluation. The study focuses on issues related to the structure, practice, and utilization of evaluation results as well as other characteristics providing the French model with a certain resistance to traditional "fast-food" or "ready-made" methodological approaches. The institutionalization of sectorial evaluation appears more promising than that of the government's centralized channel, despite the work developed by a Scientific Evaluation Council, and suggests avenues for reflection and debate pertaining to the Brazilian Unified Health System.

A Patient-Centered Framework for Evaluating Digital Maturity of Health Services: A Systematic Review

PubMed Central

Callahan, Ryan; Darzi, Ara; Mayer, Erik

2016-01-01

Background Digital maturity is the extent to which digital technologies are used as enablers to deliver a high-quality health service. Extensive literature exists about how to assess the components of digital maturity, but it has not been used to design a comprehensive framework for evaluation. Consequently, the measurement systems that do exist are limited to evaluating digital programs within one service or care setting, meaning that digital maturity evaluation is not accounting for the needs of patients across their care pathways. Objective The objective of our study was to identify the best methods and metrics for evaluating digital maturity and to create a novel, evidence-based tool for evaluating digital maturity across patient care pathways. Methods We systematically reviewed the literature to find the best methods and metrics for evaluating digital maturity. We searched the PubMed database for all papers relevant to digital maturity evaluation. Papers were selected if they provided insight into how to appraise digital systems within the health service and if they indicated the factors that constitute or facilitate digital maturity. Papers were analyzed to identify methodology for evaluating digital maturity and indicators of digitally mature systems. We then used the resulting information about methodology to design an evaluation framework. Following that, the indicators of digital maturity were extracted and grouped into increasing levels of maturity and operationalized as metrics within the evaluation framework. Results We identified 28 papers as relevant to evaluating digital maturity, from which we derived 5 themes. The first theme concerned general evaluation methodology for constructing the framework (7 papers). The following 4 themes were the increasing levels of digital maturity: resources and ability (6 papers), usage (7 papers), interoperability (3 papers), and impact (5 papers). The framework includes metrics for each of these levels at each stage of the typical patient care pathway. Conclusions The framework uses a patient-centric model that departs from traditional service-specific measurements and allows for novel insights into how digital programs benefit patients across the health system. Trial Registration N/A PMID:27080852

Predicting the Reliability of Ceramics Under Transient Loads and Temperatures With CARES/Life

NASA Technical Reports Server (NTRS)

Nemeth, Noel N.; Jadaan, Osama M.; Palfi, Tamas; Baker, Eric H.

2003-01-01

A methodology is shown for predicting the time-dependent reliability of ceramic components against catastrophic rupture when subjected to transient thermomechanical loads (including cyclic loads). The methodology takes into account the changes in material response that can occur with temperature or time (i.e., changing fatigue and Weibull parameters with temperature or time). This capability has been added to the NASA CARES/Life (Ceramic Analysis and Reliability Evaluation of Structures/Life) code. The code has been modified to have the ability to interface with commercially available finite element analysis (FEA) codes executed for transient load histories. Examples are provided to demonstrate the features of the methodology as implemented in the CARES/Life program.

The Status of Foreign Language Education: 1984.

ERIC Educational Resources Information Center

Wiley, Patricia Davis

1984-01-01

Presents a brief history of the status of foreign language instruction from 1953 to 1984. Summarizes research on effects of foreign language study. Describes the foreign language education program at the University of Tennessee--Knoxville, providing an overview of the organization, curricular content, and methodology. Evaluates Knoxville's unique…

Quantifying Interprofessional Learning in Health Professional Programs: The University of Manitoba Experience

ERIC Educational Resources Information Center

Grymonpre, Ruby E.; Dean, Heather J.; Wener, Pamela F.; Ready, A. Elizabeth; MacDonald, Laura L.; Holmqvist, Maxine E.; Fricke, Moni W.

2016-01-01

Internationally, a growing number of interprofessional education (IPE) offices are being established within academic institutions. However, few are applying educational improvement methodologies to evaluate and improve the interprofessional (IP) learning opportunities offered. The University of Manitoba IPE Initiative was established in 2008 to…

Intensive Input in Language Acquisition.

ERIC Educational Resources Information Center

Trimino, Andy; Ferguson, Nancy

This paper discusses the role of input as one of the universals in second language acquisition theory. Considerations include how language instructors can best organize and present input and when certain kinds of input are more important. A self-administered program evaluation exercise using relevant theoretical and methodological contributions…

How Many Will Choose? School Choice and Student Enrollment Planning.

ERIC Educational Resources Information Center

Chan, Tak C.

1993-01-01

Enrollment planning is the basis of all school system planning. Focuses on assessing the impact of a choice plan on student enrollment planning. Issues involved include home schooling, school employees' choice, and private kindergarten programs. Administrators are advised to evaluate existing forecasting methodologies. (MLF)

International Field Experiences Promote Professional Development for Sustainability Leaders

ERIC Educational Resources Information Center

Hull, R. Bruce; Kimmel, Courtney; Robertson, David P.; Mortimer, Michael

2016-01-01

Purpose: This paper aims to describe, explain and evaluate a graduate education program that provides international project experiences and builds competencies related to collaborative problem-solving, cultural capacity to work globally and sustainable development. Design/methodology/approach: Qualitative analysis of survey data from 28 students…

DEEP Revisited.

ERIC Educational Resources Information Center

Luker, William A.; And Others

1984-01-01

Based on an independent analysis of the data used to evaluate the Developmental Economic Education Program, questions are raised about the methodology and conclusions reached by Walstad and Soper in an article published in the Winter 1982 issue of the Journal. The original study is also defended in two replies. (Author/RM)

A pharmacy leadership action study.

PubMed

Louie, Clifton; Mertz, Elizabeth; Penfil, Brett; O'Neil, Edward

2009-01-01

To report on the creation of a leadership development program targeted exclusively at pharmacists working in management in the professional community. Large staff-model health maintenance organization (HMO) in California between 2004 and 2008. The Pharmacy Leadership Institute (PLI; a joint effort of the School of Pharmacy and the Center for the Health Professions at the University of California, San Francisco) tested a program in a large staff-model HMO with hundreds of pharmacists in leadership roles. This program included learning seminars, psychometric assessments, leadership goals, intersession activities, coaching/mentoring, and leadership projects. Not applicable. PLI collected survey data in order to evaluate the effectiveness of the institute's leadership development program. In addition, an external evaluator was hired to conduct interviews with the pharmacy directors of the organization which chose to pilot the program. The evaluations from the participants indicate that the leadership development program met many but not all of its initial objectives. Consistent with action research methodology, the faculty of the institute met to redesign some sections of the program in order to meet the established goals. Adjustments were made to different components of the program over the next 4 years. Evaluation data show that these revisions were successful. In addition, follow-up evaluations with participants showed a lasting impact of the program on both individual leadership skills and organizational outcomes. Given the positive outcomes indicated by the evaluation data used in this study, the work of PLI indicates that broader leadership skills can be identified and enhanced within a group of pharmacy managers.

Hydration: certain basic aspects for developing technical and scientific parameters into the nutrition knowledge

PubMed

Perales-García, Aránzazu; Estévez-Martínez, Isabel; Urrialde, Rafael

2016-07-12

Introduction: Hydration is defined as the water intake coming from food and beverages. Its study has become an area by itself, within the nutrition field. Meaning that in 2010 the European Food Safety Authority (EFSA) approved the water intake recommendations, but the study of this topic implies a rigorous methodology, which represents several issues. Objective: Showing as a glance the main methodological issues in hydration studies. Material and methods: Bibliographic revision of scientific literature. Results: The main methodological issues presented are: sample selection (investigation field and sample design), selection of the method to evaluate hydration status (dilution techniques, bioelectrical impedance, plasmatic and urinary indicators, changes in body composition, water losses and clinic symptoms) selection of the method to evaluate water intake (biomarker, questionnaires, informatics programs, smartphone use, 24-h register, dietary history and food frequency questionnaire), and the main sources of hydration. Conclusions: Hydration status should be understood as a routine model, with daily frequency, according to gender, age, physical activity and environmental conditions. Furthermore, the correct design of the methodology has a special importance in order to take into account all the aspects

Evaluation of the AFIT Teleteach Expanded Delivery System (TEDS) Method of Instruction for SYS 223 System Program Management.

DTIC Science & Technology

1981-03-01

tasterls I4fhesis EIEY SYSTEM (TEDS) ZETHOD OF INSTRUC- ~ - ION FOR SYS 22 J3YSTEX PROGRAM MANAGEMEN .PrFOMN ,7WRWUME // John E.Aie Captain, USAF 3...Hypotheses .. .................. 13 II. METHODOLOGY. ................... 14 Sampling Plan. .................. 14 Student and Faculty Populations ...begins with a discussion of the sampling plan and student and faculty populations . This is followed by an explanation of the experimental design and

MIT Laboratory for Computer Science Progress Report 20 - July 1982 - Jun 1983,

DTIC Science & Technology

1984-07-01

system by the Programming Technology Group. Research in the second and largest area entitled Machines, Languages , and Systems, strives to discover and...utilization and cost effectiveness . For example, the Programming Methodology Group and the Real Time Systems Group are developing languages and...100 Megabits per second when implemented with the 1.2[im. n- well cMOS process. 3. LANGUAGES 3.1. Demand Driven Evaluation In his engineer’s thesis

Behavioral patterns of environmental performance evaluation programs.

PubMed

Li, Wanxin; Mauerhofer, Volker

2016-11-01

During the past decades numerous environmental performance evaluation programs have been developed and implemented on different geographic scales. This paper develops a taxonomy of environmental management behavioral patterns in order to provide a practical comparison tool for environmental performance evaluation programs. Ten such programs purposively selected are mapped against the identified four behavioral patterns in the form of diagnosis, negotiation, learning, and socialization and learning. Overall, we found that schemes which serve to diagnose environmental abnormalities are mainly externally imposed and have been developed as a result of technical debates concerning data sources, methodology and ranking criteria. Learning oriented scheme is featured by processes through which free exchange of ideas, mutual and adaptive learning can occur. Scheme developed by higher authority for influencing behaviors of lower levels of government has been adopted by the evaluated to signal their excellent environmental performance. The socializing and learning classified evaluation schemes have incorporated dialogue, participation, and capacity building in program design. In conclusion we consider the 'fitness for purpose' of the various schemes, the merits of our analytical model and the future possibilities of fostering capacity building in the realm of wicked environmental challenges. Copyright © 2016 Elsevier Ltd. All rights reserved.

Assumptions at the philosophical and programmatic levels in evaluation.

PubMed

Mertens, Donna M

2016-12-01

Stakeholders and evaluators hold a variety of levels of assumptions at the philosophical, methodological, and programmatic levels. The use of a transformative philosophical framework is presented as a way for evaluators to become more aware of the implications of various assumptions made by themselves and program stakeholders. The argument is examined and demonstrated that evaluators who are aware of the assumptions that underlie their evaluation choices are able to provide useful support for stakeholders in the examination of the assumptions they hold with regard to the nature of the problem being addressed, the program designed to solve the problem, and the approach to evaluation that is appropriate in that context. Such an informed approach has the potential for development of more appropriate and culturally responsive programs being implemented in ways that lead to the desired impacts, as well as to lead to evaluation approaches that support effective solutions to intransigent social problems. These arguments are illustrated through examples of evaluations from multiple sectors; additional challenges are also identified. Copyright © 2016 Elsevier Ltd. All rights reserved.

Evaluation in health: participatory methodology and involvement of municipal managers

PubMed Central

de Almeida, Cristiane Andrea Locatelli; Tanaka, Oswaldo Yoshimi

2016-01-01

ABSTRACT OBJECTIVE To analyze scopes and limits of the use of participatory methodology of evaluation with municipal health managers and administrators. METHODS Qualitative research with health policymakers and managers of the Comissão Intergestores Regional (CIR – Regional Interagency Commission) of a health region of the state of Sao Paulo in Brazil. Representatives from seven member cities participated in seven workshops facilitated by the researchers, with the aim of assessing a specific problem of the care line, which would be used as a tracer of the system integrality. The analysis of the collected empirical material was based on the hermeneutic-dialectic methodology and aimed at the evaluation of the applied participatory methodology, according to its capacity of promoting a process of assessment capable to be used as a support for municipal management. RESULTS With the participatory approach of evaluation, we were able to promote in-depth discussions with the group, especially related to the construction of integral care and to the inclusion of the user’s perspective in decision-making, linked to the search for solution to concrete problems of managers. By joint exploration, the possibility of using data from electronic information systems was opened, as well as information coming directly from the users of the services, to enhance discussions and negotiations between partners. The participants were disbelievers of the replication potential of this type of evaluation without the direct monitoring of the academy, given the difficulty of organizing the process in everyday life, already taken by emergency and political issues. CONCLUSIONS Evaluations of programs and services carried out within the Regional Interagency Commission, starting from the local interest and facilitating the involvement of its members by the use of participatory methodologies, can contribute to the construction of integral care. To the extent that the act of evaluating stay invested with greater significance to the local actors, its involvement with the evaluations at the federal level can also be stimulated. PMID:27509011

Mexico City Air Quality Research Initiative; Volume 5, Strategic evaluation

DOE Office of Scientific and Technical Information (OSTI.GOV)

NONE

1994-03-01

Members of the Task HI (Strategic Evaluation) team were responsible for the development of a methodology to evaluate policies designed to alleviate air pollution in Mexico City. This methodology utilizes information from various reports that examined ways to reduce pollutant emissions, results from models that calculate the improvement in air quality due to a reduction in pollutant emissions, and the opinions of experts as to the requirements and trade-offs that are involved in developing a program to address the air pollution problem in Mexico City. The methodology combines these data to produce comparisons between different approaches to improving Mexico City`smore » air quality. These comparisons take into account not only objective factors such as the air quality improvement or cost of the different approaches, but also subjective factors such as public acceptance or political attractiveness of the different approaches. The end result of the process is a ranking of the different approaches and, more importantly, the process provides insights into the implications of implementing a particular approach or policy.« less

Presentations at the Tri-Service Cloud Modeling Workshop (2nd), Held at the Naval Surface Weapons Center, White Oak, Maryland, 26-27 June 1984. Volume 1.

DTIC Science & Technology

1984-08-01

produce even the most basic binary cloud data and methodologies needed to support the evaluation programs." In view of this recognized deficiency, the...There was an exchange of information with non - DoD agencies, with presentations made by NASA and NOAA (see pp. 537, 569). A brief report by the steering...on cloud data bases and methodologies for users. To achieve these actions requires explicit support. *See classified supplementary volume. vi CONTENTS

Findings From a Prospective Randomized Controlled Trial of an Individualized Music Listening Program for Persons With Dementia.

PubMed

Kwak, Jung; Anderson, Keith; O'Connell Valuch, Katharine

2018-06-01

Music & Memory (M&M) is a passive music intervention that uses personalized music playlists delivered on digital music players. This program has been increasingly adopted in nursing homes across the United States to facilitate communication, engagement, and socialization among persons with dementia (PWDs); however, few studies have evaluated the program's effect on PWDs' outcomes. In the present study, a randomized controlled crossover design was used to examine the impact of the M&M program on 59 PWDs in 10 nursing homes over a 14-week period. Residents' evaluated outcomes included agitation, behavioral symptoms, and use of psychotropic medications. Although trends supported the positive effects of M&M, no statistically significant differences were found in any of the outcomes measured over time. Methodological limitations withstanding, these findings call into question the effectiveness of the M&M program and the ability of facility staff to implement this intervention with fidelity.

Premium increases in state health insurance programs: lessons from a case study of the Massachusetts Medicaid buy-in program.

PubMed

Livermore, Gina A; Goodman, Nanette; Hooven, Fred; Hashemi, Lobat

In March 2003, Massachusetts increased the premiums it charges to most enrollees in its CommonHealth-Working (CH-W) program. This study evaluates the impact of the premium change on disenrollment using a comparison group methodology. The findings indicate that the premium change had only a small, but statistically significant impact on program exits. The CH-W experience differs from other state programs that saw substantial enrollment declines in response to new or increased premiums. This is likely due to factors that make CH-W different from other programs, key of which are administrative procedures intended to minimize disenrollment due to premium nonpayment.

[An educational software development proposal for nursing in neonatal cardiopulmonary resuscitation].

PubMed

Rodrigues, Rita de Cassia Vieira; Peres, Heloisa Helena Ciqueto

2013-02-01

The objective of this study was to develop an educational software program for nursing continuing education. This program was intended to incorporate applied methodological research that used the learning management system methodology created by Galvis Panqueva in association with contextualized instructional design for software design. As a result of this study, we created a computerized educational product (CEP) called ENFNET. This study describes all the necessary steps taken during its development. The creation of a CEP demands a great deal of study, dedication and investment as well as the necessity of specialized technical personnel to construct it. At the end of the study, the software was positively evaluated and shown to be a useful strategy to help users in their education, skills development and professional training.

Measuring participant rurality in Web-based interventions.

PubMed

Danaher, Brian G; Hart, L Gary; McKay, H Garth; Severson, Herbert H

2007-08-31

Web-based health behavior change programs can reach large groups of disparate participants and thus they provide promise of becoming important public health tools. Data on participant rurality can complement other demographic measures to deepen our understanding of the success of these programs. Specifically, analysis of participant rurality can inform recruitment and social marketing efforts, and facilitate the targeting and tailoring of program content. Rurality analysis can also help evaluate the effectiveness of interventions across population groupings. We describe how the RUCAs (Rural-Urban Commuting Area Codes) methodology can be used to examine results from two Randomized Controlled Trials of Web-based tobacco cessation programs: the ChewFree.com project for smokeless tobacco cessation and the Smokers' Health Improvement Program (SHIP) project for smoking cessation. Using RUCAs methodology helped to highlight the extent to which both Web-based interventions reached a substantial percentage of rural participants. The ChewFree program was found to have more rural participation which is consistent with the greater prevalence of smokeless tobacco use in rural settings as well as ChewFree's multifaceted recruitment program that specifically targeted rural settings. Researchers of Web-based health behavior change programs targeted to the US should routinely include RUCAs as a part of analyzing participant demographics. Researchers in other countries should examine rurality indices germane to their country.

An Application of Mathematical Programming to Assess Managerial Efficiency in the Houston Independent School District

DTIC Science & Technology

1981-11-01

documenting attempts to define production functions in education. Levin 2 and Hanushek characterize current methodologies as being deficient both concept- ually...Evaluation and Policy Analysis, 2, 1, (1980), 27-36. 16. Hanushek , E. A., "Conceptual and Empirical Issues in Lstimation of Educational Production

Structural Ceramics

NASA Technical Reports Server (NTRS)

1986-01-01

This publication is a compilation of abstracts and slides of papers presented at the NASA Lewis Structural Ceramics Workshop. Collectively, these papers depict the scope of NASA Lewis' structural ceramics program. The technical areas include monolithic SiC and Si3N4 development, ceramic matrix composites, tribology, design methodology, nondestructive evaluation (NDE), fracture mechanics, and corrosion.

Study program for encapsulation materials interface for low-cost solar array

NASA Technical Reports Server (NTRS)

Kaelble, D. H.; Mansfeld, F. B.; Kendig, M.; Leung, C.

1981-01-01

The service integrity of the bonded interface in solar cell modules used in solar arrays is addressed. The development of AC impedance as a nondestructive evaluation (NDE) methodology for solar arrays is reported along with development of corrosion models and materials selection criteria for corrosion resistant interfaces.

Integrating Sustainability across the University Curriculum

ERIC Educational Resources Information Center

Dmochowski, Jane Ellen; Garofalo, Dan; Fisher, Sarah; Greene, Ann; Gambogi, Danielle

2016-01-01

Purpose: Colleges and universities increasingly have the mandate and motivation to integrate sustainability into their curricula. The purpose of this paper is to share the strategy used at the University of Pennsylvania (Penn) and provide an evaluation of its success and guidance to others creating similar programs. Design/methodology/approach:…

Educational Administration and Management. Reports Studies C.95.

ERIC Educational Resources Information Center

Turner, Trevor; Persaud, Ganga

This case study reports results of research on the extent of nonformal education in Guyana; comparison of commitment in private and public enterprises; and financial support, program organization and management, learners' background and objectives, content and methodology, and evaluation strategy. The first two chapters describe the study's…

Evaluating Three Allied Health Training Programs: Exploratory Research into Curriculum Relevance and Labor Market Conditions

ERIC Educational Resources Information Center

Appel, Gary L.; And Others

1977-01-01

Presents results of a research effort designed to develop a methodology for assisting dietetic technicians, physical therapist assistants, and medical record technicians curriculum relevance, and to better understand labor market conditions affecting the utilization and distribution of midlevel technicians. (Author/LAS)

A Cost-Benefit Methodology for Summative Evaluation.

ERIC Educational Resources Information Center

Churchman, David

Although there is high interest in determining whether or not an educational program provides good value for its cost, it is difficult to make this determination, since people are not generally conceptualized as products and since educational benefits are not easily translated into financial terms. Economic principles suggest that the cost of…

Heritage Education and Heritagization Processes: SHEO Methodology for Educational Programs Evaluation

ERIC Educational Resources Information Center

Fontal, Olaia; Gómez-Redondo, Carmen

2016-01-01

The cultural approach that is developed at the beginning of this article about "heritagization" processes, allows profiling three different agencies involved in these processes, therefore, an institutional agency, an individual agency, and a community agency are drawn; the last three agencies also determine the characteristics of their…

Methodologies For Evaluating The Impact On Safety Of Intelligent Vehicle Highway Systems

DOT National Transportation Integrated Search

1994-01-01

THE IVHS PROGRAM INCLUDES A WIDE VARIETY OF TYPES OF SYSTEMS ALL OF WHICH ARE INTENDED TO IMPROVE SOME ASPECT OF HIGHWAY TRAVEL. AS THESE SYSTEMS EVOLVE, IT IS IMPORTANT TO KNOW IF THEY ARE ACCOMPLISHING THEIR DESIGN GOALS; AND IT IS ALSO IMPORTANT T...

Designing a rapid response program to support evidence-informed decision-making in the Americas region: using the best available evidence and case studies.

PubMed

Haby, Michelle M; Chapman, Evelina; Clark, Rachel; Barreto, Jorge; Reveiz, Ludovic; Lavis, John N

2016-08-18

The objective of this work was to inform the design of a rapid response program to support evidence-informed decision-making in health policy and practice for the Americas region. Specifically, we focus on the following: (1) What are the best methodological approaches for rapid reviews of the research evidence? (2) What other strategies are needed to facilitate evidence-informed decision-making in health policy and practice? and (3) How best to operationalize a rapid response program? The evidence used to inform the design of a rapid response program included (i) two rapid reviews of methodological approaches for rapid reviews of the research evidence and strategies to facilitate evidence-informed decision-making, (ii) supplementary literature in relation to the "shortcuts" that could be considered to reduce the time needed to complete rapid reviews, (iii) four case studies, and (iv) supplementary literature to identify additional operational issues for the design of the program. There is no agreed definition of rapid reviews in the literature and no agreed methodology for conducting them. Better reporting of rapid review methods is needed. The literature found in relation to shortcuts will be helpful in choosing shortcuts that maximize timeliness while minimizing the impact on quality. Evidence for other strategies that can be used concurrently to facilitate the uptake of research evidence, including evidence drawn from rapid reviews, is presented. Operational issues that need to be considered in designing a rapid response program include the implications of a "user-pays" model, the importance of recruiting staff with the right mix of skills and qualifications, and ensuring that the impact of the model on research use in decision-making is formally evaluated. When designing a new rapid response program, greater attention needs to be given to specifying the rapid review methods and reporting these in sufficient detail to allow a quality assessment. It will also be important to engage in other strategies to facilitate the uptake of the rapid reviews and to evaluate the chosen model in order to make refinements and add to the evidence base for evidence-informed decision-making.

Investigating the Gap Between Estimated and Actual Energy Efficiency and Conservation Savings for Public Buildings Projects & Programs in United States

NASA Astrophysics Data System (ADS)

Qaddus, Muhammad Kamil

The gap between estimated and actual savings in energy efficiency and conservation (EE&C) projects or programs forms the problem statement for the scope of public and government buildings. This gap has been analyzed first on impact and then on process-level. On the impact-level, the methodology leads to categorization of the gap as 'Realization Gap'. It then views the categorization of gap within the context of past and current narratives linked to realization gap. On process-level, the methodology leads to further analysis of realization gap on process evaluation basis. The process evaluation criterion, a product of this basis is then applied to two different programs (DESEU and NYC ACE) linked to the scope of this thesis. Utilizing the synergies of impact and process level analysis, it offers proposals on program development and its structure using our process evaluation criterion. Innovative financing and benefits distribution structure is thus developed and will remain part of the proposal. Restricted Stakeholder Crowd Financing and Risk-Free Incentivized return are the products of proposed financing and benefit distribution structure respectively. These products are then complimented by proposing an alternative approach in estimating EE&C savings. The approach advocates estimation based on range-allocation rather than currently utilized unique estimated savings approach. The Way Ahead section thus explores synergy between financial and engineering ranges of energy savings as a multi-discipline approach for future research. Moreover, it provides the proposed program structure with risk aversion and incentive allocation while dealing with uncertainty. This set of new approaches are believed to better fill the realization gap between estimated and actual energy efficiency savings.

Leveraging Administrative Data for Program Evaluations: A Method for Linking Data Sets Without Unique Identifiers.

PubMed

Lorden, Andrea L; Radcliff, Tiffany A; Jiang, Luohua; Horel, Scott A; Smith, Matthew L; Lorig, Kate; Howell, Benjamin L; Whitelaw, Nancy; Ory, Marcia

2016-06-01

In community-based wellness programs, Social Security Numbers (SSNs) are rarely collected to encourage participation and protect participant privacy. One measure of program effectiveness includes changes in health care utilization. For the 65 and over population, health care utilization is captured in Medicare administrative claims data. Therefore, methods as described in this article for linking participant information to administrative data are useful for program evaluations where unique identifiers such as SSN are not available. Following fuzzy matching methodologies, participant information from the National Study of the Chronic Disease Self-Management Program was linked to Medicare administrative data. Linking variables included participant name, date of birth, gender, address, and ZIP code. Seventy-eight percent of participants were linked to their Medicare claims data. Linking program participant information to Medicare administrative data where unique identifiers are not available provides researchers with the ability to leverage claims data to better understand program effects. © The Author(s) 2014.

New model framework and structure and the commonality evaluation model. [concerning unmanned spacecraft projects

NASA Technical Reports Server (NTRS)

1977-01-01

The development of a framework and structure for shuttle era unmanned spacecraft projects and the development of a commonality evaluation model is documented. The methodology developed for model utilization in performing cost trades and comparative evaluations for commonality studies is discussed. The model framework consists of categories of activities associated with the spacecraft system's development process. The model structure describes the physical elements to be treated as separate identifiable entities. Cost estimating relationships for subsystem and program-level components were calculated.

An evaluation of an urban renewal program and its effects on neighborhood resident's overall wellbeing using concept mapping.

PubMed

Mehdipanah, Roshanak; Malmusi, Davide; Muntaner, Carles; Borrell, Carme

2013-09-01

Urban renewal programs aim to improve physical and socioeconomic position of neighborhoods. However, due to the intervention's complexity, there is often little evidence of their impact on health and health inequalities. This study aimed to identify the perception of a group of neighborhood residents towards a large-scale urban renewal program in Barcelona and to explore its effects and importance on their wellbeing using concept mapping methodology. Our results indicate that the majority of urban renewal projects within the initiative, including improved walkability, construction of new public spaces and more community programs, have positive and important effects on the overall wellbeing of participants. This study presents an innovative method that diverts from traditional outcome-based evaluations studies often used within this field. Copyright © 2013 Elsevier Ltd. All rights reserved.

US Topo Maps 2014: Program updates and research

USGS Publications Warehouse

Fishburn, Kristin A.

2014-01-01

The U. S. Geological Survey (USGS) US Topo map program is now in year two of its second three-year update cycle. Since the program was launched in 2009, the product and the production system tools and processes have undergone enhancements that have made the US Topo maps a popular success story. Research and development continues with structural and content product enhancements, streamlined and more fully automated workflows, and the evaluation of a GIS-friendly US Topo GIS Packet. In addition, change detection methodologies are under evaluation to further streamline product maintenance and minimize resource expenditures for production in the future. The US Topo map program will continue to evolve in the years to come, providing traditional map users and Geographic Information System (GIS) analysts alike with a convenient, freely available product incorporating nationally consistent data that are quality assured to high standards.

Evaluating the effects of the Dietary Guidelines for Americans on consumer behavior and health: methodological challenges.

PubMed

Guthrie, Joanne F; Smallwood, David M

2003-12-01

The Dietary Guidelines for Americans is the official nutrition policy statement for the United States. Government involvement in providing information on private behavior, such as food choice, is justified by the high cost of poor diets, as measured in medical expenses and lost productivity. The Guidelines are intended to provide an up-to-date, consistent information base for federal nutrition education and information efforts and food assistance program regulations. Through these policy mechanisms, the Guidelines are assumed to improve dietary behavior, and, ultimately, health. By law, the Dietary Guidelines for Americans must be updated every five years; however, there is no mandate for evaluation. Evaluation could provide useful information to assess the extent to which the Guidelines positively influence health and provide insights into reasons for their successes and limitations. However, evaluation would also present considerable challenges. This paper discusses the critical data and methodological needs for improving evaluation of the Dietary Guidelines for Americans.

The promise and challenge of practice-research collaborations: Guiding principles and strategies for initiating, designing, and implementing program evaluation research.

PubMed

Secret, Mary; Abell, Melissa L; Berlin, Trey

2011-01-01

The authors present a set of guiding principles and strategies to facilitate the collaborative efforts of social work researchers and practitioners as they initiate, design, and implement outcome evaluations of human service interventions and programs. Beginning with an exploration of the interpersonal barriers to practice-research collaborations, and building on their experiences in successfully completing a community-based research evaluation, the authors identify specific relationship-focused principles and strategies and illustrate how these approaches can guide practice-research teams through the various sequential activities of the evaluation research process. In particular, it is suggested that practice-research collaborations can be formed, strengthened, and sustained by emphasis on a spirit of discovery and shared leadership at the start of the relationship, use of a comprehensive evaluation model to clarify and frame the evaluation and program goals, beginning where the client is when selecting research methodology and measurement tools, commitment to keeping the program first and recording everything during the implementation and data-collection stages, discussion of emerging findings and presentation of findings in graphic format at the data-analysis stage, and a total team approach at the dissemination stage.

Program logic: a framework for health program design and evaluation - the Pap nurse in general practice program.

PubMed

Hallinan, Christine M

2010-01-01

In this paper, program logic will be used to 'map out' the planning, development and evaluation of the general practice Pap nurse program in the Australian general practice arena. The incorporation of program logic into the evaluative process supports a greater appreciation of the theoretical assumptions and external influences that underpin general practice Pap nurse activity. The creation of a program logic model is a conscious strategy that results an explicit understanding of the challenges ahead, the resources available and time frames for outcomes. Program logic also enables a recognition that all players in the general practice arena need to be acknowledged by policy makers, bureaucrats and program designers when addressing through policy, issues relating to equity and accessibility of health initiatives. Logic modelling allows decision makers to consider the complexities of causal associations when developing health care proposals and programs. It enables the Pap nurse in general practice program to be represented diagrammatically by linking outcomes (short, medium and long term) with both the program activities and program assumptions. The research methodology used in the evaluation of the Pap nurse in general practice program includes a descriptive study design and the incorporation of program logic, with a retrospective analysis of Australian data from 2001 to 2009. For the purposes of gaining both empirical and contextual data for this paper, a data set analysis and literature review was performed. The application of program logic as an evaluative tool for analysis of the Pap PN incentive program facilitates a greater understanding of complex general practice activity triggers, and also allows this greater understanding to be incorporated into policy to facilitate Pap PN activity, increase general practice cervical smear and ultimately decrease burden of disease.

A systematic review of suicide prevention interventions targeting indigenous peoples in Australia, United States, Canada and New Zealand

PubMed Central

2013-01-01

Background Indigenous peoples of Australia, Canada, United States and New Zealand experience disproportionately high rates of suicide. As such, the methodological quality of evaluations of suicide prevention interventions targeting these Indigenous populations should be rigorously examined, in order to determine the extent to which they are effective for reducing rates of Indigenous suicide and suicidal behaviours. This systematic review aims to: 1) identify published evaluations of suicide prevention interventions targeting Indigenous peoples in Australia, Canada, United States and New Zealand; 2) critique their methodological quality; and 3) describe their main characteristics. Methods A systematic search of 17 electronic databases and 13 websites for the period 1981–2012 (inclusive) was undertaken. The reference lists of reviews of suicide prevention interventions were hand-searched for additional relevant studies not identified by the electronic and web search. The methodological quality of evaluations of suicide prevention interventions was assessed using a standardised assessment tool. Results Nine evaluations of suicide prevention interventions were identified: five targeting Native Americans; three targeting Aboriginal Australians; and one First Nation Canadians. The main intervention strategies employed included: Community Prevention, Gatekeeper Training, and Education. Only three of the nine evaluations measured changes in rates of suicide or suicidal behaviour, all of which reported significant improvements. The methodological quality of evaluations was variable. Particular problems included weak study designs, reliance on self-report measures, highly variable consent and follow-up rates, and the absence of economic or cost analyses. Conclusions There is an urgent need for an increase in the number of evaluations of preventive interventions targeting reductions in Indigenous suicide using methodologically rigorous study designs across geographically and culturally diverse Indigenous populations. Combining and tailoring best evidence and culturally-specific individual strategies into one coherent suicide prevention program for delivery to whole Indigenous communities and/or population groups at high risk of suicide offers considerable promise. PMID:23663493

A systematic review of suicide prevention interventions targeting indigenous peoples in Australia, United States, Canada and New Zealand.

PubMed

Clifford, Anton C; Doran, Christopher M; Tsey, Komla

2013-05-13

Indigenous peoples of Australia, Canada, United States and New Zealand experience disproportionately high rates of suicide. As such, the methodological quality of evaluations of suicide prevention interventions targeting these Indigenous populations should be rigorously examined, in order to determine the extent to which they are effective for reducing rates of Indigenous suicide and suicidal behaviours. This systematic review aims to: 1) identify published evaluations of suicide prevention interventions targeting Indigenous peoples in Australia, Canada, United States and New Zealand; 2) critique their methodological quality; and 3) describe their main characteristics. A systematic search of 17 electronic databases and 13 websites for the period 1981-2012 (inclusive) was undertaken. The reference lists of reviews of suicide prevention interventions were hand-searched for additional relevant studies not identified by the electronic and web search. The methodological quality of evaluations of suicide prevention interventions was assessed using a standardised assessment tool. Nine evaluations of suicide prevention interventions were identified: five targeting Native Americans; three targeting Aboriginal Australians; and one First Nation Canadians. The main intervention strategies employed included: Community Prevention, Gatekeeper Training, and Education. Only three of the nine evaluations measured changes in rates of suicide or suicidal behaviour, all of which reported significant improvements. The methodological quality of evaluations was variable. Particular problems included weak study designs, reliance on self-report measures, highly variable consent and follow-up rates, and the absence of economic or cost analyses. There is an urgent need for an increase in the number of evaluations of preventive interventions targeting reductions in Indigenous suicide using methodologically rigorous study designs across geographically and culturally diverse Indigenous populations. Combining and tailoring best evidence and culturally-specific individual strategies into one coherent suicide prevention program for delivery to whole Indigenous communities and/or population groups at high risk of suicide offers considerable promise.

Chile: Acceptability of a Training Program for Depression Management in Primary Care

PubMed Central

Marín, Rigoberto; Martínez, Pablo; Cornejo, Juan P.; Díaz, Berta; Peralta, José; Tala, Álvaro; Rojas, Graciela

2016-01-01

Background: In Chile, there are inconsistencies in the management of depression in primary care settings, and the National Depression Program, currently in effect, was implemented without a standardized training program. The objective of this study is to evaluate the acceptability of a training program on the management of depression for primary care health teams. Methods: The study was a randomized controlled trial, and two primary centers from the Metropolitan Region of Santiago were randomly selected to carry out the intervention training program. Pre-post surveys were applied, to evaluate expectations and satisfaction with the intervention, respectively. Descriptive and content analysis was carried out. Result: The sample consisted of 41 health professionals, 56.1% of who reported that their expectations for the intervention were met. All of the training activities were evaluated with scores higher than 6.4 (on a 1–7 scale). The trainers, the methodology, and the learning environment were considered strengths and facilitators of the program, while the limited duration of the training, the logistical problems faced during part of the program, and the lack of educational material were viewed as weaknesses. Conclusion: The intervention was well accepted by primary health care teams. However, the clinical impact in patients still has to be evaluated. PMID:27375531

The Amphibian Research and Monitoring Initiative (ARMI): 5-year report

USGS Publications Warehouse

Muths, Erin; Gallant, Alisa L.; Campbell Grant, Evan H.; Battaglin, William A.; Green, David E.; Staiger, Jennifer S.; Walls, Susan C.; Gunzburger, Margaret S.; Kearney, Rick F.

2006-01-01

This report is a 5-year retrospective of the structure, methodology, progress, and contributions to the broader scientific community that have resulted from this national USGS program. We evaluate ARMI’s success to date, with regard to the challenges faced by the program and the strengths that have emerged. We chart objectives for the next 5 years that build on current accomplishments, highlight areas meriting further research, and direct efforts to overcome existing weaknesses.

Experience factors in performing periodic physical evaluations

NASA Technical Reports Server (NTRS)

Hoffman, A. A.

1969-01-01

The lack of scientific basis in the so-called periodic health examinations on military personnel inclusive of the Executive Health Program is outlined. This latter program can well represent a management tool of the company involved in addition to being a status symbol. A multiphasic screening technique is proposed in conjunction with an automated medical history questionnaire for preventive occupational medicine methodology. The need to collate early sickness consultation or clinic visit histories with screening techniques is emphasized.

Using systems science for population health management in primary care.

PubMed

Li, Yan; Kong, Nan; Lawley, Mark A; Pagán, José A

2014-10-01

Population health management is becoming increasingly important to organizations managing and providing primary care services given ongoing changes in health care delivery and payment systems. The objective of this study is to show how systems science methodologies could be incorporated into population health management to compare different interventions and improve health outcomes. The New York Academy of Medicine Cardiovascular Health Simulation model (an agent-based model) and data from the Behavioral Risk Factor Surveillance System were used to evaluate a lifestyle program that could be implemented in primary care practice settings. The program targeted Medicare-age adults and focused on improving diet and exercise and reducing weight. The simulation results suggest that there would be significant reductions projected in the proportion of the Medicare-age population with diabetes after the implementation of the proposed lifestyle program for a relatively long term (3 and 5 years). Similar results were found for the subpopulations with high cholesterol, but the proposed intervention would not have a significant effect in the proportion of the population with hypertension over a time period of <5 years. Systems science methodologies can be useful to compare the health outcomes of different interventions. These tools can become an important component of population health management because they can help managers and other decision makers evaluate alternative programs in primary care settings. © The Author(s) 2014.

Interviewer as instrument: accounting for human factors in evaluation research.

PubMed

Brown, Joel H

2006-04-01

This methodological study examines an original data collection model designed to incorporate human factors and enhance data richness in qualitative and evaluation research. Evidence supporting this model is drawn from in-depth youth and adult interviews in one of the largest policy/program evaluations undertaken in the United States, the Drug, Alcohol, and Tobacco Education evaluation (77 districts, 118 schools). When applying the explicit observation technique (EOT)--the strategic and nonjudgmental disclosure of nonverbal human factor cues by the interviewer to the respondent during interview--data revealed the observation disclosure pattern. Here, respondents linked perceptions with policy or program implementation or effectiveness evidence. Although more research is needed, it is concluded that the EOT yields richer data when compared with traditional semistructured interviews and, thus, holds promise to enhance qualitative and evaluation research methods. Validity and reliability as well as qualitative and evaluation research considerations are discussed.

A methodology for evaluation of an interactive multispectral image processing system

NASA Technical Reports Server (NTRS)

Kovalick, William M.; Newcomer, Jeffrey A.; Wharton, Stephen W.

1987-01-01

Because of the considerable cost of an interactive multispectral image processing system, an evaluation of a prospective system should be performed to ascertain if it will be acceptable to the anticipated users. Evaluation of a developmental system indicated that the important system elements include documentation, user friendliness, image processing capabilities, and system services. The criteria and evaluation procedures for these elements are described herein. The following factors contributed to the success of the evaluation of the developmental system: (1) careful review of documentation prior to program development, (2) construction and testing of macromodules representing typical processing scenarios, (3) availability of other image processing systems for referral and verification, and (4) use of testing personnel with an applications perspective and experience with other systems. This evaluation was done in addition to and independently of program testing by the software developers of the system.

An application of multiattribute decision analysis to the Space Station Freedom program. Case study: Automation and robotics technology evaluation

NASA Technical Reports Server (NTRS)

Smith, Jeffrey H.; Levin, Richard R.; Carpenter, Elisabeth J.

1990-01-01

The results are described of an application of multiattribute analysis to the evaluation of high leverage prototyping technologies in the automation and robotics (A and R) areas that might contribute to the Space Station (SS) Freedom baseline design. An implication is that high leverage prototyping is beneficial to the SS Freedom Program as a means for transferring technology from the advanced development program to the baseline program. The process also highlights the tradeoffs to be made between subsidizing high value, low risk technology development versus high value, high risk technology developments. Twenty one A and R Technology tasks spanning a diverse array of technical concepts were evaluated using multiattribute decision analysis. Because of large uncertainties associated with characterizing the technologies, the methodology was modified to incorporate uncertainty. Eight attributes affected the rankings: initial cost, operation cost, crew productivity, safety, resource requirements, growth potential, and spinoff potential. The four attributes of initial cost, operations cost, crew productivity, and safety affected the rankings the most.

Development and preliminary evaluation of a behavioural HIV prevention program for teenage girls of Latino descent in the USA

PubMed Central

Davidson, Tatiana M.; Lopez, Cristina M.; Saulson, Raelle; Borkman, April L.; Soltis, Kathryn; Ruggiero, Kenneth J.; de Arellano, Michael; Wingood, Gina M.; DiClemente, Ralph J.; Danielson, Carla Kmett

2014-01-01

National data suggests that teenage girls of Latino descent in the USA are disproportionately affected by HIV with the rate of new infections being approximately 4 times higher compared to White women of comparable age (Centers for Disease Control and Prevention 2013). This paper highlights the need for an effective single-sex HIV prevention program for teenage girls of Latino descent and describes the development and preliminary evaluation of Chicas Healing, Informing, Living and Empowering (CHILE), a culturally-tailored, HIV prevention programme exclusively for teenage girls of Latino descent that was adapted from Sisters Informing, Healing, Living, and Empowering (SiHLE), an evidence-based HIV prevention program that is culturally tailored for African American young women. Theatre testing, a pre-testing methodology to assess consumer response to a demonstration of a product, was utilised to evaluate the relevance and utility of the HIV program as well as opportunities for the integration of cultural constructs. Future directions for the evaluation of CHILE are discussed. PMID:24697607

[Evaluation of the first training on clinical research methodology in Chile].

PubMed

Espinoza, Manuel; Cabieses, Báltica; Pedreros, César; Zitko, Pedro

2011-03-01

This paper describes the evaluation of the first training on clinical research methodology in Chile (EMIC-Chile) 12 months after its completion. An online survey was conducted for students and the Delphi method was used for the teaching team. Among the students, the majority reported that the program had contributed to their professional development and that they had shared some of the knowledge acquired with colleagues in their workplace. Forty-one percent submitted a project to obtain research funding through a competitive grants process once they had completed the course. Among the teachers, the areas of greatest interest were the communication strategy, teaching methods, the characteristics of the teaching team, and potential strategies for making the EMIC-Chile permanent in the future. This experience could contribute to future research training initiatives for health professionals. Recognized challenges are the involvement of nonmedical professions in clinical research, the complexities associated with the distance learning methodology, and the continued presence of initiatives of this importance at the national and regional level.

Development and application of course-embedded assessment system for program outcome evaluation in the Korean nursing education: A pilot study.

PubMed

Park, Jee Won; Seo, Eun Ji; You, Mi-Ae; Song, Ju-Eun

2016-03-01

Program outcome evaluation is important because it is an indicator for good quality of education. Course-embedded assessment is one of the program outcome evaluation methods. However, it is rarely used in Korean nursing education. The study purpose was to develop and apply preliminarily a course-embedded assessment system to evaluate one program outcome and to share our experiences. This was a methodological study to develop and apply the course-embedded assessment system based on the theoretical framework in one nursing program in South Korea. Scores for 77 students generated from the three practicum courses were used. The course-embedded assessment system was developed following the six steps suggested by Han's model as follows. 1) One program outcome in the undergraduate program, "nursing process application ability", was selected and 2) the three clinical practicum courses related to the selected program outcome were identified. 3) Evaluation tools including rubric and items were selected for outcome measurement and 4) performance criterion, the educational goal level for the program, was established. 5) Program outcome was actually evaluated using the rubric and evaluation items in the three practicum courses and 6) the obtained scores were analyzed to identify the achievement rate, which was compared with the performance criterion. Achievement rates for the selected program outcome in adult, maternity, and pediatric nursing practicum were 98.7%, 100%, and 66.2% in the case report and 100% for all three in the clinical practice, and 100%, 100%, and 87% respectively for the conference. These are considered as satisfactory levels when compared with the performance criterion of "at least 60% or more". Course-embedded assessment can be used as an effective and economic method to evaluate the program outcome without running an integrative course additionally. Further studies to develop course-embedded assessment systems for other program outcomes in nursing education are needed. Copyright © 2016 Elsevier Ltd. All rights reserved.

Industrial methodology for process verification in research (IMPROVER): toward systems biology verification

PubMed Central

Meyer, Pablo; Hoeng, Julia; Rice, J. Jeremy; Norel, Raquel; Sprengel, Jörg; Stolle, Katrin; Bonk, Thomas; Corthesy, Stephanie; Royyuru, Ajay; Peitsch, Manuel C.; Stolovitzky, Gustavo

2012-01-01

Motivation: Analyses and algorithmic predictions based on high-throughput data are essential for the success of systems biology in academic and industrial settings. Organizations, such as companies and academic consortia, conduct large multi-year scientific studies that entail the collection and analysis of thousands of individual experiments, often over many physical sites and with internal and outsourced components. To extract maximum value, the interested parties need to verify the accuracy and reproducibility of data and methods before the initiation of such large multi-year studies. However, systematic and well-established verification procedures do not exist for automated collection and analysis workflows in systems biology which could lead to inaccurate conclusions. Results: We present here, a review of the current state of systems biology verification and a detailed methodology to address its shortcomings. This methodology named ‘Industrial Methodology for Process Verification in Research’ or IMPROVER, consists on evaluating a research program by dividing a workflow into smaller building blocks that are individually verified. The verification of each building block can be done internally by members of the research program or externally by ‘crowd-sourcing’ to an interested community. www.sbvimprover.com Implementation: This methodology could become the preferred choice to verify systems biology research workflows that are becoming increasingly complex and sophisticated in industrial and academic settings. Contact: gustavo@us.ibm.com PMID:22423044

Evaluating local food programs: the case of Select Nova Scotia.

PubMed

Knight, Andrew J

2013-02-01

This study evaluated the effectiveness of the buy local food program Select Nova Scotia; a government program with the goal to increase awareness and consumption of Nova Scotia produced and processed agri-food products by Nova Scotians and visitors. The evaluation methodology was based on prior evaluation resources and local food consumer research. Data were gathered through a web panel survey; 877 respondents completed the survey in June 2010. The results suggest that the program is reaching a wider audience than just those predisposed to local food initiatives. In addition, awareness of Select Nova was related to perceptions of local benefits and barriers, as well as purchase motivation and behavior. Respondents who were aware of Select Nova Scotia rated societal benefits as more important and viewed location and price as less of a barrier; they were also more likely to be highly motivated to purchase local foods. This study also informs results found in previous consumer research studies and identifies marketing opportunities to enhance the effectiveness of local food programs. The results suggest that societal benefits might be used as a way to differentiate products with similar attributes. Copyright © 2012 Elsevier Ltd. All rights reserved.

Implementing Impact Evaluations of Malaria Control Interventions: Process, Lessons Learned, and Recommendations

PubMed Central

Hershey, Christine L.; Bhattarai, Achuyt; Florey, Lia S.; McElroy, Peter D.; Nielsen, Carrie F.; Yé, Yazoume; Eckert, Erin; Franca-Koh, Ana Cláudia; Shargie, Estifanos; Komatsu, Ryuichi; Smithson, Paul; Thwing, Julie; Mihigo, Jules; Herrera, Samantha; Taylor, Cameron; Shah, Jui; Mouzin, Eric; Yoon, Steven S.; Salgado, S. René

2017-01-01

Abstract. As funding for malaria control increased considerably over the past 10 years resulting in the expanded coverage of malaria control interventions, so did the need to measure the impact of these investments on malaria morbidity and mortality. Members of the Roll Back Malaria (RBM) Partnership undertook impact evaluations of malaria control programs at a time when there was little guidance in terms of the process for conducting an impact evaluation of a national-level malaria control program. The President’s Malaria Initiative (PMI), as a member of the RBM Partnership, has provided financial and technical support for impact evaluations in 13 countries to date. On the basis of these experiences, PMI and its partners have developed a streamlined process for conducting the evaluations with a set of lessons learned and recommendations. Chief among these are: to ensure country ownership and involvement in the evaluations; to engage stakeholders throughout the process; to coordinate evaluations among interested partners to avoid duplication of efforts; to tailor the evaluation to the particular country context; to develop a standard methodology for the evaluations and a streamlined process for completion within a reasonable time; and to develop tailored dissemination products on the evaluation for a broad range of stakeholders. These key lessons learned and resulting recommendations will guide future impact evaluations of malaria control programs and other health programs. PMID:28990921

Cost allocation methodology applicable to the temporary assistance for needy families program. Final rule.

PubMed

2008-07-23

This final rule applies to the Temporary Assistance for Needy Families (TANF) program and requires States, the District of Columbia and the Territories (hereinafter referred to as the "States") to use the "benefiting program" cost allocation methodology in U.S. Office of Management and Budget (OMB) Circular A-87 (2 CFR part 225). It is the judgment and determination of HHS/ACF that the "benefiting program" cost allocation methodology is the appropriate methodology for the proper use of Federal TANF funds. The Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996 gave federally-recognized Tribes the opportunity to operate their own Tribal TANF programs. Federally-recognized Indian tribes operating approved Tribal TANF programs have always followed the "benefiting program" cost allocation methodology in accordance with OMB Circular A-87 (2 CFR part 225) and the applicable regulatory provisions at 45 CFR 286.45(c) and (d). This final rule contains no substantive changes to the proposed rule published on September 27, 2006.

Assessing the role of mini-applications in predicting key performance characteristics of scientific and engineering applications

DOE PAGES

Barrett, R. F.; Crozier, P. S.; Doerfler, D. W.; ...

2014-09-28

Computational science and engineering application programs are typically large, complex, and dynamic, and are often constrained by distribution limitations. As a means of making tractable rapid explorations of scientific and engineering application programs in the context of new, emerging, and future computing architectures, a suite of miniapps has been created to serve as proxies for full scale applications. Each miniapp is designed to represent a key performance characteristic that does or is expected to significantly impact the runtime performance of an application program. In this paper we introduce a methodology for assessing the ability of these miniapps to effectively representmore » these performance issues. We applied this methodology to four miniapps, examining the linkage between them and an application they are intended to represent. Herein we evaluate the fidelity of that linkage. This work represents the initial steps required to begin to answer the question, ''Under what conditions does a miniapp represent a key performance characteristic in a full app?''« less

The Best Children's Museums--According to "Child" Magazine.

ERIC Educational Resources Information Center

West, Robert Mac

2002-01-01

Reports the first known survey to evaluate and rank children's museums. The survey was conducted by Child magazine but questions the methodology used to derive the rankings. Identifies the top 50 museums in the rankings. Notes that dinosaurs remain a popular topic and many new programs are designed for very young children. (DLH)

76 FR 32971 - Proposed Priority for the Disability and Rehabilitation Research Projects and Centers Program

Federal Register 2010, 2011, 2012, 2013, 2014

2011-06-07

... and systematic search of the research literature on a topic for relevant studies, which are then evaluated using pre-determined, objective criteria for relevance and methodological rigor. In a systematic review, the evidence from relevant studies that meet the pre-determined criteria is then analyzed and...

English Language Teachers' Ideology of ELT Assessment Literacy

ERIC Educational Resources Information Center

Hakim, Badia

2015-01-01

Deep understanding, clear perception and accurate use of assessment methodology play an integral role in the success of a language program. Use of various assessment techniques to evaluate and improve the performance of learners has been the focal point of interest in the field of English Language Teaching (ELT). Equally researchers are interested…

Mortuary Science Programs: Examination of the External Evaluation Team

ERIC Educational Resources Information Center

Reinhard, D. Elaine

2010-01-01

The purpose of this study is to expand the literature on mortuary science accreditation site visit teams. This study used a mixed methodology design to examine: (1) who serves on the American Board of Funeral Service Education accreditation external site visit teams; (2) reasons for involvement in accreditation; (3) perceptions of important site…

Maintenance Resources Evaluation Technique.

DTIC Science & Technology

1999-03-01

variable Z; P-6 BIBLIOGRAPHY Argentine Air Force. Reglamento de Conduction Logistica RAC 9 (Logistics Management Regulation). Estado Mayor General de ...other hand, too many resources are expensive to acquire and maintain and difficult to transport . 1-1 Sizing the means needed to accomplish its...functional areas (maintenance, supply/inventory, transportation , etc), methodologies of operations research (simulation, mathematical programming

An Evaluation of Management Training and Coaching

ERIC Educational Resources Information Center

Berg, Morten Emil; Karlsen, Jan Terje

2012-01-01

Purpose: The focus of this paper is on management training and development. The purpose has been to address how coaching can be applied to learn about leadership tools and what effect this has on management behaviour and development. Design/methodology/approach: This is a qualitative case study of a management development program. The empirical…

Tool to Prioritize Energy Efficiency Investments

DOE Office of Scientific and Technical Information (OSTI.GOV)

Farese, P.; Gelman, R.; Hendron, R.

2012-08-01

To provide analytic support of the U.S. Department of Energy's Office of the Building Technology Program (BTP), NREL developed a Microsoft Excel-based tool to provide an open and objective comparison of the hundreds of investment opportunities available to BTP. This tool uses established methodologies to evaluate the energy savings and cost of those savings.

Evaluating the Validity of Computerized Content Analysis Programs for Identification of Emotional Expression in Cancer Narratives

ERIC Educational Resources Information Center

Bantum, Erin O'Carroll; Owen, Jason E.

2009-01-01

Psychological interventions provide linguistic data that are particularly useful for testing mechanisms of action and improving intervention methodologies. For this study, emotional expression in an Internet-based intervention for women with breast cancer (n = 63) was analyzed via rater coding and 2 computerized coding methods (Linguistic Inquiry…

Teaching Environmental Education through PBL: Evaluation of a Teaching Intervention Program

ERIC Educational Resources Information Center

Vasconcelos, Clara

2012-01-01

If our chosen aim in science education is to be inclusive and to improve students' learning achievements, then we must identify teaching methodologies that are appropriate for teaching and learning specific knowledge. Karagiorgi and Symeo (2005) remind us that instructional designers are thus challenged to translate the philosophy of…

Safeguards Technology Development Program 1st Quarter FY 2018 Report

DOE Office of Scientific and Technical Information (OSTI.GOV)

Prasad, Manoj K.

LLNL will evaluate the performance of a stilbene-based scintillation detector array for IAEA neutron multiplicity counting (NMC) applications. This effort will combine newly developed modeling methodologies and recently acquired high-efficiency stilbene detector units to quantitatively compare the prototype system performance with the conventional He-3 counters and liquid scintillator alternatives.

Stress-Prevention in Secondary Schools: Online- versus Face-to-Face-Training

ERIC Educational Resources Information Center

Fridrici, Mirko; Lohaus, Arnold

2009-01-01

Purpose: The purpose of this paper is to focus on the evaluation of an internet-delivered stress-prevention program for adolescents as a possible alternative for school-based implementation of mental health promotion. Design/methodology/approach: A total of 904 adolescents in grades eight and nine were assigned to four treatment conditions…

Effective Research into Experiential Education: A Critical Resource in Its Own Right.

ERIC Educational Resources Information Center

Gray, Tonia; Patterson, John

This paper discusses the use of quantitative methodologies to evaluate extended-stay outdoor education school programs (ESOESPs) in Australia. A recurring theme in the literature of outdoor education is the paucity of empirical data to support its continuation and development. Proponents argue that the outdoor education experience leads somehow to…

78 FR 57885 - Report on the Criteria and Methodology for Determining the Eligibility of Candidate Countries for...

Federal Register 2010, 2011, 2012, 2013, 2014

2013-09-20

...: Sources: Sources: Quarterly reporting Indicator tracking Quarterly tables reporting. Survey of MCC staff Quarterly reporting GAO Audits. Impact evaluations Survey of MCC staff. Similarly, the Board may consider a... reporting, documentation of changes in timing or scope of a threshold program in implementation, a survey of...

Knaves or Knights, Pawns or Queens?: An Evaluation of Australian Higher Education Reform Policy

ERIC Educational Resources Information Center

Dollery, Brian; Murray, David; Crase, Lin

2006-01-01

Purpose: To invoke Julian Le Grand's conceptual model of the interaction between human motivation and policy formulation in order to explain how motivational endogeneity in the university environment has distorted policy outcomes in the Australian higher education reform program. Design/methodology/approach: Le Grand contends that changes in the…

Retrained Teachers and School Culture: Complex Interactions

ERIC Educational Resources Information Center

Aubusson, Peter; Watson, Kevin; Vozzo, Les; Steele, Frances

2005-01-01

This article reports the experiences of three retrained teachers as they made the transition to science teaching. The stories of their survival in a new school context were constructed from data obtained in an evaluation of the retraining program and interpreted using a reflexive methodology. These mature professionals had developed an approach to…

Successes and Challenges in Triangulating Methodologies in Evaluations of Exemplary Urban Schools

ERIC Educational Resources Information Center

Towns, Donna Penn; Serpell, Zewelanji

2004-01-01

The Exemplary Schools study at Howard University's Center for Research on the Education of Students Placed At Risk (CRESPAR) was designed to aid the program in its goal toward developing and implementing a reform model that "overdetermines" success for all students. The Talent Development (TD) principle of "overdetermination of success" argues…

[Tail Plane Icing

NASA Technical Reports Server (NTRS)

1997-01-01

The Aviation Safety Program initiated by NASA in 1997 has put greater emphasis in safety related research activities. Ice-contaminated-tailplane stall (ICTS) has been identified by the NASA Lewis Icing Technology Branch as an important activity for aircraft safety related research. The ICTS phenomenon is characterized as a sudden, often uncontrollable aircraft nose- down pitching moment, which occurs due to increased angle-of-attack of the horizontal tailplane resulting in tailplane stall. Typically, this phenomenon occurs when lowering the flaps during final approach while operating in or recently departing from icing conditions. Ice formation on the tailplane leading edge can reduce tailplane angle-of-attack range and cause flow separation resulting in a significant reduction or complete loss of aircraft pitch control. In 1993, the Federal Aviation Authority (FAA) and NASA embarked upon a four-year research program to address the problem of tailplane stall and to quantify the effect of tailplane ice accretion on aircraft performance and handling characteristics. The goals of this program, which was completed in March 1998, were to collect aerodynamic data for an aircraft tail with and without ice contamination and to develop analytical methods for predicting the effects of tailplane ice contamination. Extensive dry air and icing tunnel tests which resulted in a database of the aerodynamic effects associated with tailplane ice contamination. Although the FAA/NASA tailplane icing program generated some answers regarding ice-contaminated-tailplane stall (ICTS) phenomena, NASA researchers have found many open questions that warrant further investigation into ICTS. In addition, several aircraft manufacturers have expressed interest in a second research program to expand the database to other tail configurations and to develop experimental and computational methodologies for evaluating the ICTS phenomenon. In 1998, the icing branch at NASA Lewis initiated a second multi-phase research program for tailplane icing (TIP II) to develop test methodologies and tailplane performance and handling qualities evaluation tools. The main objectives of this new NASA/Industry/Academia collaborative research programs were: (1) define and evaluate a sub-scale wind tunnel test methodology for determining tailplane performance degradation due to icing. (2) develop an experimental database of tailplane aerodynamic performance with and without ice contamination for a range of tailplane configurations. Wind tunnel tests were planned with representative general aviation aircraft, i.e., the Learjet 45, and a twin engine low speed aircraft. This report summarizes the research performed during the first year of the study, and outlines the work tasks for the second year.

Organizational Change Efforts: Methodologies for Assessing Organizational Effectiveness and Program Costs versus Benefits.

ERIC Educational Resources Information Center

Macy, Barry A.; Mirvis, Philip H.

1982-01-01

A standardized methodology for identifying, defining, and measuring work behavior and performance rather than production, and a methodology that estimates the costs and benefits of work innovation are presented for assessing organizational effectiveness and program costs versus benefits in organizational change programs. Factors in a cost-benefit…

Climate Model Diagnostic Analyzer Web Service System

NASA Astrophysics Data System (ADS)

Lee, S.; Pan, L.; Zhai, C.; Tang, B.; Kubar, T. L.; Li, J.; Zhang, J.; Wang, W.

2015-12-01

Both the National Research Council Decadal Survey and the latest Intergovernmental Panel on Climate Change Assessment Report stressed the need for the comprehensive and innovative evaluation of climate models with the synergistic use of global satellite observations in order to improve our weather and climate simulation and prediction capabilities. The abundance of satellite observations for fundamental climate parameters and the availability of coordinated model outputs from CMIP5 for the same parameters offer a great opportunity to understand and diagnose model biases in climate models. In addition, the Obs4MIPs efforts have created several key global observational datasets that are readily usable for model evaluations. However, a model diagnostic evaluation process requires physics-based multi-variable comparisons that typically involve large-volume and heterogeneous datasets, making them both computationally- and data-intensive. In response, we have developed a novel methodology to diagnose model biases in contemporary climate models and implementing the methodology as a web-service based, cloud-enabled, provenance-supported climate-model evaluation system. The evaluation system is named Climate Model Diagnostic Analyzer (CMDA), which is the product of the research and technology development investments of several current and past NASA ROSES programs. The current technologies and infrastructure of CMDA are designed and selected to address several technical challenges that the Earth science modeling and model analysis community faces in evaluating and diagnosing climate models. In particular, we have three key technology components: (1) diagnostic analysis methodology; (2) web-service based, cloud-enabled technology; (3) provenance-supported technology. The diagnostic analysis methodology includes random forest feature importance ranking, conditional probability distribution function, conditional sampling, and time-lagged correlation map. We have implemented the new methodology as web services and incorporated the system into the Cloud. We have also developed a provenance management system for CMDA where CMDA service semantics modeling, service search and recommendation, and service execution history management are designed and implemented.

Report of Workshop on Methodology for Evaluating Potential Lunar Resources Sites

NASA Technical Reports Server (NTRS)

Williams, R. J.; Hubbard, N.

1981-01-01

The type and quantity of lunar materials needed to support a space power satellite program was used to define the type and quality of geological information required to certify a site for exploitation. The existing geological, geochemical, and geophysical data are summarized. The difference between these data and the required data for exploitation is used to define program requirements. Most of these requirements involve linear extensions of existing capabilities, fuller utilization of existing data, or expanded use of automated systems.

Flexible thermal protection materials for entry systems

NASA Astrophysics Data System (ADS)

Kourtides, Demetrius A.

1993-02-01

Current programs addressed in aeroassist flight experiment are: (1) evaluation of thermal performance of advanced rigid and flexible insulations and reflective coating; (2) investigation of lighter than baseline materials; (3) investigation of rigid insulations which perform well; (4) study of flexible insulations which require ceramic coating; and (5) study of reflective coating effective at greater than 15 percent. In National Aerospace Plane (NASP), the programs addressed are: (1) high and low temperature insulations; and (2) attachment/standoff methodology critical which affects thermal performance.

Prospective Investigation of Injury Rates and Injury Risk Factors Among Federal Bureau of Investigation New Agent Trainees, Quantico, Virginia, 2009-2010

DTIC Science & Technology

2010-08-01

assistance of the United States Army Public Health Command (USAPHC) in injury prevention efforts at the FBI New Agent Training Program at the FBI...personnel to apply their experience in injury prevention programs in military training to assist in injury prevention efforts in the FBI New Agent... prevention recommendations USAPHC had made Army-wide. USAPHC personnel proposed a methodology to evaluate injury incidence and injury risk factors

Flexible thermal protection materials for entry systems

NASA Technical Reports Server (NTRS)

Kourtides, Demetrius A.

1993-01-01

Current programs addressed in aeroassist flight experiment are: (1) evaluation of thermal performance of advanced rigid and flexible insulations and reflective coating; (2) investigation of lighter than baseline materials; (3) investigation of rigid insulations which perform well; (4) study of flexible insulations which require ceramic coating; and (5) study of reflective coating effective at greater than 15 percent. In National Aerospace Plane (NASP), the programs addressed are: (1) high and low temperature insulations; and (2) attachment/standoff methodology critical which affects thermal performance.

Evaluation of automated decisionmaking methodologies and development of an integrated robotic system simulation. Appendix B: ROBSIM programmer's guide

NASA Technical Reports Server (NTRS)

Haley, D. C.; Almand, B. J.; Thomas, M. M.; Krauze, L. D.; Gremban, K. D.; Sanborn, J. C.; Kelly, J. H.; Depkovich, T. M.; Wolfe, W. J.; Nguyen, T.

1986-01-01

The purpose of the Robotic Simulation (ROBSIM) program is to provide a broad range of computer capabilities to assist in the design, verification, simulation, and study of robotic systems. ROBSIM is programmed in FORTRAM 77 and implemented on a VAX 11/750 computer using the VMS operating system. The programmer's guide describes the ROBSIM implementation and program logic flow, and the functions and structures of the different subroutines. With the manual and the in-code documentation, an experienced programmer can incorporate additional routines and modify existing ones to add desired capabilities.

Evaluation of automated decisionmaking methodologies and development of an integrated robotic system simulation, appendix B

NASA Technical Reports Server (NTRS)

Haley, D. C.; Almand, B. J.; Thomas, M. M.; Krauze, L. D.; Gremban, K. D.; Sanborn, J. C.; Kelly, J. H.; Depkovich, T. M.

1984-01-01

The purpose of the Robotics Simulation (ROBSIM) program is to provide a broad range of computer capabilities to assist in the design, verification, simulation, and study of robotic systems. ROBSIM is programmed in FORTRAN 77 and implemented on a VAX 11/750 computer using the VMS operating system. This programmer's guide describes the ROBSIM implementation and program logic flow, and the functions and structures of the different subroutines. With this manual and the in-code documentation, and experienced programmer can incorporate additional routines and modify existing ones to add desired capabilities.

A counterfactual impact evaluation of a bilingual program on students' grade point average at a spanish university.

PubMed

Arco-Tirado, J L; Fernández-Martín, F; Ramos-García, A M; Littvay, L; Villoria, J; Naranjo, J A

2018-06-01

This observational study intends to estimate the causal effects of an English as a Medium of Instruction (EMI) program (as predictor) on students Grade Point Average (GPA) (as outcome) at a particular University in Spain by using a Counterfactual Impact Evaluation (CIE). The need to address the crucial question of causal inferences in EMI programs to produce credible evidences of successful interventions contrasts, however, with the absence of experimental or quasi-experimental research and evaluation designs in the field. CIE approach is emerging as a methodologically viable solution to bridge that gap. The program evaluated here consisted in delivering an EMI program in a Primary Education Teacher Training Degree group. After achieving balance on the observed covariates and recreating a situation that would have been expected in a randomized experiment, three matching approaches such as genetic matching, nearest neighbor matching and Coarsened Exact Matching were used to analyze observational data from a total of 1288 undergraduate students, including both treatment and control group. Results show unfavorable effects of the bilingual group treatment condition. Potential interpretations and recommendations are provided in order to strengthen future causal evidences of bilingual education programs' effectiveness in Higher Education. Copyright © 2018 The Authors. Published by Elsevier Ltd.. All rights reserved.

Using baldrige performance excellence program approaches in the pursuit of radiation oncology quality care, patient satisfaction, and workforce commitment.

PubMed

Sternick, Edward S

2011-01-01

The Malcolm Baldrige National Quality Improvement Act was signed into law in 1987 to advance US business competitiveness and economic growth. Administered by the National Institute of Standards and Technology, the Act created the Baldrige National Quality Program, recently renamed the Baldrige Performance Excellence Program. The comprehensive analytical approaches referred to as the Baldrige Healthcare Criteria, are very well-suited for the evaluation and sustainable improvement of radiation oncology management and operations. A multidisciplinary self-assessment approach is used for radiotherapy program evaluation and development in order to generate a fact-based, knowledge-driven system for improving quality of care, increasing patient satisfaction, enhancing leadership effectiveness, building employee engagement, and boosting organizational innovation. This methodology also provides a valuable framework for benchmarking an individual radiation oncology practice's operations and results against guidelines defined by accreditation and professional organizations and regulatory agencies.

Teaching clinical research methodology to the academic medical community: a fifteen-year retrospective of a comprehensive curriculum.

PubMed

Supino, Phyllis G; Borer, Jeffrey S

2007-05-01

Due to inadequate preparation, many medical professionals are unable to critically evaluate published research articles or properly design, execute and present their own research. To increase exposure among physicians, medical students, and allied health professionals to diverse methodological issues involved in performing research. A comprehensive course on research methodology was newly designed for physicians and other members of an academic medical community, and has been successfully implemented beginning 1991. The role of the study hypothesis is highlighted; interactive pedagogical techniques are employed to promote audience engagement. Participants complete an annual evaluation to assess course quality and perceived outcomes. Outcomes also are assessed qualitatively by faculty. More than 500 physicians/other professionals have participated. Ratings have been consistently high. Topics deemed most valuable are investigational planning, hypothesis construction and study designs. An enhancement of capacity to define hypotheses and apply methodological concepts in the criticism of scientific papers and development of protocols/manuscripts has been observed. Participants and faculty believe the course improves critical appraisal skills and ability to conduct research. Our experience shows it is feasible to accomplish these objectives, with a high level of satisfaction, through a didactic program targeted to the general academic community.

A systems-based food safety evaluation: an experimental approach.

PubMed

Higgins, Charles L; Hartfield, Barry S

2004-11-01

Food establishments are complex systems with inputs, subsystems, underlying forces that affect the system, outputs, and feedback. Building on past exploration of the hazard analysis critical control point concept and Ludwig von Bertalanffy General Systems Theory, the National Park Service (NPS) is attempting to translate these ideas into a realistic field assessment of food service establishments and to use information gathered by these methods in efforts to improve food safety. Over the course of the last two years, an experimental systems-based methodology has been drafted, developed, and tested by the NPS Public Health Program. This methodology is described in this paper.

Do employee health management programs work?

PubMed

Serxner, Seth; Gold, Daniel; Meraz, Angela; Gray, Ann

2009-01-01

Current peer review literature clearly documents the economic return and Return-on-Investment (ROI) for employee health management (EHM) programs. These EHM programs are defined as: health promotion, self-care, disease management, and case management programs. The evaluation literature for the sub-set of health promotion and disease management programs is examined in this article for specific evidence of the level of economic return in medical benefit cost reduction or avoidance. The article identifies the methodological challenges associated with determination of economic return for EHM programs and summarizes the findings from 23 articles that included 120 peer review study results. The article identifies the average ROI and percent health plan cost impact to be expected for both types of EHM programs, the expected time period for its occurrence, and caveats related to its measurement.

Methodological Framework for Analysis of Buildings-Related Programs with BEAMS, 2008

DOE Office of Scientific and Technical Information (OSTI.GOV)

Elliott, Douglas B.; Dirks, James A.; Hostick, Donna J.

The U.S. Department of Energy’s (DOE’s) Office of Energy Efficiency and Renewable Energy (EERE) develops official “benefits estimates” for each of its major programs using its Planning, Analysis, and Evaluation (PAE) Team. PAE conducts an annual integrated modeling and analysis effort to produce estimates of the energy, environmental, and financial benefits expected from EERE’s budget request. These estimates are part of EERE’s budget request and are also used in the formulation of EERE’s performance measures. Two of EERE’s major programs are the Building Technologies Program (BT) and the Weatherization and Intergovernmental Program (WIP). Pacific Northwest National Laboratory (PNNL) supports PAEmore » by developing the program characterizations and other market information necessary to provide input to the EERE integrated modeling analysis as part of PAE’s Portfolio Decision Support (PDS) effort. Additionally, PNNL also supports BT by providing line-item estimates for the Program’s internal use. PNNL uses three modeling approaches to perform these analyses. This report documents the approach and methodology used to estimate future energy, environmental, and financial benefits using one of those methods: the Building Energy Analysis and Modeling System (BEAMS). BEAMS is a PC-based accounting model that was built in Visual Basic by PNNL specifically for estimating the benefits of buildings-related projects. It allows various types of projects to be characterized including whole-building, envelope, lighting, and equipment projects. This document contains an overview section that describes the estimation process and the models used to estimate energy savings. The body of the document describes the algorithms used within the BEAMS software. This document serves both as stand-alone documentation for BEAMS, and also as a supplemental update of a previous document, Methodological Framework for Analysis of Buildings-Related Programs: The GPRA Metrics Effort, (Elliott et al. 2004b). The areas most changed since the publication of that previous document are those discussing the calculation of lighting and HVAC interactive effects (for both lighting and envelope/whole-building projects). This report does not attempt to convey inputs to BEAMS or the methodology of their derivation.« less

Space Shuttle Program: STS-1 Medical Report

NASA Technical Reports Server (NTRS)

1981-01-01

The necessity for developing medical standards addressing individual classes of Shuttle crew positions is discussed. For the U.S. manned program the conclusion of the Apollo era heralded the end of water recovery operations and the introduction of land-based medical operations. This procedural change marked a significant departure from the accepted postflight medical recovery and evaluation techniques. All phases of the missions required careful re-evaluation, identification of potential impact on preexisting medical operational techniques, and development of new methodologies which were carefully evaluated and tested under simulated conditions. Significant coordination was required between the different teams involved in medical operations. Additional dimensions were added to the concepts of medical operations, by the introduction of different toxic substances utilized by the Space Transportation Systems especially during ground operations.

A methodological approach and framework for sustainability assessment in NGO-implemented primary health care programs.

PubMed

Sarriot, Eric G; Winch, Peter J; Ryan, Leo J; Bowie, Janice; Kouletio, Michelle; Swedberg, Eric; LeBan, Karen; Edison, Jay; Welch, Rikki; Pacqué, Michel C

2004-01-01

An estimated 10.8 million children under 5 continue to die each year in developing countries from causes easily treatable or preventable. Non governmental organizations (NGOs) are frontline implementers of low-cost and effective child health interventions, but their progress toward sustainable child health gains is a challenge to evaluate. This paper presents the Child Survival Sustainability Assessment (CSSA) methodology--a framework and process--to map progress towards sustainable child health from the community level and upward. The CSSA was developed with NGOs through a participatory process of research and dialogue. Commitment to sustainability requires a systematic and systemic consideration of human, social and organizational processes beyond a purely biomedical perspective. The CSSA is organized around three interrelated dimensions of evaluation: (1) health and health services; (2) capacity and viability of local organizations; (3) capacity of the community in its social ecological context. The CSSA uses a participatory, action-planning process, engaging a 'local system' of stakeholders in the contextual definition of objectives and indicators. Improved conditions measured in the three dimensions correspond to progress toward a sustainable health situation for the population. This framework opens new opportunities for evaluation and research design and places sustainability at the center of primary health care programming.

Software Requirements Engineering Methodology (Development)

DTIC Science & Technology

1979-06-01

Higher Order Software [20]; and the Michael Jackson Design Methodology [21]. Although structured programming constructs have proven to be more useful...reviewed here. Similarly, the manual techniques for software design (e.g., HIPO Diagrams, Nassi-Schneidermann charts, Top-Down Design, the Michael ... Jackson Design Methodology, Yourdon’s Structured Design) are not addressed. 6.1.3 Research Programs There are a number of research programs underway

Industrial demand side management: A status report

DOE Office of Scientific and Technical Information (OSTI.GOV)

Hopkins, M.F.; Conger, R.L.; Foley, T.J.

This report provides an overview of and rationale for industrial demand side management (DSM) programs. Benefits and barriers are described, and data from the Manufacturing Energy Consumption Survey are used to estimate potential energy savings in kilowatt hours. The report presents types and examples of programs and explores elements of successful programs. Two in-depth case studies (from Boise Cascade and Eli Lilly and Company) illustrate two types of effective DSM programs. Interviews with staff from state public utility commissions indicate the current thinking about the status and future of industrial DSM programs. A comprehensive bibliography is included, technical assistance programsmore » are listed and described, and a methodology for evaluating potential or actual savings from projects is delineated.« less

A methodology for the evaluation of program cost and schedule risk for the SEASAT program

NASA Technical Reports Server (NTRS)

Abram, P.; Myers, D.

1976-01-01

An interactive computerized project management software package (RISKNET) is designed to analyze the effect of the risk involved in each specific activity on the results of the total SEASAT-A program. Both the time and the cost of each distinct activity can be modeled with an uncertainty interval so as to provide the project manager with not only the expected time and cost for the completion of the total program, but also with the expected range of costs corresponding to any desired level of significance. The nature of the SEASAT-A program is described. The capabilities of RISKNET and the implementation plan of a RISKNET analysis for the development of SEASAT-A are presented.

[Evaluating a blended-learning program on developing teamwork competence].

PubMed

Aguado, David; Arranz, Virginia; Valera-Rubio, Ana; Marín-Torres, Susana

2011-08-01

The knowledge, skills and abilities that are required to work optimally in teams are critical for many types of work. Organizations can provide access to these skills by means of training programs. Diverse studies show how traditional in-site training methodologies can improve teamwork knowledge, skills and abilities. Nevertheless, in-site methods can be complemented with on-line strategies that result in blended-learning programs. The aim of this work is to analyze, following Kirkpatrick's assessment levels, the effectiveness of a blended-learning program of teamwork training in an organizational context. Carried out with 102 professionals, the results show participants' satisfaction with the program, high level of learning (of both declarative and procedural knowledge), and a moderate level of transfer of learning to the job.

School Climate in American Secondary Schools: A Psychometric Examination of PISA 2009 School Climate Scale

ERIC Educational Resources Information Center

Sun, Letao; Royal, Kenneth

2017-01-01

The purpose of this study was to examine the quality of the Program for International Student Assessment (PISA) 2009 school climate survey instrument and evaluate perceptions of secondary school principals' located in the United States about school climate using an Item Response Theory (IRT) methodological approach. In particular, this study…

77 FR 38072 - Submission for OMB Review; Comment Request; The National Diabetes Education Program Survey of the...

Federal Register 2010, 2011, 2012, 2013, 2014

2012-06-26

... effectively manage their disease to prevent or delay complications and improve quality of life; (3) decrease... behavior change, improved quality of life, and better diabetes outcomes; (3) expanding NDEP reach and... validity of the methodology and assumptions used; (3) Evaluate the quality, utility, and clarity of the...

A Methodological Study of a Computer-Managed Instructional Program in High School Physics.

ERIC Educational Resources Information Center

Denton, Jon James

The purpose of this study was to develop and evaluate an instructional model which utilized the computer to produce individually prescribed instructional guides in physics at the secondary school level. The sample consisted of three classes. Of these, two were randomly selected to serve as the treatment groups, e.g., individualized instruction and…

PERT and CPM: Workshop Material.

ERIC Educational Resources Information Center

Burroughs Corp., Detroit, MI.

This is a workbook containing problems in PERT (program evaluation review technique). It is intended to be used in a workshop or classroom to train management personnel in the basic methodology and capability of PERT. This material is not adequate in depth to create an expert in these techniques, but it is felt that the material is adequate to…

Project #138. Coronary Care Education of Health Care Team. Final Report.

ERIC Educational Resources Information Center

Saint Joseph Hospital, MO.

The goal of this project was to develop, establish, and implement a system for the educational development of health care team members of the St. Joseph region in emergency and coronary care. Programs, curricula, and evaluation methodology were devised for four levels of critical care personnel: R.N.s emphasizing emergency and coronary care;…

Predicate Matching in NLP: A Review of Research on the Preferred Representational System.

ERIC Educational Resources Information Center

Sharpley, Christopher F.

1984-01-01

Reviews 15 studies that have investigated the use of the Preferred Representational System (PRS) in Neurolinguistic Programming (NLP). Aspects of design, methodology, population and dependent measures are evaluated, with comments on the outcomes obtained. Results suggested that there is little supportive evidence for the use of PRS in the NLP.…

METHODOLOGY FOR PROJECTION OF OCCUPATIONAL TRENDS IN THE DENVER STANDARD METROPOLITAN STATISTICAL AREA.

ERIC Educational Resources Information Center

FISHMAN, LESLIE; AND OTHERS

THE FINAL STAGE OF A PROGRAM FOR ACHIEVING A BALANCE BETWEEN THE AGGREGATE SUPPLY AND DEMAND FOR LABOR IS THE DISTRIBUTION OF THE INDUSTRY EMPLOYMENT ESTIMATES INTO OCCUPATIONAL REQUIREMENTS BASED ON PROJECTIONS OF OCCUPATIONAL PATTERNS BY INDUSTRY. THIS CAN BE USED TO EVALUATE POTENTIAL AREAS OF SUBSTANTIAL SURPLUS OR SHORTAGE, AND PROVIDE THE…

Contemporary Instructor-Librarian Collaboration: A Case Study of an Online Embedded Librarian Implementation

ERIC Educational Resources Information Center

Edwards, Mary E.; Black, Erik W.

2012-01-01

This paper reports the results of a case study evaluation of an embedded librarian project at a large, land-grant, research institution. The case is comprised of learners who are full-time academic health care professionals enrolled in an online graduate educational technology program. The mixed methods methodology focused on assessing the…

Coordination and Evaluation of Videodisc and Microcomputer Programs. Final Technical Report.

ERIC Educational Resources Information Center

McLaughlin, Donald H.; And Others

This report provides information on the background, methodology, and findings of the Videodisc-Microcomputer (ViM) Network project, which brought together over 40 schools in 15 states to explore the potential of a new technology for basic skills instruction. Initiated in 1981 and completed at the end of 1983, the project involved a wide variety of…

Beyond Academic and Social Integration: Understanding the Impact of a STEM Enrichment Program on the Retention and Degree Attainment of Underrepresented Students

PubMed Central

Lane, Tonisha B.

2016-01-01

The current study used a case study methodological approach, including document analysis, semistructured interviews, and participant observations, to investigate how a science, technology, engineering, and mathematics (STEM) enrichment program supported retention and degree attainment of underrepresented students at a large, public, predominantly white institution. From this study, a model emerged that encompassed four components: proactive care, holistic support, community building, and catalysts for STEM identity development. These components encompassed a number of strategies and practices that were instrumental in the outcomes of program participants. This paper concludes with implications for practice, such as using models to inform program planning, assessment, and evaluation. PMID:27543638

Reconsidering community-based health promotion: promise, performance, and potential.

PubMed

Merzel, Cheryl; D'Afflitti, Joanna

2003-04-01

Contemporary public health emphasizes a community-based approach to health promotion and disease prevention. The evidence from the past 20 years indicates, however, that many community-based programs have had only modest impact, with the notable exception of a number of HIV prevention programs. To better understand the reasons for these outcomes, we conducted a systematic literature review of 32 community-based prevention programs. Reasons for poor performance include methodological challenges to study design and evaluation, concurrent secular trends, smaller-than-expected effect sizes, limitations of the interventions, and limitations of theories used. The effectiveness of HIV programs appears to be related in part to extensive formative research and an emphasis on changing social norms.