Qualitative Evaluation.

ERIC Educational Resources Information Center

Stone, James C., Ed.; James, Raymond A., Ed.

1981-01-01

"Qualitative evaluation" is the theme of this issue of the California Journal of Teacher Education. Ralph Tyler states that evaluation is essentially descriptive, and using numbers does not solve basic problems. Martha Elin Vernazza examines the issue of objectivity in history and its implications for evaluation. She posits that the…

Miramar College Program Evaluation: Aviation Maintenance.

ERIC Educational Resources Information Center

Moriyama, Bruce; Brumley, Leslie

Qualitative and quantitative data are presented in this evaluation of the curricular, personnel, and financial status of Miramar College's program in aviation maintenance. The report first provides the results of an interview with the program chairperson, which sought information on program objectives and goals and their determination, the extent…

Miramar College Program Evaluation: Criminal Justice.

ERIC Educational Resources Information Center

Moriyama, Bruce; Brumley, Leslie

Qualitative and quantitative data are presented in this evaluation of the curricular, personnel, and financial status of Miramar College's program in criminal justice. The report first outlines the information gathered in an interview with the program chairperson, conducted to determine program objectives and goals and how they were determined,…

Miramar College Program Evaluation: Fire Science.

ERIC Educational Resources Information Center

Moriyama, Bruce; Brumley, Leslie

Qualitative and quantitative data are presented in this evaluation of the curricular, personnel, and financial status of Miramar College's program in fire sciences. The report first provides the results of an interview with the program chairperson, which sought information on program objectives and goals and their determination, the extent to…

Using Qualitative Methods to Evaluate a Family Behavioral Intervention for Type 1 Diabetes

PubMed Central

Herbert, Linda Jones; Sweenie, Rachel; Kelly, Katherine Patterson; Holmes, Clarissa; Streisand, Randi

2013-01-01

Introduction The objectives of this study were to qualitatively evaluate a dyadic adolescent-parent type 1 diabetes (T1D) program developed to prevent deterioration in diabetes care among adolescents with T1D and provide recommendations for program refinement. Method Thirteen adolescent-parent dyads who participated in the larger RCT, the TeamWork Project, were interviewed regarding their perceptions of their participation in the program and current T1D challenges. Interviews were transcribed and coded to establish broad themes. Results Adolescents and parents thought the TeamWork Project sessions were helpful and taught them new information. Five themes catalog findings from the qualitative interviews: TeamWork content, TeamWork structure, transition of responsibility, current and future challenges, and future intervention considerations. Discussion Addressing T1D challenges as a parent-adolescent dyad via a behavioral clinic program is helpful to families during adolescence. Findings highlight the utility of qualitative evaluation to tailor interventions for the unique challenges related to pediatric chronic illness. PMID:24269281

Qualitative Approaches to Evaluating Education.

ERIC Educational Resources Information Center

Fetterman, David M.

This paper explores the variety of qualitative methods available, in the context of a larger quantitative-qualitative debate in the field of educational evaluation. Each approach is reviewed in terms of the work of its major proponents. The dominant forms of qualitative evaluation include: (1) ethnography; (2) naturalistic inquiry; (3) generic…

Program Evaluation: A Review and Synthesis.

ERIC Educational Resources Information Center

Webber, Charles F.

This paper reviews models of program evaluation. Major topics and issues found in the evaluation literature include quantitative versus qualitative approaches, identification and involvement of stakeholders, formulation of research questions, collection of data, analysis and interpretation of data, reporting of results, evaluation utilization, and…

New Teacher Induction: A Program Evaluation

ERIC Educational Resources Information Center

Hunter, J. Warren

2016-01-01

The purpose of this qualitative program evaluation was to examine the impact a two-year new teacher induction program had on teachers' feelings of support, satisfaction, and self-efficacy. The program purports that higher feelings of support, satisfaction, and self-efficacy in teachers will lead to lower teacher attrition. In turn, research shows…

A qualitative evaluation of the 2005-2011 National Academic Centers of Excellence in Youth Violence Prevention Program.

PubMed

Holland, Kristin M; Vivolo-Kantor, Alana M; Dela Cruz, Jason; Massetti, Greta M; Mahendra, Reshma

2015-12-01

The Centers for Disease Control and Prevention's Division of Violence Prevention (DVP) funded eight National Academic Centers of Excellence (ACEs) in Youth Violence Prevention from 2005 to 2010 and two Urban Partnership Academic Centers of Excellence (UPACEs) in Youth Violence Prevention from 2006 to 2011. The ACEs and UPACEs constitute DVP's 2005-2011 ACE Program. ACE Program goals include partnering with communities to promote youth violence (YV) prevention and fostering connections between research and community practice. This article describes a qualitative evaluation of the 2005-2011 ACE Program using an innovative approach for collecting and analyzing data from multiple large research centers via a web-based Information System (ACE-IS). The ACE-IS was established as an efficient mechanism to collect and document ACE research and programmatic activities. Performance indicators for the ACE Program were established in an ACE Program logic model. Data on performance indicators were collected through the ACE-IS biannually. Data assessed Centers' ability to develop, implement, and evaluate YV prevention activities. Performance indicator data demonstrate substantial progress on Centers' research in YV risk and protective factors, community partnerships, and other accomplishments. Findings provide important lessons learned, illustrate progress made by the Centers, and point to new directions for YV prevention research and programmatic efforts. Published by Elsevier Ltd.

Systematic review of qualitative evaluations of reentry programs addressing problematic drug use and mental health disorders amongst people transitioning from prison to communities.

PubMed

Kendall, Sacha; Redshaw, Sarah; Ward, Stephen; Wayland, Sarah; Sullivan, Elizabeth

2018-03-02

The paper presents a systematic review and metasynthesis of findings from qualitative evaluations of community reentry programs. The programs sought to engage recently released adult prison inmates with either problematic drug use or a mental health disorder. Seven biomedical and social science databases, Cinahl, Pubmed, Scopus, Proquest, Medline, Sociological abstracts and Web of Science and publisher database Taylor and Francis were searched in 2016 resulting in 2373 potential papers. Abstract reviews left 140 papers of which 8 were included after detailed review. Major themes and subthemes were identified through grounded theory inductive analysis of results from the eight papers. Of the final eight papers the majority (6) were from the United States. In total, the papers covered 405 interviews and included 121 (30%) females and 284 (70%) males. Findings suggest that the interpersonal skills of case workers; access to social support and housing; and continuity of case worker relationships throughout the pre-release and post-release period are key social and structural factors in program success. Evaluation of community reentry programs requires qualitative data to contextualize statistical findings and identify social and structural factors that impact on reducing incarceration and improving participant health. These aspects of program efficacy have implications for reentry program development and staff training and broader social and health policy and services.

Right timing in formative program evaluation.

PubMed

Hall, Jori; Freeman, Melissa; Roulston, Kathy

2014-08-01

Since many educational researchers and program developers have limited knowledge of formative evaluation, formative data may be underutilized during the development and implementation of an educational program. The purpose of this article is to explain how participatory, responsive, educative, and qualitative approaches to formative evaluation can facilitate a partnership between evaluators and educational researchers and program managers to generate data useful to inform program implementation and improvement. This partnership is critical, we argue, because it enables an awareness of when to take appropriate action to ensure successful educational programs or "kairos". To illustrate, we use examples from our own evaluation work to highlight how formative evaluation may facilitate opportune moments to (1) define the substance and purpose of a program, (2) develop understanding and awareness of the cultural interpretations of program participants, and (3) show the relevance of stakeholder experiences to program goals. Copyright © 2014 Elsevier Ltd. All rights reserved.

Qualitative Analysis of Written Reflections during a Teaching Certificate Program

PubMed Central

Castleberry, Ashley N.; Payakachat, Nalin; Ashby, Sarah; Nolen, Amanda; Carle, Martha; Neill, Kathryn K.

2016-01-01

Objective. To evaluate the success of a teaching certificate program by qualitatively evaluating the content and extent of participants’ reflections. Methods. Two investigators independently identified themes within midpoint and final reflection essays across six program years. Each essay was evaluated to determine the extent of reflection in prompted teaching-related topic areas (strengths, weaknesses, assessment, feedback). Results. Twenty-eight themes were identified within 132 essays. Common themes encompassed content delivery, student assessment, personal successes, and challenges encountered. Deep reflection was exhibited, with 48% of essays achieving the highest level of critical reflection. Extent of reflection trended higher from midpoint to final essays, with significant increases in the strengths and feedback areas. Conclusion. The teaching certificate program fostered critical reflection and self-reported positive behavior change in teaching, thus providing a high-quality professional development opportunity. Such programs should strongly consider emphasizing critical reflection through required reflective exercises at multiple points within program curricula. PMID:26941436

What Do Students Learn when We Teach Peace? A Qualitative Assessment of a Theater Peace Program

ERIC Educational Resources Information Center

Duckworth, Cheryl Lynn; Allen, Barb; Williams, Teri Triguba

2012-01-01

This is a qualitative assessment of a theater arts peace education program for high-school students. We present the results of qualitative interviews with students who participated in a peace education program. They tell us in their own words what they believe they learned. Given that most peace education evaluation is quantitative or focuses on…

The relevance of qualitative research for clinical programs in psychiatry.

PubMed

Goering, Paula; Boydell, Katherine M; Pignatiello, Antonio

2008-03-01

It is time to move beyond education about qualitative research theory and methods to using them to understand and improve psychiatric practice. There is a good fit between this agenda and current thinking about research use that broadens definitions of evidence beyond the results of experiments. This paper describes a qualitative program evaluation to illustrate what kind of useful knowledge is generated and how it can be created through a clinician-researcher partnership. The linkage and exchange model of effective knowledge translation described involves interaction between clinicians and researchers throughout the research process and results in mutual learning through the planning, disseminating, and application of existing or new qualitative research in decision making.

Patient and Nurse Experiences in a Rural Chronic Disease Management Program: A Qualitative Evaluation.

PubMed

Davisson, Erica A; Swanson, Elizabeth A

Rural status confounds chronic disease self-management. The purpose of this qualitative, descriptive study was to evaluate the nurse-led "Living Well" chronic disease management program reporting patient recruitment and retention issues since program initiation in 2013. The Chronic Care Model (CCM) was the guiding framework used to reinforce that interdisciplinary teams must have productive patient interactions for their program(s) to be sustainable. A rural, Midwest county clinic's chronic disease management program. Observations, interviews, and within- and across-case coding were used. Patients' responses were analyzed to identify (1) reasons for recruitment and retention problems and (2) program elements that were viewed as successful or needing improvement. A convenience sample of 6 rural, English-speaking adults (65 years or older, with no severe cognitive impairment) with at least one chronic condition was recruited and interviewed. Themes emerged related to nurse knowledge, availability, and value; peer support; overcoming barriers; adherence enhancement; and family/friends' involvement. Patients reported engagement in self-management activities because of program elements such as support groups and productive nurse-patient interactions. Interdisciplinary communication, commitment, and patient referral processes were identified as reasons for recruitment and retention issues. Findings substantiated that certain elements must be present and improved upon for future rural programs to be successful. Interdisciplinary communication may need to be improved to address recruitment and retention problems. It was clear from patient interviews that the nurse coordinators played a major role in patients' self-management adherence and overall satisfaction with the program. This is important to case management because results revealed the need for programs of this nature that incorporate the vital role of nurse coordinators and align with the CCM value of providing a

Alumni Perspectives on Career Preparation during a Postdoctoral Training Program: A Qualitative Study

ERIC Educational Resources Information Center

Faupel-Badger, Jessica M.; Raue, Kimberley; Nelson, David E.; Tsakraklides, Sophia

2015-01-01

Published evaluations of career preparation of alumni from long-standing postdoctoral fellowship programs in the biomedical sciences are limited and often focus on quantitative analysis of data from extant publicly available sources. Qualitative methods provide the opportunity to gather robust information about specific program elements from…

Evaluation of a Research Mentorship Program in Community Care

ERIC Educational Resources Information Center

Ploeg, Jenny; de Witt, Lorna; Hutchison, Brian; Hayward, Lynda; Grayson, Kim

2008-01-01

This article describes the results of a qualitative case study evaluating a research mentorship program in community care settings in Ontario, Canada. The purpose of the program was to build evaluation and research capacity among staff of community care agencies through a mentorship program. Data were collected through in-depth, semi-structured…

Evaluating the BK 21 Program. Research Brief

ERIC Educational Resources Information Center

Seong, Somi; Popper, Steven W.; Goldman, Charles A.; Evans, David K.; Grammich, Clifford A.

2008-01-01

The Brain Korea 21 program (BK21), an effort to improve Korean universities and research, has attracted a great deal of attention in Korea, producing the need to understand how well the program is meeting its goals. RAND developed a logic model for identifying program goals and dynamics, suggested quantitative and qualitative evaluation methods,…

Working with Science Teachers to Transform the Opportunity Landscape for Regional and Rural Youth: A Qualitative Evaluation of the Science in Schools Program

ERIC Educational Resources Information Center

Sheehan, Grania R.; Mosse, Jennifer

2013-01-01

This article reports on a qualitative evaluation of the Science in Schools program; a suite of science based activities delivered by staff of a regional university campus and designed to provide professional development for science teachers working in non-metropolitan schools in a socioeconomically disadvantaged region of Australia. The research…

Qualitative Evaluation of Emotional Intelligence In-Service Program for Secondary School Teachers

ERIC Educational Resources Information Center

Fer, Seval

2004-01-01

This paper is an attempt to evaluate the Emotional Intelligence (EQ) In-Service Program on the basis of experiences of 20 secondary school teachers who attended the program in a private school in Turkey. A phenomenological approach, with a focus group method was used. The first objective of this study was to evaluate EQ program on the basis of…

Evaluation Processes Used to Assess the Effectiveness of Vocational-Technical Programs.

ERIC Educational Resources Information Center

Bruhns, Arthur E.

Evaluation is quantitative or qualitative, the criteria determined by or given to the student. The criteria show how close he has come to the program's objectives and the ranking of individual performance. Vocational education programs susceptible to evaluation are listed and relevant evaluative techniques discussed. Graduate interviews concerning…

Building for the Future: Community College Leadership Development Program Evaluation

ERIC Educational Resources Information Center

Bresso, Michele

2012-01-01

This qualitative descriptive study examines and evaluates an internal, grow-your-own, community college leadership development program. Participants in a community college leadership development program self-reported their leadership knowledge, attitudes, and practices (KAP) before, during, and after participation in the program. Study…

Evaluating Writing Programs: Paradigms, Problems, Possibilities.

ERIC Educational Resources Information Center

McLeod, Susan H.

1992-01-01

Describes two methodological approaches (qualitative and quantitative) that grow out of two different research examples. Suggests the problems these methods present. Discusses the ways in which an awareness of these problems can help teachers to understand how to work with researchers in designing useful evaluations of writing programs. (PRA)

Commentary: Writing and evaluating qualitative research reports

USDA-ARS?s Scientific Manuscript database

An overview of qualitative methods is provided, particularly for reviewers and authors who may be less familiar with qualitative research. A question and answer format is used to address considerations for writing and evaluating qualitative research. When producing qualitative research, individuals ...

Residential Wilderness Programs: The Role of Social Support in Influencing Self-Evaluations of Male Adolescents

ERIC Educational Resources Information Center

Cook, Emily C.

2008-01-01

This qualitative study explores the aspects of a residential wilderness experience that informed self-evaluations in male adolescents, ages 12-16. To assess change in self-evaluations and program factors associated with change, qualitative interviews were conducted with adolescents upon entry to the program and four months later. Participants'…

Harnessing health information to foster disadvantaged teens' community engagement, leadership skills, and career plans: a qualitative evaluation of the Teen Health Leadership Program.

PubMed

Keselman, Alla; Ahmed, Einas A; Williamson, Deborah C; Kelly, Janice E; Dutcher, Gale A

2015-04-01

This paper describes a qualitative evaluation of a small-scale program aiming to improve health information literacy, leadership skills, and interest in health careers among high school students in a low-income, primarily minority community. Graduates participated in semi-structured interviews, transcripts of which were coded with a combination of objectives-driven and data-driven categories. The program had a positive impact on the participants' health information competency, leadership skills, academic orientation, and interest in health careers. Program enablers included a supportive network of adults, novel experiences, and strong mentorship. The study suggests that health information can provide a powerful context for enabling disadvantaged students' community engagement and academic success.

Positive Effects of a Stress Reduction Program Based on Mindfulness Meditation in Brazilian Nursing Professionals: Qualitative and Quantitative Evaluation.

PubMed

dos Santos, Teresa Maria; Kozasa, Elisa Harumi; Carmagnani, Isabel Sampaio; Tanaka, Luiza Hiromi; Lacerda, Shirley Silva; Nogueira-Martins, Luiz Antonio

2016-01-01

Mindfulness meditation has been shown to effectively mitigate the negative effects of stress among nursing professionals, but in countries like Brazil, these practices are relatively unexplored. To evaluate the effects of a Stress Reduction Program (SRP) including mindfulness and loving kindness meditation among nursing professionals working in a Brazilian hospital setting. Pilot study with a mixed model using quantitative and qualitative methods was used to evaluate a group of participants. The quantitative data were analyzed at three different time points: pre-intervention, post-intervention, and follow-up. The qualitative data were analyzed at post-intervention. Hospital São Paulo (Brazil). Sample 13 nursing professionals, including nurses, technicians, and nursing assistants working in a hospital. Participants underwent mindfulness and loving kindness meditation during a period of six weeks. Perceived Stress Scale (PSS), Maslach Burnout Inventory (MBI), Beck Depression Inventory (BDI), State-Trait Anxiety Inventory (STAI), Satisfaction With Life Scale (SWLS), Self-Compassion Scale (SCS), WHOQOL-BREF quality of life assessment, and Work Stress Scale (WSS). Qualitative data were collected via a group interview following six weeks participation in the SRP. The quantitative analyses revealed a significant reduction (P < .05) between pre-intervention and post-intervention scores for perceived stress, burnout, depression, and anxiety (trait). These variables showed no significant differences between post-intervention and follow-up scores. The WHOQOL-BREF revealed significant increase (P < .05) just in the physical and psychological domains at post-intervention scores, which remained at the follow-up. Qualitative results showed improvement in the reactivity to inner experience; a more attentive perception of internal and external experiences; greater attention and awareness of actions and attitudes at every moment; and a positive influence of the SRP in nursing

Qualitative Evaluation Methods in Ethics Education: A Systematic Review and Analysis of Best Practices.

PubMed

Watts, Logan L; Todd, E Michelle; Mulhearn, Tyler J; Medeiros, Kelsey E; Mumford, Michael D; Connelly, Shane

2017-01-01

Although qualitative research offers some unique advantages over quantitative research, qualitative methods are rarely employed in the evaluation of ethics education programs and are often criticized for a lack of rigor. This systematic review investigated the use of qualitative methods in studies of ethics education. Following a review of the literature in which 24 studies were identified, each study was coded based on 16 best practices characteristics in qualitative research. General thematic analysis and grounded theory were found to be the dominant approaches used. Researchers are effectively executing a number of best practices, such as using direct data sources, structured data collection instruments, non-leading questioning, and expert raters. However, other best practices were rarely present in the courses reviewed, such as collecting data using multiple sources, methods, raters, and timepoints, evaluating reliability, and employing triangulation analyses to assess convergence. Recommendations are presented for improving future qualitative research studies in ethics education.

A Mobile Health App-Based Postnatal Educational Program (Home-but not Alone): Descriptive Qualitative Study.

PubMed

Shorey, Shefaly; Yang, Yen Yen; Dennis, Cindy-Lee

2018-04-19

The postnatal period poses numerous challenges for new parents. Various educational programs are available to support new parents during this stressful period. However, the usefulness of educational programs must be evaluated to ascertain their credibility. The aim of this descriptive, qualitative study was to explore the views of parents of newborns with regard to the content and delivery of a mobile health (mHealth) app-based postnatal educational program. A qualitative semistructured interview guide was used to collect data from 17 participants who belonged to the intervention group of a randomized controlled trial. The intervention, a 4-week-long access to a mHealth app-based educational program, was evaluated. The interviews were conducted in English and at the participants' homes. Thematic analysis was used to analyze the data. The Consolidated Criteria for Reporting Qualitative Research checklist was used to report the findings. The interviews revealed 4 main themes: (1) positive features of the mHealth app, (2) advice from midwives, (3) experiences gained from using the mHealth app, and (4) recommendations for the future. The participants evaluated the educational program to be a good source of information that was tailored to the local context. The different modes of delivery, including audio and video, accentuated the accessibility of information. The parents evaluated that the facilitator of the featured communication platform, a midwife, provided trustworthy advice. Belongingness to a virtual community beyond the hospital endowed the parents the confidence that they were not alone and were supported by other parents and health care professionals. According to the parents, the mHealth app-based educational program was helpful in supporting a multi-ethnic sample of parents during the postnatal period. This insight indicates that the program could be implemented in a wide community of parents in the postnatal period. The helpfulness of the educational

A Mobile Health App–Based Postnatal Educational Program (Home-but not Alone): Descriptive Qualitative Study

PubMed Central

Yang, Yen Yen; Dennis, Cindy-Lee

2018-01-01

Background The postnatal period poses numerous challenges for new parents. Various educational programs are available to support new parents during this stressful period. However, the usefulness of educational programs must be evaluated to ascertain their credibility. Objective The aim of this descriptive, qualitative study was to explore the views of parents of newborns with regard to the content and delivery of a mobile health (mHealth) app–based postnatal educational program. Methods A qualitative semistructured interview guide was used to collect data from 17 participants who belonged to the intervention group of a randomized controlled trial. The intervention, a 4-week-long access to a mHealth app–based educational program, was evaluated. The interviews were conducted in English and at the participants’ homes. Thematic analysis was used to analyze the data. The Consolidated Criteria for Reporting Qualitative Research checklist was used to report the findings. Results The interviews revealed 4 main themes: (1) positive features of the mHealth app, (2) advice from midwives, (3) experiences gained from using the mHealth app, and (4) recommendations for the future. The participants evaluated the educational program to be a good source of information that was tailored to the local context. The different modes of delivery, including audio and video, accentuated the accessibility of information. The parents evaluated that the facilitator of the featured communication platform, a midwife, provided trustworthy advice. Belongingness to a virtual community beyond the hospital endowed the parents the confidence that they were not alone and were supported by other parents and health care professionals. Conclusions According to the parents, the mHealth app–based educational program was helpful in supporting a multi-ethnic sample of parents during the postnatal period. This insight indicates that the program could be implemented in a wide community of parents in the

Supportive supervision and constructive relationships with healthcare workers support CHW performance: Use of a qualitative framework to evaluate CHW programming in Uganda.

PubMed

Ludwick, Teralynn; Turyakira, Eleanor; Kyomuhangi, Teddy; Manalili, Kimberly; Robinson, Sheila; Brenner, Jennifer L

2018-02-13

While evidence supports community health worker (CHW) capacity to improve maternal and newborn health in less-resourced countries, key implementation gaps remain. Tools for assessing CHW performance and evidence on what programmatic components affect performance are lacking. This study developed and tested a qualitative evaluative framework and tool to assess CHW team performance in a district program in rural Uganda. A new assessment framework was developed to collect and analyze qualitative evidence based on CHW perspectives on seven program components associated with effectiveness (selection; training; community embeddedness; peer support; supportive supervision; relationship with other healthcare workers; retention and incentive structures). Focus groups were conducted with four high/medium-performing CHW teams and four low-performing CHW teams selected through random, stratified sampling. Content analysis involved organizing focus group transcripts according to the seven program effectiveness components, and assigning scores to each component per focus group. Four components, 'supportive supervision', 'good relationships with other healthcare workers', 'peer support', and 'retention and incentive structures' received the lowest overall scores. Variances in scores between 'high'/'medium'- and 'low'-performing CHW teams were largest for 'supportive supervision' and 'good relationships with other healthcare workers.' Our analysis suggests that in the Bushenyi intervention context, CHW team performance is highly correlated with the quality of supervision and relationships with other healthcare workers. CHWs identified key performance-related issues of absentee supervisors, referral system challenges, and lack of engagement/respect by health workers. Other less-correlated program components warrant further study and may have been impacted by relatively consistent program implementation within our limited study area. Applying process-oriented measurement tools are

Integrative Reiki for cancer patients: a program evaluation.

PubMed

Fleisher, Kimberly A; Mackenzie, Elizabeth R; Frankel, Eitan S; Seluzicki, Christina; Casarett, David; Mao, Jun J

2014-01-01

This mixed methods study sought to evaluate the outcomes of an integrative Reiki volunteer program in an academic medical oncology center setting. We used de-identified program evaluation data to perform both quantitative and qualitative analyses of participants' experiences of Reiki sessions. The quantitative data were collected pre- and postsession using a modified version of the distress thermometer. The pre- and postsession data from the distress assessment were analyzed using a paired Student's : test. The qualitative data were derived from written responses to open-ended questions asked after each Reiki session and were analyzed for key words and recurring themes. Of the 213 pre-post surveys of first-time sessions in the evaluation period, we observed a more than 50% decrease in self-reported distress (from 3.80 to 1.55), anxiety (from 4.05 to 1.44), depression (from 2.54 to 1.10), pain (from 2.58 to 1.21), and fatigue (from 4.80 to 2.30) with P < .001 for all. Using conservative estimates that treat missing data as not endorsing Reiki, we found 176 (82.6%) of participants liked the Reiki session, 176 (82.6%) found the Reiki session helpful, 157 (73.7%) plan to continue using Reiki, and 175 (82.2%) would recommend Reiki to others. Qualitative analyses found that individuals reported that Reiki induced relaxation and enhanced spiritual well-being. An integrative Reiki volunteer program shows promise as a component of supportive care for cancer patients. More research is needed to evaluate and understand the impact that Reiki may have for patients, caregivers, and staff whose lives have been affected by cancer.

Evaluation of National Institute for Learning Development and Discovery Educational Therapy Program

ERIC Educational Resources Information Center

Frimpong, Prince Christopher

2014-01-01

In Maryland, some Christian schools have enrolled students with learning disabilities (LDs) but do not have any interventional programs at the school to help them succeed academically. The purpose of this qualitative program evaluation was to evaluate the National Institute for Learning Development (NILD) and Discovery Therapy Educational Program…

The Evaluation of Health Education Program (HEP) of 9th Graders

ERIC Educational Resources Information Center

Erdogan, Polat; Gürol, Mehmet

2016-01-01

The main purpose of evaluation is to improve the quality of program. So, the purpose of this study was to evaluate the Health Education Program of 9th graders (HEP) with Educational Criticism Model of Eisner. The study was conducted with the qualitative research method. A holistic single case design was employed in this study. The typical case…

Promoting and evaluating scientific rigour in qualitative research.

PubMed

Baillie, Lesley

2015-07-15

This article explores perspectives on qualitative research and the variety of views concerning rigour in the research process. Evaluating and ensuring the quality of research are essential considerations for practitioners who are appraising evidence to inform their practice or research. Several criteria and principles for evaluating quality in qualitative research are presented, recognising that their application in practice is influenced by the qualitative methodology used. The article examines a range of techniques that a qualitative researcher can use to promote rigour and apply it to practice.

An economic analysis methodology for project evaluation and programming.

DOT National Transportation Integrated Search

2013-08-01

Economic analysis is a critical component of a comprehensive project or program evaluation methodology that considers all key : quantitative and qualitative impacts of highway investments. It allows highway agencies to identify, quantify, and value t...

The Practice of Health Program Evaluation.

PubMed

Lewis, Sarah R

2017-11-01

The Practice of Health Program Evaluation provides an overview of the evaluation process for public health programs while diving deeper to address select advanced concepts and techniques. The book unfolds evaluation as a three-phased process consisting of identification of evaluation questions, data collection and analysis, and dissemination of results and recommendations. The text covers research design, sampling methods, as well as quantitative and qualitative approaches. Types of evaluation are also discussed, including economic assessment and systems research as relative newcomers. Aspects critical to conducting a successful evaluation regardless of type or research design are emphasized, such as stakeholder engagement, validity and reliability, and adoption of sound recommendations. The book encourages evaluators to document their approach by developing an evaluation plan, a data analysis plan, and a dissemination plan, in order to help build consensus throughout the process. The evaluative text offers a good bird's-eye view of the evaluation process, while offering guidance for evaluation experts on how to navigate political waters and advocate for their findings to help affect change.

VoiceThread: A Useful Program Evaluation Tool

ERIC Educational Resources Information Center

Mott, Rebecca

2018-01-01

With today's technology, Extension professionals have a variety of tools available for program evaluation. This article describes an innovative platform called VoiceThread that has been used in many classrooms but also is useful for conducting virtual focus group research. I explain how this tool can be used to collect qualitative participant…

Cooking, Healthy Eating, Fitness and Fun (CHEFFs): Qualitative Evaluation of a Nutrition Education Program for Children Living at Urban Family Homeless Shelters

PubMed Central

Applebaum, Jo; Stephenson-Hunter, Cara; Tinio, Andrea; Shapiro, Alan

2013-01-01

Objectives. We assessed the feasibility of a 15-week nutrition education, physical activity, and media literacy program for children living in urban family homeless shelters. Methods. We developed a qualitative monitoring tool to evaluate program process and impact at 2 shelter sites in the Bronx, New York, from 2009 to 2012. Facilitators recorded indications of participants’ understanding of intended messages and demonstrations of changes in attitudes and behaviors. Comments, insights, and actions were recorded as they occurred. Facilitators also documented barriers to delivery of content and activities as intended. We used content analysis to examine data for patterns and identify themes. Results. A total of 162 children participated at the 2 shelter sites. Analysis of qualitative data yielded 3 themes: (1) children’s knowledge and understanding of content, (2) children’s shift in attitudes or intentions, and (3) interpretations through children’s life experience. Food insecurity as well as shelter food service and policies were important influences on children’s choices, hunger, and sense of well-being. Conclusions. Children’s experiences highlighted the need to advocate for shelter policies that adequately provide for children’s nutritional and physical activity requirements and foster academic development. PMID:24148062

Cooking, healthy eating, fitness and fun (CHEFFs): qualitative evaluation of a nutrition education program for children living at urban family homeless shelters.

PubMed

Rodriguez, Jacqueline; Applebaum, Jo; Stephenson-Hunter, Cara; Tinio, Andrea; Shapiro, Alan

2013-12-01

We assessed the feasibility of a 15-week nutrition education, physical activity, and media literacy program for children living in urban family homeless shelters. We developed a qualitative monitoring tool to evaluate program process and impact at 2 shelter sites in the Bronx, New York, from 2009 to 2012. Facilitators recorded indications of participants' understanding of intended messages and demonstrations of changes in attitudes and behaviors. Comments, insights, and actions were recorded as they occurred. Facilitators also documented barriers to delivery of content and activities as intended. We used content analysis to examine data for patterns and identify themes. A total of 162 children participated at the 2 shelter sites. Analysis of qualitative data yielded 3 themes: (1) children's knowledge and understanding of content, (2) children's shift in attitudes or intentions, and (3) interpretations through children's life experience. Food insecurity as well as shelter food service and policies were important influences on children's choices, hunger, and sense of well-being. Children's experiences highlighted the need to advocate for shelter policies that adequately provide for children's nutritional and physical activity requirements and foster academic development.

Supplemental Literacy Instruction for Students with Down Syndrome: A Program Evaluation

ERIC Educational Resources Information Center

Regan, Lisa Michelle

2013-01-01

The study utilizes an inductive, qualitative approach to program evaluation to understand the nature of an afterschool literacy tutoring program for students with Down syndrome. Two research questions guide this study: (a) What are the curricular and instructional elements of the Let's Read Now (LRN) literacy tutoring program for students…

Supporting aboriginal knowledge and practice in health care: lessons from a qualitative evaluation of the strong women, strong babies, strong culture program.

PubMed

Lowell, Anne; Kildea, Sue; Liddle, Marlene; Cox, Barbara; Paterson, Barbara

2015-02-05

The Strong Women, Strong Babies, Strong Culture Program (the Program) evolved from a recognition of the value of Aboriginal knowledge and practice in promoting maternal and child health (MCH) in remote communities of the Northern Territory (NT) of Australia. Commencing in 1993 it continues to operate today. In 2008, the NT Department of Health commissioned an evaluation to identify enabling factors and barriers to successful implementation of the Program, and to identify potential pathways for future development. In this paper we focus on the evaluation findings related specifically to the role of Aborignal cultural knowledge and practice within the Program. A qualitative evaluation utilised purposive sampling to maximise diversity in program history and Aboriginal culture. Semi-structured, in-depth interviews with 76 participants were recorded in their preferred language with a registered Interpreter when required. Thematic analysis of data was verified or modified through further discussions with participants and members of the evaluation team. Although the importance of Aboriginal knowledge and practice as a fundamental component of the Program is widely acknowledged, there has been considerable variation across time and location in the extent to which these cultural dimensions have been included in practice. Factors contributing to this variation are complex and relate to a number of broad themes including: location of control over Program activities; recognition and respect for Aboriginal knowledge and practice as a legitimate component of health care; working in partnership; communication within and beyond the Program; access to transport and working space; and governance and organisational support. We suggest that inclusion of Aboriginal knowledge and practice as a fundamental component of the Program is key to its survival over more than twenty years despite serious challenges. Respect for the legitimacy of Aboriginal knowledge and practice within health

Developing a Research Program Using Qualitative and Quantitative Approaches.

ERIC Educational Resources Information Center

Beck, Cheryl Tatano

1997-01-01

A research program on postpartum depression is used to illustrate the use of both qualitative and quantitative approaches. The direction of a research program is thus not limited by the type of methods in which a researcher has expertise. (SK)

Commentary: Writing and Evaluating Qualitative Research Reports

PubMed Central

Thompson, Deborah; Aroian, Karen J.; McQuaid, Elizabeth L.; Deatrick, Janet A.

2016-01-01

Objective To provide an overview of qualitative methods, particularly for reviewers and authors who may be less familiar with qualitative research. Methods A question and answer format is used to address considerations for writing and evaluating qualitative research. Results and Conclusions When producing qualitative research, individuals are encouraged to address the qualitative research considerations raised and to explicitly identify the systematic strategies used to ensure rigor in study design and methods, analysis, and presentation of findings. Increasing capacity for review and publication of qualitative research within pediatric psychology will advance the field’s ability to gain a better understanding of the specific needs of pediatric populations, tailor interventions more effectively, and promote optimal health. PMID:27118271

A combined impact-process evaluation of a program promoting active transport to school: understanding the factors that shaped program effectiveness.

PubMed

Crawford, S; Garrard, J

2013-01-01

This mixed methods study was a comprehensive impact-process evaluation of the Ride2School program in metropolitan and regional areas in Victoria, Australia. The program aimed to promote transport to school for primary school children. Qualitative and quantitative data were collected at baseline and followup from two primary schools involved in the pilot phase of the program and two matched comparison schools, and a further 13 primary schools that participated in the implementation phase of the program. Classroom surveys, structured and unstructured observations, and interviews with Ride2School program staff were used to evaluate the pilot program. For the 13 schools in the second phase of the program, parents and students completed questionnaires at baseline (N = 889) and followup (N = 761). Based on the quantitative data, there was little evidence of an overall increase in active transport to school across participating schools, although impacts varied among individual schools. Qualitative data in the form of observations, interviews, and focus group discussions with students, school staff, and program staff provided insight into the reasons for variable program impacts. This paper highlights the benefits of undertaking a mixed methods approach to evaluating active transport to school programs that enables both measurement and understanding of program impacts.

A Combined Impact-Process Evaluation of a Program Promoting Active Transport to School: Understanding the Factors That Shaped Program Effectiveness

PubMed Central

Crawford, S.; Garrard, J.

2013-01-01

This mixed methods study was a comprehensive impact-process evaluation of the Ride2School program in metropolitan and regional areas in Victoria, Australia. The program aimed to promote transport to school for primary school children. Qualitative and quantitative data were collected at baseline and followup from two primary schools involved in the pilot phase of the program and two matched comparison schools, and a further 13 primary schools that participated in the implementation phase of the program. Classroom surveys, structured and unstructured observations, and interviews with Ride2School program staff were used to evaluate the pilot program. For the 13 schools in the second phase of the program, parents and students completed questionnaires at baseline (N = 889) and followup (N = 761). Based on the quantitative data, there was little evidence of an overall increase in active transport to school across participating schools, although impacts varied among individual schools. Qualitative data in the form of observations, interviews, and focus group discussions with students, school staff, and program staff provided insight into the reasons for variable program impacts. This paper highlights the benefits of undertaking a mixed methods approach to evaluating active transport to school programs that enables both measurement and understanding of program impacts. PMID:23606865

Qualitative Approaches to Evaluating Education.

ERIC Educational Resources Information Center

Fetterman, David M.

1988-01-01

Discusses qualitative research and its application to educational evaluation. Approaches discussed include the following: (1) ethnography; (2) naturalistic inquiry; (3) generic pragmatic (sociological) inquiry; (4) connoisseurship/criticism; (5) metaphors; and (6) phenomenography. (FMW)

Impact of the Social Café Meals program: a qualitative investigation.

PubMed

Allen, Louise; O'Connor, Jacklin; Amezdroz, Emily; Bucello, Pieta; Mitchell, Hannah; Thomas, Arabella; Kleve, Sue; Bernardi, Anthony; Wallis, Liza; Palermo, Claire

2014-01-01

Social Café Meals Programs aim to reduce food insecurity and social exclusion by providing participants access to subsidised meals in mainstream local cafés. This study aimed to explore the program's ability to address social exclusion and food insecurity and the impact of the program on the community. A qualitative evaluation approach was utilised whereby in-depth interviews were conducted with café owners, café staff and current program members of two Social Café Meals Programs operating in the south-eastern suburbs of Melbourne. Twelve program members and six café staff completed an in-depth interview at the local cafés. Data were analysed using a thematic analysis approach focusing on the lived experience of the café owners, staff and program members. Four key themes were identified. The program (i) improved food access for vulnerable groups and (ii) created community cohesiveness. (iii) The café environment was important in facilitating program use by community members. (iv) Café owners felt rewarded for their community contribution via the program. Social Café Meals Programs may provide a solution to improving food security and reducing social exclusion and may be considered as a strategy for improving nutrition and social health for at-risk and vulnerable groups.

Methodology for the evaluation of the Stephanie Alexander Kitchen Garden program.

PubMed

Gibbs, L; Staiger, P K; Townsend, M; Macfarlane, S; Gold, L; Block, K; Johnson, B; Kulas, J; Waters, E

2013-04-01

Community and school cooking and gardening programs have recently increased internationally. However, despite promising indications, there is limited evidence of their effectiveness. This paper presents the evaluation framework and methods negotiated and developed to meet the information needs of all stakeholders for the Stephanie Alexander Kitchen Garden (SAKG) program, a combined cooking and gardening program implemented in selectively funded primary schools across Australia. The evaluation used multiple aligned theoretical frameworks and models, including a public health ecological approach, principles of effective health promotion and models of experiential learning. The evaluation is a non-randomised comparison of six schools receiving the program (intervention) and six comparison schools (all government-funded primary schools) in urban and rural areas of Victoria, Australia. A mixed-methods approach was used, relying on qualitative measures to understand changes in school cultures and the experiential impacts on children, families, teachers, parents and volunteers, and quantitative measures at baseline and 1 year follow up to provide supporting information regarding patterns of change. The evaluation study design addressed the limitations of many existing evaluation studies of cooking or garden programs. The multistrand approach to the mixed methodology maintained the rigour of the respective methods and provided an opportunity to explore complexity in the findings. Limited sensitivity of some of the quantitative measures was identified, as well as the potential for bias in the coding of the open-ended questions. The SAKG evaluation methodology will address the need for appropriate evaluation approaches for school-based kitchen garden programs. It demonstrates the feasibility of a meaningful, comprehensive evaluation of school-based programs and also demonstrates the central role qualitative methods can have in a mixed-method evaluation. So what? This paper

A Qualitative Study of College-Based Peace Education Programs

ERIC Educational Resources Information Center

Boudreau, Will

2017-01-01

The purpose of this exploratory research study was to examine the perceptions of seven northeast United States, college-based, Peace Education program directors regarding their respective programs' characteristics and the challenges they face. This qualitative study was designed to fill a gap in the literature by examining the perceptions of…

[Reconstituting evaluation methods based on both qualitative and quantitative paradigms].

PubMed

Miyata, Hiroaki; Okubo, Suguru; Yoshie, Satoru; Kai, Ichiro

2011-01-01

Debate about the relationship between quantitative and qualitative paradigms is often muddled and confusing and the clutter of terms and arguments has resulted in the concepts becoming obscure and unrecognizable. In this study we conducted content analysis regarding evaluation methods of qualitative healthcare research. We extracted descriptions on four types of evaluation paradigm (validity/credibility, reliability/credibility, objectivity/confirmability, and generalizability/transferability), and classified them into subcategories. In quantitative research, there has been many evaluation methods based on qualitative paradigms, and vice versa. Thus, it might not be useful to consider evaluation methods of qualitative paradigm are isolated from those of quantitative methods. Choosing practical evaluation methods based on the situation and prior conditions of each study is an important approach for researchers.

Qualitative Evaluation of a Role Play Bullying Simulation.

PubMed

Gillespie, Gordon L; Brown, Kathryn; Grubb, Paula; Shay, Amy; Montoya, Karen

Bullying against nurses is becoming a pervasive problem. In this article, a role play simulation designed for undergraduate nursing students is described. In addition, the evaluation findings from a subsample of students who participated in a role play simulation addressing bullying behaviors are reported. Focus group sessions were completed with a subset of eight students who participated in the intervention. Sessions were audiorecorded, transcribed verbatim, and analyzed using Colaizzi's procedural steps for qualitative analysis. Themes derived from the data were "The Experience of Being Bullied", "Implementation of the Program", "Desired Outcome of the Program", and "Context of Bullying in the Nursing Profession". Role play simulation was an effective and active learning strategy to diffuse education on bullying in nursing practice. Bullying in nursing was identified as a problem worthy of incorporation into the undergraduate nursing curriculum. To further enhance the learning experience with role play simulation, adequate briefing instructions, opportunity to opt out of the role play, and comprehensive debriefing are essential.

A qualitative evaluation of strategies to increase colorectal cancer screening uptake

PubMed Central

Tinmouth, Jill; Ritvo, Paul; McGregor, S. Elizabeth; Claus, Danielle; Pasut, George; Myers, Ronald E.; Guglietti, Crissa; Paszat, Lawrence F.; Hilsden, Robert J.; Rabeneck, Linda

2011-01-01

Abstract Objective To obtain data that could be used to optimize the content and design of the targeted, mailed invitations that Ontario’s provincewide colorectal cancer (CRC) screening program plans to use to increase screening uptake; to identify other strategies to increase CRC screening uptake; and to describe the effects of this qualitative work on a subsequent quantitative pilot study. Design Qualitative study using semistructured focus groups. Setting Four different Ontario communities. Participants Six focus groups comprising a total of 62 participants. Methods Six focus groups were conducted in 4 different Ontario communities. For 3 of the communities, participants were recruited from the general population by a private marketing firm, using random-digit dialing, and received a small honorarium for participating. In Sault Ste Marie, participants were convenience samples recruited from a large primary care practice and were not offered compensation. Responses were elicited regarding various strategies for promoting CRC screening. Findings represent all responses observed as well as recommendations to program planners based on focus groups observations. Main findings Key themes identified included the importance of receiving a CRC screening invitation from one’s family physician; a desire for personalized, brief communications; and a preference for succinct information in mailed materials. Strong support was indicated for direct mailing of the CRC screening kit (fecal occult blood test). Our findings substantially influenced the final design and content of the envelope and letter to be mailed in the subsequent quantitative pilot study. Conclusion We report strong support from our focus groups for a succinct, personalized invitation for CRC screening from one’s own family physician. We have also shown that qualitative evaluation can be used to provide decision makers with pertinent and timely knowledge. Our study is highly relevant to other public

Commentary: Writing and Evaluating Qualitative Research Reports.

PubMed

Wu, Yelena P; Thompson, Deborah; Aroian, Karen J; McQuaid, Elizabeth L; Deatrick, Janet A

2016-06-01

To provide an overview of qualitative methods, particularly for reviewers and authors who may be less familiar with qualitative research. A question and answer format is used to address considerations for writing and evaluating qualitative research. When producing qualitative research, individuals are encouraged to address the qualitative research considerations raised and to explicitly identify the systematic strategies used to ensure rigor in study design and methods, analysis, and presentation of findings. Increasing capacity for review and publication of qualitative research within pediatric psychology will advance the field's ability to gain a better understanding of the specific needs of pediatric populations, tailor interventions more effectively, and promote optimal health. © The Author 2016. Published by Oxford University Press on behalf of the Society of Pediatric Psychology. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

In It Together: A Qualitative Evaluation of Participant Experiences of a 10-Week, Group-Based, Workplace HIIT Program for Insufficiently Active Adults.

PubMed

Kinnafick, Florence-Emilie; Thøgersen-Ntoumani, Cecilie; Shepherd, Sam O; Wilson, Oliver J; Wagenmakers, Anton J M; Shaw, Christopher S

2018-02-01

Using guidance from the reach, efficacy, adoption, implementation, and maintenance evaluation framework, we aimed to qualitatively evaluate the participant experiences of a workplace high-intensity interval training (HIIT) intervention. Twelve previously insufficiently active individuals (four males and eight females) were interviewed once as part of three focus groups. Perceptions of program satisfaction, barriers to and facilitators of adherence, and persistence to exercise were explored. HIIT initiates interest because of its novelty, provides a sense of accomplishment, and overcomes the barriers of perceived lack of time. The feeling of relatedness between the participants can attenuate negative unpleasant responses during the HIIT sessions. HIIT, in this workplace setting, is an acceptable intervention for physically inactive adults. However, participants were reluctant to maintain the same mode of exercise, believing that HIIT sessions were for the very fit.

Evaluating the Healthy Start program. Design development to evaluative assessment.

PubMed

Raykovich, K S; McCormick, M C; Howell, E M; Devaney, B L

1996-09-01

The national evaluation of the federally funded Healthy Start program involved translating a design for a process and outcomes evaluation and standard maternal and infant data set, both developed prior to the national evaluation contract award, into an evaluation design and client data collection protocol that could be used to evaluate 15 diverse grantees. This article discusses the experience of creating a process and outcomes evaluation design that was both substantively and methodologically appropriate given such issues as the diversity of grantees and their community-based intervention strategies; the process of accessing secondary data sources, including vital records; the quality of client level data submissions; and the need to incorporate both qualitative and quantitative approaches into the evaluation design. The relevance of this experience for the conduct of other field studies of public health interventions is discussed.

Graduate Program Evaluation in the Area Leading Educational, Outlying and Backward

ERIC Educational Resources Information Center

Prasetyono, Hendro

2016-01-01

This study aimed to evaluate the Bachelor program Educating in the Regions Frontier, Outermost and Disadvantaged (SM-3T). The research location is at the Jakarta State University. The evaluation method used is Cippo consisting of components Context, Input, Process, Product and Outcomes. The method used is qualitative descriptive data analysis…

Applying Program Theory-Driven Approach to Design and Evaluate a Teacher Professional Development Program

ERIC Educational Resources Information Center

Lin, Su-ching; Wu, Ming-sui

2016-01-01

This study was the first year of a two-year project which applied a program theory-driven approach to evaluating the impact of teachers' professional development interventions on students' learning by using a mix of methods, qualitative inquiry, and quasi-experimental design. The current study was to show the results of using the method of…

Qualitative evaluation and economic estimates of an infection control champions program.

PubMed

Lloyd-Smith, Elisa; Curtin, Jim; Gilbart, Wayne; Romney, Marc G

2014-12-01

In many North American hospitals, conventional infection control operational models often struggle to provide sufficient support to frontline health care workers. The objective of this study was to describe a sustainable infection control champion (ICC) program based on findings from focus groups. A distributed model of infection control was established by placing infection prevention and control-trained ICCs in 3 Canadian hospitals for a period of 12 months. Subsequently, semistructured focus groups were conducted to describe overall feasibility and impeding and critical factors affecting sustainability. An economic estimate of the ICC program compared with the cost of hiring a new infection control practitioner was also calculated. Focus group participants considered the program feasible. Barriers included lack of time and staff turnover. Themes critical for the successful implementation of an ICC program included defined ICC roles and goals, adequate support and resources for the ICC, engagement with all levels of staff, flexible structure, and program evaluation. The cost per bed of the ICC program was less than the cost per bed of hiring a new infection control practitioner. A distributed model of providing infection prevention and control services may have benefit when hospital infection control teams are underresourced, as is often the case. Several key factors are needed for the successful implementation of an ICC program. Copyright © 2014 Association for Professionals in Infection Control and Epidemiology, Inc. Published by Elsevier Inc. All rights reserved.

Methods for evaluating a mature substance abuse prevention/early intervention program.

PubMed

Becker, L R; Hall, M; Fisher, D A; Miller, T R

2000-05-01

The authors describe methods for work in progress to evaluate four workplace prevention and/or early intervention programs designed to change occupational norms and reduce substance abuse at a major U.S. transportation company. The four programs are an employee assistance program, random drug testing, managed behavioral health care, and a peer-led intervention program. An elaborate mixed-methods evaluation combines data collection and analysis techniques from several traditions. A process-improvement evaluation focuses on the peer-led component to describe its evolution, document the implementation process for those interested in replicating it, and provide information for program improvement. An outcome-assessment evaluation examines impacts of the four programs on job performance measures (e.g., absenteeism, turnover, injury, and disability rates) and includes a cost-offset and employer cost-savings analysis. Issues related to using archival data, combining qualitative and quantitative designs, and working in a corporate environment are discussed.

African American perspectives: A qualitative study of an informal science enrichment program

NASA Astrophysics Data System (ADS)

Simpson, Jamila Rashida

The purposes of this study were to determine what program characteristics African American parents consider when they enroll their children into an informal science education enrichment program, the parents' evaluation of a program called Jordan Academy in which they enrolled their children, and the alignment of the parents' perspectives with Black Cultural Ethos (BCE). BCE refers to nine dimensions posited by Wade Boykin, a psychologist, as comprising African American culture. Participants were parents of students that attended Jordan Academy, an informal science enrichment program designed for third through sixth grade students from underserved populations. Qualitative methodologies were utilized to perform a thorough assessment of parents' perspectives. Data sources included classroom observations, student surveys, academy curriculum, photos and video-taped class sessions. Data included teachers and parents' responses to semi-structured, audio recorded interviews and students' written responses to open-ended items on the program's evaluation instrument. The data were analyzed for themes and the findings compared to Black Cultural Ethos. Findings revealed that the participants believed that informal science education offered their children opportunities not realized in the formal school setting - a means of impacting their children holistically. The parents expressed the academic, cultural, and personal development of their children in their characterizations of the ideal informal science education experience and in their evaluations of Jordan Academy. Overall, the parents' views emphasized the BCE values of harmony, affect, verve, movement, orality and communalism. The study has important implications for practices within and research on informal science education.

Improving clinical and translational research training: a qualitative evaluation of the Atlanta Clinical and Translational Science Institute KL2-mentored research scholars program

PubMed Central

Comeau, Dawn L; Escoffery, Cam; Freedman, Ariela; Ziegler, Thomas R; Blumberg, Henry M

2017-01-01

A major impediment to improving the health of communities is the lack of qualified clinical and translational research (CTR) investigators. To address this workforce shortage, the National Institutes of Health (NIH) developed mechanisms to enhance the career development of CTR physician, PhD, and other doctoral junior faculty scientists including the CTR-focused K12 program and, subsequently, the KL2-mentored CTR career development program supported through the Clinical and Translational Science Awards (CTSAs). Our evaluation explores the impact of the K12/KL2 program embedded within the Atlanta Clinical and Translational Science Institute (ACTSI), a consortium linking Emory University, Morehouse School of Medicine and the Georgia Institute of Technology. We conducted qualitative interviews with program participants to evaluate the impact of the program on career development and collected data on traditional metrics (number of grants, publications). 46 combined K12/KL2 scholars were supported between 2002 and 2016. 30 (65%) of the 46 K12/KL2 scholars are women; 24 (52%) of the trainees are minorities, including 10 (22%) scholars who are members of an underrepresented minority group. Scholars reported increased research skills, strong mentorship experiences, and positive impact on their career trajectory. Among the 43 scholars who have completed the program, 39 (91%) remain engaged in CTR and received over $89 000 000 as principal investigators on federally funded awards. The K12/KL2 funding provided the training and protected time for successful career development of CTR scientists. These data highlight the need for continued support for CTR training programs for junior faculty. PMID:27591319

Utilization-focused evaluation of a STEM enrichment program

NASA Astrophysics Data System (ADS)

Carter, Sally

The purpose of this study was to determine the impact and utilization of a STEM enrichment program (hereafter referred to as The Program). The Program consisted of two parts. First an educator resource center provided free educational materials throughout The Program’s home state. The second part of The Program was a network of education specialists who provided professional development for teachers, modeled lessons with students, and provided presentations for the general public. The problem addressed by this study was a lack of knowledge regarding the impact of The Program. The Program’s director requested a utilization-focused program evaluation to answer thirteen questions. Questions covered Program impact for five areas: overall impact on teachers, overall impact on students, overall impact of materials, overall impact of Program personnel, and overall impact on STEM education. A mixed-methods case study was designed to gather data. Quantitative data included Program archival data regarding the number of contacts and a survey distributed to teachers who had used The Program’s services on at least one occasion. Qualitative data included written comments gathered from the teacher surveys, seven teacher focus groups, and four Program personnel interviews. Data found an overall positive Program impact in all five areas. Both quantitative and qualitative data showed favorable perceptions by teachers and Program personnel. It is not known if data from this case study can be generalized to other STEM enrichment programs. Future research might include a study to determine if The Program’s model could be used to generate new STEM enrichment programs.

Are Validity and Reliability "Relevant" in Qualitative Evaluation Research?

ERIC Educational Resources Information Center

Goodwin, Laura D.; Goodwin, William L.

1984-01-01

The views of prominant qualitative methodologists on the appropriateness of validity and reliability estimation for the measurement strategies employed in qualitative evaluations are summarized. A case is made for the relevance of validity and reliability estimation. Definitions of validity and reliability for qualitative measurement are presented…

An Interprofessional Program Evaluation Case Study: Utilizing Multiple Measures To Assess What Matters. AIR 1997 Annual Forum Paper.

ERIC Educational Resources Information Center

Delaney, Anne Marie

This paper reviews the first two years of a model program-evaluation case study which is intended to show: (1) how program evaluation can contribute to academic and professional degree programs; (2) how qualitative and quantitative techniques can be used to produce reliable measures for evaluation studies; and (3) how the role of the institutional…

A Qualitative Evaluation of an Online Expert-Facilitated Course on Tobacco Dependence Treatment.

PubMed

Ebn Ahmady, Arezoo; Barker, Megan; Dragonetti, Rosa; Fahim, Myra; Selby, Peter

2017-01-01

Qualitative evaluations of courses prove difficult due to low response rates. Online courses may permit the analysis of qualitative feedback provided by health care providers (HCPs) during and after the course is completed. This study describes the use of qualitative methods for an online continuing medical education (CME) course through the analysis of HCP feedback for the purpose of quality improvement. We used formative and summative feedback from HCPs about their self-reported experiences of completing an online expert-facilitated course on tobacco dependence treatment (the Training Enhancement in Applied Cessation Counselling and Health [TEACH] Project). Phenomenological, inductive, and deductive approaches were applied to develop themes. QSR NVivo 11 was used to analyze the themes derived from free-text comments and responses to open-ended questions. A total of 277 out of 287 participants (96.5%) completed the course evaluations and provided 690 comments focused on how to improve the program. Five themes emerged from the formative evaluations: overall quality, content, delivery method, support, and time. The majority of comments (22.6%) in the formative evaluation expressed satisfaction with overall course quality. Suggestions for improvement were mostly for course content and delivery method (20.4% and 17.8%, respectively). Five themes emerged from the summative evaluation: feedback related to learning objectives, interprofessional collaboration, future topics of relevance, overall modifications, and overall satisfaction. Comments on course content, website function, timing, and support were the identified areas for improvement. This study provides a model to evaluate the effectiveness of online educational interventions. Significantly, this constructive approach to evaluation allows CME providers to take rapid corrective action.

A Qualitative Evaluation of an Online Expert-Facilitated Course on Tobacco Dependence Treatment

PubMed Central

Ebn Ahmady, Arezoo; Barker, Megan; Dragonetti, Rosa; Fahim, Myra; Selby, Peter

2017-01-01

Qualitative evaluations of courses prove difficult due to low response rates. Online courses may permit the analysis of qualitative feedback provided by health care providers (HCPs) during and after the course is completed. This study describes the use of qualitative methods for an online continuing medical education (CME) course through the analysis of HCP feedback for the purpose of quality improvement. We used formative and summative feedback from HCPs about their self-reported experiences of completing an online expert-facilitated course on tobacco dependence treatment (the Training Enhancement in Applied Cessation Counselling and Health [TEACH] Project). Phenomenological, inductive, and deductive approaches were applied to develop themes. QSR NVivo 11 was used to analyze the themes derived from free-text comments and responses to open-ended questions. A total of 277 out of 287 participants (96.5%) completed the course evaluations and provided 690 comments focused on how to improve the program. Five themes emerged from the formative evaluations: overall quality, content, delivery method, support, and time. The majority of comments (22.6%) in the formative evaluation expressed satisfaction with overall course quality. Suggestions for improvement were mostly for course content and delivery method (20.4% and 17.8%, respectively). Five themes emerged from the summative evaluation: feedback related to learning objectives, interprofessional collaboration, future topics of relevance, overall modifications, and overall satisfaction. Comments on course content, website function, timing, and support were the identified areas for improvement. This study provides a model to evaluate the effectiveness of online educational interventions. Significantly, this constructive approach to evaluation allows CME providers to take rapid corrective action. PMID:28992759

The Evaluator's Perspective: Evaluating the State Capacity Building Program.

ERIC Educational Resources Information Center

Madey, Doren L.

A historical antagonism between the advocates of quantitative evaluation methods and the proponents of qualitative evaluation methods has stymied the recognition of the value to be gained by utilizing both methodologies in the same study. The integration of quantitative and qualitative methods within a single evaluation has synergistic effects in…

Impact of Participation in TimeSlips, a Creative Group-Based Storytelling Program, on Medical Student Attitudes Toward Persons With Dementia: A Qualitative Study

PubMed Central

George, Daniel R.; Stuckey, Heather L.; Dillon, Caroline F.; Whitehead, Megan M.

2011-01-01

Purpose: To evaluate whether medical student participation in TimeSlips (TS), a creative group-based storytelling program, with persons affected by dementia would improve student attitudes toward this patient population. Design and Methods: Fifteen fourth-year medical students from Penn State College of Medicine participated in a month-long regimen of TS sessions at a retirement community. Student course evaluations were analyzed at the conclusion of the program to examine perceived qualitative changes in attitude. Findings: Qualitative data revealed insights into the manner in which student attitudes toward a geriatric patient population became more positive. Implications: This is the first known pilot study to suggest that participation in a creative group-based storytelling program might improve medical student attitudes toward persons with dementia. PMID:21665958

A Qualitative Evaluation of Student Learning and Skills Use in a School-Based Mindfulness and Yoga Program

PubMed Central

Mirabal-Beltran, Roxanne; Cluxton-Keller, Fallon; Gould, Laura Feagans; Greenberg, Mark T.; Mendelson, Tamar

2016-01-01

Previous studies on school-based mindfulness and yoga programs have focused primarily on quantitative measurement of program outcomes. This study used qualitative data to investigate program content and skills that students remembered and applied in their daily lives. Data were gathered following a 16-week mindfulness and yoga intervention delivered at three urban schools by a community non-profit organization. We conducted focus groups and interviews with nine classroom teachers who did not participate in the program and held six focus groups with 22 fifth and sixth grade program participants. This study addresses two primary research questions: (1) What skills did students learn, retain, and utilize outside the program? and (2) What changes did classroom teachers expect and observe among program recipients? Four major themes related to skill learning and application emerged as follows: (1) youths retained and utilized program skills involving breath work and poses; (2) knowledge about health benefits of these techniques promoted self-utilization and sharing of skills; (3) youths developed keener emotional appraisal that, coupled with new and improved emotional regulation skills, helped de-escalate negative emotions, promote calm, and reduce stress; and (4) youths and teachers reported realistic and optimistic expectations for future impact of acquired program skills. We discuss implications of these findings for guiding future research and practice. PMID:26918064

Health Care Merged With Senior Housing: Description and Evaluation of a Successful Program.

PubMed

Barry, Theresa Teta

2017-01-01

Objective: This article describes and evaluates a successful partnership between a large health care organization and housing for seniors. The program provides on-site, primary care visits by a physician and a nurse in addition to intensive social services to residents in an affordable senior housing apartment building located in Pennsylvania. Per Donabedian's "Structure-Process-Outcome" model, the program demonstrated positive health care outcomes for its participants via a prescribed structure. To provide guidance for replication in similar settings, we qualitatively evaluated the processes by which successful outcomes were obtained. Methods: With program structures in place and outcomes measured, this case study collected and analyzed qualitative information taken from key informant interviews on care processes involved in the program. Themes were extracted from semistructured interviews and used to describe the processes that helped and hindered the program. Results and Discussion: Common processes were identified across respondents; however, the nuanced processes that lead to successful outcomes suggest that defined structures and processes may not be sufficient to produce similar outcomes in other settings. Further research is needed to determine the program's replicability and policy implications.

Problems and challenges of nursing students’ clinical evaluation: A qualitative study

PubMed Central

Rafiee, Ghazanfar; Moattari, Marzieh; Nikbakht, Alireza N; Kojuri, Javad; Mousavinasab, Masoud

2014-01-01

Background: The purpose of this qualitative exploratory study was to explore the views of nursing trainers and students about nursing students’ clinical evaluation problems and drawbacks in Shiraz Nursing and Midwifery School. Materials and Methods: A qualitative exploratory approach was used in this study at Shiraz Nursing and Midwifery School in 2012. A purposeful sample of 8 nursing instructors and 40 nursing students was interviewed and the data on their opinions about the problems of the clinical evaluation were collected through semi-structured deep interviews. Initially, four open-ended questions, which were related to the clinical evaluation status, problems, were used to stimulate discussions in the interview sessions. Content analysis was employed in order to analyze the transcribed data. The recorded interviews were initially transcribed, read, and reread on a number of occasions to get an overall feeling of what the participants were saying. Each line or incident was described, and then a code, which reflected the essence of the participants’ comments, was given. Results: The codes were compared for similarity and differences, merged together, and categorized. Finally, five themes emerged: In appropriate clinical evaluation method, problems of clinical evaluation Process, problems related to clinical instructors, unsuitable programming of clinical education, and organizational shortcomings. Conclusion: Besides focusing on upgrading the current clinical evaluation forms, nursing trainers should improve their knowledge about a complete and comprehensive clinical evaluation. They should also apply other appropriate and objective clinical evaluation methods and tools, and perform a formative and summative clinical evaluation. Also, workload adjustment of the nursing trainers needs revision. Therefore, despite using traditional and sometimes limited evaluation methods for assessing nursing students, a co mprehensive and appropriate evaluation of nursing

Evaluating a physician leadership development program - a mixed methods approach.

PubMed

Throgmorton, Cheryl; Mitchell, Trey; Morley, Tom; Snyder, Marijo

2016-05-16

Purpose - With the extent of change in healthcare today, organizations need strong physician leaders. To compensate for the lack of physician leadership education, many organizations are sending physicians to external leadership programs or developing in-house leadership programs targeted specifically to physicians. The purpose of this paper is to outline the evaluation strategy and outcomes of the inaugural year of a Physician Leadership Academy (PLA) developed and implemented at a Michigan-based regional healthcare system. Design/methodology/approach - The authors applied the theoretical framework of Kirkpatrick's four levels of evaluation and used surveys, observations, activity tracking, and interviews to evaluate the program outcomes. The authors applied grounded theory techniques to the interview data. Findings - The program met targeted outcomes across all four levels of evaluation. Interview themes focused on the significance of increasing self-awareness, building relationships, applying new skills, and building confidence. Research limitations/implications - While only one example, this study illustrates the importance of developing the evaluation strategy as part of the program design. Qualitative research methods, often lacking from learning evaluation design, uncover rich themes of impact. The study supports how a PLA program can enhance physician learning, engagement, and relationship building throughout and after the program. Physician leaders' partnership with organization development and learning professionals yield results with impact to individuals, groups, and the organization. Originality/value - Few studies provide an in-depth review of evaluation methods and outcomes of physician leadership development programs. Healthcare organizations seeking to develop similar in-house programs may benefit applying the evaluation strategy outlined in this study.

Challenges in evaluating clinical governance systems in iran: a qualitative study.

PubMed

Hooshmand, Elaheh; Tourani, Sogand; Ravaghi, Hamid; Ebrahimipour, Hossein

2014-04-01

In spite of the pivotal role of clinical governance in enhancing quality of services provided by hospitals across the country, a scientific framework with specific criteria for evaluating hospitals has not been developed so far. This study was conducted with the aim to identify the challenges involved in evaluating systems of clinical governance in Iran. For the purposes of this qualitative study, 15 semi-structured interviews with experts in the field were conducted in 2011 and the data were analyzed using framework analysis method. Five major challenges in evaluating clinical governance include managing human resources, improving clinical quality, managing development, organizing clinical governance, and providing patient-oriented healthcare system. Healthcare system in Iran requires a clinical governance program which has a patient-oriented approach in philosophy, operation, and effectiveness in order to meet the challenges ahead.

An Evaluation of a Program to Help Dual-Earner Couples Share the Second Shift.

ERIC Educational Resources Information Center

Hawkins, Alan J.; And Others

1994-01-01

Used both traditional scientific and feminist methodologies to evaluate effectiveness of family life education program designed to help dual-earner couples (n=14 couples) share domestic labor. Both quantitative and qualitative data suggest that program produced small increases in husbands' involvement in both housework and child care and large…

Pre-Placement Program for Severely Multi-Handicapped Deaf-Blind Children, 1980-1981. Final Evaluation Report.

ERIC Educational Resources Information Center

Tobias, Robert; And Others

Evaluation of the sixth and final year of operation for a preplacement program for 13 severely multiply handicapped deaf blind children, located in the Industrial Home for the Blind, is reported. The program is explained to prepare students for entrance into their existing special education programs. Qualitative findings on the physical setting,…

A qualitative study exploring adolescents' experiences with a school-based mental health program.

PubMed

Garmy, Pernilla; Berg, Agneta; Clausson, Eva K

2015-10-21

Supporting positive mental health development in adolescents is a major public health concern worldwide. Although several school-based programs aimed at preventing depression have been launched, it is crucial to evaluate these programs and to obtain feedback from participating adolescents. This study aimed to explore adolescents' experiences with a -based cognitive-behavioral depression prevention program. Eighty-nine adolescents aged 13-15 years were divided into 12 focus groups. The focus group interviews were analyzed using qualitative content analysis. Three categories and eight subcategories were found to be related to the experience of the school-based program. The first category, intrapersonal strategies, consisted of the subcategories of directed thinking, improved self-confidence, stress management, and positive activities. The second category, interpersonal awareness, consisted of the subcategories of trusting the group and considering others. The third category, structural constraints, consisted of the subcategories of negative framing and emphasis on performance. The school-based mental health program was perceived as beneficial and meaningful on both individual and group levels, but students expressed a desire for a more health-promoting approach.

Motivation for Evaluation: A roadmap for Improving Program Efficacy

NASA Astrophysics Data System (ADS)

Taber, J. J.; Bohon, W.; Bravo, T. K.; Dorr, P. M.; Hubenthal, M.; Johnson, J. A.; Sumy, D. F.; Welti, R.; Davis, H. B.

2016-12-01

Over the past year, the Incorporated Research Institutions for Seismology (IRIS) Education and Public Outreach (EPO) program has undertaken a new effort to increase the rigor with which it evaluates its programs and products. More specifically we sought to make evaluation an integral part of our EPO staff's work, enable staff to demonstrate why we do the activities we do, enhance the impact or our products and programs, and empower staff to be able to make evidence-based claims. The challenges we faced included a modest budget, finding an applicable approach to both new and legacy programs ranging from formal and informal education to public outreach, and implementing the process without overwhelming staff. The Collaborative Impact Analysis Method (IAM; Davis and Scalice, 2015) was selected as it allowed us to combine the EPO staff's knowledge of programs, audiences and content with the expertise of an outside evaluation expert, through consultations and a qualitative rubric assessing the initial state of each product/program's evaluation. Staff then developed action plans to make incremental improvements to the evaluation of programs over time. We have found that this approach promotes the development of staff knowledge and skills regarding evaluation, provides a common language among staff, increases enthusiasm to collect and share data, encourages discussions of evaluative approaches when planning new activities, and improves each program's ability to capture the intended and unintended effects on the behaviors, attitudes, skills, interests, and/or knowledge of users/participants. We will share the initial IAM Scores for products and programs in the EPO portfolio, along with examples of the action plans for several key products and programs, and the impact that implementing those actions plans has had on our evaluations. Davis, H. & Scalice, D. (2015). Evaluate the Impact of your Education and Outreach Program Using the Quantitative Collaborative Impact Analysis

Use of Evidence in the Implementation of Social Programs: A Qualitative Study from Chile.

PubMed

Flores, Rodrigo; Naranjo, Carola; Hein, Andreas

2016-01-01

Through this qualitative, empirical study the authors aim to explore and describe the sources of knowledge that are used to guide intervention practice by social workers in Chile. Particular attention was paid to factors that may facilitate or hinder the use of research-based evidence to guide social interventions design, implementation, and outcome evaluation. In order to explore these issues, 25 semi-structured interviews with social workers from Chilean social service non-profit organizations were conducted. The main findings suggest that social workers do not use research-based evidence to support their social interventions due to various personal organizational constraints (e.g., lack of time, lack of access to resources for disseminating evidence, lack of English command). In addition, no evaluation processes of social programs which will support evidence-based effectiveness could be found. One key barrier to support use of evidence and evidence production may be related to the fact that most non-governmental organizations maintain a hierarchical and vertical relationship with state agencies (program design, oversight, and funding) for social program development.

Evaluating the effectiveness of clinical medical librarian programs: a systematic review of the literature*

PubMed Central

Wagner, Kay Cimpl; Byrd, Gary D.

2004-01-01

Objective: This study was undertaken to determine if a systematic review of the evidence from thirty years of literature evaluating clinical medical librarian (CML) programs could help clarify the effectiveness of this outreach service model. Methods: A descriptive review of the CML literature describes the general characteristics of these services as they have been implemented, primarily in teaching-hospital settings. Comprehensive searches for CML studies using quantitative or qualitative evaluation methods were conducted in the medical, allied health, librarianship, and social sciences literature. Findings: Thirty-five studies published between 1974 and 2001 met the review criteria. Most (30) evaluated single, active programs and used descriptive research methods (e.g., use statistics or surveys/questionnaires). A weighted average of 89% of users in twelve studies found CML services useful and of high quality, and 65% of users in another overlapping, but not identical, twelve studies said these services contributed to improved patient care. Conclusions: The total amount of research evidence for CML program effectiveness is not great and most of it is descriptive rather than comparative or analytically qualitative. Standards are needed to consistently evaluate CML or informationist programs in the future. A carefully structured multiprogram study including three to five of the best current programs is needed to define the true value of these services. PMID:14762460

Health in the urban environment: a qualitative review of the Brighton and Hove WHO Healthy City Program.

PubMed

Hall, Caroline; Davies, John Kenneth; Sherriff, Nigel

2010-01-01

Phase IV of the WHO European Region's Healthy Cities Program ended in December 2008. This article presents the findings from a recently completed review of Brighton and Hove's Healthy City Program which aimed to scope whether added value had accrued from the city's role as a WHO Healthy City during phase IV. In contrast to most other evaluations of healthy cities, this review adopted a qualitative approach representing an appraisal of the Brighton and Hove Healthy City Program from the internal viewpoint of its local stakeholders. In addition to documentary analysis and a facilitated workshop, a series of in-depth interviews (N = 27) were conducted with stakeholders from the Brighton and Hove Healthy City Partnership representing each of the sectors reflected in the Local Strategic Partnership (public, statutory, elected, community and voluntary, neighborhood and communities, business). The key findings of the review are presented in a way which reflects the three key areas of the review including (1) the healthy cities approach, (2) participation in phase IV of the WHO Healthy Cities Program, and (3) the Brighton and Hove Healthy City Partnership. These findings are discussed, and recommendations for action at local, national, and European levels are proposed. In particular, we argue that there is an urgent need to develop a suitable monitoring and evaluation system for the WHO Healthy Cities Program with appropriate indicators that are meaningful and relevant to local stakeholders. Moreover, it would be important for any such system to capitalize on the benefits that qualitative methodologies can offer alongside more traditional quantitative indicators.

Emerging Themes in Youth Apprenticeship Programs: A Qualitative Study.

ERIC Educational Resources Information Center

Evanciew, Cheryl E. P.

A qualitative study examined experiences between mentors and youth apprentices from the standpoint of situated cognition theory. The youth apprenticeship program studied was located in a rural southeastern community. High school youth apprentices worked with adult mentors and management personnel in selected businesses in the fields of business…

Challenges in Evaluating Clinical Governance Systems in Iran: A Qualitative Study

PubMed Central

Hooshmand, Elaheh; Tourani, Sogand; Ravaghi, Hamid; Ebrahimipour, Hossein

2014-01-01

Background: In spite of the pivotal role of clinical governance in enhancing quality of services provided by hospitals across the country, a scientific framework with specific criteria for evaluating hospitals has not been developed so far. Objectives: This study was conducted with the aim to identify the challenges involved in evaluating systems of clinical governance in Iran. Materials and Methods: For the purposes of this qualitative study, 15 semi-structured interviews with experts in the field were conducted in 2011 and the data were analyzed using framework analysis method. Results: Five major challenges in evaluating clinical governance include managing human resources, improving clinical quality, managing development, organizing clinical governance, and providing patient-oriented healthcare system. Conclusions: Healthcare system in Iran requires a clinical governance program which has a patient-oriented approach in philosophy, operation, and effectiveness in order to meet the challenges ahead. PMID:24910799

Patient Perspectives on a Text Messaging Program to Support Asthma Management: A Qualitative Study.

PubMed

Doyle, Reina; Albright, Karen; Hurley, Laura P; Chávez, Catia; Stowell, Melanie; Dircksen, Suzanne; Havranek, Edward P; Anderson, Mark

2018-05-01

This study investigated participants' acceptance of a short messaging service (SMS) intervention designed to support asthma management, including suggestions regarding program delivery and message content. Individual and group interviews were conducted with patients from a safety-net health care system in Denver, Colorado. Eligible participants were English or Spanish speakers between the ages of 13 and 40 years, with diagnosed persistent asthma. All individual and group interviews were digitally recorded, transcribed, translated from Spanish to English (where applicable), and analyzed for thematic content by experienced analysts using established qualitative content techniques. The qualitative software package ATLAS.ti was used for data analysis and management. This study included a total of 43 participants. In general, participants were receptive toward the SMS program and supported the use of tailored and interactive messages. Adolescents supported the idea of enhancing care by sending messages to a support person, such as a parent or guardian. However, adults were less receptive toward this idea. Participants also preferred directive educational messages and cues to action, while general messages reminding them of their asthma diagnosis were viewed less favorably. The results from this study will inform a randomized control trial evaluating the efficacy of the SMS intervention.

Implementing the Baby One Program: a qualitative evaluation of family-centred child health promotion in remote Australian Aboriginal communities.

PubMed

Campbell, Sandra; McCalman, Janya; Redman-MacLaren, Michelle; Canuto, Karla; Vine, Kristina; Sewter, Jenny; McDonald, Malcolm

2018-03-24

A healthy start predicts better health in later life. Many remote-living Aboriginal and Torres Strait Islander Australian families lack access to consistent, culturally-safe health services. This paper presents a study of implementation of the Baby One Program (BOP). The BOP was designed as a family-centred, Indigenous Healthworker-led, home-visiting model of care focused on promoting family health to give children the best start to life. It was developed by Aboriginal community-controlled Apunipima Cape York Health Council and delivered in Queensland Cape York remote communities. We aimed to determine how the BOP was implemented, enablers, strategies used and formative implementation outcomes. The qualitative approach utilised theoretical and purposive sampling to explore people's experiences of a program implementation process. Data were generated from semi-structured interviews with four family members enrolled in the BOP and 24 Apunipima staff members. In addition, twenty community members, including two program users, participated in a men's community focus group. The findings are presented according to themes arising from the data. The BOP was rolled out in nine remote Cape York communities between July 2014 and December 2015 and there was high uptake. Indigenous Healthworkers were supported by midwives and maternal and child health nurses to deliver health education to 161 eligible families. The key to effective implementation of family-centred care appeared to be the relationships formed between health practitioners, especially Indigenous Healthworkers, and families. The data revealed the following themes: challenging environments for new families and valuing cultural ways, resourcing program delivery, working towards a team approach, negotiating the cultural interface, engaging families, exchanging knowledge through 'yarning', strengthening the workforce, and seeing health changes in families. Healthworker education and training, and knowledge exchange

A Hospice Rotation for Military Medical Residents: A Mixed Methods, Multi-Perspective Program Evaluation

PubMed Central

Boyden, Jackelyn Y.; Kalish, Virginia B.; Muir, J. Cameron; Richardson, Suzanne; Connor, Stephen R.

2016-01-01

Abstract Background: An estimated 6,000 to 18,000 additional hospice and palliative medicine (HPM) physicians are needed in the United States. A source could be the military graduate medical education system where 15% of U.S. medical residents are trained. A community-based hospice and palliative care organization created a one-week rotation for military residents including participation in interdisciplinary group visits at patients' homes, facilities, and an inpatient hospice unit. Objective: Our goal was to evaluate the effectiveness of a one-week community HPM rotation for military medical residents. Methods: A mixed-methods, multi-stakeholder perspective program evaluation model was used for program years 2011 to 2013. Data were managed and analyzed using Microsoft Excel and Atlas.ti. Participants in the rotation were residents training at two local military hospitals. Program evaluation data were collected from residents, military program liaisons, and hospice clinical preceptors. Quantitative data included pre- and post-tests based on Accreditation Council for Graduate Medical Education competencies completed by residents. Qualitative data included resident essays and semi-structured interviews with hospice preceptors and military program liaisons. Results: Quantitative and qualitative data suggested that the rotation increased military residents' knowledge, attitudes, and comfort level with HPM. Quantitative analysis of test scores indicated improvements from pre- to post-tests in each of five areas of learning. Qualitative data indicated the rotation created a greater appreciation for the overall importance of HPM and increased understanding of eligibility and methods for pain and symptom management. Conclusions: A one-week community hospice rotation for medical military residents impacts participant's knowledge of and attitudes toward HPM. PMID:27139524

Using Mixed Methods and Collaboration to Evaluate an Education and Public Outreach Program (Invited)

NASA Astrophysics Data System (ADS)

Shebby, S.; Shipp, S. S.

2013-12-01

Traditional indicators (such as the number of participants or Likert-type ratings of participant perceptions) are often used to provide stakeholders with basic information about program outputs and to justify funding decisions. However, use of qualitative methods can strengthen the reliability of these data and provide stakeholders with more meaningful information about program challenges, successes, and ultimate impacts (Stern, Stame, Mayne, Forss, David & Befani, 2012). In this session, presenters will discuss how they used a mixed methods evaluation to determine the impact of an education and public outreach (EPO) program. EPO efforts were intended to foster more effective, sustainable, and efficient utilization of science discoveries and learning experiences through three main goals 1) increase engagement and support by leveraging of resources, expertise, and best practices; 2) organize a portfolio of resources for accessibility, connectivity, and strategic growth; and 3) develop an infrastructure to support coordination. The evaluation team used a mixed methods design to conduct the evaluation. Presenters will first discuss five potential benefits of mixed methods designs: triangulation of findings, development, complementarity, initiation, and value diversity (Greene, Caracelli & Graham, 2005). They will next demonstrate how a 'mix' of methods, including artifact collection, surveys, interviews, focus groups, and vignettes, was included in the EPO project's evaluation design, providing specific examples of how alignment between the program theory and the evaluation plan was best achieved with a mixed methods approach. The presentation will also include an overview of different mixed methods approaches and information about important considerations when using a mixed methods design, such as selection of data collection methods and sources, and the timing and weighting of quantitative and qualitative methods (Creswell, 2003). Ultimately, this presentation will

Qualitative Parameters for Evaluation Procedures of Non-Formal and Informal Learning Achievements

ERIC Educational Resources Information Center

Stasiunaitiene, Egle; Kaminskiene, Lina

2009-01-01

The article introduces evaluation principles of non-formal and informal learning that determine the quality of evaluation, describes stages of the evaluation procedure, differentiates their qualitative parameters and defines their criteria and indicators. It also brings in the discussion that consideration of qualitative parameters for the…

The Design and Pilot Evaluation of an Interactive Learning Environment for Introductory Programming Influenced by Cognitive Load Theory and Constructivism

ERIC Educational Resources Information Center

Moons, Jan; De Backer, Carlos

2013-01-01

This article presents the architecture and evaluation of a novel environment for programming education. The design of this programming environment, and the way it is used in class, is based on the findings of constructivist and cognitivist learning paradigms. The environment is evaluated based on qualitative student and teacher evaluations and…

Perceptions of ESL Program Management in Canadian Higher Education: A Qualitative Case Study

ERIC Educational Resources Information Center

Eaton, Sarah Elaine

2017-01-01

ESL programs at post-secondary institutions must often generate revenue in addition to teaching students English. Institutions often impose explicit expectations on these programs to generate profit, creating unique challenges for those who administer them. This qualitative case study investigated challenges faced by ESL program directors at one…

Computing in Qualitative Analysis: A Healthy Development?

ERIC Educational Resources Information Center

Richards, Lyn; Richards, Tom

1991-01-01

Discusses the potential impact of computers in qualitative health research. Describes the original goals, design, and implementation of NUDIST, a qualitative computing software. Argues for evaluation of the impact of computer techniques and for an opening of debate among program developers and users to address the purposes and power of computing…

Features of a Health-Oriented Education Program during Daily Commutes: A Qualitative Study.

PubMed

Ramezankhani, Ali; Heydarabadi, Akbar Babaei; Ghaffari, Mohtasham; Mehrabi, Yadollah; Kazemi, Sadegh

2016-06-01

Today, despite scientific advances, many people spend more time and distance between home and their workplaces because of various economic and population reasons. The aim of this study was to identify features of an appropriate health education program during commutes for factory staff at Ardakan county (Yazd province, Iran). This qualitative study was conducted via the phenomenological method in 2014. The population of this study was members of the staff of Ardakan Steel Company. Nineteen specialists and 11 members of the factory's staff were invited to participate in the study, and data were collected using semi-structured interviews. The interviews took 20 to 40 minutes, and their content was analyzed using content analysis. Extraction of codes and themes and their placement in this study showed that an educational program during commutes should have nine features to have the desired effectiveness, i.e., the program must be audience-oriented, repeatable, participatory, technology-based, combinational, supportive, and motivational and interesting. Also, the program should have environmental and organizational support, and it must be evaluated for its effectiveness. Considering appropriate features of a health education program in educational situations, especially interventions related to daily commutes, is very important because the effectiveness of such health-oriented educational programs must be ensured.

Alumni Perspectives on Career Preparation during a Postdoctoral Training Program: A Qualitative Study

PubMed Central

Faupel-Badger, Jessica M.; Raue, Kimberley; Nelson, David E.; Tsakraklides, Sophia

2015-01-01

Published evaluations of career preparation of alumni from long-standing postdoctoral fellowship programs in the biomedical sciences are limited and often focus on quantitative analysis of data from extant publicly available sources. Qualitative methods provide the opportunity to gather robust information about specific program elements from structured postdoctoral training programs and the influence of this training on subsequent career paths of alumni. In-depth interviews with a subset of the National Cancer Institute’s Cancer Prevention Fellowship Program (CPFP) alumni (n = 27), representing more than 25 years of the program’s history and multiple career sectors, were conducted to assess alumni reflections on the training environment and career preparation during their time in the CPFP. NVivo software was used to analyze data and identify major themes. Four main themes emerged from these interviews, including: the value of structured training curriculum, mentorship, transdisciplinary environment, and professional identity. Even when reflecting on training that occurred one to two decades earlier, alumni were able to highlight specific components of a structured postdoctoral training program as influencing their research and career trajectories. These results may have relevance for those interested in assessing how postdoctoral training can influence fellows throughout their careers and understanding salient features of structured programs. PMID:25673353

A Qualitative Examination of Challenges Influencing Doctoral Students in an Online Doctoral Program

ERIC Educational Resources Information Center

Deshpande, Anant

2016-01-01

The main purpose of the study was to investigate the challenges faced by students in completion of an online doctoral program at the University of Liverpool, Online Doctoral Business Administration program. We analyse the responses of 91 doctoral students in an online DBA program. Based on the exploratory qualitative study themes were developed…

Education Model Program on Water-Energy Research: A New STEM Graduate Program from Development through Evaluation

NASA Astrophysics Data System (ADS)

McCay, D.; Fiorenza, P.; Lautz, L.

2017-12-01

More than half of Ph.D. scientists and engineers find employment in non-academic sectors. Recognizing the range of career options for graduate degree holders and the need to align graduate education with the expectations of prospective employers, the National Science Foundation (NSF) created the NSF Research Traineeship (NRT) program. To date, over 100 NRT programs have been funded. As these programs are implemented, it is important to assess their progress, successes, and challenges. This presentation describes the ongoing evaluation of one NRT program, "Education Model Program on Water-Energy Research" (or EMPOWER) at Syracuse University. Through seminars, mini-grants, professional development activities, field courses, internship opportunities, and coursework, EMPOWER's goal is to equip students with the skills needed for the range of career options in water and energy. In collaboration with an external evaluator, EMPOWER is examining the fidelity of the program to proposed goals, providing feedback to inform project improvement (formative assessment) and assessing the effectiveness of achieving program goals (summative assessment). Using a convergent parallel mixed method design, qualitative and quantitative data were collected to develop a full assessment of the first year of the program. Evaluation findings have resulted in several positive changes to the program. For example, EMPOWER students perceive themselves to have high technical skills, but the data show that the students do not believe that they have a strong professional network. Based on those findings, EMPOWER offered several professional development events focused on building one's professional network. Preliminary findings have enabled the EMPOWER leadership team to make informed decisions about the ways the program elements can be redesigned to better meet student needs, about how to the make the program more effective, and determine the program elements that may be sustained beyond the funding

Qualitative evaluation of a physical activity-based chronic disease prevention program in a low-income, rural South African setting.

PubMed

Draper, Catherine E; Nemutandani, Simon M; Grimsrud, Anna T; Rudolph, Michael; Kolbe-Alexander, Tracy L; de Kock, Lauren; Lambert, Estelle V

2010-01-01

Chronic diseases, an increasing global concern, are prevalent in the low-income communities of South Africa, where rural health systems bear the double burden of infectious and chronic diseases. The Discovery Healthy Lifestyle Programme (DHLP) is a physical activity-based chronic disease prevention program that has been implemented in a low-income, rural setting in South Africa. The DHLP consists of both school- and primary healthcare clinic-based interventions for learners (Healthnutz) and adults (Live it Up), facilitated by teachers, nurses and community volunteers. The aim of this evaluation was to qualitatively assess the process by which the DHLP was implemented, identifying enabling factors and barriers. Data were collected in target communities at schools and clinics from semi-structured focus groups of program leaders and members, teachers and community members (n = 45), situational analyses of the school physical activity environment, informal community observations and informal interviews with program coordinators. The target communities faced socioeconomic and health inequalities and remained under-resourced and under-served. In spite of these and other challenges, the DHLP was well received by community members and stakeholders. It was valued by respondents for its health and psychosocial outcomes, evidenced by increased knowledge and awareness of the importance of physical activity and healthy lifestyles, and positively altered perceptions of physical activity. Program implementers believed the Live it Up component was growing, and this suggested the sustainability of the program. There were, however, some concerns about the fidelity of the Healthnutz intervention, due to timetabling difficulties. Despite this, teachers were positive about the program and its value for their learners, staff and school. The community characteristics of being under-resourced and under-served appeared to positively influence DHLP implementation. Local government

Potential pros and cons of external healthcare performance evaluation systems: real-life perspectives on Iranian hospital evaluation and accreditation program.

PubMed

Jaafaripooyan, Ebrahim

2014-09-01

Performance evaluation is essential to quality improvement in healthcare. The current study has identified the potential pros and cons of external healthcare evaluation programs, utilizing them subsequently to look into the merits of a similar case in a developing country. A mixed method study employing both qualitative and quantitative data collection and analysis techniques was adopted to achieve the study end. Subject Matter Experts (SMEs) and professionals were approached for two-stage process of data collection. Potential advantages included greater attractiveness of high accreditation rank healthcare organizations to their customers/purchasers and boosted morale of their personnel. Downsides, as such, comprised the programs' over-reliance on value judgment of surveyors, routinization and incurring undue cost on the organizations. In addition, the improved, standardized care processes as well as the judgmental nature of program survey were associated, as pros and cons, to the program investigated by the professionals. Besides rendering a tentative assessment of Iranian hospital evaluation program, the study provides those running external performance evaluations with a lens to scrutinize the virtues of their own evaluation systems through identifying the potential advantages and drawbacks of such programs. Moreover, the approach followed could be utilized for performance assessment of similar evaluation programs.

Experienced quality factors: qualitative evaluation approach to audiovisual quality

NASA Astrophysics Data System (ADS)

Jumisko-Pyykkö, Satu; Häkkinen, Jukka; Nyman, Göte

2007-02-01

Subjective evaluation is used to identify impairment factors of multimedia quality. The final quality is often formulated via quantitative experiments, but this approach has its constraints, as subject's quality interpretations, experiences and quality evaluation criteria are disregarded. To identify these quality evaluation factors, this study examined qualitatively the criteria participants used to evaluate audiovisual video quality. A semi-structured interview was conducted with 60 participants after a subjective audiovisual quality evaluation experiment. The assessment compared several, relatively low audio-video bitrate ratios with five different television contents on mobile device. In the analysis, methodological triangulation (grounded theory, Bayesian networks and correspondence analysis) was applied to approach the qualitative quality. The results showed that the most important evaluation criteria were the factors of visual quality, contents, factors of audio quality, usefulness - followability and audiovisual interaction. Several relations between the quality factors and the similarities between the contents were identified. As a research methodological recommendation, the focus on content and usage related factors need to be further examined to improve the quality evaluation experiments.

Formative Qualitative Evaluation for "Exploratory" ITS Research.

ERIC Educational Resources Information Center

Murray, Tom

1993-01-01

Discusses evaluation methods applicable to exploratory research areas, provides an overview of qualitative and formative methods for exploratory research on intelligent tutoring systems (ITS) and describes an exploratory study in ITS knowledge acquisition which involved working with three educators to build an ITS for high school physics.…

Physiotherapists supporting self-management through health coaching: a mixed methods program evaluation.

PubMed

Dufour, Sinéad Patricia; Graham, Shane; Friesen, Josh; Rosenblat, Michael; Rous, Colin; Richardson, Julie

2015-01-01

To evaluate a program in support of chronic disease self-management (CDSM) that is founded on a health coaching (HC) approach, includes supervised exercise and mindfulness-based stress reduction components and is delivered within a private practice physiotherapy setting. An explanatory mixed method design, framed by theory-based program evaluation, was employed to evaluate an eight-week group-based program. Standardized self-rated and performance measures were evaluated pre- and post intervention. Additionally, participant focus groups were conducted following the intervention period. An inductive thematic approach was undertaken to analyze the qualitative data. Seventeen participants (N = 17) completed the study. Improvements were seen in both self-report and performance outcomes. Participants explained how and why they felt the program was beneficial. Six themes were generated: (1) group dynamic; (2) learning versus doing; (3) holism and comprehensive care; (4) self-efficacy and empowerment; (5) previous solutions versus new management strategies; and (6) healthcare provider support. This study established that a group program in support of CDSM founded on a HC approach demonstrated potential value from participants as well as favorable outcomes. A pragmatic randomized control trial is required to determine efficacy of this intervention.

PULSAR: A Qualitative Study of a Substance Abuse Prevention Program

ERIC Educational Resources Information Center

Martino-McAllister, Jeanne M.

2004-01-01

The purpose of this study was to explore the risk, protective factors, and resiliency characteristics of students selected to participate in the Police, Public Educators and Peers Utilizing the Leadership Skills of Students At Risk/As Resources (PULSAR) program. The study is significant as it employed qualitative methods and a resiliency-focused…

Potential pros and cons of external healthcare performance evaluation systems: real-life perspectives on Iranian hospital evaluation and accreditation program

PubMed Central

Jaafaripooyan, Ebrahim

2014-01-01

Background: Performance evaluation is essential to quality improvement in healthcare. The current study has identified the potential pros and cons of external healthcare evaluation programs, utilizing them subsequently to look into the merits of a similar case in a developing country. Methods: A mixed method study employing both qualitative and quantitative data collection and analysis techniques was adopted to achieve the study end. Subject Matter Experts (SMEs) and professionals were approached for two-stage process of data collection. Results: Potential advantages included greater attractiveness of high accreditation rank healthcare organizations to their customers/purchasers and boosted morale of their personnel. Downsides, as such, comprised the programs’ over-reliance on value judgment of surveyors, routinization and incurring undue cost on the organizations. In addition, the improved, standardized care processes as well as the judgmental nature of program survey were associated, as pros and cons, to the program investigated by the professionals. Conclusion: Besides rendering a tentative assessment of Iranian hospital evaluation program, the study provides those running external performance evaluations with a lens to scrutinize the virtues of their own evaluation systems through identifying the potential advantages and drawbacks of such programs. Moreover, the approach followed could be utilized for performance assessment of similar evaluation programs. PMID:25279381

Process Evaluation for Improving K12 Program Effectiveness: Case Study of a National Institutes of Health Building Interdisciplinary Research Careers in Women's Health Research Career Development Program.

PubMed

Raymond, Nancy C; Wyman, Jean F; Dighe, Satlaj; Harwood, Eileen M; Hang, Mikow

2018-06-01

Process evaluation is an important tool in quality improvement efforts. This article illustrates how a systematic and continuous evaluation process can be used to improve the quality of faculty career development programs by using the University of Minnesota's Building Interdisciplinary Research Careers in Women's Health (BIRCWH) K12 program as an exemplar. Data from a rigorous process evaluation incorporating quantitative and qualitative measurements were analyzed and reviewed by the BIRCWH program leadership on a regular basis. Examples are provided of how this evaluation model and processes were used to improve many aspects of the program, thereby improving scholar, mentor, and advisory committee members' satisfaction and scholar outcomes. A rigorous evaluation plan can increase the effectiveness and impact of a research career development plan.

An Evaluation of the Federal Class-Size Reduction Program in Wake County, North Carolina--1999-2000.

ERIC Educational Resources Information Center

Scudder, David F.

An empirical evaluation of the federal class-size reduction (CSR) program in Wake County, North Carolina, during the 1999-2000 school year is presented. The qualitative process evaluation showed implementation issues involving the mechanics and the meaning of CSR. Often, schools did not understand where CSR occurred because of changing enrollment…

Career Development in Language Education Programs

ERIC Educational Resources Information Center

Shawer, Saad Fathy; Alkahtani, Saad Ali

2013-01-01

This study assesses the influence of a two-year language program evaluation on program directors and faculty career development. The study makes use of mixed-paradigms (positivism and qualitative interpretive), mixed-strategies (survey research and qualitative evaluation), one-way analysis of variance (ANOVA) and a post-hoc test of multiple…

Cochrane Qualitative and Implementation Methods Group guidance series-paper 6: reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses.

PubMed

Flemming, Kate; Booth, Andrew; Hannes, Karin; Cargo, Margaret; Noyes, Jane

2018-05-01

To outline contemporary and novel developments for the presentation and reporting of syntheses of qualitative, implementation, and process evaluation evidence and provide recommendations for the use of reporting guidelines. An overview of reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses drawing on current international literature and the collective expert knowledge of the Cochrane Qualitative and Implementation Methods Group. Several reporting guidelines exist that can be used or adapted to report syntheses of qualitative, implementation, and process evaluation evidence. Methods to develop individual guidance varied. The use of a relevant reporting guideline can enhance the transparency, consistency, and quality of reporting. Guidelines that exist are generic, method specific, and for particular aspects of the reviewing process, searching. Caution is expressed over the potential for reporting guidelines to produce a mechanistic approach moving the focus away from the content and toward the procedural aspects of the review. The use of a reporting guideline is recommended and a five-step decision flowchart to guide the choice of reporting guideline is provided. Gaps remain in method-specific reporting guidelines such as mixed-study, implementation, and process evaluation evidence syntheses. Copyright © 2017 Elsevier Inc. All rights reserved.

Process Evaluation of a Workers' Health Surveillance Program for Meat Processing Workers.

PubMed

van Holland, Berry J; Brouwer, Sandra; de Boer, Michiel R; Reneman, Michiel F; Soer, Remko

2017-09-01

Objective To evaluate the implementation process of a workers' health surveillance (WHS) program in a Dutch meat processing company. Methods Workers from five plants were eligible to participate in the WHS program. The program consisted of four evaluative components and an intervention component. Qualitative and quantitative methods were used to evaluate seven process aspects. Data were gathered by interviews with stakeholders, participant questionnaires, and from registries of the company and occupational health service. Results Two recruitment strategies were used: open invitation or automatic participation. Of the 986 eligible workers, 305 participated in the program. Average reach was 53 %. Two out of five program components could not be assessed on dose delivered, dose received and fidelity. If components were assessable, 85-100 % of the components was delivered, 66-100 % of the components was received by participants, and fidelity was 100 %. Participants were satisfied with the WHS program (mean score 7.6). Contextual factors that facilitated implementation were among others societal developments and management support. Factors that formed barriers were program novelty and delayed follow-up. Conclusion The WHS program was well received by participants. Not all participants were offered the same number of program components, and not all components were performed according to protocol. Deviation from protocol is an indication of program failure and may affect program effectiveness.

Facing the fear of failure: An explorative qualitative study of client experiences in a mindfulness-based stress reduction program for university students with academic evaluation anxiety.

PubMed

Hjeltnes, Aslak; Binder, Per-Einar; Moltu, Christian; Dundas, Ingrid

2015-01-01

The aim of this qualitative study was to investigate the subjective experiences of 29 university students who participated in an 8-week mindfulness-based stress reduction (MBSR) program for academic evaluation anxiety. Participants who self-referred to the Student Counseling Service underwent individual semi-structured interviews about how they experienced the personal relevance and practical usefulness of taking the MBSR program. Interviews were transcribed and analyzed through a team-based explorative-reflective thematic approach based on a hermeneutic-phenomenological epistemology. Five salient patterns of meaning (themes) were found: (1) finding an inner source of calm, (2) sharing a human struggle, (3) staying focused in learning situations, (4) moving from fear to curiosity in academic learning, and (5) feeling more self-acceptance when facing difficult situations. We contextualize these findings in relation to existing research, discuss our own process of reflexivity, highlight important limitations of this study, and suggest possible implications for future research.

Facing the fear of failure: An explorative qualitative study of client experiences in a mindfulness-based stress reduction program for university students with academic evaluation anxiety

PubMed Central

Hjeltnes, Aslak; Binder, Per-Einar; Moltu, Christian; Dundas, Ingrid

2015-01-01

The aim of this qualitative study was to investigate the subjective experiences of 29 university students who participated in an 8-week mindfulness-based stress reduction (MBSR) program for academic evaluation anxiety. Participants who self-referred to the Student Counseling Service underwent individual semi-structured interviews about how they experienced the personal relevance and practical usefulness of taking the MBSR program. Interviews were transcribed and analyzed through a team-based explorative–reflective thematic approach based on a hermeneutic-phenomenological epistemology. Five salient patterns of meaning (themes) were found: (1) finding an inner source of calm, (2) sharing a human struggle, (3) staying focused in learning situations, (4) moving from fear to curiosity in academic learning, and (5) feeling more self-acceptance when facing difficult situations. We contextualize these findings in relation to existing research, discuss our own process of reflexivity, highlight important limitations of this study, and suggest possible implications for future research. PMID:26297629

Qualitative process evaluation of an Australian alcohol media literacy study: recommendations for designing culturally responsive school-based programs.

PubMed

Gordon, Chloe S; Kervin, Lisa K; Jones, Sandra C; Howard, Steven J

2017-02-02

Alcohol media literacy programs seek to mitigate the potentially harmful effects of alcohol advertising on children's drinking intentions and behaviours through equipping them with skills to challenge media messages. In order for such programs to be effective, the teaching and learning experiences must be tailored to their specific cultural context. Media in the Spotlight is an alcohol media literacy program aimed at 9 to 12 year old Australian children. This study evaluates the process and implementation of the program, outlining the factors that facilitated and inhibited implementation. From this evaluation, a pedagogical framework has been developed for health professionals implementing culturally responsive programs in school settings. Process measures included: semi-structured interviews with teachers before and after the program was implemented (n = 11 interviews), program evaluation questionnaires completed by children (n = 166), lesson observations completed by teachers (n = 35 observations), and reflective journal entries completed by the researcher (n = 44 entries). A thematic analysis approach was used to analyse all of the data sets using NVivo. Inductive coding was used, whereby the findings were derived from the research objectives and multiple readings and interpretations of the data. Five key pedagogical considerations were identified that facilitated implementation. These were: connecting to the students' life worlds to achieve cultural significance; empowering students with real-world skills to ensure relevance; ensuring programs are well structured with strong connections to the school curriculum; creating developmentally appropriate activities while providing a range of assessment opportunities; and including hands-on and interactive activities to promote student engagement. Three potential inhibitors to implementing the alcohol media literacy program in upper-elementary school classrooms were identified. These included topic

Evaluating the Effectiveness of READ 180: A Reading Program for Struggling Middle School Students

ERIC Educational Resources Information Center

Johnson, Latonya R.

2011-01-01

This study was conducted using a qualitative approach to evaluate the effectiveness of READ 180 Program in improving students reading interests and achievement at West Bolivar Middle School (WBMS). A number of interviews and observations were conducted. Two teachers and nine students were interviewed, and nine classroom observations were…

Program evaluation of Protovation Camp

NASA Astrophysics Data System (ADS)

Healy, Laurel Lynell Martin

The purpose of this program evaluation was to determine the extent to which Protovation Camp utilized the combined resources of multiple institutions to impact student learning in science, technology, engineering, and math. The partnership consisted of multiple institutions: the university, providing graduate students to facilitate inquiry-based lessons; the science center, allowing the use of their facilities and resources; and the elementary school, contributing rising third through fifth grade campers. All of these components were examined. The mixed-methods approach used post hoc quantitative data for campers, which consisted of pre-test and post-test scores on the Test of Science-Related Attitudes (TOSRA), the Draw-A-Scientist Test, and content tests based on the camp activities. Additionally, TOSRA scores and current survey results for the graduate students were used along with qualitative data collected from plusdelta charts to determine the impact of participation in Protovation Camp on teachers and students. Results of the program evaluation indicated that when students were taught inquiry-based lessons that ignite wonder, both their attitudes toward science and their knowledge about science improved. An implication for teacher preparation programs was that practicing inquiry-based lessons on actual students (campers) was an important component for teachers (graduate students) as they prepare to positively impact student learning in their own classrooms. Immediate feedback from the campers in the form of pre-test and post-test scores and from peers on plusdelta charts allowed the graduate students the opportunity to make needed adjustments to improve effectiveness before using the lesson with a new set of campers or later in their own classrooms. Keywords. Teacher preparation, Inquiry-based instruction, STEM instructions, University and museum partnerships

Teachers' Perceptions of Georgia's Early Reading Intervention Program: A Qualitative Study

ERIC Educational Resources Information Center

Hobby, Patti Tennant

2012-01-01

This qualitative study used an ethnographic method to investigate teachers' perceptions of reading interventions delivered in a state-funded early intervention program (EIP). Academically at-risk students struggle to meet grade-level standards year after year, even with interventions involving small group, targeted assistance. Teacher perceptions…

Evaluation of a Passive Nature Viewing Program Set to Music.

PubMed

Cadman, Sally J

2014-09-01

Research has revealed that passive nature viewing (viewing nature scenes without actually being in nature) has many health benefits but little is known about the best method of offering this complementary modality. The purpose of this pilot program was to evaluate the impact of a passive nature viewing program set to music on stress reduction in adults living in the community. A pre- and postsurvey design along with weekly recordings of stress and relaxation levels were used to evaluate the effect of this passive nature viewing program on stress reduction. Participants watched one of three preselected nature scenes for 5 minutes a day over 1 month and rated their stress and relaxation levels weekly on a 100-mm Visual Analogue Scale before and after viewing the nature DVD. Quantitative analysis were not performed because of the less number of subjects (n = 10) completing the study. Qualitative analysis found five key categories that have an impact on program use: (a) technology, (b) personal preferences, (c) time, (d) immersion, and (e) use of the program. Holistic nurses may consider integrating patient preferences and immersion strategies in the design of future passive nature viewing programs to reduce attrition and improve success. © The Author(s) 2013.

Evaluating Educational Programs.

ERIC Educational Resources Information Center

Ball, Samuel

The activities of Educational Testing Service (ETS) in evaluating educational programs are described. Program evaluations are categorized as needs assessment, formative evaluation, or summative evaluation. Three classic efforts which illustrate the range of ETS' participation are the Pennsylvania Goals Study (1965), the Coleman Report--Equality of…

A Qualitative Evaluation of Engagement and Attrition in a Nurse Home Visiting Program: From the Participant and Provider Perspective.

PubMed

Beasley, Lana O; Ridings, Leigh E; Smith, Tyler J; Shields, Jennifer D; Silovsky, Jane F; Beasley, William; Bard, David

2018-05-01

Beginning parenting programs in the prenatal and early postnatal periods have a large potential for impact on later child and maternal outcomes. Home-based parenting programs, such as the Nurse Family Partnership (NFP), have been established to help address this need. Program reach and impact is dependent on successful engagement of expecting mothers with significant risks; however, NFP attrition rates remain high. The current study qualitatively examined engagement and attrition from the perspectives of NFP nurses and mothers in order to identify mechanisms that enhance service engagement. Semi-structured interviews were conducted in focus groups composed of either engaged (27 total mothers) or unengaged (15 total mothers) mothers from the NFP program. NFP nurses (25 total nurses) were recruited for individual semi-structured interviews. Results suggest that understanding engagement in the NFP program requires addressing both initial and sustained engagement. Themes associated with enhanced initial engagement include nurse characteristics (e.g., flexible, supportive, caring) and establishment of a solid nurse-family relationship founded on these characteristics. Factors impacting sustained engagement include nurse characteristics, provision of educational materials on child development, individualized services for families, and available family support. Identified barriers to completing services include competing demands and lack of support. Findings of this study have direct relevance for workforce planning, including hiring and training through integrating results regarding effective nurse characteristics. Additional program supports to enhance parent engagement may be implemented across home-based parenting programs in light of the current study's findings.

Students' experiences of embedded academic literacy support in a graduate entry nursing program: A qualitative study.

PubMed

Ramjan, Lucie M; Maneze, Della; Everett, Bronwyn; Glew, Paul; Trajkovski, Suza; Lynch, Joan; Salamonson, Yenna

2018-01-01

Graduate entry nursing (GEN) programs were designed to address the predicted nursing shortfall. In Australia, although these programs attract students from culturally and linguistically diverse (CALD) backgrounds, the workload is compounded by cultural differences and a new academic learning environment which presents additional challenges. This qualitative descriptive study explored the experiences of GEN students enrolled in the introductory unit of their nursing program with embedded academic literacy support in Sydney, Australia. Twenty-four commencing GEN students were interviewed in January 2016. Interviews were transcribed verbatim and thematically analysed. Three main themes emerged which illustrated that GEN students were 'diamonds in the rough'. They possessed a raw natural beauty that required some shaping and polishing to ensure academic needs were met. To ensure retention is high, institutions need to evaluate how best to support and harness the potential of these unique students. Copyright © 2017 Elsevier Ltd. All rights reserved.

Evaluative criteria for qualitative research in health care: controversies and recommendations.

PubMed

Cohen, Deborah J; Crabtree, Benjamin F

2008-01-01

We wanted to review and synthesize published criteria for good qualitative research and develop a cogent set of evaluative criteria. We identified published journal articles discussing criteria for rigorous research using standard search strategies then examined reference sections of relevant journal articles to identify books and book chapters on this topic. A cross-publication content analysis allowed us to identify criteria and understand the beliefs that shape them. Seven criteria for good qualitative research emerged: (1) carrying out ethical research; (2) importance of the research; (3) clarity and coherence of the research report; (4) use of appropriate and rigorous methods; (5) importance of reflexivity or attending to researcher bias; (6) importance of establishing validity or credibility; and (7) importance of verification or reliability. General agreement was observed across publications on the first 4 quality dimensions. On the last 3, important divergent perspectives were observed in how these criteria should be applied to qualitative research, with differences based on the paradigm embraced by the authors. Qualitative research is not a unified field. Most manuscript and grant reviewers are not qualitative experts and are likely to embrace a generic set of criteria rather than those relevant to the particular qualitative approach proposed or reported. Reviewers and researchers need to be aware of this tendency and educate health care researchers about the criteria appropriate for evaluating qualitative research from within the theoretical and methodological framework from which it emerges.

Evaluative Criteria for Qualitative Research in Health Care: Controversies and Recommendations

PubMed Central

Cohen, Deborah J.; Crabtree, Benjamin F.

2008-01-01

PURPOSE We wanted to review and synthesize published criteria for good qualitative research and develop a cogent set of evaluative criteria. METHODS We identified published journal articles discussing criteria for rigorous research using standard search strategies then examined reference sections of relevant journal articles to identify books and book chapters on this topic. A cross-publication content analysis allowed us to identify criteria and understand the beliefs that shape them. RESULTS Seven criteria for good qualitative research emerged: (1) carrying out ethical research; (2) importance of the research; (3) clarity and coherence of the research report; (4) use of appropriate and rigorous methods; (5) importance of reflexivity or attending to researcher bias; (6) importance of establishing validity or credibility; and (7) importance of verification or reliability. General agreement was observed across publications on the first 4 quality dimensions. On the last 3, important divergent perspectives were observed in how these criteria should be applied to qualitative research, with differences based on the paradigm embraced by the authors. CONCLUSION Qualitative research is not a unified field. Most manuscript and grant reviewers are not qualitative experts and are likely to embrace a generic set of criteria rather than those relevant to the particular qualitative approach proposed or reported. Reviewers and researchers need to be aware of this tendency and educate health care researchers about the criteria appropriate for evaluating qualitative research from within the theoretical and methodological framework from which it emerges. PMID:18626033

NVivo 8 and consistency in data analysis: reflecting on the use of a qualitative data analysis program.

PubMed

Bergin, Michael

2011-01-01

Qualitative data analysis is a complex process and demands clear thinking on the part of the analyst. However, a number of deficiencies may obstruct the research analyst during the process, leading to inconsistencies occurring. This paper is a reflection on the use of a qualitative data analysis program, NVivo 8, and its usefulness in identifying consistency and inconsistency during the coding process. The author was conducting a large-scale study of providers and users of mental health services in Ireland. He used NVivo 8 to store, code and analyse the data and this paper reflects some of his observations during the study. The demands placed on the analyst in trying to balance the mechanics of working through a qualitative data analysis program, while simultaneously remaining conscious of the value of all sources are highlighted. NVivo 8 as a qualitative data analysis program is a challenging but valuable means for advancing the robustness of qualitative research. Pitfalls can be avoided during analysis by running queries as the analyst progresses from tree node to tree node rather than leaving it to a stage whereby data analysis is well advanced.

The Utility of the Memorable Messages Framework as an Intermediary Evaluation Tool for Fruit and Vegetable Consumption in a Nutrition Education Program

ERIC Educational Resources Information Center

Davis, LaShara A.; Morgan, Susan E.; Mobley, Amy R.

2016-01-01

Additional strategies to evaluate the impact of community nutrition education programs on low-income individuals are needed. The objective of this qualitative study was to examine the use of the Memorable Messages Framework as an intermediary nutrition education program evaluation tool to determine what fruit and vegetable messages were reported…

Implementation and evaluation of a multisite drug usage evaluation program across Australian hospitals - a quality improvement initiative

PubMed Central

2011-01-01

Background With the use of medicines being a broad and extensive part of health management, mechanisms to ensure quality use of medicines are essential. Drug usage evaluation (DUE) is an evidence-based quality improvement methodology, designed to improve the quality, safety and cost-effectiveness of drug use. The purpose of this paper is to describe a national DUE methodology used to improve health care delivery across the continuum through multi-faceted intervention involving audit and feedback, academic detailing and system change, and a qualitative assessment of the methodology, as illustrated by the Acute Postoperative Pain Management (APOP) project. Methods An established methodology, consisting of a baseline audit of inpatient medical records, structured patient interviews and general practitioner surveys, followed by an educational intervention and follow-up audit, is used. Australian hospitals, including private, public, metropolitan and regional, are invited to participate on a voluntary basis. De-identified data collected by hospitals are collated and evaluated nationally to provide descriptive comparative analyses. Hospitals benchmark their practices against state and national results to facilitate change. The educational intervention consists of academic detailing, group education, audit and feedback, point-of-prescribing prompts and system changes. A repeat data collection is undertaken to assess changes in practice. An online qualitative survey was undertaken to evaluate the APOP program. Qualitative assessment of hospitals' perceptions of the effectiveness of the overall DUE methodology and changes in procedure/prescribing/policy/clinical practice which resulted from participation were elicited. Results 62 hospitals participated in the APOP project. Among 23 respondents to the evaluation survey, 18 (78%) reported improvements in the documentation of pain scores at their hospital. 15 (65%) strongly agreed or agreed that participation in APOP directly

Implementation and evaluation of a multisite drug usage evaluation program across Australian hospitals - a quality improvement initiative.

PubMed

Pulver, Lisa K; Wai, Angela; Maxwell, David J; Robertson, Marion B; Riddell, Steven

2011-08-29

With the use of medicines being a broad and extensive part of health management, mechanisms to ensure quality use of medicines are essential. Drug usage evaluation (DUE) is an evidence-based quality improvement methodology, designed to improve the quality, safety and cost-effectiveness of drug use. The purpose of this paper is to describe a national DUE methodology used to improve health care delivery across the continuum through multi-faceted intervention involving audit and feedback, academic detailing and system change, and a qualitative assessment of the methodology, as illustrated by the Acute Postoperative Pain Management (APOP) project. An established methodology, consisting of a baseline audit of inpatient medical records, structured patient interviews and general practitioner surveys, followed by an educational intervention and follow-up audit, is used. Australian hospitals, including private, public, metropolitan and regional, are invited to participate on a voluntary basis. De-identified data collected by hospitals are collated and evaluated nationally to provide descriptive comparative analyses. Hospitals benchmark their practices against state and national results to facilitate change. The educational intervention consists of academic detailing, group education, audit and feedback, point-of-prescribing prompts and system changes. A repeat data collection is undertaken to assess changes in practice.An online qualitative survey was undertaken to evaluate the APOP program. Qualitative assessment of hospitals' perceptions of the effectiveness of the overall DUE methodology and changes in procedure/prescribing/policy/clinical practice which resulted from participation were elicited. 62 hospitals participated in the APOP project. Among 23 respondents to the evaluation survey, 18 (78%) reported improvements in the documentation of pain scores at their hospital. 15 (65%) strongly agreed or agreed that participation in APOP directly resulted in increased

Qualitative Comparative Analysis: A Hybrid Method for Identifying Factors Associated with Program Effectiveness.

PubMed

Cragun, Deborah; Pal, Tuya; Vadaparampil, Susan T; Baldwin, Julie; Hampel, Heather; DeBate, Rita D

2016-07-01

Qualitative comparative analysis (QCA) was developed over 25 years ago to bridge the qualitative and quantitative research gap. Upon searching PubMed and the Journal of Mixed Methods Research , this review identified 30 original research studies that utilized QCA. Perceptions that QCA is complex and provides few relative advantages over other methods may be limiting QCA adoption. Thus, to overcome these perceptions, this article demonstrates how to perform QCA using data from fifteen institutions that implemented universal tumor screening (UTS) programs to identify patients at high risk for hereditary colorectal cancer. In this example, QCA revealed a combination of conditions unique to effective UTS programs. Results informed additional research and provided a model for improving patient follow-through after a positive screen.

Qualitative Comparative Analysis: A Hybrid Method for Identifying Factors Associated with Program Effectiveness

PubMed Central

Cragun, Deborah; Pal, Tuya; Vadaparampil, Susan T.; Baldwin, Julie; Hampel, Heather; DeBate, Rita D.

2015-01-01

Qualitative comparative analysis (QCA) was developed over 25 years ago to bridge the qualitative and quantitative research gap. Upon searching PubMed and the Journal of Mixed Methods Research, this review identified 30 original research studies that utilized QCA. Perceptions that QCA is complex and provides few relative advantages over other methods may be limiting QCA adoption. Thus, to overcome these perceptions, this article demonstrates how to perform QCA using data from fifteen institutions that implemented universal tumor screening (UTS) programs to identify patients at high risk for hereditary colorectal cancer. In this example, QCA revealed a combination of conditions unique to effective UTS programs. Results informed additional research and provided a model for improving patient follow-through after a positive screen. PMID:27429602

Evaluation of a community health representative program among the Cree of northern Quebec.

PubMed

Lavallée, C; James, C A; Robinson, E J

1991-01-01

The James Bay Cree Community Health Representative (CHR) program was implemented in 1984 to train persons from the Cree population of northern Quebec to act as health care advocates and educators, as intermediaries between the Cree population, health services and local organizations, and as participants in assessing health needs. A formative evaluation was initiated which included quantitative analysis of the daily tasks of CHRs and a qualitative component based on documentary research, observation and semi-structured interviews. The evaluation revealed that CHRs actively participated in the ongoing community health programs mainly through health education; the people interviewed showed a high level of satisfaction. However, direct supervision, sufficient continuing education for the CHRs and better integration into health care teams are long-term goals which need to be emphasized in the program.

A qualitative evaluation of scalpel skill teaching of podiatry students.

PubMed

Causby, Ryan S; McDonnell, Michelle N; Reed, Lloyd; Fryer, Caroline E; Hillier, Susan L

2017-01-01

Degrees in health disciplines need a balance of theoretical knowledge and sufficient clinical practice to meet registration requirements, in particular those requiring specialist skills such as the use of scalpels and other small instruments, such as podiatry. However, despite this requirement there is a scarcity of literature and research to inform teaching of these particular manual clinical skills. Therefore, the aims of this study were to determine the current approaches being used to teach manual skills, in particular scalpel skills, in university podiatry programs in Australia and New Zealand, and to explore what issues, challenges and innovations exist. A qualitative study, consisting of semi-structured interviews with staff at eight university podiatry programs in Australia and New Zealand was undertaken to determine how these skills are taught and evaluated, and how poor performers are managed. A conventional content analysis technique was used to analyse and code interview data, with the resultant categories reported. Approaches to teaching manual clinical skills, in particular scalpel skills, appear to be consistent between university programs in Australia and New Zealand in utilising didactic-style content, demonstration, physical practice on inanimate objects and real skin, and often the use of supplementary audio-visual material. The main reported differences between programs were in methods and processes of practice, with controversy regarding the use of inanimate objects versus real skin for practice. Despite a lack of research and literature surrounding this topic, the approach to teaching is relatively consistent between programs with greatest disparity being the structure and duration of practice. Key issues for teaching staff in teaching manual skills were students' clinical exposure, motivation, levels of anxiety and dexterity.

Language Program Evaluation

ERIC Educational Resources Information Center

Norris, John M.

2016-01-01

Language program evaluation is a pragmatic mode of inquiry that illuminates the complex nature of language-related interventions of various kinds, the factors that foster or constrain them, and the consequences that ensue. Program evaluation enables a variety of evidence-based decisions and actions, from designing programs and implementing…

Process evaluation of two home-based bimanual training programs in children with unilateral cerebral palsy (the COAD-study): protocol for a mixed methods study.

PubMed

Beckers, Laura; van der Burg, Jan; Janssen-Potten, Yvonne; Rameckers, Eugène; Aarts, Pauline; Smeets, Rob

2018-04-24

As part of the COAD-study two home-based bimanual training programs for young children with unilateral Cerebral Palsy (uCP) have been developed, both consisting of a preparation phase and a home-based training phase. Parents are coached to use either an explicit or implicit motor learning approach while teaching bimanual activities to their child. A process evaluation of these complex interventions is crucial in order to draw accurate conclusions and provide recommendations for implementation in clinical practice and further research. The aim of the process evaluation is to systematically assess fidelity of the home-based training programs, to examine the mechanisms that contribute to their effects on child-related and parent-related outcomes, and to explore the influence of contextual factors. A mixed methods embedded design is used that emerges from a pragmatism paradigm. The qualitative strand involves a generic qualitative approach. The process evaluation components fidelity (quality), dose delivered (completeness), dose received (exposure and satisfaction), recruitment and context will be investigated. Data collection includes registration of attendance of therapists and remedial educationalists to a course regarding the home-based training programs; a questionnaire to evaluate this course by the instructor; a report form concerning the preparation phase to be completed by the therapist; registration and video analyses of the home-based training; interviews with parents and questionnaires to be filled out by the therapist and remedial educationalist regarding the process of training; and focus groups with therapists and remedial educationalists as well as registration of drop-out rates and reasons, to evaluate the overall home-based training programs. Inductive thematic analysis will be used to analyse qualitative data. Qualitative and quantitative findings are merged through meta-inference. So far, effects of home-based training programs in paediatric

Building capacity for change: evaluation of an organisation-wide leadership development program.

PubMed

Schultz, Tim; Shoobridge, Jodie; Harvey, Gill; Carter, Libby; Kitson, Alison

2018-05-23

Objective. This study evaluated Leading 4 Change, a change leadership development program designed to support healthcare middle managers through a period of significant organisational change and enhance workplace resilience. Methods. A mixed methods evaluation was conducted within the program's framework of a quality improvement activity. Quantitative measures were participant responses (n=160) to online questionnaires, which were compared before and after the program, using an uncontrolled pre-post study design. Four questionnaires were used: Resilience @ Work, General Self-Efficacy, the 11-item Learning Organization Survey and organisational climate. Differences between the pre- and post-program periods were compared using linear mixed-effects models, incorporating repeated measures between 'pre' and 'post' periods. Qualitative data were obtained by interviewing four participants on three occasions during the program, and through text responses provided by participants during a presentation session after the program finished. Both interview data and textual data were subjected to thematic analysis. Results. Integration of data from quantitative and qualitative analyses generated three main findings: (1) participants were satisfied and engaged with the program, which met their learning objectives; (2) the program led to increased workplace resilience, in particular the ability to manage stress and self-efficacy for individuals; and (3) organisational learning perceptions were unchanged. Conclusion. Although conducted during a period of intense internal and external pressure, Leading 4 Change led to demonstrable effects. It effectively engaged middle managers across a health system. However, there was no evidence that the effect of the program extended beyond individual participants to their perceptions of their work environment as a learning organisation. What is known about the topic? Although much has been written about change management and change leadership

Clients' experiences of a community based lifestyle modification program: a qualitative study.

PubMed

Chan, Ruth S M; Lok, Kris Y W; Sea, Mandy M M; Woo, Jean

2009-10-01

There is little information about how clients attending lifestyle modification programs view the outcomes. This qualitative study examined the clients' experience of a community based lifestyle modification program in Hong Kong. Semi-structured interviews were conducted with 25 clients attending the program. Clients perceived the program had positive impacts on their health and nutrition knowledge. They experienced frustration, negative emotion, lack of motivation, and pressure from others during the program. Working environment and lack of healthy food choices in restaurants were the major perceived environmental barriers for lifestyle modification. Clients valued nutritionists' capability of providing professional information and psychological support in the program. Our results suggest that nutritionist's capability of providing quality consultations and patient-centered care are important for empowering clients achieve lifestyle modification.

Meeting the needs of new graduates in the emergency department: a qualitative study evaluating a new graduate internship program.

PubMed

Glynn, Penelope; Silva, Sheila

2013-03-01

The purpose of this study was to explore the experiences of new graduate emergency nurses participating in a structured internship program. In order to meet the needs of new graduate nurses in emergency departments, these departments have developed a variety of orientation programs, some more successful than others. One type of program involves a combination of didactic content and hands-on clinical experience. This study examines the experiences of new graduate nurses in an internship program at a 200-bed community hospital. A qualitative design was used. Interviews with 8 of 9 nurses who participated in the new graduate internship program between 2006 and 2007 were conducted. Content analysis was used to analyze the data. Three themes were identified from the experiences and expectations reported in the interviews: (1) the acquisition of new knowledge and skills in a specialty area, (2) becoming more proficient, and (3) assistance with role transition. The significant role and importance of the unit-base clinical nurse specialist (CNS) and the nurse preceptors were also identified. The findings of this study suggest that a structured internship program is helpful to new graduate nurses when orienting to a critical care area such as the emergency department. Furthermore, the combination of didactic and clinical content, and the roles of the preceptor and unit-base CNS appear to be key factors in the successful transition from new graduate to emergency nurse. Copyright © 2013 Emergency Nurses Association. Published by Mosby, Inc. All rights reserved.

Counseling Skills Pre-Practicum Training at Guidance and Counseling Undergraduate Programs: A Qualitative Investigation

ERIC Educational Resources Information Center

Aladag, Mine

2013-01-01

This study was aimed to describe counseling skills pre-practicum training at guidance and counseling undergraduate programs in Turkey. A descriptive study was conducted based upon qualitative data. The research group of this study consisted of 11 guidance and counseling undergraduate programs voluntarily participate into study. The Survey Form…

Making Sense of Alternative Assessment in a Qualitative Evaluation System

ERIC Educational Resources Information Center

Rojas Serrano, Javier

2017-01-01

In a Colombian private English institution, a qualitative evaluation system has been incorporated. This type of evaluation poses challenges to students who have never been evaluated through a system that eliminates exams or quizzes and, as a consequence, these students have to start making sense of it. This study explores the way students face the…

Evaluation of an implementation model: a national investigation of VA residential programs.

PubMed

Cook, Joan M; Dinnen, Stephanie; Coyne, James C; Thompson, Richard; Simiola, Vanessa; Ruzek, Josef; Schnurr, Paula P

2015-03-01

This national investigation utilizes qualitative data to evaluate an implementation model regarding factors influencing provider use of two evidence-based treatments for posttraumatic stress disorder (PTSD). Semi-structured qualitative interviews with 198 mental health providers from 38 Department of Veterans Affairs' (VA) residential treatment programs were used to explore these issues regarding prolonged exposure (PE) and cognitive processing therapy (CPT) in VA residential PTSD programs. Several unique and some overlapping predictors emerged. Leadership was viewed as an influence on implementation for both CPT and PE, while a lack of dedicated time and resources was viewed as a deterrent for both. Compatibility of CPT with providers' existing practices and beliefs, the ability to observe noticeable patient improvement, a perceived relative advantage of CPT over alternative treatments, and the presence of a supportive peer network emerged as influential on CPT implementation. Leadership was associated with PE implementation. Implications for the design and improvement of training and implementation efforts are discussed.

Faculty Development at One Midwestern Dental School: A Program Evaluation.

PubMed

Gadbury-Amyot, Cynthia C; Smith, Deborah B; Overman, Pamela R; Bunce, Larry

2015-10-01

Most dental school faculty members arrive on campus with a wealth of clinical experience but little to no teacher training. For the past two decades, there has been a call for schools to educate their faculty on a wide variety of topics including educational methodology and cutting-edge educational techniques through faculty development programs. Drawing on theories of general program evaluation as well as evaluation specific to educational programming, the aim of this study was to investigate outcomes of the Faculty Development Program at the University of Missouri-Kansas City School of Dentistry between 2007 and 2014. A mixed-methods research design gathered quantitative data via email survey sent to all eligible teaching faculty members; it received an overall response rate of 54% (N=51). Qualitative data came from open-ended survey questions and a focus group with seven volunteer faculty participants. The survey data suggested that the stated outcomes of faculty development were being met for all stakeholder groups with varying degrees of success. Focus group results indicated a need for a more formal new faculty orientation and better communication with all about the specific charge of faculty development within the school. Evaluation of faculty development activities in academic dental institutions is a necessary component of the ongoing improvement of dental education. Suggestions for future evaluations include the idea of collaborating with other dental schools to increase sample sizes, which would increase participants' perception of the level of confidentiality and make statistical analyses more robust.

Developing medical educators – a mixed method evaluation of a teaching education program

PubMed Central

Roos, Marco; Kadmon, Martina; Kirschfink, Michael; Koch, Eginhard; Jünger, Jana; Strittmatter-Haubold, Veronika; Steiner, Thorsten

2014-01-01

Background It is well accepted that medical faculty teaching staff require an understanding of educational theory and pedagogical methods for effective medical teaching. The purpose of this study was to evaluate the effectiveness of a 5-day teaching education program. Methods An open prospective interventional study using quantitative and qualitative instruments was performed, covering all four levels of the Kirkpatrick model: Evaluation of 1) ‘Reaction’ on a professional and emotional level using standardized questionnaires; 2) ‘Learning’ applying a multiple choice test; 3) ‘Behavior’ by self-, peer-, and expert assessment of teaching sessions with semistructured interviews; and 4) ‘Results’ from student evaluations. Results Our data indicate the success of the educational intervention at all observed levels. 1) Reaction: The participants showed a high acceptance of the instructional content. 2) Learning: There was a significant increase in knowledge (P<0.001) as deduced from a pre-post multiple-choice questionnaire, which was retained at 6 months (P<0.001). 3) Behavior: Peer-, self-, and expert-assessment indicated a transfer of learning into teaching performance. Semistructured interviews reflected a higher level of professionalism in medical teaching by the participants. 4) Results: Teaching performance ratings improved in students’ evaluations. Conclusions Our results demonstrate the success of a 5-day education program in embedding knowledge and skills to improve performance of medical educators. This multimethodological approach, using both qualitative and quantitative measures, may serve as a model to evaluate effectiveness of comparable interventions in other settings. PMID:24679671

Marketing of Revenue-Generating ESL Programs at the University of Calgary: A Qualitative Study

ERIC Educational Resources Information Center

Eaton, Sarah Elaine

2009-01-01

This qualitative case study endeavored to understand how program managers at one post-secondary Canadian university define, understand and undertake the marketing of their revenue-generating English as a Second Language (ESL) programs. Data were collected through interviews with three managers of distinct English as a Second Language (ESL)…

Readiness for Delivering Digital Health at Scale: Lessons From a Longitudinal Qualitative Evaluation of a National Digital Health Innovation Program in the United Kingdom

PubMed Central

Lennon, Marilyn R; Bouamrane, Matt-Mouley; Devlin, Alison M; O'Connor, Siobhan; O'Donnell, Catherine; Chetty, Ula; Agbakoba, Ruth; Bikker, Annemieke; Grieve, Eleanor; Finch, Tracy; Watson, Nicholas; Wyke, Sally

2017-01-01

Background Digital health has the potential to support care delivery for chronic illness. Despite positive evidence from localized implementations, new technologies have proven slow to become accepted, integrated, and routinized at scale. Objective The aim of our study was to examine barriers and facilitators to implementation of digital health at scale through the evaluation of a £37m national digital health program: ‟Delivering Assisted Living Lifestyles at Scale” (dallas) from 2012-2015. Methods The study was a longitudinal qualitative, multi-stakeholder, implementation study. The methods included interviews (n=125) with key implementers, focus groups with consumers and patients (n=7), project meetings (n=12), field work or observation in the communities (n=16), health professional survey responses (n=48), and cross program documentary evidence on implementation (n=215). We used a sociological theory called normalization process theory (NPT) and a longitudinal (3 years) qualitative framework analysis approach. This work did not study a single intervention or population. Instead, we evaluated the processes (of designing and delivering digital health), and our outcomes were the identified barriers and facilitators to delivering and mainstreaming services and products within the mixed sector digital health ecosystem. Results We identified three main levels of issues influencing readiness for digital health: macro (market, infrastructure, policy), meso (organizational), and micro (professional or public). Factors hindering implementation included: lack of information technology (IT) infrastructure, uncertainty around information governance, lack of incentives to prioritize interoperability, lack of precedence on accountability within the commercial sector, and a market perceived as difficult to navigate. Factors enabling implementation were: clinical endorsement, champions who promoted digital health, and public and professional willingness. Conclusions

Monitoring and evaluating transition and sustainability of donor-funded programs: Reflections on the Avahan experience.

PubMed

Bennett, Sara; Ozawa, Sachiko; Rodriguez, Daniela; Paul, Amy; Singh, Kriti; Singh, Suneeta

2015-10-01

In low and middle-income countries, programs funded and implemented by international donors frequently transition to local funding and management, yet such processes are rarely evaluated. We reflect upon experience evaluating the transition of a large scale HIV/AIDS prevention program in India, known as Avahan, in order to draw lessons about transition evaluation approaches and implementation challenges. In terms of conceptualizing the transition theory, the evaluation team identified tensions between the idea of institutionalizing key features of the Avahan program, and ensuring program flexibility to promote sustainability. The transition was planned in three rounds allowing for adaptations to transition intervention and program design during the transition period. The assessment team found it important to track these changes in order to understand which strategies and contextual features supported transition. A mixed methods evaluation was employed, combining semi-structured surveys of transitioning entities (conducted pre and post transition), with longitudinal case studies. Qualitative data helped explain quantitative findings. Measures of transition readiness appeared robust, but we were uncertain of the robustness of institutionalization measures. Finally, challenges to the implementation of such an evaluation are discussed. Given the scarceness of transition evaluations, the lessons from this evaluation may have widespread relevance. Copyright © 2015 The Authors. Published by Elsevier Ltd.. All rights reserved.

A program evaluation of Protovation Camp at an elementary school in North Carolina

NASA Astrophysics Data System (ADS)

Cavoly, Denise Y.

The purpose of this program evaluation was to investigate the impact over time teachers' self-efficacies and the outcome expectancies of those who participated in an inquiry-based, hands-on, constructivist professional development program to learn science content. The hope was that after active participation in this inquiry-based professional development program that provides science inquiry experiences, the teachers, graduate students and elementary students would gain content knowledge, increase self-efficacies, and provide the outcome expectancies of the learning development program that provides science inquiry experiences. The mixed-methods approach used quantitative and qualitative data for campers, which consisted of pre-test and post-test scores on the Test of Science-Related Attitudes (TOSRA), the Draw-A-Scientist Test, Science Process Skills Inventory (SPSI) and content tests based on the camp activities. Additionally, TOSRA scores, Teacher Sense of Efficacy Scale (TSES), and Thinking about Science Survey (TSSI) results for the graduate students and elementary teachers were used along with qualitative data collected from plusdelta charts and interviews to determine the impact of participation in Protovation Camp on teachers and students. Results of the program evaluation indicated that when students were taught inquiry-based lessons that ignite wonder, both their attitudes toward science and their knowledge about science improved. An implication for teacher preparation programs was that practicing inquiry-based lessons on actual elementary students was an important component for teachers and graduate students as they prepare to positively impact student learning in their own classrooms. The findings of this study suggest that it is not just the length of the professional development program that is crucial, but the need for an implementation period while teachers work to transfer the learning to the classroom to their own students is critical to the

Pathways to Program Success: A Qualitative Comparative Analysis (QCA) of Communities Putting Prevention to Work Case Study Programs

PubMed Central

Kane, Heather; Hinnant, Laurie; Day, Kristine; Council, Mary; Tzeng, Janice; Soler, Robin; Chambard, Megan; Roussel, Amy; Heirendt, Wendy

2016-01-01

Objective To examine the elements of capacity, a measure of organizational resources supporting program implementation that result in successful completion of public health program objectives in a public health initiative serving 50 communities. Design We used crisp set Qualitative Comparative Analysis (QCA) to analyze case study and quantitative data collected during the evaluation of the Communities Putting Prevention to Work (CPPW) program. Setting CPPW awardee program staff and partners implemented evidence-based public health improvements in counties, cities, and organizations (eg, worksites, schools). Participants Data came from case studies of 22 CPPW awardee programs that implemented evidence-based, community-and organizational-level public health improvements. Intervention Program staff implemented a range of evidence-based public health improvements related to tobacco control and obesity prevention. Main Outcome Measure The outcome measure was completion of approximately 60% of work plan objectives. Results Analysis of the capacity conditions revealed 2 combinations for completing most work plan objectives: (1) having experience implementing public health improvements in combination with having a history of collaboration with partners; and (2) not having experience implementing public health improvements in combination with having leadership support. Conclusion Awardees have varying levels of capacity. The combinations identified in this analysis provide important insights into how awardees with different combinations of elements of capacity achieved most of their work plan objectives. Even when awardees lack some elements of capacity, they can build it through strategies such as hiring staff and engaging new partners with expertise. In some instances, lacking 1 or more elements of capacity did not prevent an awardee from successfully completing objectives. Implications for Policy & Practice These findings can help funders and practitioners recognize and

Dyadic Interviews as a Tool for Qualitative Evaluation

ERIC Educational Resources Information Center

Morgan, David L.; Eliot, Susan; Lowe, Robert A.; Gorman, Paul

2016-01-01

Although evaluation researchers frequently make use of focus groups and individual interviews as sources of qualitative data, there has been far less attention to dyadic interviews that create a conversation between two research participants. This article describes dyadic interviews as a format that shares many of the advantages of focus groups,…

Quantitative and Qualitative Effectiveness Dimensions of the U.S. Coast Guard Recruiting Program.

DTIC Science & Technology

1983-06-01

of the recruiting program’s ou-put in qualitative form as well as the practical exercise of this thesis’ recommendations. In order to describe...i- nually investigating the relationships between the vaicus elements of the recruiting program and their octent-ial ’t 41 postively impact the...to identify and promote desirable recruiting behaviors, the recruiting management (Headquarters level) can exercise considerable control over the

Development and Evaluation of Innovative Peer-Led Physical Activity Programs for Mental Health Service Users

PubMed Central

Graham, Candida R.; Larstone, Roseann; Griffiths, Brenda; de Leeuw, Sarah; Anderson, Lesley; Powell-Hellyer, Stephanie; Long, Nansi

2017-01-01

Abstract Mental health service users (MHSUs) have elevated rates of cardiometabolic disturbance. Improvements occur with physical activity (PA) programs. We report the development and evaluation of three innovative peer-developed and peer-led PA programs: 1) walking; 2) fitness; and 3) yoga. Qualitative evaluation with 33 MHSUs in British Columbia, Canada, occurred. These programs yielded improvements for participants, highlighted by powerful narratives of health improvement, and improved social connections. The feasibility and acceptability of innovative peer-developed and peer-led programs were shown. Analyses revealed concepts related to engagement and change. Relating core categories, we theorize effective engagement of MHSUs requires accessibility on three levels (geographic, cost, and program flexibility) and health behavior change occurs within co-constituent relationships (to self, to peers, and to the wider community). This study highlights the benefits of peer involvement in developing and implementing PA programs and provides a theoretical framework of understanding engagement and behavior change in health programs for MHSUs. PMID:28953007

Alumni perspectives on career preparation during a postdoctoral training program: a qualitative study.

PubMed

Faupel-Badger, Jessica M; Raue, Kimberley; Nelson, David E; Tsakraklides, Sophia

2015-03-02

Published evaluations of career preparation of alumni from long-standing postdoctoral fellowship programs in the biomedical sciences are limited and often focus on quantitative analysis of data from extant publicly available sources. Qualitative methods provide the opportunity to gather robust information about specific program elements from structured postdoctoral training programs and the influence of this training on subsequent career paths of alumni. In-depth interviews with a subset of the National Cancer Institute's Cancer Prevention Fellowship Program (CPFP) alumni (n=27), representing more than 25 years of the program's history and multiple career sectors, were conducted to assess alumni reflections on the training environment and career preparation during their time in the CPFP. NVivo software was used to analyze data and identify major themes. Four main themes emerged from these interviews, including: the value of structured training curriculum, mentorship, transdisciplinary environment, and professional identity. Even when reflecting on training that occurred one to two decades earlier, alumni were able to highlight specific components of a structured postdoctoral training program as influencing their research and career trajectories. These results may have relevance for those interested in assessing how postdoctoral training can influence fellows throughout their careers and understanding salient features of structured programs. © 2015 J. M. Faupel-Badger et al. CBE—Life Sciences Education © 2015 The American Society for Cell Biology. This article is distributed by The American Society for Cell Biology under license from the author(s). It is available to the public under an Attribution–Noncommercial–Share Alike 3.0 Unported Creative Commons License (http://creativecommons.org/licenses/by-nc-sa/3.0).

Increasing the use of evaluation data collection in an EPO program

NASA Astrophysics Data System (ADS)

Taber, J. J.; Bohon, W.; Bravo, T. K.; Dordevic, M.; Dorr, P. M.; Hubenthal, M.; Johnson, J. A.; Sumy, D.; Welti, R.; Davis, H. B.

2017-12-01

Over the past two years, the Incorporated Research Institutions for Seismology Education and Public Outreach (EPO) program has sought to increase the evaluation rigor of its programs and products. Specifically we sought to make evaluation an integral part of our work; enabling staff to demonstrate why we do the activities we do, enhancing the impact or our products/programs, and empowering staff to make evidence-based claims. The Collaborative Impact Analysis Method (Davis and Scalice, 2015) was selected as it allowed us to combine staff's knowledge of programs, audiences and content with the expertise of an outside evaluation expert, through consultations and a qualitative rubric assessing the initial state of each product/program's evaluation. Staff then developed action plans to make improvements to the programs over time. A key part of the initial action plans has been the collection and analysis of new evaluation data. The most frequently used tools were surveys as they were relatively straightforward to implement and analyze, and could be adapted for different situations. Examples include: brand awareness, value of booth interactions, assessing community interest in a data app, and user surveys of social media and specific web pages. Other evaluation activities included beta testing of new software, and interviews with students and faculty involved in summer field experiences. The surveys have allowed us to document increased impact in some areas, to improve the usability of products and activities, and to provide baseline impact data. The direct involvement of staff in the process has helped staff appreciate the value of evaluation, but there are also challenges to this approach. Since many of the surveys are developed and conducted by EPO staff, rather than being primarily handled by the evaluator, the process takes considerably more staff time to implement. We are still determining how to best manage and present the data and analysis; our current approach

Integrating Oncology Massage Into Chemoinfusion Suites: A Program Evaluation.

PubMed

Mao, Jun J; Wagner, Karen E; Seluzicki, Christina M; Hugo, Audra; Galindez, Laura K; Sheaffer, Heather; Fox, Kevin R

2017-03-01

This article reports on the development, implementation, and evaluation of an integrative clinical oncology massage program for patients undergoing chemotherapy for breast cancer in a large academic medical center. We describe the development and implementation of an oncology massage program embedded into chemoinfusion suites. We used deidentified program evaluation data to identify specific reasons individuals refuse massage and to evaluate the immediate impact of massage treatments on patient-reported outcomes using a modified version of the Distress Thermometer delivered via iPad. We analyzed premassage and postmassage data from the Distress Thermometer using paired t test and derived qualitative data from participants who provided written feedback on their massage experiences. Of the 1,090 massages offered, 692 (63%) were accepted. We observed a significant decrease in self-reported anxiety (from 3.9 to 1.7), nausea (from 2.5 to 1.2), pain (from 3.3 to 1.9), and fatigue (from 4.8 to 3.0) premassage and postmassage, respectively (all P < .001). We found that 642 survey participants (93%) were satisfied with their massage, and 649 (94%) would recommend it to another patient undergoing treatment. Spontaneous patient responses overwhelmingly endorsed the massage as relaxing. No adverse events were reported. Among the 398 patients (36%) who declined a massage, top reasons were time concerns and lack of interest. A clinical oncology massage program can be safely and effectively integrated into chemoinfusion units to provide symptom control for patients with breast cancer. This integrative approach overcomes patient-level barriers of cost, time, and travel, and addresses the institutional-level barrier of space.

Medical Students' and Tutors' Experiences of Directed and Self-Directed Learning Programs in Evidence-Based Medicine: A Qualitative Evaluation Accompanying a Randomized Controlled Trial

ERIC Educational Resources Information Center

Bradley, Peter; Oterholt, Christina; Nordheim, Lena; Bjorndal, Arild

2005-01-01

This qualitative study aims to interpret the results of a randomized controlled trial comparing two educational programs (directed learning and self-directed learning) in evidence-based medicine (EBM) for medical students at the University of Oslo from 2002 to 2003. There is currently very little comparative educational research in this field. In…

Qualitative evaluation of a self-management intervention for people in the early stage of dementia.

PubMed

Martin, Faith; Turner, Andrew; Wallace, Louise M; Stanley, Damian; Jesuthasan, Jana; Bradbury, Nicola

2015-07-01

Self-management programs are effective for people living with chronic illnesses. However, there has been little research addressing self-management for people with dementia in the early stages. This study presents a qualitative evaluation of the experiences of attending a novel self-management program and initial process evaluation. The program was designed with and for people with dementia. It addresses: (a) relationship with family, (b) maintenance of an active lifestyle, (c) psychological well-being, (d) techniques to cope with memory changes and (e) information about dementia. Six participants with early stage dementia completed the intervention that was co-delivered by lay and clinical professional tutors. Participants and tutors attended focus group and interviews at the end of the program to explore their perceptions of the intervention. These were audio-recorded, transcribed verbatim and analysed thematically. Participants reported enjoyment and benefits from the intervention. This was despite some reporting concerns relating to their memory difficulties. The program's flexible nature, focus on strengths and the opportunity to spend time with other people living with dementia were particularly well received. Participants and tutors outlined areas for further improvement. The program was feasible and its flexible delivery appeared to facilitate participant benefit. Emphasis should be placed on maintaining activity and relationships, improving positive well-being and social interaction during the program. Memory of the pleasant experience and strengths focus was evidenced, which may impact positively on quality of life. The results highlight the usefulness and acceptability of self-management for people with early stage dementia and provide initial support for the program's structure and content. © The Author(s) 2013.

Measurement issues in the evaluation of chronic disease self-management programs.

PubMed

Nolte, Sandra; Elsworth, Gerald R; Newman, Stanton; Osborne, Richard H

2013-09-01

To provide an in-depth analysis of outcome measures used in the evaluation of chronic disease self-management programs consistent with the Stanford curricula. Based on a systematic review on self-management programs, effect sizes derived from reported outcome measures are categorized according to the quality of life appraisal model developed by Schwartz and Rapkin which classifies outcomes from performance-based measures (e.g., clinical outcomes) to evaluation-based measures (e.g., emotional well-being). The majority of outcomes assessed in self-management trials are based on evaluation-based methods. Overall, effects on knowledge--the only performance-based measure observed in selected trials--are generally medium to large. In contrast, substantially more inconsistent results are found for both perception- and evaluation-based measures that mostly range between nil and small positive effects. Effectiveness of self-management interventions and resulting recommendations for health policy makers are most frequently derived from highly variable evaluation-based measures, that is, types of outcomes that potentially carry a substantial amount of measurement error and/or bias such as response shift. Therefore, decisions regarding the value and efficacy of chronic disease self-management programs need to be interpreted with care. More research, especially qualitative studies, is needed to unravel cognitive processes and the role of response shift bias in the measurement of change.

An Investigation into the Development, Implementation, and Evaluation of Online Learning Programs for Small Rural Public Schools in Texas

ERIC Educational Resources Information Center

Tabor, Bruce

2012-01-01

This qualitative study investigated the experiences of 10 superintendents in Pre-K-8, 1A, and small 2A schools located in rural areas of Texas, who have implemented an online learning program or programs. Narrative techniques were used to explore the challenges that they faced in the pre-planning, planning, implementation and evaluation of their…

Corporate Mentors and Undergraduate Students: A Qualitative Study of the Advancing Women in Construction Mentorship Program

ERIC Educational Resources Information Center

Eicher, Matthew

2013-01-01

In a conscious effort to combat the low enrollment of women in construction management, a program was created to retain women through a mentorship program--Advancing Women in Construction. A qualitative analysis, facilitated through a grounded theory approach, sought to understand if the program was indeed successful, and what value did the…

Implementing and evaluating a program to facilitate chronic disease prevention and screening in primary care: a mixed methods program evaluation.

PubMed

Manca, Donna Patricia; Aubrey-Bassler, Kris; Kandola, Kami; Aguilar, Carolina; Campbell-Scherer, Denise; Sopcak, Nicolette; O'Brien, Mary Ann; Meaney, Christopher; Faria, Vee; Baxter, Julia; Moineddin, Rahim; Salvalaggio, Ginetta; Green, Lee; Cave, Andrew; Grunfeld, Eva

2014-10-08

The objectives of this paper are to describe the planned implementation and evaluation of the Building on Existing Tools to Improve Chronic Disease Prevention and Screening in Primary Care (BETTER 2) program which originated from the BETTER trial. The pragmatic trial, informed by the Chronic Care Model, demonstrated the effectiveness of an approach to Chronic Disease Prevention and Screening (CDPS) involving the use of a new role, the prevention practitioner. The desired goals of the program are improved clinical outcomes, reduction in the burden of chronic disease, and improved sustainability of the health-care system through improved CDPS in primary care. The BETTER 2 program aims to expand the implementation of the intervention used in the original BETTER trial into communities across Canada (Alberta, Ontario, Newfoundland and Labrador, the Northwest Territories and Nova Scotia). This proactive approach provides at-risk patients with an intervention from the prevention practitioner, a health-care professional. Using the BETTER toolkit, the prevention practitioner determines which CDPS actions the patient is eligible to receive, and through shared decision-making and motivational interviewing, develops a unique and individualized 'prevention prescription' with the patient. This intervention is 1) personalized; 2) addressing multiple conditions; 3) integrated through linkages to local, regional, or national resources; and 4) longitudinal by assessing patients over time. The BETTER 2 program brings together primary care providers, policy/decision makers and researchers to work towards improving CDPS in primary care. The target patient population is adults aged 40-65. The reach, effectiveness, adoption, implementation, maintain (RE-AIM) framework will inform the evaluation of the program through qualitative and quantitative methods. A composite index will be used to quantitatively assess the effectiveness of the prevention practitioner intervention. The CDPS actions

A leadership development program for surgeons: First-year participant evaluation.

PubMed

Pradarelli, Jason C; Jaffe, Gregory A; Lemak, Christy Harris; Mulholland, Michael W; Dimick, Justin B

2016-08-01

In a dynamic health care system, strong leadership has never been more important for surgeons. Little is known about how to design and conduct effectively a leadership program specifically for surgeons. We sought to evaluate critically a Leadership Development Program for practicing surgeons by exploring how the program's strengths and weaknesses affected the surgeons' development as physician-leaders. At a large academic institution, we conducted semistructured interviews with 21 surgical faculty members who applied voluntarily, were selected, and completed a newly created Leadership Development Program in December 2012. Interview transcripts underwent qualitative descriptive analysis with thematic coding based on grounded theory. Themes were extracted regarding surgeons' evaluations of the program on their development as physician-leaders. After completing the program, surgeons reported personal improvements in the following 4 areas: self-empowerment to lead, self-awareness, team-building skills, and knowledge in business and leadership. Surgeons felt "more confident about stepping up as a leader" and more aware of "how others view me and my interactions." They described a stronger grasp on "giving feedback" as well as a better understanding of "business/organizational issues." Overall, surgeon-participants reported positive impacts of the program on their day-to-day work activities and general career perspective as well as on their long-term career development plans. Surgeons also recommended areas where the program could potentially be improved. These interviews detailed self-reported improvements in leadership knowledge and capabilities for practicing surgeons who completed a Leadership Development Program. A curriculum designed specifically for surgeons may enable future programs to equip surgeons better for important leadership roles in a complex health care environment. Copyright © 2016 Elsevier Inc. All rights reserved.

A Leadership Development Program for Surgeons: First-Year Participant Evaluation

PubMed Central

Pradarelli, Jason C.; Jaffe, Gregory A.; Lemak, Christy Harris; Mulholland, Michael W.; Dimick, Justin B.

2016-01-01

Structured Abstract Background In a dynamic healthcare system, strong leadership has never been more important for surgeons. Little is known about how to effectively design and conduct a leadership program specifically for surgeons. We sought to critically evaluate a Leadership Development Program for practicing surgeons by exploring the strengths and weaknesses of program components on surgeons’ development as physician-leaders. Methods At a large academic institution, we conducted semi-structured interviews with 21 surgical faculty members who voluntarily applied, were selected, and completed a newly-created Leadership Development Program in December 2012. Interview transcripts underwent qualitative descriptive analysis with thematic coding based on grounded theory. Themes were extracted regarding surgeons’ evaluations of the program on their development as physician-leaders. Results After completing the program, surgeons reported personal improvements in the following 4 areas: self-empowerment to lead, self-awareness, team-building skills, and business and leadership knowledge. Surgeons felt “more confident about stepping up as a leader” and more aware about “how others view me and my interactions.” They described a stronger grasp on “giving feedback” as well as “business/organizational issues.” Overall, surgeon participants reported positive impacts of the program on their day-to-day work activities, general career perspective, as well as their long-term career development plans. Surgeons also recommended areas for potential improvement for the program. Conclusions These interviews detailed self-reported improvements in leadership knowledge and capabilities for practicing surgeons who completed a Leadership Development Program. A curriculum designed specifically for surgeons may enable future programs to better equip surgeons for important leadership roles in a complex healthcare environment. PMID:27138180

Evaluating Afterschool Programs

ERIC Educational Resources Information Center

Little, Priscilla M.

2014-01-01

Well-implemented afterschool programs can promote a range of positive learning and developmental outcomes. However, not all research and evaluation studies have shown the benefits of participation, in part because programs and their evaluation were out of sync. This chapter provides practical guidance on how to foster that alignment between…

[Qualitative evaluation of TAMARPP, the relapse prevention program for substance abusers at the Mental Health and Welfare Center].

PubMed

Taniai, Tomoko; Yottsuji, Naomi; Okuda, Hidemi; Karibe, Haruo; Miura, Kasumi; Hiraga, Masasi; Kondo, Ayumi; Matsumoto, Toshihiko

2014-12-01

The development of an effective treatment system for patients with addiction-related problems like substance abuse or pathological gambling is urgently needed. The purpose of this study was to clarify the therapeutic factors of the Tama Mental Health and Welfare Center Relapse Prevention Program (TAMARPP). The program is provided at the Tama Comprehensive Center for Mental Health and Welfare, which is operated publicly by the Tokyo Metropolitan Government. Seven personnel (3 clinical psychologists, 2 public health nurses, and 2 recovering counselors) searched the consultation records of 31 participants who had continued treatment longer than 1 year, and made a list of qualitative factors that may positively impact participant outcome. The following six factors were extracted: 1) a "no-blame" atmosphere that makes participants feel safe and free; 2) a mutual helping process that enables participants to empathize with one another; 3) an individual-centered approach that focuses on a participant's motivation to recover; 4) a psycho-educational process that establishes treatment goals; 5) a treatment goal that involves some type of social role; and 6) the referral of graduates to another long-term treatment program such as Narcotics Anonymous. Our results suggest the need of personnel to provide treatment as follows: 1) keep the circumstances safe; 2) use a method that matches the participant's motivation; 3) refer graduates to another in-depth treatment program; 4) emphasize the relationship between thinking, emotion, and behavior; and 5) improve the quality of life of the participant. The missions of our center, an official institution that runs such a program, are as follows: 1) to be the first place at which addicts seek aid during the early stage; 2) to cooperate with other facilities in order to provide social resources that support recovery; and 3) to support patients suffering from both addiction and comorbid psychiatric diseases.

Optimizing Implementation of Obesity Prevention Programs: A Qualitative Investigation Within a Large-Scale Randomized Controlled Trial.

PubMed

Kozica, Samantha L; Teede, Helena J; Harrison, Cheryce L; Klein, Ruth; Lombard, Catherine B

2016-01-01

The prevalence of obesity in rural and remote areas is elevated in comparison to urban populations, highlighting the need for interventions targeting obesity prevention in these settings. Implementing evidence-based obesity prevention programs is challenging. This study aimed to investigate factors influencing the implementation of obesity prevention programs, including adoption, program delivery, community uptake, and continuation, specifically within rural settings. Nested within a large-scale randomized controlled trial, a qualitative exploratory approach was adopted, with purposive sampling techniques utilized, to recruit stakeholders from 41 small rural towns in Australia. In-depth semistructured interviews were conducted with clinical health professionals, health service managers, and local government employees. Open coding was completed independently by 2 investigators and thematic analysis undertaken. In-depth interviews revealed that obesity prevention programs were valued by the rural workforce. Program implementation is influenced by interrelated factors across: (1) contextual factors and (2) organizational capacity. Key recommendations to manage the challenges of implementing evidence-based programs focused on reducing program delivery costs, aided by the provision of a suite of implementation and evaluation resources. Informing the scale-up of future prevention programs, stakeholders highlighted the need to build local rural capacity through developing supportive university partnerships, generating local program ownership and promoting active feedback to all program partners. We demonstrate that the rural workforce places a high value on obesity prevention programs. Our results inform the future scale-up of obesity prevention programs, providing an improved understanding of strategies to optimize implementation of evidence-based prevention programs. © 2015 National Rural Health Association.

Maximizing the Impact of Program Evaluation: A Discrepancy-Based Process for Educational Program Evaluation.

ERIC Educational Resources Information Center

Cantor, Jeffrey A.

This paper describes a formative/summative process for educational program evaluation, which is appropriate for higher education programs and is based on M. Provus' Discrepancy Evaluation Model and the principles of instructional design. The Discrepancy Based Methodology for Educational Program Evaluation facilitates systematic and detailed…

MATERNAL PERCEPTIONS OF PARENTING FOLLOWING AN EVIDENCE-BASED PARENTING PROGRAM: A QUALITATIVE STUDY OF LEGACY FOR CHILDRENTM.

PubMed

Hartwig, Sophie A; Robinson, Lara R; Comeau, Dawn L; Claussen, Angelika H; Perou, Ruth

2017-07-01

This article presents the findings of a qualitative study of maternal perceptions of parenting following participation in Legacy for Children TM (Legacy), an evidence-based parenting program for low-income mothers of young children and infants. To further examine previous findings and better understand participant experiences, we analyzed semistructured focus-group discussions with predominantly Hispanic and Black, non-Hispanic Legacy mothers at two sites (n = 166) using thematic analysis and grounded theory techniques. The qualitative study presented here investigated how mothers view their parenting following participation in Legacy, allowing participants to describe their experience with the program in their own words, thus capturing an "insider" perspective. Mothers at both sites communicated knowledge and use of positive parenting practices targeted by the goals of Legacy; some site-specific differences emerged related to these parenting practices. These findings align with the interpretation of quantitative results from the randomized controlled trials and further demonstrate the significance of the Legacy program in promoting positive parenting for mothers living in poverty. This study emphasizes the importance of understanding real-world context regarding program efficacy and the benefit of using qualitative research to understand participant experiences. © 2017 Michigan Association for Infant Mental Health.

Online versus Paper Evaluations: Differences in Both Quantitative and Qualitative Data

ERIC Educational Resources Information Center

Burton, William B.; Civitano, Adele; Steiner-Grossman, Penny

2012-01-01

This study sought to determine if differences exist in the quantitative and qualitative data collected with paper and online versions of a medical school clerkship evaluation form. Data from six-and-a-half years of clerkship evaluations were used, some collected before and some after the conversion from a paper to an online evaluation system. The…

Building Successful Home Visitor-Mother Relationships and Reaching Program Goals in Two Early Head Start Programs: A Qualitative Look at Contributing Factors

ERIC Educational Resources Information Center

Brookes, Sheila J.; Summers, Jean Ann; Thornburg, Kathy R.; Ispa, Jean M.; Lane, Valeri J.

2006-01-01

This paper presents the results of two qualitative studies, conducted independently in two Early Head Start programs, exploring the reasons given by mothers and home visitors for family success or lack of success in achieving program goals and for engagement in the mother-home visitor relationship. Several patterns pertaining to family issues and…

A qualitative study of programs for parents with serious mental illness and their children: building practice-based evidence.

PubMed

Nicholson, Joanne; Hinden, Beth R; Biebel, Kathleen; Henry, Alexis D; Katz-Leavy, Judith

2007-10-01

The rationale for the development of effective programs for parents with serious mental illness and their children is compelling. Using qualitative methods and a grounded theory approach with data obtained in site visits, seven existing programs for parents with mental illness and their children in the United States are described and compared across core components: target population, theory and assumptions, funding, community and agency contexts, essential services and intervention strategies, moderators, and outcomes. The diversity across programs is strongly complemented by shared characteristics, the identification of which provides the foundation for future testing and the development of an evidence base. Challenges in program implementation and sustainability are identified. Qualitative methods are useful, particularly when studying existing programs, in taking steps toward building the evidence base for effective programs for parents with serious mental illness and their children.

Integrating Oncology Massage Into Chemoinfusion Suites: A Program Evaluation

PubMed Central

Wagner, Karen E.; Seluzicki, Christina M.; Hugo, Audra; Galindez, Laura K.; Sheaffer, Heather; Fox, Kevin R.

2017-01-01

Objective: This article reports on the development, implementation, and evaluation of an integrative clinical oncology massage program for patients undergoing chemotherapy for breast cancer in a large academic medical center. Materials and Methods: We describe the development and implementation of an oncology massage program embedded into chemoinfusion suites. We used deidentified program evaluation data to identify specific reasons individuals refuse massage and to evaluate the immediate impact of massage treatments on patient-reported outcomes using a modified version of the Distress Thermometer delivered via iPad. We analyzed premassage and postmassage data from the Distress Thermometer using paired t test and derived qualitative data from participants who provided written feedback on their massage experiences. Results: Of the 1,090 massages offered, 692 (63%) were accepted. We observed a significant decrease in self-reported anxiety (from 3.9 to 1.7), nausea (from 2.5 to 1.2), pain (from 3.3 to 1.9), and fatigue (from 4.8 to 3.0) premassage and postmassage, respectively (all P < .001). We found that 642 survey participants (93%) were satisfied with their massage, and 649 (94%) would recommend it to another patient undergoing treatment. Spontaneous patient responses overwhelmingly endorsed the massage as relaxing. No adverse events were reported. Among the 398 patients (36%) who declined a massage, top reasons were time concerns and lack of interest. Conclusion: A clinical oncology massage program can be safely and effectively integrated into chemoinfusion units to provide symptom control for patients with breast cancer. This integrative approach overcomes patient-level barriers of cost, time, and travel, and addresses the institutional-level barrier of space. PMID:28045616

A Systematic Quantitative-Qualitative Model: How To Evaluate Professional Services

ERIC Educational Resources Information Center

Yoda, Koji

1973-01-01

The proposed evaluation model provides for the assignment of relative weights to each criterion, and establishes a weighting system for calculating a quantitative-qualitative raw score for each service activity of a faculty member being reviewed. (Author)

A qualitative evaluation of the 2005–2011 National Academic Centers of Excellence in Youth Violence Prevention Program☆

PubMed Central

Holland, Kristin M.; Vivolo-Kantor, Alana M.; Cruz, Jason Dela; Massetti, Greta M.; Mahendra, Reshma

2018-01-01

The Centers for Disease Control and Prevention’s Division of Violence Prevention (DVP) funded eight National Academic Centers of Excellence (ACEs) in Youth Violence Prevention from 2005 to 2010 and two Urban Partnership Academic Centers of Excellence (UPACEs) in Youth Violence Prevention from 2006 to 2011. The ACEs and UPACEs constitute DVP’s 2005–2011 ACE Program. ACE Program goals include partnering with communities to promote youth violence (YV) prevention and fostering connections between research and community practice. This article describes a qualitative evaluation of the 2005–2011 ACE Program using an innovative approach for collecting and analyzing data from multiple large research centers via a web-based Information System (ACE-IS). The ACE-IS was established as an efficient mechanism to collect and document ACE research and programmatic activities. Performance indicators for the ACE Program were established in an ACE Program logic model. Data on performance indicators were collected through the ACE-IS biannually. Data assessed Centers’ ability to develop, implement, and evaluate YV prevention activities. Performance indicator data demonstrate substantial progress on Centers’ research in YV risk and protective factors, community partnerships, and other accomplishments. Findings provide important lessons learned, illustrate progress made by the Centers, and point to new directions for YV prevention research and programmatic efforts. PMID:26319174

The Evaluation Handbook: Guidelines for Evaluating Dropout Prevention Programs.

ERIC Educational Resources Information Center

Smink, Jay; Stank, Peg

This manual, developed in an effort to take the mysticism out of program evaluation, discusses six phases of the program evaluation process. The introduction discusses reasons for evaluation, process and outcome evaluation, the purpose of the handbook, the evaluation process, and the Sequoia United School District Dropout Prevention Program. Phase…

A Qualitative Exploration of Community-Based Organization Programs, Resources, and Training to Promote Adolescent Sexual Health

ERIC Educational Resources Information Center

McCarthy, Molly A.; Fisher, Christopher M.; Zhou, Junmin; Zhu, He; Pelster, Aja Kneip; Schober, Daniel J.; Baldwin, Kathleen; Fortenberry, J. Dennis; Goldsworthy, Richard

2015-01-01

Youth development professionals (YDPs) working at community-based organizations (CBOs) can promote adolescent sexual health through programs. This study explored the programs and resources that youth access at CBOs and training YDPs receive. Twenty-one semi-structured interviews were conducted with YDPs. Qualitative content analyses were conducted…

A Yoga Intervention Program for Patients Suffering from Symptoms of Posttraumatic Stress Disorder: A Qualitative Descriptive Study.

PubMed

Jindani, Farah A; Khalsa, G F S

2015-07-01

To understand how individuals with symptoms of posttraumatic stress disorder (PTSD) perceive a trauma-sensitive Kundalini yoga (KY) program. Digitally recorded telephone interviews 30-60 minutes in duration were conducted with 40 individuals with PTSD participating in an 8-week KY treatment program. Interviews were transcribed verbatim and analyzed using qualitative thematic analysis techniques. Qualitative analysis identifies three major themes: self-observed changes, new awareness, and the yoga program itself. Findings suggest that participants noted changes in areas of health and well-being, lifestyle, psychosocial integration, and perceptions of self in relation to the world. Presented are practical suggestions for trauma-related programming. There is a need to consider alternative and potentially empowering approaches to trauma treatment. Yoga-related self-care or self-management strategies are widely accessible, are empowering, and may address the mind-body elements of PTSD.

Key ingredients of anti-stigma programs for health care providers: a data synthesis of evaluative studies.

PubMed

Knaak, Stephanie; Modgill, Geeta; Patten, Scott B

2014-10-01

As part of its ongoing effort to combat stigma against mental illness among health care providers, the Mental Health Commission of Canada partnered with organizations conducting anti-stigma interventions. Our objective was to evaluate program effectiveness and to better understand what makes some programs more effective than others. Our paper reports the elements of these programs found to be most strongly associated with favourable outcomes. Our study employed a multi-phased, mixed-methods design. First, a grounded theory qualitative study was undertaken to identify key program elements. Next, each program (n = 22) was coded according to the presence or absence of the identified key program ingredients. Then, random-effects, meta-regression modelling was used to examine the association between program outcomes and the key ingredients. The qualitative analysis led to a 6-ingredient model of key program elements. Results of the quantitative analysis showed that programs that included all 6 of these ingredients performed significantly better than those that did not. Individual analyses of each of the 6 ingredients showed that including multiple forms of social contact and emphasizing recovery were characteristics of the most effective programs. The results provide a validation of a 6-ingredient model of key program elements for anti-stigma programming for health care providers. Emphasizing recovery and including multiple types of social contact are of particular importance for maximizing the effectiveness of anti-stigma programs for health care providers.

Evaluating Evaluations: The Case of Parent Involvement Programs

ERIC Educational Resources Information Center

Mattingly, Doreen J.; Prislin, Radmila; McKenzie, Thomas L.; Rodriguez, James L.; Kayzar, Brenda

2002-01-01

This article analyzes 41 studies that evaluated K-12 parent involvement programs in order to assess claims that such programs are an effective means of improving student learning. It examines the characteristics of the parent involvement programs, as well as the research design, data, and analytical techniques used in program evaluation. Our…

Faculty performance evaluation in accredited U.S. public health graduate schools and programs: a national study.

PubMed

Gimbel, Ronald W; Cruess, David F; Schor, Kenneth; Hooper, Tomoko I; Barbour, Galen L

2008-10-01

To provide baseline data on evaluation of faculty performance in U.S. schools and programs of public health. The authors administered an anonymous Internet-based questionnaire using PHP Surveyor. The invited sample consisted of individuals listed in the Council on Education for Public Health (CEPH) Directory of Accredited Schools and Programs of Public Health. The authors explored performance measures in teaching, research, and service, and assessed how faculty performance measures are used. A total of 64 individuals (60.4%) responded to the survey, with 26 (40.6%) reporting accreditation/reaccreditation by CEPH within the preceding 24 months. Although all schools and programs employ faculty performance evaluations, a significant difference exists between schools and programs in the use of results for merit pay increases and mentoring purposes. Thirty-one (48.4%) of the organizations published minimum performance expectations. Fifty-nine (92.2%) of the respondents counted number of publications, but only 22 (34.4%) formally evaluated their quality. Sixty-two (96.9%) evaluated teaching through student course evaluations, and only 29 (45.3%) engaged in peer assessment. Although aggregate results of teaching evaluation are available to faculty and administrators, this information is often unavailable to students and the public. Most schools and programs documented faculty service activities qualitatively but neither assessed it quantitatively nor evaluated its impact. This study provides insight into how schools and programs of public health evaluate faculty performance. Results suggest that although schools and programs do evaluate faculty performance on a basic level, many do not devote substantial attention to this process.

Advancing the study of violence against women using mixed methods: integrating qualitative methods into a quantitative research program.

PubMed

Testa, Maria; Livingston, Jennifer A; VanZile-Tamsen, Carol

2011-02-01

A mixed methods approach, combining quantitative with qualitative data methods and analysis, offers a promising means of advancing the study of violence. Integrating semi-structured interviews and qualitative analysis into a quantitative program of research on women's sexual victimization has resulted in valuable scientific insight and generation of novel hypotheses for testing. This mixed methods approach is described and recommendations for integrating qualitative data into quantitative research are provided.

Indigenous health program evaluation design and methods in Australia: a systematic review of the evidence.

PubMed

Lokuge, Kamalini; Thurber, Katherine; Calabria, Bianca; Davis, Meg; McMahon, Kathryn; Sartor, Lauren; Lovett, Raymond; Guthrie, Jill; Banks, Emily

2017-10-01

Indigenous Australians experience a disproportionately higher burden of disease compared to non-Indigenous Australians. High-quality evaluation of Indigenous health programs is required to inform health and health services improvement. We aimed to quantify methodological and other characteristics of Australian Indigenous health program evaluations published in the peer-reviewed literature. Systematic review of peer-reviewed literature (November 2009-2014) on Indigenous health program evaluation. We identified 118 papers describing evaluations of 109 interventions; 72.0% were university/research institution-led. 82.2% of evaluations included a quantitative component; 49.2% utilised quantitative data only and 33.1% used both quantitative and qualitative data. The most common design was a before/after comparison (30.5%, n=36/118). 7.6% of studies (n=9/118) used an experimental design: six individual-level and three cluster-randomised controlled trials. 56.8% (67/118) reported on service delivery/process outcomes (versus health or health risk factor outcomes) only. Given the number of Indigenous health programs that are implemented, few evaluations overall are published in the peer-reviewed literature and, of these, few use optimal methodologies such as mixed methods and experimental design. Implications for public health: Multiple strategies are required to increase high-quality, accessible evaluation in Indigenous health, including supporting stronger research-policy-practice partnerships and capacity building for evaluation by health services and government. © 2017 The Authors.

Customizing Process to Align with Purpose and Program: The 2003 MS PHD'S in Ocean Sciences Program Evaluative Case Study

NASA Astrophysics Data System (ADS)

Williamson, V. A.; Pyrtle, A. J.

2004-12-01

How did the 2003 Minorities Striving and Pursuing Higher Degrees of Success (MS PHD'S) in Ocean Sciences Program customize evaluative methodology and instruments to align with program goals and processes? How is data captured to document cognitive and affective impact? How are words and numbers utilized to accurately illustrate programmatic outcomes? How is compliance with implicit and explicit funding regulations demonstrated? The 2003 MS PHD'S in Ocean Sciences Program case study provides insightful responses to each of these questions. MS PHD'S was developed by and for underrepresented minorities to facilitate increased and sustained participation in Earth system science. Key components of this initiative include development of a community of scholars sustained by face-to-face and virtual mentoring partnerships; establishment of networking activities between and among undergraduate, graduate, postgraduate students, scientists, faculty, professional organization representatives, and federal program officers; and provision of forums to address real world issues as identified by each constituent group. The evaluative case study of the 2003 MS PHD'S in Ocean Sciences Program consists of an analysis of four data sets. Each data set was aligned to document progress in the achievement of the following program goals: Goal 1: The MS PHD'S Ocean Sciences Program will successfully market, recruit, select, and engage underrepresented student and non-student participants with interest/ involvement in Ocean Sciences; Goal 2: The MS PHD'S Ocean Sciences Program will provide meaningful engagement for participants as determined by quantitative analysis of user-feedback; Goal 3: The MS PHD'S Ocean Sciences Program will provide meaningful engagement for participants as determined by qualitative analysis of user-feedback, and; Goal 4: The MS PHD'S Ocean Sciences Program will develop a constituent base adequate to demonstrate evidence of interest, value, need and sustainability in

ADVANCING THE STUDY OF VIOLENCE AGAINST WOMEN USING MIXED METHODS: INTEGRATING QUALITATIVE METHODS INTO A QUANTITATIVE RESEARCH PROGRAM

PubMed Central

Testa, Maria; Livingston, Jennifer A.; VanZile-Tamsen, Carol

2011-01-01

A mixed methods approach, combining quantitative with qualitative data methods and analysis, offers a promising means of advancing the study of violence. Integrating semi-structured interviews and qualitative analysis into a quantitative program of research on women’s sexual victimization has resulted in valuable scientific insight and generation of novel hypotheses for testing. This mixed methods approach is described and recommendations for integrating qualitative data into quantitative research are provided. PMID:21307032

Active thermography in qualitative evaluation of protective materials.

PubMed

Gralewicz, Grzegorz; Wiecek, Bogusław

2009-01-01

This is a study of the possibilities of a qualitative evaluation of protective materials with active thermography. It presents a simulation of a periodic excitation of a multilayer composite material. Tests were conducted with lock-in thermography on Kevlar composite consisting of 16 layers of Kevlar fabric reinforced with formaldehyde resin with implanted delamination defects. Lock-in thermography is a versatile tool for nondestructive evaluation. It is a fast, remote and nondestructive procedure. Hence, it was used to detect delaminations in the composite structure of materials used in the production of components designed for personal protection. This method directly contributes to an improvement in safety.

Measuring Success: Evaluating Educational Programs

ERIC Educational Resources Information Center

Fisher, Yael

2010-01-01

This paper reveals a new evaluation model, which enables educational program and project managers to evaluate their programs with a simple and easy to understand approach. The "index of success model" is comprised of five parameters that enable to focus on and evaluate both the implementation and results of an educational program. The…

Evaluating meta-ethnography: systematic analysis and synthesis of qualitative research.

PubMed

Campbell, R; Pound, P; Morgan, M; Daker-White, G; Britten, N; Pill, R; Yardley, L; Pope, C; Donovan, J

2011-12-01

Methods for reviewing and synthesising findings from quantitative research studies in health care are well established. Although there is recognition of the need for qualitative research to be brought into the evidence base, there is no consensus about how this should be done and the methods for synthesising qualitative research are at a relatively early stage of development. To evaluate meta-ethnography as a method for synthesising qualitative research studies in health and health care. Two full syntheses of qualitative research studies were conducted between April 2002 and September 2004 using meta-ethnography: (1) studies of medicine-taking and (2) studies exploring patients' experiences of living with rheumatoid arthritis. Potentially relevant studies identified in multiple literature searches conducted in July and August 2002 (electronically and by hand) were appraised using a modified version of the Critical Appraisal Skills Programme questions for understanding qualitative research. Candidate papers were excluded on grounds of lack of relevance to the aims of the synthesis or because the work failed to employ qualitative methods of data collection and analysis. Thirty-eight studies were entered into the medicine-taking synthesis, one of which did not contribute to the final synthesis. The synthesis revealed a general caution about taking medicine, and that the practice of lay testing of medicines was widespread. People were found to take their medicine passively or actively or to reject it outright. Some, in particular clinical areas, were coerced into taking it. Those who actively accepted their medicine often modified the regimen prescribed by a doctor, without the doctor's knowledge. The synthesis concluded that people often do not take their medicines as prescribed because of concern about the medicines themselves. 'Resistance' emerged from the synthesis as a concept that best encapsulated the lay response to prescribed medicines. It was suggested that a

Toward an agenda for evaluation of qualitative research.

PubMed

Stige, Brynjulf; Malterud, Kirsti; Midtgarden, Torjus

2009-10-01

Evaluation is essential for research quality and development, but the diversity of traditions that characterize qualitative research suggests that general checklists or shared criteria for evaluation are problematic. We propose an approach to research evaluation that encourages reflexive dialogue through use of an evaluation agenda. In proposing an evaluation agenda we shift attention from rule-based judgment to reflexive dialogue. Unlike criteria, an agenda may embrace pluralism, and does not request consensus on ontological, epistemological, and methodological issues, only consensus on what themes warrant discussion. We suggest an evaluation agenda-EPICURE-with two dimensions communicated through use of two acronyms.The first, EPIC, refers to the challenge of producing rich and substantive accounts based on engagement, processing, interpretation, and (self-)critique. The second-CURE-refers to the challenge of dealing with preconditions and consequences of research, with a focus on (social) critique, usefulness, relevance, and ethics. The seven items of the composite agenda EPICURE are presented and exemplified. Features and implications of the agenda approach to research evaluation are then discussed.

Experiences of School Teachers on Participation in a Brief School Mental Health Program: A Qualitative Inquiry

ERIC Educational Resources Information Center

Shrinivasa, Basavaraj; Reshma, B. K.; Virupaksha, H. G.; Chaithra, Chandrakanth; Vidya, Naik; Nithyananda, S.; Joseph Arthur, Julian Anthony; Amaresha, Anekal C.

2016-01-01

Background: School mental health programs (SMHPs) aim to strengthen school teachers' understanding about issues related to child and adolescent mental health and their management. Many studies have looked at outcomes of such programs quantitatively. However, there is a lack of studies on the qualitative effects of SMHPs. With this in mind, the aim…

Empowerment Evaluation: A Form of Self-Evaluation.

ERIC Educational Resources Information Center

Fetterman, David M.

Empowerment evaluation is an innovative approach that uses evaluation concepts and techniques to foster improvement and self-determination. Empowerment evaluation employs qualitative and quantitative methodologies. Although it can be applied to individuals and organizations, the usual focus is on programs. The value orientation of empowerment…

Developing, Implementing, and Evaluating a Formal Pharmacist Mentorship Program

PubMed Central

Nieuwstraten, Carmine; Huh, Alice; Liu, Gwen; Davis, Karen; Dolovich, Lisa

2011-01-01

Background: Mentoring is associated with positive professional and personal outcomes. However, there are few published data on mentoring programs for pharmacists. Objective: To develop and evaluate a mentorship program for hospital pharmacists that was implemented at St Joseph’s Healthcare Hamilton, in Hamilton, Ontario, by identifying the benefits and challenges that participants experienced and determining whether the program provided the necessary skills for a successful mentoring relationship. Methods: A descriptive pilot study was performed between June 2007 and November 2008. Focus groups and self-administered questionnaires were conducted at two time points (after 3–4 months and at the end of the study period). The focus groups were conducted separately for mentors and mentees. Data were summarized by predefined categories. Quantitative data from the questionnaires were summarized as medians, minimums, and maximums, and qualitative survey data were transcribed and reviewed. Results: Three mentors were each paired with a mentee. The mentees identified an average of 4 learning objectives. All of the mentees reported improvements in their self-perceived level of competency and skill within the mentoring relationship and their confidence in their ability to perform the functions of a hospital pharmacist. The job satisfaction of both mentors and mentees improved. Reported challenges were related to scheduling and documentation. Mentors and mentees reported high levels of overall satisfaction with the program, at both of the evaluation time points. Participants spent less than 60 min/week each on mentoring activities. Conclusions: Both mentors and mentees benefited from the mentoring relationship. PMID:22479041

Applying the chronic care model to an employee benefits program: a qualitative inquiry.

PubMed

Schauer, Gillian L; Wilson, Mark; Barrett, Barbara; Honeycutt, Sally; Hermstad, April K; Kegler, Michelle C

2013-12-01

To assess how employee benefits programs may strengthen and/or complement elements of the chronic care model (CCM), a framework used by health systems to improve chronic illness care. A qualitative inquiry consisting of semi-structured interviews with employee benefit administrators and partners from a self-insured, self-administered employee health benefits program was conducted at a large family-owned business in southwest Georgia. Results indicate that the employer adapted and used many health system-related elements of the CCM in the design of their benefit program. Data also suggest that the employee benefits program contributed to self-management skills and to informing and activating patients to interact with the health system. Findings suggest that employee benefits programs can use aspects of the CCM in their own benefit design, and can structure their benefits to contribute to patient-related elements from the CCM.

The Spiral-Interactive Program Evaluation Model.

ERIC Educational Resources Information Center

Khaleel, Ibrahim Adamu

1988-01-01

Describes the spiral interactive program evaluation model, which is designed to evaluate vocational-technical education programs in secondary schools in Nigeria. Program evaluation is defined; utility oriented and process oriented models for evaluation are described; and internal and external evaluative factors and variables that define each…

The portfolio as an evaluation tool: an analysis of its use in an undergraduate nursing program.

PubMed

Friedrich, Denise Barbosa de Castro; Gonçalves, Angela Maria Corrêa; de Sá, Tatiana Santos; Sanglard, Leticia Ribeiro; Duque, Débora Ribeiro; de Oliveira, Gabriela Mota Antunes

2010-01-01

This qualitative study was carried out between April and August 2007. It analyzed the use of portfolios in the academic community. A total of nine full-time professors and 119 students enrolled in their third semester were interviewed through a semi-structured interview. Content analysis was used to analyze data. Learning evaluations are seen as a verification of knowledge and efficacy of pedagogical method, and also as an incentive to study. Evaluations are procedural, that is, evaluation is continuous, or one-time, e.g. semester end tests. The portfolio is defined as a gradual and continuous evaluation tool. The faculty members and students need to accept the use of portfolios and evaluate the possibilities of this resource. This study is a first attempt to appraise the evaluation process of an undergraduate program, and the use of portfolios and other strategies needs to be consolidated in order to improve the educational process in undergraduate nursing programs.

Black Teachers Matter: Qualitative Study of Factors Influencing African American Candidates Success in a Teacher Preparation Program

ERIC Educational Resources Information Center

Dinkins, Elizabeth; Thomas, Kevin

2016-01-01

This qualitative study examined the perspectives and experiences of ten African American students at a predominantly White institution to understand why students persisted or discontinued in the teacher preparation program. Findings indicate three predominant factors influence Black candidates' decision to complete or leave the program: the role…

Program evaluation of an Integrated Basic Science Medical Curriculum in Shiraz Medical School, Using CIPP Evaluation Model

PubMed Central

ROOHOLAMINI, AZADEH; AMINI, MITRA; BAZRAFKAN, LEILA; DEHGHANI, MOHAMMAD REZA; ESMAEILZADEH, ZOHREH; NABEIEI, PARISA; REZAEE, RITA; KOJURI, JAVAD

2017-01-01

Introduction: In recent years curriculum reform and integration was done in many medical schools. The integrated curriculum is a popular concept all over the world. In Shiraz medical school, the reform was initiated by stablishing the horizontal basic science integration model and Early Clinical Exposure (ECE) for undergraduate medical education. The purpose of this study was to provide the required data for the program evaluation of this curriculum for undergraduate medical students, using CIPP program evaluation model. Methods: This study is an analytic descriptive and triangulation mixed method study which was carried out in Shiraz Medical School in 2012, based on the views of professors of basic sciences courses and first and second year medical students. The study evaluated the quality of the relationship between basic sciences and clinical courses and the method of presenting such courses based on the Context, Input, Process and Product (CIPP) model. The tools for collecting data, both quantitatively and qualitatively, were some questionnaires, content analysis of portfolios, semi- structured interview and brain storming sessions. For quantitative data analysis, SPSS software, version 14, was used. Results: In the context evaluation by modified DREEM questionnaire, 77.75%of the students believed that this educational system encourages them to actively participate in classes. Course schedule and atmosphere of class were reported suitable by 87.81% and 83.86% of students. In input domain that was measured by a researcher made questionnaire, the facilities for education were acceptable except for shortage of cadavers. In process evaluation, the quality of integrated modules presentation and Early Clinical Exposure (ECE) was good from the students’ viewpoint. In product evaluation, students’ brain storming, students’ portfolio and semi-structured interview with faculties were done, showing some positive aspects of integration and some areas that need

A qualitative analysis of immunization programs with sustained high coverage, 2000-2005.

PubMed

Kennedy, Allison; Groom, Holly; Evans, Victoria; Fasano, Nancy

2010-01-01

Despite record-high immunization coverage nationally, there is considerable variation across state and local immunization programs, which are responsible for the implementation of vaccine recommendations in their jurisdictions. The objectives of this study were to describe activities of state and local immunization programs that sustained high coverage levels across several years and to identify common themes and practical examples for sustaining childhood vaccination coverage rates that could be applied elsewhere. We conducted 95 semi-structured key informant interviews with internal staff members and external partners at the 10 immunization programs with the highest sustained childhood immunization coverage from 2000 to 2005, as measured by the National Immunization Survey. Interview transcripts were analyzed qualitatively using a general inductive approach. Common themes across the 10 programs included maintaining a strong program infrastructure, using available data to drive planning and decision making, a commitment to building and sustaining relationships, and a focus on education and communication. Given the challenges of an increasingly complex immunization system, the lessons learned from these programs may help inform others who are working to improve childhood immunization delivery and coverage in their own programs.

Program Theory Evaluation: Logic Analysis

ERIC Educational Resources Information Center

Brousselle, Astrid; Champagne, Francois

2011-01-01

Program theory evaluation, which has grown in use over the past 10 years, assesses whether a program is designed in such a way that it can achieve its intended outcomes. This article describes a particular type of program theory evaluation--logic analysis--that allows us to test the plausibility of a program's theory using scientific knowledge.…

Barriers to Engagement in a Workplace Weight Management Program: A Qualitative Study.

PubMed

Clancy, Shayna M; Stroo, Marissa; Schoenfisch, Ashley; Dabrera, Thushani; Østbye, Truls

2018-03-01

To investigate (1) why some participants in a workplace weight management program were more engaged in the program, (2) specific barriers and facilitators for engagement and weight loss, and (3) suggest how workplaces may better engage employees in these programs to improve their effectiveness. Qualitative study (8 focus groups). A large academic university and medical system. Twenty-six (5%) of the 550 employees who participated in a weight management program as part of the Steps to Health study. A trained moderator guided the audio-recorded focus groups. Transcripts were analyzed using the directed content analysis approach. Participants faced numerous barriers to engagement in workplace weight management programs, both within and outside the workplace. Participants viewed the coaches positively and reported that the coaches had a strong influence on their engagement in the program. Participants suggested increased frequency and variety of contact by coaches, on-site group exercise classes, and tailored educational materials. Workplace weight management programs may be improved by being more flexible around participants' schedules and changing needs, by increasing access to affordable, convenient exercise facilities, and by implementing institutional changes that encourage healthy eating and physical activity during the workday. Employers should measure program engagement and solicit participant feedback to ensure that the programs are appropriate and delivered in an optimal manner.

Readiness for Delivering Digital Health at Scale: Lessons From a Longitudinal Qualitative Evaluation of a National Digital Health Innovation Program in the United Kingdom.

PubMed

Lennon, Marilyn R; Bouamrane, Matt-Mouley; Devlin, Alison M; O'Connor, Siobhan; O'Donnell, Catherine; Chetty, Ula; Agbakoba, Ruth; Bikker, Annemieke; Grieve, Eleanor; Finch, Tracy; Watson, Nicholas; Wyke, Sally; Mair, Frances S

2017-02-16

Digital health has the potential to support care delivery for chronic illness. Despite positive evidence from localized implementations, new technologies have proven slow to become accepted, integrated, and routinized at scale. The aim of our study was to examine barriers and facilitators to implementation of digital health at scale through the evaluation of a £37m national digital health program: ‟Delivering Assisted Living Lifestyles at Scale" (dallas) from 2012-2015. The study was a longitudinal qualitative, multi-stakeholder, implementation study. The methods included interviews (n=125) with key implementers, focus groups with consumers and patients (n=7), project meetings (n=12), field work or observation in the communities (n=16), health professional survey responses (n=48), and cross program documentary evidence on implementation (n=215). We used a sociological theory called normalization process theory (NPT) and a longitudinal (3 years) qualitative framework analysis approach. This work did not study a single intervention or population. Instead, we evaluated the processes (of designing and delivering digital health), and our outcomes were the identified barriers and facilitators to delivering and mainstreaming services and products within the mixed sector digital health ecosystem. We identified three main levels of issues influencing readiness for digital health: macro (market, infrastructure, policy), meso (organizational), and micro (professional or public). Factors hindering implementation included: lack of information technology (IT) infrastructure, uncertainty around information governance, lack of incentives to prioritize interoperability, lack of precedence on accountability within the commercial sector, and a market perceived as difficult to navigate. Factors enabling implementation were: clinical endorsement, champions who promoted digital health, and public and professional willingness. Although there is receptiveness to digital health

Evaluating an employee wellness program.

PubMed

Mukhopadhyay, Sankar; Wendel, Jeanne

2013-12-01

What criteria should be used to evaluate the impact of a new employee wellness program when the initial vendor contract expires? Published academic literature focuses on return-on-investment as the gold standard for wellness program evaluation, and a recent meta-analysis concludes that wellness programs can generate net savings after one or two years. In contrast, surveys indicate that fewer than half of these programs report net savings, and actuarial analysts argue that return-on-investment is an unrealistic metric for evaluating new programs. These analysts argue that evaluation of new programs should focus on contract management issues, such as the vendor's ability to: (i) recruit employees to participate and (ii) induce behavior change. We compute difference-in-difference propensity score matching estimates of the impact of a wellness program implemented by a mid-sized employer. The analysis includes one year of pre-implementation data and three years of post-implementation data. We find that the program successfully recruited a broad spectrum of employees to participate, and it successfully induced short-term behavior change, as manifested by increased preventive screening. However, the effects on health care expenditures are positive (but insignificant). If it is unrealistic to expect new programs to significantly reduce healthcare costs in a few years, then focusing on return-on-investment as the gold standard metric may lead to early termination of potentially useful wellness programs. Focusing short-term analysis of new programs on short-term measures may provide a more realistic evaluation strategy.

Wiisokotaatiwin: development and evaluation of a community-based palliative care program in Naotkamegwanning First Nation.

PubMed

Nadin, Shevaun; Crow, Maxine; Prince, Holly; Kelley, Mary Lou

2018-04-01

Approximately 474 000 Indigenous people live in 617 First Nations communities across Canada; 125 of those communities are located in Ontario, primarily in rural and remote areas. Common rural health challenges, including for palliative care, involve quality and access. The need for culturally relevant palliative care programs in First Nations communities is urgent because the population is aging with a high burden of chronic and terminal disease. Because local palliative care is lacking, most First Nations people now leave their culture, family and community to receive care in distant hospitals or long-term care homes. Due to jurisdictional issues, a policy gap exists where neither federal nor provincial governments takes responsibility for funding palliative care in First Nations communities. Further, no Canadian program models existed for how different levels of government can collaborate to fund and deliver palliative care in First Nations communities. This article describes an innovative, community-based palliative care program (Wiisokotaatiwin) developed in rural Naotkamegwanning, and presents the results of a process evaluation of its pilot implementation. The evaluation aimed to (i) document the program's pilot implementation, (ii) assess progress toward intended program outcomes and (iii) assess the perceived value of the program. The Wiisokotaatiwin Program was developed and implemented over 5 years using participatory action research (http://www.eolfn.lakeheadu.ca). A mixed-method evaluation approach was adopted. Descriptive data were extracted from program documents (eg client registration forms). Client tracking forms documented service provision data for a 4-month sample period. Quantitative and qualitative data were collected through client and family member questionnaires (n=7) and healthcare provider questionnaires (n=22). A focus group was conducted with the program leadership team responsible for program development. Quantitative data were

Qualitative Evaluation of a New Tobacco Cessation Training Curriculum for Patient Navigators

PubMed Central

Shuk, Elyse; Krebs, Paul; Lu, Wei-Hsin; Burkhalter, Jack; Cortez-Weir, Jeralyn; Rodriguez, Rian; Burnside, Vanessa N.; Lubetkin, Erica I.

2011-01-01

Treatments for tobacco dependence exist but are underutilized, particularly among low-income and minority smokers. Patient navigation has been shown to help patients overcome barriers to quality care. In preparation for testing the feasibility of integrating tobacco cessation patient navigation into primary care, this paper describes the development and qualitative evaluation of a new curriculum for training patient navigators to address cessation treatment barriers faced by low-income, minority smokers who are advised to quit by their physicians. Thematic text analysis of transcripts obtained from focus groups with experienced patient navigators (n = 19) was conducted. Participants endorsed patient navigation as a relevant strategy for addressing tobacco cessation treatment barriers and made several recommendations regarding the knowledge, core competencies, and skills needed to conduct tobacco cessation patient navigation. This curriculum could be used by existing patient navigation training centers or made available as a self-guided continuing education program for experienced navigators who wish to expand their navigation interventions to include a tobacco cessation focus. PMID:21553331

Qualitative methods in PhD theses from general practice in Scandinavia.

PubMed

Malterud, Kirsti; Hamberg, Katarina; Reventlow, Susanne

2017-12-01

Qualitative methodology is gaining increasing attention and esteem in medical research, with general practice research taking a lead. With these methods, human and social interaction and meaning can be explored and shared by systematic interpretation of text from talk, observation or video. Qualitative studies are often included in Ph.D. theses from general practice in Scandinavia. Still, the Ph.D. programs across nations and institutions offer only limited training in qualitative methods. In this opinion article, we draw upon our observations and experiences, unpacking and reflecting upon values and challenges at stake when qualitative studies are included in Ph.D. theses. Hypotheses to explain these observations are presented, followed by suggestions for standards of evaluation and improvement of Ph.D. programs. The authors conclude that multimethod Ph.D. theses should be encouraged in general practice research, in order to offer future researchers an appropriate toolbox.

Qualitative methods in PhD theses from general practice in Scandinavia

PubMed Central

Malterud, Kirsti; Hamberg, Katarina; Reventlow, Susanne

2017-01-01

Qualitative methodology is gaining increasing attention and esteem in medical research, with general practice research taking a lead. With these methods, human and social interaction and meaning can be explored and shared by systematic interpretation of text from talk, observation or video. Qualitative studies are often included in Ph.D. theses from general practice in Scandinavia. Still, the Ph.D. programs across nations and institutions offer only limited training in qualitative methods. In this opinion article, we draw upon our observations and experiences, unpacking and reflecting upon values and challenges at stake when qualitative studies are included in Ph.D. theses. Hypotheses to explain these observations are presented, followed by suggestions for standards of evaluation and improvement of Ph.D. programs. The authors conclude that multimethod Ph.D. theses should be encouraged in general practice research, in order to offer future researchers an appropriate toolbox. PMID:29094644

A Qualitative Study of Fitness Instructors' Experiences Leading an Exercise Program for Children with Juvenile Idiopathic Arthritis

ERIC Educational Resources Information Center

Hutzal, Carolyn E.; Wright, F. Virginia; Stephens, Samantha; Schneiderman-Walker, Jane; Feldman, Brian M.

2009-01-01

Children with arthritis face challenges when they try to increase their physical activity. The study's objective was to identify elements of a successful community-based exercise program for children with arthritis by investigating the perspectives of fitness instructors who led the program. This qualitative study used a phenomenological approach.…

Understanding Acid-Base Concepts: Evaluating the Efficacy of a Senior High School Student-Centred Instructional Program in Indonesia

ERIC Educational Resources Information Center

Rahayu, Sri; Chandrasegaran, A. L.; Treagust, David F.; Kita, Masakazu; Ibnu, Suhadi

2011-01-01

This study was a mixed quantitative-qualitative research to evaluate the efficacy of a designed student-centred instructional (DSCI) program for teaching about acids and bases. The teaching innovation was designed based on constructivist, hands-on inquiry and context-based approaches and implemented in seven 45-min lessons with a class of 36 grade…

Reported credibility techniques in higher education evaluation studies that use qualitative methods: A research synthesis.

PubMed

Liao, Hongjing; Hitchcock, John

2018-06-01

This synthesis study examined the reported use of credibility techniques in higher education evaluation articles that use qualitative methods. The sample included 118 articles published in six leading higher education evaluation journals from 2003 to 2012. Mixed methods approaches were used to identify key credibility techniques reported across the articles, document the frequency of these techniques, and describe their use and properties. Two broad sets of techniques were of interest: primary design techniques (i.e., basic), such as sampling/participant recruitment strategies, data collection methods, analytic details, and additional qualitative credibility techniques (e.g., member checking, negative case analyses, peer debriefing). The majority of evaluation articles reported use of primary techniques although there was wide variation in the amount of supporting detail; most of the articles did not describe the use of additional credibility techniques. This suggests that editors of evaluation journals should encourage the reporting of qualitative design details and authors should develop strategies yielding fuller methodological description. Copyright © 2018 Elsevier Ltd. All rights reserved.

A Qualitative Exploration of Implementation Factors in a School-Based Mindfulness and Yoga Program: Lessons Learned from Students and Teachers

PubMed Central

Dariotis, Jacinda K.; Mirabal-Beltran, Roxanne; Cluxton-Keller, Fallon; Gould, Laura Feagans; Greenberg, Mark T.; Mendelson, Tamar

2016-01-01

Identifying factors relevant for successful implementation of school-based interventions is essential to ensure that programs are provided in an effective and engaging manner. The perspectives of two key stakeholders critical for identifying implementation barriers and facilitators – students and their classroom teachers – merit attention in this context and have rarely been explored using qualitative methods. This study reports qualitative perspectives of fifth and sixth grade participants and their teachers of a 16-week school-based mindfulness and yoga program in three public schools serving low-income urban communities. Four themes related to program implementation barriers and facilitators emerged: program delivery factors, program buy-in, implementer communication with teachers, and instructor qualities. Feedback from students and teachers is discussed in the context of informing implementation, adaptation, and future development of school-based mindfulness and yoga programming in urban settings. PMID:28670007

A Qualitative Exploration of Implementation Factors in a School-Based Mindfulness and Yoga Program: Lessons Learned from Students and Teachers.

PubMed

Dariotis, Jacinda K; Mirabal-Beltran, Roxanne; Cluxton-Keller, Fallon; Gould, Laura Feagans; Greenberg, Mark T; Mendelson, Tamar

2017-01-01

Identifying factors relevant for successful implementation of school-based interventions is essential to ensure that programs are provided in an effective and engaging manner. The perspectives of two key stakeholders critical for identifying implementation barriers and facilitators - students and their classroom teachers - merit attention in this context and have rarely been explored using qualitative methods. This study reports qualitative perspectives of fifth and sixth grade participants and their teachers of a 16-week school-based mindfulness and yoga program in three public schools serving low-income urban communities. Four themes related to program implementation barriers and facilitators emerged: program delivery factors, program buy-in, implementer communication with teachers, and instructor qualities. Feedback from students and teachers is discussed in the context of informing implementation, adaptation, and future development of school-based mindfulness and yoga programming in urban settings.

42 CFR 491.11 - Program evaluation.

Code of Federal Regulations, 2010 CFR

2010-10-01

... 42 Public Health 5 2010-10-01 2010-10-01 false Program evaluation. 491.11 Section 491.11 Public... Certification; and FQHCs Conditions for Coverage § 491.11 Program evaluation. (a) The clinic or center carries out, or arranges for, an annual evaluation of its total program. (b) The evaluation includes review of...

High Resolution Qualitative and Quantitative MR Evaluation of the Glenoid Labrum

PubMed Central

Iwasaki, Kenyu; Tafur, Monica; Chang, Eric Y.; SherondaStatum; Biswas, Reni; Tran, Betty; Bae, Won C.; Du, Jiang; Bydder, Graeme M.; Chung, Christine B.

2015-01-01

Objective To implement qualitative and quantitative MR sequences for the evaluation of labral pathology. Methods Six glenoid labra were dissected and the anterior and posterior portions were divided into normal, mildly degenerated, or severely degenerated groups using gross and MR findings. Qualitative evaluation was performed using T1-weighted, proton density-weighted (PD), spoiled gradient echo (SPGR) and ultra-short echo time (UTE) sequences. Quantitative evaluation included T2 and T1rho measurements as well as T1, T2*, and T1rho measurements acquired with UTE techniques. Results SPGR and UTE sequences best demonstrated labral fiber structure. Degenerated labra had a tendency towards decreased T1 values, increased T2/T2* values and increased T1 rho values. T2* values obtained with the UTE sequence allowed for delineation between normal, mildly degenerated and severely degenerated groups (p<0.001). Conclusion Quantitative T2* measurements acquired with the UTE technique are useful for distinguishing between normal, mildly degenerated and severely degenerated labra. PMID:26359581

Evaluating efforts to diversify the biomedical workforce: the role and function of the Coordination and Evaluation Center of the Diversity Program Consortium.

PubMed

McCreath, Heather E; Norris, Keith C; Calderόn, Nancy E; Purnell, Dawn L; Maccalla, Nicole M G; Seeman, Teresa E

2017-01-01

The National Institutes of Health (NIH)-funded Diversity Program Consortium (DPC) includes a Coordination and Evaluation Center (CEC) to conduct a longitudinal evaluation of the two signature, national NIH initiatives - the Building Infrastructure Leading to Diversity (BUILD) and the National Research Mentoring Network (NRMN) programs - designed to promote diversity in the NIH-funded biomedical, behavioral, clinical, and social sciences research workforce. Evaluation is central to understanding the impact of the consortium activities. This article reviews the role and function of the CEC and the collaborative processes and achievements critical to establishing empirical evidence regarding the efficacy of federally-funded, quasi-experimental interventions across multiple sites. The integrated DPC evaluation is particularly significant because it is a collaboratively developed Consortium Wide Evaluation Plan and the first hypothesis-driven, large-scale systemic national longitudinal evaluation of training programs in the history of NIH/National Institute of General Medical Sciences. To guide the longitudinal evaluation, the CEC-led literature review defined key indicators at critical training and career transition points - or Hallmarks of Success. The multidimensional, comprehensive evaluation of the impact of the DPC framed by these Hallmarks is described. This evaluation uses both established and newly developed common measures across sites, and rigorous quasi-experimental designs within novel multi-methods (qualitative and quantitative). The CEC also promotes shared learning among Consortium partners through working groups and provides technical assistance to support high-quality process and outcome evaluation internally of each program. Finally, the CEC is responsible for developing high-impact dissemination channels for best practices to inform peer institutions, NIH, and other key national and international stakeholders. A strong longitudinal evaluation across

Utility of the Living (Well Through) Intergenerational Fitness and Exercise Program as a County-Delivered Extension Program

ERIC Educational Resources Information Center

Sowle, Ashleigh J.; Francis, Sarah L.; Margrett, Jennifer A.; Franke, Warren D.

2016-01-01

Rural-residing older adults are not participating in regular physical activity. Extension is in an excellent position to fill this programming void through transdisciplinary programming such as the Living (well through) Intergenerational Fitness and Exercise (LIFE) program. Qualitative evaluation was conducted to assess the LIFE program's utility…

The Houston Community College Eligible Legalized Alien Program. Evaluation Program. Evaluation Report.

ERIC Educational Resources Information Center

Seaman, Don F.; Cuellar, Sylvia

The Houston Community College (Texas) program (TOTAL ACCESS) designed in response to the Immigration Reform and Control Act of 1986, is described and evaluated. The program offers classes to eligible aliens (97% Hispanic Americans from Mexico, El Salvador, and Guatemala) wishing to pursue the educational program required for legalization. Program…

Program Performance Inventory: Six Juvenile Offender Programs.

ERIC Educational Resources Information Center

Thomalla, Terri Groff; Dougherty, Victoria J.

This report describes the performance of 6 Connecticut juvenile justice alternative sanction programs in 14 qualitative areas: community reintegration; outcomes and evaluation; assessment methods; risk factors; escalation of criminal activity; family involvement; community involvement; work ethic and vocational training; education and life skills;…

Socioeconomic impact assessment in ex ante evaluations: a case study on the rural development programs of the European Union.

PubMed

Vidueira, Pablo; Díaz-Puente, José M; Rivera, María

2014-08-01

Ex ante impact assessment has become a fundamental tool for effective program management, and thus, a compulsory task when establishing a new program in the European Union (EU). This article aims to analyze benefits from ex ante impact assessment, methodologies followed, and difficulties encountered. This is done through the case study on the rural development programs (RDPs) in the EU. Results regarding methodologies are then contrasted with the international context in order to provide solid insights to evaluators and program managing authorities facing ex ante impact assessment. All European RDPs from the period 2007 through 2013 (a total of 88) and their corresponding available ex ante evaluations (a total of 70) were analyzed focusing on the socioeconomic impact assessment. Only 46.6% of the regions provide quantified impact estimations on socioeconomic impacts in spite of it being a compulsory task demanded by the European Commission (EC). Recommended methods by the EC are mostly used, but there is a lack of mixed method approaches since qualitative methods are used in substitution of quantitative ones. Two main difficulties argued were the complexity of program impacts and the lack of needed program information. Qualitative approaches on their own have been found as not suitable for ex ante impact assessment, while quantitative approaches-such as microsimulation models-provide a good approximation to actual impacts. However, time and budgetary constraints make that quantitative and mixed methods should be mainly applied on the most relevant impacts for the program success. © The Author(s) 2014.

Evaluating Dermatology Residency Program Websites.

PubMed

Ashack, Kurt A; Burton, Kyle A; Soh, Jonathan M; Lanoue, Julien; Boyd, Anne H; Milford, Emily E; Dunnick, Cory; Dellavalle, Robert P

2016-03-16

Internet resources play an important role in how medical students access information related to residency programs.Evaluating program websites is necessary in order to provide accurate information for applicants and provide information regarding areas of website improvement for programs. To date, dermatology residency websites (D  WS) have not been evaluated.This paper evaluates dermatology residency websites based on availability of predefined measures. Using the FREIDA (Fellowship and Residency Electronic Interactive Database) Online database, authors searched forall accredited dermatology program websites. Eligible programs were identified through the FREIDA Online database and had a functioning website. Two authors independently extracted data with consensus or third researcher resolution of differences. This data was accessed and archived from July 15th to July 17th, 2015.Primary outcomes measured were presence of content on education, resident and faculty information, program environment, applicant recruitment, schedule, salary, and website quality evaluated using an online tool (WooRank.com). Out of 117 accredited dermatology residencies, 115 had functioning webpages. Of these, 76.5% (75) had direct links found on the FRIEDA Online database. Most programs contained information on education, faculty, program environment, and applicant recruitment. However, website quality and marketing effectiveness were highly variable; most programs were deemed to need improvements in the functioning of their webpages. Also, additional information on current residents and about potential away rotations were lacking from most websites with only 52.2% (60) and 41.7% (48) of programs providing this content, respectively. A majority of dermatology residency websites contained adequate information on many of the factors we evaluated. However, many were lacking in areas that matter to applicants. We hope this report will encourage dermatology residencyprograms

Evaluation of a pilot promotora program for Latino forest workers in southern Oregon.

PubMed

Bush, Diane E; Wilmsen, Carl; Sasaki, Timothy; Barton-Antonio, Dinorah; Steege, Andrea L; Chang, Charlotte

2014-07-01

Forest work, an occupation with some of the highest injury and illness rates, is conducted primarily by Latino immigrant workers. This study evaluates a pilot program where promotoras (lay community health educators) provided occupational health and safety trainings for Latino forest workers. Evaluation methods included a focus group, post-tests, and qualitative feedback. Community capacity to address working conditions increased through (i) increased leadership and community access to information and resources; and (ii) increased worker awareness of workplace health and safety rights and resources. Fear of retaliation remains a barrier to workers taking action; nevertheless, the promotoras supported several workers in addressing-specific workplace issues. For working conditions to significantly improve, major structural influences need to be addressed. A long-term, organizationally supported promotora program can play a key role in linking and supporting change at the individual, interpersonal and community levels, contributing to and supporting structural change. © 2014 Wiley Periodicals, Inc.

A qualitative assessment of program characteristics for preventing secondary conditions in individuals with fetal alcohol spectrum disorders.

PubMed

Patrenko, Christie L M; Tahir, Naira; Mahoney, Erin C; Chin, Nancy P

2014-01-01

Fetal alcohol spectrum disorders (FASD) are a major public health problem that affects 2 to 5 percent of the population. Individuals with FASD are at high risk for secondary conditions, such as mental health problems, school disruptions, and trouble with the law. Evidence-based intervention programs are needed to prevent and treat secondary conditions in this population. The purpose of this study was to identify intervention program characteristics for preventing secondary conditions in individuals with FASD from the perspectives of parents and service providers. This qualitative study utilized a phenomenological approach to identify program characteristics for preventing secondary conditions. Twenty-five parents of children (ages 3 to 33) with FASD and 18 service providers participated in focus groups or individual interviews. Data was systematically analyzed using a framework approach. Themes did not differ by participant type. Participants emphasized five primary characteristics of intervention programs for individuals with FASD. Programs need to 1) be available to individuals across the lifespan, 2) have a prevention focus, 3) be individualized, 4) be comprehensive, and 5) be coordinated across systems and developmental stages. Participants discussed a variety of specific intervention strategies for each developmental stage and setting. Program characteristics identified in this study are consistent with a positive behavior support framework. This framework is discussed in the context of research on existing interventions for individuals with FASD, and recommendations for future intervention development and evaluation are highlighted.

Evaluation of Turkish Education Programs

ERIC Educational Resources Information Center

Durmuscelebi, Mustafa

2010-01-01

The aim of this study is to evaluate the new Turkish education program that has been being implemented since 2005 gradually in light of teacher suggestions. The study has been done in scanning model. In this study which has been conducted with the purpose of evaluating the newly prepared Turkish education programs, the program has been tried to be…

General Criteria for Evaluating Social Programs.

ERIC Educational Resources Information Center

Shipman, Stephanie

1989-01-01

A framework of general evaluation criteria for ensuring the comprehensiveness of program reviews and appropriate and fair comparison of children's programs is outlined. It has two components: (1) descriptive; and (2) evaluative. The framework was developed by researchers at the General Accounting Office for evaluation of federal programs. (TJH)

Methodological Reflections on the Contribution of Qualitative Research to the Evaluation of Clinical Ethics Support Services.

PubMed

Wäscher, Sebastian; Salloch, Sabine; Ritter, Peter; Vollmann, Jochen; Schildmann, Jan

2017-05-01

This article describes a process of developing, implementing and evaluating a clinical ethics support service intervention with the goal of building up a context-sensitive structure of minimal clinical-ethics in an oncology department without prior clinical ethics structure. Scholars from different disciplines have called for an improvement in the evaluation of clinical ethics support services (CESS) for different reasons over several decades. However, while a lot has been said about the concepts and methodological challenges of evaluating CESS up to the present time, relatively few empirical studies have been carried out. The aim of this article is twofold. On the one hand, it describes a process of development, modifying and evaluating a CESS intervention as part of the ETHICO research project, using the approach of qualitative-formative evaluation. On the other hand, it provides a methodological analysis which specifies the contribution of qualitative empirical methods to the (formative) evaluation of CESS. We conclude with a consideration of the strengths and limitations of qualitative evaluation research with regards to the evaluation and development of context sensitive CESS. We further discuss our own approach in contrast to rather traditional consult or committee models. © 2017 John Wiley & Sons Ltd.

IMAGINE-ing interprofessional education: program evaluation of a novel inner city health educational experience

PubMed Central

Hu, Tina; Cox, Kelly Anne; Nyhof-Young, Joyce

2017-01-01

Background Poverty is a key determinant of health that leads to poor health outcomes. Although most healthcare providers will work with patients experiencing poverty, surveys among healthcare students have reported a curriculum gap in this area. This study aims to introduce and evaluate a novel, student-run interprofessional inner city health educational program that combines both practical and didactic educational components. Methods Students participating in the program answered pre- and post-program surveys. Wilcoxon signed-rank tests and descriptive thematic analysis were used for quantitative and qualitative data, respectively. Results A total of 28 out of 35 participants responded (response rate: 80%). Student knowledge about issues facing underserved populations and resources for underserved populations significantly increased after program participation. Student comfort working with underserved populations also significantly increased after program participation. Valued program elements included workshops, shadowing, and a focus on marginalized populations. Conclusion Interprofessional inner city health educational programs are beneficial for students to learn about poverty intervention and resources, and may represent a strategy to address a gap in the healthcare professional curriculum. PMID:28344718

A Qualitative Analysis of Common Concerns about Challenges Facing Pharmacy Experiential Education Programs

PubMed Central

Craddick, Karen; Eccles, Dayl; Kwasnik, Abigail; O’Sullivan, Teresa A.

2015-01-01

Objective. To qualitatively analyze free-text responses gathered as part of a previously published survey in order to systematically identify common concerns facing pharmacy experiential education (EE) programs. Methods. In 2011, EE directors at all 118 accredited pharmacy schools in the US were asked in a survey to describe the most pressing issues facing their programs. Investigators performed qualitative, thematic analysis of responses and compared results against demographic data (institution type, class size, number of practice sites, number and type of EE faculty member/staff). Expert and novice investigators identified common themes via an iterative process. To check validity, additional expert and novice reviewers independently coded responses. The Cohen kappa coefficient was calculated and showed good agreement between investigators and reviewers. Results. Seventy-eight responses were received (66% response rate) representing 75% of publicly funded institutions and 71% of schools with class sizes 51-150. Themes identified as common concerns were site capacity, workload/financial support, quality assurance, preceptor development, preceptor stipends, assessment, onboarding, and support/recognition from administration. Good agreement (mean percent agreement 93%, ƙ range=0.59-0.92) was found between investigators and reviewers. Conclusion. Site capacity for student placements continues to be the foremost concern for many experiential education programs. New concerns about preceptor development and procedures for placing and orienting students at individual practice sites (ie, “onboarding”) have emerged and must be addressed as new accreditation standards are implemented. PMID:25741022

Mentoring doctoral students for qualitative research: interviews with experienced nursing faculty in Japan.

PubMed

Kayama, Mami; Gregg, Misuzu F; Asahara, Kiyomi; Yamamoto-Mitani, Noriko; Okuma, Keiko; Ohta, Kikuko; Kinoshita, Yasuhito

2013-05-01

This study aimed to describe the process of mentoring doctoral students for qualitative research in Japanese graduate programs in nursing. Nine experienced faculty-seven nurse researchers and two sociologists-were interviewed. Participants were asked about their process of mentoring students for qualitative nursing dissertations. Data analysis was conducted using a qualitative descriptive method. Participants' age ranged from 48 to 60 years. The first theme in the mentoring process is about the individualized, one-on-one mentorship process. The second theme occurs in a group process. The third theme is coordinating mentors and establishing a network to support the evaluation system. The mentoring processes identified in this study will be useful for future faculty development. The study elucidated much room for improvement in doctoral education programs for qualitative research methods in nursing science. Copyright 2013, SLACK Incorporated.

A Qualitative Case Study Exploring an Early Childhood Educator Professional Development Program Linking Teacher Efficacy, Engagement, and Retention

ERIC Educational Resources Information Center

Guerrant, Lisa Y.

2016-01-01

This qualitative study explored the lived experiences of eleven early childhood educators who participated in a professional development program. The study was guided by the central research question, "What are the perceptions of early childhood educators on the professional development program as it relates to teacher efficacy, engagement,…

Evaluation of a rural chronic disease self-management program.

PubMed

Stone, Genevieve R; Packer, Tanya L

2010-01-01

Internationally, the prevalence of long-term health conditions is at epidemic proportions. Australia is no exception. The Australian Government's 'Better Health Initiative' has 5 key strategies to build better health care, one of which is the adoption of self-management and self-management support. Self-management allows people to manage their condition and the consequences it brings to their lives in partnership with their health providers. The purpose of this article was to report both the process and patient outcomes following the introduction of the Stanford Chronic Disease Self-Management Program (CDSMP) into an existing service in an Australian rural setting. Implementation processes were evaluated using semi-structured interviews conducted with managers, lay and health professional course leaders and participants about positive and negative aspects of providing the CDSMP. Participant outcomes were evaluated using a modified pre-test, post-test design to evaluate changes in activity participation and self-management knowledge and skills. Both negative and positive aspects of providing the program were represented by two key themes: (1) program content and quality; and (2) logistics of delivery. Throughout the interviews, managers and leaders, and course participants offered recommendations that were thematically grouped into 3 categories: (1) enhancing quality; (2) improving the logistics; and (3) providing resources. Comparison of activity levels with a community sample indicated that participants had significantly decreased participation levels. Scores on the Health Education Impact Questionnaire v2 (heiQ - RETRO) demonstrated statistically better scores at post-test on the domains of 'self monitoring', 'insight' and 'health service navigation' with a trend towards significance on 3 other domains. Future implementation of CDSMPs in rural areas will be encouraged by these patient outcomes, and informed by the qualitative findings from managers, leaders and

Staff Development Program Evaluation.

ERIC Educational Resources Information Center

Ashur, Nina E.; And Others

An evaluation of the staff development program at College of the Canyons (California) was conducted in 1991 to provide information applicable to program improvement. Questionnaires were distributed to all faculty, classified staff, and flexible calendar program committee and staff development advisory committee members, resulting in response rates…

Implementation evaluation of the Telephone Lifestyle Coaching (TLC) program: organizational factors associated with successful implementation.

PubMed

Damschroder, Laura J; Reardon, Caitlin M; Sperber, Nina; Robinson, Claire H; Fickel, Jacqueline J; Oddone, Eugene Z

2017-06-01

The Telephone Lifestyle Coaching (TLC) program provided telephone-based coaching for six lifestyle behaviors to 5321 Veterans at 24 Veterans Health Administration (VHA) medical facilities. The purpose of the study was to conduct an evaluation of the TLC program to identify factors associated with successful implementation. A mixed-methods study design was used. Quantitative measures of organizational readiness for implementation and facility complexity were used to purposively select a subset of facilities for in-depth evaluation. Context assessments were conducted using interview transcripts. The Consolidated Framework for Implementation Research (CFIR) was used to guide qualitative data collection and analysis. Factors most strongly correlated with referral rates included having a skilled implementation leader who used effective multi-component strategies to engage primary care clinicians as well as general clinic structures that supported implementation. Evaluation findings pointed to recommendations for local and national leaders to help anticipate and mitigate potential barriers to successful implementation.

Solar energy program evaluation: an introduction

DOE Office of Scientific and Technical Information (OSTI.GOV)

deLeon, P.

The Program Evaluation Methodology provides an overview of the practice and methodology of program evaluation and defines more precisely the evaluation techniques and methodologies that would be most appropriate to government organizations which are actively involved in the research, development, and commercialization of solar energy systems. Formal evaluation cannot be treated as a single methodological approach for assessing a program. There are four basic types of evaluation designs - the pre-experimental design; the quasi-experimental design based on time series; the quasi-experimental design based on comparison groups; and the true experimental design. This report is organized to first introduce the rolemore » and issues of evaluation. This is to provide a set of issues to organize the subsequent sections detailing the national solar energy programs. Then, these two themes are integrated by examining the evaluation strategies and methodologies tailored to fit the particular needs of the various individual solar energy programs. (MCW)« less

Implementing a collaborative return-to-work program: Lessons from a qualitative study in a large Canadian healthcare organization.

PubMed

Skivington, Kathryn; Lifshen, Marni; Mustard, Cameron

2016-11-22

Comprehensive workplace return-to-work policies, applied with consistency, can reduce length of time out of work and the risk of long-term disability. This paper reports on the findings from a qualitative study exploring managers' and return-to-work-coordinators' views on the implementation of their organization's new return-to-work program. To provide practical guidance to organizations in designing and implementing return-to-work programs for their employees. Semi-structured qualitative interviews were undertaken with 20 managers and 10 return-to-work co-ordinators to describe participants' perspectives on the progress of program implementation in the first 18 months of adoption. The study was based in a large healthcare organization in Ontario, Canada. Thematic analysis of the data was conducted. We identified tensions evident in the early implementation phase of the organization's return-to-work program. These tensions were attributed to uncertainties concerning roles and responsibilities and to circumstances where objectives or principles appeared to be in conflict. The implementation of a comprehensive and collaborative return-to-work program is a complex challenge. The findings described in this paper may provide helpful guidance for organizations embarking on the development and implementation of a return-to-work program.

Critiquing qualitative research.

PubMed

Beck, Cheryl Tatano

2009-10-01

The ability to critique research is a valuable skill that is fundamental to a perioperative nurse's ability to base his or her clinical practice on evidence derived from research. Criteria differ for critiquing a quantitative versus a qualitative study (ie, statistics are evaluated in a quantitative study, but not in a qualitative study). This article provides on guidelines for assessing qualitative research. Excerpts from a published qualitative research report are summarized and then critiqued. Questions are provided that help evaluate different sections of a research study (eg, sample, data collection methods, data analysis).

How Service Works. Summary of the W. K. Kellogg Foundation's "Service Works": A Retrospective Evaluation of Higher Education Service Programs.

ERIC Educational Resources Information Center

Kellogg Foundation, Battle Creek, MI.

Service learning is an undergraduate education strategy found across four program areas funded by the W. K. Kellogg Foundation (WKKF). This cross-foundation, retrospective evaluation examines 35 projects of $100,000 or more funded between 1985-95. Study information was based on qualitative data and approaches, with the majority of project…

Monitoring and evaluation framework for hypertension programs. A collaboration between the Pan American Health Organization and World Hypertension League.

PubMed

Campbell, Norm R C; Ordunez, Pedro; DiPette, Donald J; Giraldo, Gloria P; Angell, Sonia Y; Jaffe, Marc G; Lackland, Dan; Martinez, Ramón; Valdez, Yamilé; Maldonado Figueredo, Javier I; Paccot, Melanie; Santana, Maria J; Whelton, Paul K

2018-06-01

The Pan American Health Organization (PAHO)-World Hypertension League (WHL) Hypertension Monitoring and Evaluation Framework is summarized. Standardized indicators are provided for monitoring and evaluating national or subnational hypertension control programs. Five core indicators from the World Health Organization hearts initiative and a single PAHO-WHL core indicator are recommended to be used in all hypertension control programs. In addition, hypertension control programs are encouraged to select from 14 optional qualitative and 33 quantitative indicators to facilitate progress towards enhanced hypertension control. The intention is for hypertension programs to select quantitative indicators based on the current surveillance mechanisms that are available and what is feasible and to use the framework process indicators as a guide to program management. Programs may wish to increase or refine the number of indicators they use over time. With adaption the indicators can also be implemented at a community or clinic level. The standardized indicators are being pilot tested in Cuba, Colombia, Chile, and Barbados. ©2018 Wiley Periodicals, Inc.

What influences success in family medicine maternity care education programs? Qualitative exploration.

PubMed

Biringer, Anne; Forte, Milena; Tobin, Anastasia; Shaw, Elizabeth; Tannenbaum, David

2018-05-01

To ascertain how program leaders in family medicine characterize success in family medicine maternity care education and determine which factors influence the success of training programs. Qualitative research using semistructured telephone interviews. Purposive sample of 6 family medicine programs from 5 Canadian provinces. Eighteen departmental leaders and program directors. Semistructured telephone interviews were conducted with program leaders in family medicine maternity care. Departmental leaders identified maternity care programs deemed to be "successful." Interviews were audiorecorded and transcribed verbatim. Team members conducted thematic analysis. Participants considered their education programs to be successful in family medicine maternity care if residents achieved competency in intrapartum care, if graduates planned to include intrapartum care in their practices, and if their education programs were able to recruit and retain family medicine maternity care faculty. Five key factors were deemed to be critical to a program's success in family medicine maternity care: adequate clinical exposure, the presence of strong family medicine role models, a family medicine-friendly hospital environment, support for the education program from multiple sources, and a dedicated and supportive community of family medicine maternity care providers. Training programs wishing to achieve greater success in family medicine maternity care education should employ a multifaceted strategy that considers all 5 of the interdependent factors uncovered in our research. By paying particular attention to the informal processes that connect these factors, program leaders can preserve the possibility that family medicine residents will graduate with the competence and confidence to practise full-scope maternity care. Copyright© the College of Family Physicians of Canada.

Designing an evaluation for a multiple-strategy community intervention: the North Coast Stay on Your Feet program.

PubMed

van Beurden, E; Kempton, A; Sladden, T; Garner, E

1998-02-01

Evaluation of the North Coast Stay on Your Feet falls prevention program is described as a case study of a comprehensive evaluation design for multi-strategic community interventions. Qualitative and quantitative methods were used to evaluate the program at formative, process and outcome levels. Formative evaluation used literature review, focus groups, mail-out and telephone survey methods to gather evidence from publications, older people, health workers, local business, media and government bodies. It included an analysis of demographic and hospital databases and identified incidence, causal pathways, knowledge, attitudes, behaviour, consequences and effectiveness of potential strategies. Process evaluation employed auditing, monitoring and telephone surveys to maintain an inventory of intervention activities and to track the reach of the program. Outcome evaluation involved a longitudinal study of intervention and control cohorts, surveyed before, during and after the intervention by telephone to monitor changes in knowledge, attitudes, risk and falls incidence. The survey instrument was designed for both formative and outcome evaluation, and analysis reflected the research design by incorporating repeat measures and adjusting for bias and confounding. Outcome validity was cross-checked via hospital admission rates. A novel, integrated framework for presenting inputs, activities and outcomes from all stages of the program is described. This framework facilitated feedback to stakeholders and enabled subsequent rapid adjustment of the intervention. Rigorous evaluation combined with clear presentation of findings helped to engender intersectoral support and obtain funding grants for extended implementation and evaluation. It also helped Stay on Your Feet to become a model for other falls prevention programs within Australia and internationally.

Critical evaluation of international health programs: Reframing global health and evaluation.

PubMed

Chi, Chunhuei; Tuepker, Anaïs; Schoon, Rebecca; Núñez Mondaca, Alicia

2018-04-01

Striking changes in the funding and implementation of international health programs in recent decades have stimulated debate about the role of communities in deciding which health programs to implement. An important yet neglected piece of that discussion is the need to change norms in program evaluation so that analysis of community ownership, beyond various degrees of "participation," is seen as central to strong evaluation practices. This article challenges mainstream evaluation practices and proposes a framework of Critical Evaluation with 3 levels: upstream evaluation assessing the "who" and "how" of programming decisions; midstream evaluation focusing on the "who" and "how" of selecting program objectives; and downstream evaluation, the focus of current mainstream evaluation, which assesses whether the program achieved its stated objectives. A vital tenet of our framework is that a community possesses the right to determine the path of its health development. A prerequisite of success, regardless of technical outcomes, is that programs must address communities' high priority concerns. Current participatory methods still seldom practice community ownership of program selection because they are vulnerable to funding agencies' predetermined priorities. In addition to critiquing evaluation practices and proposing an alternative framework, we acknowledge likely challenges and propose directions for future research. Copyright © 2018 John Wiley & Sons, Ltd.

Execution of a participatory supportive return to work program within the Dutch social security sector: a qualitative evaluation of stakeholders' perceptions.

PubMed

Lammerts, Lieke; Schaafsma, Frederieke G; van Mechelen, Willem; Anema, Johannes R

2016-04-14

A process evaluation of a participatory supportive return to work program, aimed at workers without a (permanent) employment contract who are sick-listed due to a common mental disorder, revealed that this program was executed less successfully than similar programs evaluated in earlier studies. The program consisted of a participatory approach, integrated care and direct placement in competitive employment. Aim of this study was to get a better understanding of the execution of the program by evaluating stakeholders' perceptions. In the absence of an employer, the program was applied by the Dutch Social Security Agency, in collaboration with vocational rehabilitation agencies. Together with the sick-listed workers, these were the main stakeholders. Our research questions involved stakeholders' perceptions of the function(s) of the program, and their perceptions of barriers and facilitators for a successful execution of the program within the Dutch social security sector. Semi-structured interviews were held with five sick-listed workers, eight professionals of the Social Security Agency, and two case managers of vocational rehabilitation agencies. Interview topics were related to experiences with different components of the program. Selection of respondents was based on purposive sampling and continued until data saturation was reached. Content analysis was applied to identify patterns in the data. Two researchers developed a coding system, based on predefined topics and themes emerging from the data. Although perceived functions of some components of the program were as intended, all stakeholders stressed that the program often had not resulted in return to work. Perceived barriers for a successful execution were related to a poor collaboration between the Dutch Social Security Agency, vocational rehabilitation agencies and healthcare providers, the type of experienced (health) problems, time constraints, and limited job opportunities. For future implementation

Integrating Program Theory and Systems-Based Procedures in Program Evaluation: A Dynamic Approach to Evaluate Educational Programs

ERIC Educational Resources Information Center

Grammatikopoulos, Vasilis

2012-01-01

The current study attempts to integrate parts of program theory and systems-based procedures in educational program evaluation. The educational program that was implemented, called the "Early Steps" project, proposed that physical education can contribute to various educational goals apart from the usual motor skills improvement. Basic…

Accelerated Training of Skilled Birth Attendants in a Marginalized Population on the Thai-Myanmar Border: A Multiple Methods Program Evaluation.

PubMed

White, Adrienne Lynne; Min, Thaw Htwe; Gross, Mechthild M; Kajeechiwa, Ladda; Thwin, May Myo; Hanboonkunupakarn, Borimas; Than, Hla Hla; Zin, Thet Wai; Rijken, Marcus J; Hoogenboom, Gabie; McGready, Rose

2016-01-01

To evaluate a skilled birth attendant (SBA) training program in a neglected population on the Thai-Myanmar border, we used multiple methods to show that refugee and migrant health workers can be given effective training in their own environment to become SBAs and teachers of SBAs. The loss of SBAs through resettlement to third countries necessitated urgent training of available workers to meet local needs. All results were obtained from student records of theory grades and clinical log books. Qualitative evaluation of both the SBA and teacher programs was obtained using semi-structured interviews with supervisors and teachers. We also reviewed perinatal indicators over an eight-year period, starting prior to the first training program until after the graduation of the fourth cohort of SBAs. Four SBA training programs scheduled between 2009 and 2015 resulted in 79/88 (90%) of students successfully completing a training program of 250 theory hours and 625 supervised clinical hours. All 79 students were able to: achieve pass grades on theory examination (median 80%, range [70-89]); obtain the required clinical experience within twelve months; achieve clinical competence to provide safe care during childbirth. In 2010-2011, five experienced SBAs completed a train-the-trainer (TOT) program and went on to facilitate further training programs. Perinatal indicators within Shoklo Malaria Research Unit (SMRU), such as place of birth, maternal and newborn outcomes, showed no significant differences before and after introduction of training or following graduate deployment in the local maternity units. Confidence, competence and teamwork emerged from qualitative evaluation by senior SBAs working with and supervising students in the clinics. We demonstrate that in resource-limited settings or in marginalized populations, it is possible to accelerate training of skilled birth attendants to provide safe maternity care. Education needs to be tailored to local needs to ensure

Evaluation: Review of the Past, Preview of the Future.

ERIC Educational Resources Information Center

Smith, M. F.

1994-01-01

This paper summarized contributors' ideas about evaluation as a field and where it is going. Topics discussed were qualitative versus quantitative debate; evaluation's purpose; professionalization; program failure; program development; evaluators as advocates; evaluation knowledge; evaluation expansion; and methodology and design. (SLD)

Use of program logic models in the Southern Rural Access Program evaluation.

PubMed

Pathman, Donald; Thaker, Samruddhi; Ricketts, Thomas C; Albright, Jennifer B

2003-01-01

The Southern Rural Access Program (SRAP) evaluation team used program logic models to clarify grantees' activities, objectives, and timelines. This information was used to benchmark data from grantees' progress reports to assess the program's successes. This article presents a brief background on the use of program logic models--essentially charts or diagrams specifying a program's planned activities, objectives, and goals--for evaluating and managing a program. It discusses the structure of the logic models chosen for the SRAP and how the model concept was introduced to the grantees to promote acceptance and use of the models. The article describes how the models helped clarify the program's objectives and helped lead agencies plan and manage the many program initiatives and subcontractors in their states. Models also provided a framework for grantees to report their progress to the National Program Office and evaluators and promoted the evaluators' visibility and acceptance by the grantees. Program logics, however, increased grantees' reporting requirements and demanded substantial time of the evaluators. Program logic models, on balance, proved their merit in the SRAP through their contributions to its management and evaluation and by providing a better understanding of the program's initiatives, successes, and potential impact.

Tailored program evaluation: Past, present, future.

PubMed

Suggs, L Suzanne; Cowdery, Joan E; Carroll, Jennifer B

2006-11-01

This paper discusses measurement issues related to the evaluation of computer-tailored health behavior change programs. As the first generation of commercially available tailored products is utilized in health promotion programming, programmers and researchers are becoming aware of the unique challenges that the evaluation of these programs presents. A project is presented that used an online tailored health behavior assessment (HBA) in a worksite setting. Process and outcome evaluation methods are described and include the challenges faced, and strategies proposed and implemented, for meeting them. Implications for future research in tailored program development, implementation, and evaluation are also discussed.

How To Design a Program Evaluation. CSE Program Evaluation Kit, Volume 3. Second Edition.

ERIC Educational Resources Information Center

Fitz-Gibbon, Carol Taylor; Morris, Lynn Lyons

The "CSE Program Evaluation Kit" is a series of nine books intended to assist people conducting program evaluations. This volume, the third in the kit, discusses the logic underlying the use of quantitative research designs, including the pretest-posttest design, and supplies step-by-step procedures for setting up and interpreting the…

Developing an Integrative Treatment Program for Cancer-Related Fatigue Using Stakeholder Engagement - A Qualitative Study.

PubMed

Canella, Claudia; Mikolasek, Michael; Rostock, Matthias; Beyer, Jörg; Guckenberger, Matthias; Jenewein, Josef; Linka, Esther; Six, Claudia; Stoll, Sarah; Stupp, Roger; Witt, Claudia M

2017-11-01

Although cancer-related fatigue (CRF) has gained increased attention in the past decade, it remains difficult to treat. An integrative approach combining conventional and complementary medicine interventions seems highly promising. Treatment programs are more likely to be effective if the needs and interests of the people involved are well represented. This can be achieved through stakeholder engagement. The aim of the study was to develop an integrative CRF treatment program using stakeholder engagement and to compare it to an expert version. In a qualitative study, a total of 22 stakeholders (4 oncologists, 1 radiation-oncologist, 1 psycho-oncologist, 5 nurses/nurse experts, 9 patients, 1 patient family member, 1 representative of a local Swiss Cancer League) were interviewed either face-to-face or in a focus group setting. For data analysis, qualitative content analysis was used. With stakeholder engagement, the integrative CRF treatment program was adapted to usual care using a prioritizing approach and allowing more patient choice. Unlike the expert version, in which all intervention options were on the same level, the stakeholder engagement process resulted in a program with 3 different levels. The first level includes mandatory nonpharmacological interventions, the second includes nonpharmacological choice-based interventions, and the third includes pharmacological interventions for severe CRF. The resulting stakeholder based integrative CRF treatment program was implemented as clinical practice guideline at our clinic (Institute for Complementary and Integrative Medicine, University Hospital Zurich). Through the stakeholder engagement approach, we integrated the needs and preferences of people who are directly affected by CRF. This resulted in an integrative CRF treatment program with graded recommendations for interventions and therefore potentially greater sustainability in a usual care setting.

Teacher Evaluation: A Qualitative Study on Growth-Producing Practices in K-8 Schools

ERIC Educational Resources Information Center

Manso, Phoebe Hechanova

2017-01-01

Literature on teacher evaluation describes a flawed system that evokes wide-ranging attempts to "get it right." This qualitative study is another effort that seeks to understand its legal and traditional constructs. The goal of this research is to identify growth-producing practices that will transform teacher evaluation into an organic…

[Reconsidering evaluation criteria regarding health care research: toward an integrative framework of quantitative and qualitative criteria].

PubMed

Miyata, Hiroaki; Kai, Ichiro

2006-05-01

Debate about the relationship between quantitative and qualitative paradigms is often muddled and confused and the clutter of terms and arguments has resulted in the concepts becoming obscure and unrecognizable. It is therefore very important to reconsider evaluation criteria regarding rigor in social science. As Lincoln & Guba have already compared quantitative paradigms (validity, reliability, neutrality, generalizability) with qualitative paradigms (credibility, dependability, confirmability, transferability), we have discuss use of evaluation criteria based on pragmatic perspective. Validity/Credibility is the paradigm concerned to observational framework, while Reliability/Dependability refer to the range of stability in observations, Neutrality/Confirmability reflect influences between observers and subjects, Generalizability/Transferability have epistemological difference in the way findings are applied. Qualitative studies, however, does not always chose the qualitative paradigms. If we assume the stability to some extent, it is better to use the quantitative paradigm (reliability). Moreover as a quantitative study can not always guarantee a perfect observational framework, with stability in all phases of observations, it is useful to use qualitative paradigms to enhance the rigor in the study.

An Evaluation Program for the Eckerd Foundation Therapeutic Wilderness Camping Program: An Evaluation of an Atypical Alternative Education Program.

ERIC Educational Resources Information Center

Griffin, William H.; Carter, James D.

The strategy used in evaluating an out-of-doors resident camping program for emotionally disturbed children is outlined. This strategy calls for examining the following elements in the program: (1) program goals and objectives; (2) collection and processing program data; (3) camper progress assessment; (4) program audit; (5) assessment of past…

Evaluating adolescent pregnancy programs: rethinking our priorities.

PubMed

Stahler, G J; DuCette, J P

1991-01-01

Noting that impact evaluations of adolescent pregnancy programs are characterized by poor quality, the authors recommend using a different standard in assessing the value of programs. While the number of adolescent pregnancy programs has multiplied during the last 3 decades, little is known about their impact in ameliorating the negative consequences of too-early childbearing. An ideal evaluation of these programs would randomly select and randomly assign subjects to experimental and control groups. But evaluations conducted by individual program generally face obstacles that limit the randomness of the study. most individual programs lack the financial resources and do not employ the full-time professional evaluators needed to carry out a valid evaluation. These factors result in too short an evaluation period, incomplete and inaccurate data, and lack of randomness in the assignment of control groups. To more accurately assess the impact of the programs, the authors recommend that individual programs focus on process evaluation and collection of complete and reliable data on their clients. From the onset, a program should have a clear description of its content, logic of intervention, and method of implementation. It should maintain thorough records on client characteristics, service utilization, and should conduct long-term follow-ups. For rigorous impact evaluations, programs should rely on 3rd party entities. These independent organizations -- universities or research institutes -- do not have a stake in the outcome of the evaluation, making the study all the more objective. Furthermore, they provide experienced researchers.

Using Computational Text Classification for Qualitative Research and Evaluation in Extension

ERIC Educational Resources Information Center

Smith, Justin G.; Tissing, Reid

2018-01-01

This article introduces a process for computational text classification that can be used in a variety of qualitative research and evaluation settings. The process leverages supervised machine learning based on an implementation of a multinomial Bayesian classifier. Applied to a community of inquiry framework, the algorithm was used to identify…

Program Fighter - An Evaluation.

ERIC Educational Resources Information Center

Hull, David G.; Fowler, Wallace T.

Described is a computer program for the sizing of subsonic and supersonic fighters which has been adapted for use in an aerospace engineering design course. Following a description of the program, an evaluation of its use in the university is presented. It is concluded that computer programs for the conceptual design of aerospace vehicles can play…

Qualitative Comparison of Streamflow Information Programs of the U.S. Geological Survey and Three Non-Federal Agencies

USGS Publications Warehouse

Norris, J. Michael; Lewis, Michael; Dorsey, Michael; Kimbrough, Robert; Holmes, Robert R.; Staubitz, Ward

2008-01-01

A qualitative comparison was made of the streamgaging programs of the U.S. Geological Survey (USGS) and three non-Federal agencies in terms of approximate costs and streamflow-information products produced. The three non-Federal agencies provided the USGS with detailed information on their streamgaging program and related costs, and the USGS explored, through publicly available Web sites and one-on-one discussions, the comparability of the streamflow information produced. The type and purpose of streamgages operated, the quality of streamflow record produced, and cost-accounting methods have a great effect on streamgaging costs. There are many uses of streamflow information, and the information requirements for streamgaging programs differ greatly across this range of purposes. A premise of the USGS streamgaging program is that the network must produce consistent data of sufficient quality to support the broadest range of possible uses. Other networks may have a narrower range of purposes; as a consequence, the method of operation, data-quality objectives, and information delivery may be different from those for a multipurpose network. As a result, direct comparison of the overall cost (or of the cost per streamgage) among these programs is not possible. The analysis is, nonetheless, very instructive and provides USGS program managers, agency leadership, and other agency streamgaging program managers useful insight to influence future decisions. Even though the comparison of streamgaging costs and streamflow information products was qualitative, this analysis does offer useful insights on longstanding questions of USGS streamgaging costs.

Mothers’ experiences in the Nurse-Family Partnership program: a qualitative case study

PubMed Central

2012-01-01

Background Few studies have explored the experiences of low income mothers participating in nurse home visiting programs. Our study explores and describes mothers' experiences participating in the Nurse-Family Partnership (NFP) Program, an intensive home visiting program with demonstrated effectiveness, from the time of program entry before 29 weeks gestation until their infant's first birthday. Methods A qualitative case study approach was implemented. A purposeful sample of 18 low income, young first time mothers participating in a pilot study of the NFP program in Hamilton, Ontario, Canada partook in one to two face to face in-depth interviews exploring their experiences in the program. All interviews were digitally recorded and transcribed verbatim. Conventional content analysis procedures were used to analyze all interviews. Data collection and initial analysis were implemented concurrently. Results The mothers participating in the NFP program were very positive about their experiences in the program. Three overarching themes emerged from the data: 1. Getting into the NFP program; 2. The NFP nurse is an expert, but also like a friend providing support; and 3. Participating in the NFP program is making me a better parent. Conclusions Our findings provide vital information to home visiting nurses and to planners of home visiting programs about mothers' perspectives on what is important to them in their relationships with their nurses, how nurses and women are able to develop positive therapeutic relationships, and how nurses respond to mothers' unique life situations while home visiting within the NFP Program. In addition our findings offer insights into why and under what circumstances low income mothers will engage in nurse home visiting and how they expect to benefit from their participation. PMID:22953748

[Advantages and disadvantages of incorporating qualitative methodology in the evaluation of health services. A practical case: evaluation of a high-resolution clinic].

PubMed

Alvarez Del Arco, D; Rodríguez Rieiro, C; Sanchidrián De Blás, C; Alejos, B; Plá Mestre, R

2012-01-01

We examined the usefulness of incorporating a qualitative phase in the evaluation of the quality of care in a high-resolution medical service carried out with quantitative methods. A quantitative research was performed using a structured questionnaire and selecting interviewees by systematic randomized sampling methods (n=320). In addition, a qualitative research was carried on through semi-structured interviews with patients selected by convenience criteria (n=11), observations in the care assistance circuit, and a group interview with health professionals working in the service. A multidisciplinary research team conducted an individual analysis of the information collected in both quantitative and qualitative phases. Subsequently, three meetings based on group brainstorming techniques were held to identify the diverse contributions of each of the methodologies employed to the research, using affinity graphs to analyse the different results obtained in both phases and evaluate possible bias arising from the use of qualitative methods. Qualitative research allowed examining specific aspects of the health care service that had been collected in the quantitative phase, harmonizing the results obtained in the previous phase, giving in-depth data on the reasons for patient dissatisfaction with specific aspects, such as waiting times and available infrastructures, and identifying emerging issues of the service which had not been previously assessed. Overall, the qualitative phase enriched the results of the research. It is appropriate and recommendable to incorporate this methodological approach in research aimed at evaluating the quality of the service in specific health care settings, since it is provided first hand, by the voice of the customer. Copyright © 2011 SECA. Published by Elsevier Espana. All rights reserved.

Feedback Improvement in Automatic Program Evaluation Systems

ERIC Educational Resources Information Center

Skupas, Bronius

2010-01-01

Automatic program evaluation is a way to assess source program files. These techniques are used in learning management environments, programming exams and contest systems. However, use of automated program evaluation encounters problems: some evaluations are not clear for the students and the system messages do not show reasons for lost points.…

Implementing a collaborative return-to-work program: Lessons from a qualitative study in a large Canadian healthcare organization

PubMed Central

Skivington, Kathryn; Lifshen, Marni; Mustard, Cameron

2016-01-01

BACKGROUND: Comprehensive workplace return-to-work policies, applied with consistency, can reduce length of time out of work and the risk of long-term disability. This paper reports on the findings from a qualitative study exploring managers’ and return-to-work-coordinators’ views on the implementation of their organization’s new return-to-work program. OBJECTIVES: To provide practical guidance to organizations in designing and implementing return-to-work programs for their employees. METHODS: Semi-structured qualitative interviews were undertaken with 20 managers and 10 return-to-work co-ordinators to describe participants’ perspectives on the progress of program implementation in the first 18 months of adoption. The study was based in a large healthcare organization in Ontario, Canada. Thematic analysis of the data was conducted. RESULTS: We identified tensions evident in the early implementation phase of the organization’s return-to-work program. These tensions were attributed to uncertainties concerning roles and responsibilities and to circumstances where objectives or principles appeared to be in conflict. CONCLUSIONS: The implementation of a comprehensive and collaborative return-to-work program is a complex challenge. The findings described in this paper may provide helpful guidance for organizations embarking on the development and implementation of a return-to-work program. PMID:27792035

[Qualitative Research in Health Services Research - Discussion Paper, Part 3: Quality of Qualitative Research].

PubMed

Stamer, M; Güthlin, C; Holmberg, C; Karbach, U; Patzelt, C; Meyer, T

2015-12-01

The third and final discussion paper of the German Network of Health Services Research's (DNVF) "Qualitative Methods Working Group" demonstrates methods for the evaluation and quality of qualitative research in health services research. In this paper we discuss approaches described in evaluating qualitative studies, including: an orientation to the general principles of empirical research, an approach-specific course of action, as well as procedures based on the research-process and criteria-oriented approaches. Divided into general and specific aspects to be considered in a qualitative study quality evaluation, the central focus of the discussion paper undertakes an extensive examination of the process and criteria-oriented approaches. The general aspects include the participation of relevant groups in the research process as well as ethical aspects of the research and data protection issues. The more specific aspects in evaluating the quality of qualitative research include considerations about the research interest, research questions, and the selection of data collection methods and types of analyses. The formulated questions are intended to guide reviewers and researchers to evaluate and to develop qualitative research projects appropriately. The intention of this discussion paper is to ensure a transparent research culture, and to reflect on and discuss the methodological and research approach of qualitative studies in health services research. With this paper we aim to initiate a discussion on high quality evaluation of qualitative health services research. © Georg Thieme Verlag KG Stuttgart · New York.

Revolutionizing volunteer interpreter services: an evaluation of an innovative medical interpreter education program.

PubMed

Hasbún Avalos, Oswaldo; Pennington, Kaylin; Osterberg, Lars

2013-12-01

In our ever-increasingly multicultural, multilingual society, medical interpreters serve an important role in the provision of care. Though it is known that using untrained interpreters leads to decreased quality of care for limited English proficiency patients, because of a short supply of professionals and a lack of formalized, feasible education programs for volunteers, community health centers and internal medicine practices continue to rely on untrained interpreters. To develop and formally evaluate a novel medical interpreter education program that encompasses major tenets of interpretation, tailored to the needs of volunteer medical interpreters. One-armed, quasi-experimental retro-pre-post study using survey ratings and feedback correlated by assessment scores to determine educational intervention effects. Thirty-eight students; 24 Spanish, nine Mandarin, and five Vietnamese. The majority had prior interpreting experience but no formal medical interpreter training. Students completed retrospective pre-test and post-test surveys measuring confidence in and perceived knowledge of key skills of interpretation. Primary outcome measures were a 10-point Likert scale for survey questions of knowledge, skills, and confidence, written and oral assessments of interpreter skills, and qualitative evidence of newfound knowledge in written reflections. Analyses showed a statistically significant (P <0.001) change of about two points in mean self-ratings on knowledge, skills, and confidence, with large effect sizes (d > 0.8). The second half of the program was also quantitatively and qualitatively shown to be a vital learning experience, resulting in 18 % more students passing the oral assessments; a 19 % increase in mean scores for written assessments; and a newfound understanding of interpreter roles and ways to navigate them. This innovative program was successful in increasing volunteer interpreters' skills and knowledge of interpretation, as well as confidence

Guidelines for the Evaluation of Bilingual Education Programs.

ERIC Educational Resources Information Center

Cardoza, Desdemona

Principles of program evaluation research are outlined so that bilingual education program coordinators can conduct methodologically acceptable program evaluations. The three basic principles of evaluation research are: identification of the program participants, definition of the program intervention, and assessment of program effectiveness.…

Evaluation of NASA space grant consortia programs

NASA Technical Reports Server (NTRS)

Eisenberg, Martin A.

1990-01-01

The meaningful evaluation of the NASA Space Grant Consortium and Fellowship Programs must overcome unusual difficulties: (1) the program, in its infancy, is undergoing dynamic change; (2) the several state consortia and universities have widely divergent parochial goals that defy a uniform evaluative process; and (3) the pilot-sized consortium programs require that the evaluative process be economical in human costs less the process of evaluation comprise the effectiveness of the programs they are meant to assess. This paper represents an attempt to assess the context in which evaluation is to be conducted, the goals and limitations inherent to the evaluation, and to recommend appropriate guidelines for evaluation.

Planning Evaluation through the Program Life Cycle

ERIC Educational Resources Information Center

Scheirer, Mary Ann; Mark, Melvin M.; Brooks, Ariana; Grob, George F.; Chapel, Thomas J.; Geisz, Mary; McKaughan, Molly; Leviton, Laura

2012-01-01

Linking evaluation methods to the several phases of a program's life cycle can provide evaluation planners and funders with guidance about what types of evaluation are most appropriate over the trajectory of social and educational programs and other interventions. If methods are matched to the needs of program phases, evaluation can and should…

A Program Evaluation Tool for Dual Enrollment Transition Programs

ERIC Educational Resources Information Center

Grigal, Meg; Dwyre, Amy; Emmett, Joyce; Emmett, Richard

2012-01-01

This article describes the development and use of a program evaluation tool designed to support self-assessment of college-based dual enrollment transition programs serving students with intellectual disabilities between the ages of 18-21 in college settings. The authors describe the need for such an evaluation tool, outline the areas addressed by…

Expediting the Analysis of Qualitative Data in Evaluation: A Procedure for the Rapid Identification of Themes from Audio Recordings (RITA)

ERIC Educational Resources Information Center

Neal, Jennifer Watling; Neal, Zachary P.; VanDyke, Erika; Kornbluh, Mariah

2015-01-01

Qualitative data offer advantages to evaluators, including rich information about stakeholders' perspectives and experiences. However, qualitative data analysis is labor-intensive and slow, conflicting with evaluators' needs to provide punctual feedback to their clients. In this method note, we contribute to the literature on rapid evaluation and…

Monitoring and Evaluating the Transition of Large-Scale Programs in Global Health

PubMed Central

Bao, James; Rodriguez, Daniela C; Paina, Ligia; Ozawa, Sachiko; Bennett, Sara

2015-01-01

Purpose: Donors are increasingly interested in the transition and sustainability of global health programs as priorities shift and external funding declines. Systematic and high-quality monitoring and evaluation (M&E) of such processes is rare. We propose a framework and related guiding questions to systematize the M&E of global health program transitions. Methods: We conducted stakeholder interviews, searched the peer-reviewed and gray literature, gathered feedback from key informants, and reflected on author experiences to build a framework on M&E of transition and to develop guiding questions. Findings: The conceptual framework models transition as a process spanning pre-transition and transition itself and extending into sustained services and outcomes. Key transition domains include leadership, financing, programming, and service delivery, and relevant activities that drive the transition in these domains forward include sustaining a supportive policy environment, creating financial sustainability, developing local stakeholder capacity, communicating to all stakeholders, and aligning programs. Ideally transition monitoring would begin prior to transition processes being implemented and continue for some time after transition has been completed. As no set of indicators will be applicable across all types of health program transitions, we instead propose guiding questions and illustrative quantitative and qualitative indicators to be considered and adapted based on the transition domains identified as most important to the particular health program transition. The M&E of transition faces new and unique challenges, requiring measuring constructs to which evaluators may not be accustomed. Many domains hinge on measuring “intangibles” such as the management of relationships. Monitoring these constructs may require a compromise between rigorous data collection and the involvement of key stakeholders. Conclusion: Monitoring and evaluating transitions in global

Evaluation of ridesharing programs in Michigan

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kulp, G.; Tsao, H.J.; Webber, R.E.

1982-10-01

The design, implementation, and results of a carpool and vanpool evaluation are described. Objectives of the evaluation were: to develop credible estimates of the energy savings attributable to the ridesharing program, to provide information for improving the performance of the ridesharing program, and to add to a general understanding of the ridesharing process. Previous evaluation work is critiqued and the research methodology adopted for this study is discussed. The ridesharing program in Michigan is described and the basis for selecting Michigan as the evaluation site is discussed. The evaluation methodology is presented, including research design, sampling procedure, data collection, andmore » data validation. Evaluation results are analyzed. (LEW)« less

Two Feathers Endowment Scholarship Program: Program Evaluation

ERIC Educational Resources Information Center

Pierce, Alexandra

2004-01-01

The Saint Paul Foundation contracted with the Wilder Research Center to conduct an evaluation of their Two Feathers Scholarship Program. The Two Feathers Scholarship Program is funded through the Two Feathers Endowment, which is one part of the Foundation's SpectrumTrust. SpectrumTrust is a unique partnership between communities of color and The…

Program Evaluation at Foothill College.

ERIC Educational Resources Information Center

Lowe, Irel D.

Evaluation systems were designed for the assessment of the instructional and non-instructional programs at Foothill College (California) to determine whether the objectives of the programs were being met and to ascertain the direct and indirect costs of the programs. For the instructional programs, criteria for determining effectiveness, revenue…

Library Programs. Evaluating Federally Funded Public Library Programs.

ERIC Educational Resources Information Center

Office of Educational Research and Improvement (ED), Washington, DC.

Following an introduction by Betty J. Turock, nine reports examine key issues in library evaluation: (1) "Output Measures and the Evaluation Process" (Nancy A. Van House) describes measurement as a concept to be understood in the larger context of planning and evaluation; (2) "Adapting Output Measures to Program Evaluation"…

Design of a study evaluating the effects, health economics, and stakeholder perspectives of a multi-component occupational rehabilitation program with an added workplace intervention - a  study protocol.

PubMed

Rise, Marit B; Skagseth, Martin; Klevanger, Nina E; Aasdahl, Lene; Borchgrevink, Petter; Jensen, Chris; Tenggren, Hanne; Halsteinli, Vidar; Jacobsen, Trym N; Løland, Svein B; Johnsen, Roar; Fimland, Marius S

2018-02-05

Recent research has suggested that interventions at the workplace might be the most potent ingredient in return to work interventions, but few studies have investigated the different effects of workplace interventions as part of occupational rehabilitation programs. The comprehensive design described in this article includes effect (on return to work and health outcomes), and health economic evaluations of a workplace intervention added to a multicomponent rehabilitation program. Qualitative and mixed method studies will investigate sick-listed persons', rehabilitation therapists' and employers' perspectives on the usability and outcomes of the rehabilitation program and the workplace intervention. The program and intervention are provided to patients with musculoskeletal, psychological or general and unspecified diagnoses. The program is multi-component and includes Acceptance and Commitment Therapy, physical exercise, patient education and creating a plan for increased work participation. Persons who are employed, aged from 18 to 60 years, with a current sick leave status of 50% or more and a diagnosis within the musculoskeletal, psychological or general and unspecified chapters of International Classification of Primary Care-2 (ICPC-2) will be recruited to a researcher-blinded parallel-group randomized controlled trial. All participants take part in an in-patient occupational rehabilitation program, while the intervention group also takes part in an intervention at the workplace. The effect and economic evaluation will investigate the effect of the added workplace intervention. The primary outcome measures will be time until full sustainable return to work and total number of sickness absence days in the 12 months after inclusion. Health economic evaluations will investigate the cost-effectiveness and cost-utility. Qualitative studies will investigate rehabilitation therapists' experiences with working towards return to work within an ACT-approach and

[Design, implementation and evaluation of a health education program for the elderly].

PubMed

Pino, Margarita; Ricoy, Maria Carmen; Portela, Julio

2010-09-01

The objective was to design, implement and evaluate a health education program, using the analysis of the habits that harm the health of people over 65 years old. An evaluative research was carried out from a multiple case study in the North-West area of Spain, combining both the quantitative and the qualitative approach. A questionnaire and interview were used as tools for data collection. The elderly take a lot of medicines and also treat themselves. A small group smoke and drink alcohol. Over 25% have sedentary habits and their average body mass rate was 30.55. The implementation of the programme has significantly influenced their quality of life. Elderly people have deep rooted unhealthy habits. The achievement of educational contents improved their quality of life. However, they are reluctant to assume new habits, even though they are healthy.

How To: Evaluate Education Programs

ERIC Educational Resources Information Center

Fink, Arlene; Kosecoff, Jacqueline

This book presents a compilation of 28 issues of the newsletter, "How To Evaluate Education Programs" from the first one published in September 1977 through the issue of December 1979 on the topic of evaluating educational programs. The subject is covered in the following chapters: (1) How to Choose a Test; (2) How to Rate and Compare…

The Evaluation and Research of Multi-Project Programs: Program Component Analysis.

ERIC Educational Resources Information Center

Baker, Eva L.

1977-01-01

It is difficult to base evaluations on concepts irrelevant to state policy making. Evaluation of a multiproject program requires both time and differentiation of method. Data from the California Early Childhood Program illustrate process variables for program component analysis, and research questions for intraprogram comparison. (CP)

Development and Implementation of a Mental Health Work Rehabilitation Program: Results of a Developmental Evaluation.

PubMed

Sylvain, Chantal; Durand, Marie-José; Velasquez Sanchez, Astrid; Lessard, Nathalie; Maillette, Pascale

2018-05-23

Purpose Long-term work disability due to common mental disorders (CMDs) is a growing problem. Yet optimal interventions remain unclear and little is known about implementation challenges in everyday practice. This study aimed to support and evaluate, in real time, the development and implementation of a work rehabilitation program (WRP) designed to promote post-CMD return-to-work (RTW). Methods A 2-year developmental evaluation was performed using a participatory approach. At program outset, the researchers held five work meetings to revise the program's logic model and discuss its underlying change theory with clinicians. Data collection tools used throughout the study period were structured charts of activities conducted with workers (n = 41); in-depth interviews with program clinicians and managers (n = 9); and participant observation during work meetings. Quantitative data were analyzed using descriptive statistics. Qualitative data underwent thematic analysis using a processual approach. Results Three types of activity were developed and implemented: individual and group interventions targeting workers, and joint activities targeting partners (physicians, employers, others). While worker-targeted activities were generally implemented as planned, joint activities were sporadic. Analysis of the implementation process revealed five challenges faced by clinicians. Determinants included clinicians, host organization, sociopolitical context and resources provided by the evaluation. Conclusion The program studied is original in that it is based on the best available scientific knowledge, yet adapted to contextual particularities. The identified implementation challenges highlight the need for greater importance to be placed on the external, non-program context to ensure sustainable implementation in everyday practice.

What Makes a Good School Leadership Program? A Qualitative Study of the Lookstein Center Educational Leadership Advancement Initiative (ELAI)

ERIC Educational Resources Information Center

Burstein, Alissa Beth; Kohn, Eli

2017-01-01

How do educators become successful leaders? This qualitative study set out to learn more about The Lookstein Center ELAI program as well as mentoring and leadership training in general, with the hope of offering insights to other school leadership programs. The mentor-mentee relationship was seen to develop into a collaborative partnership, with…

Program Evaluation: Two Management-Oriented Samples

ERIC Educational Resources Information Center

Alford, Kenneth Ray

2010-01-01

Two Management-Oriented Samples details two examples of the management-oriented approach to program evaluation. Kenneth Alford, a doctorate candidate at the University of the Cumberlands, details two separate program evaluations conducted in his school district and seeks to compare and contrast the two evaluations based upon the characteristics of…

The CHAP-EMS health promotion program: a qualitative study on participants' views of the role of paramedics.

PubMed

Brydges, Madison; Denton, Margaret; Agarwal, Gina

2016-08-24

Expanded roles for paramedics, commonly termed community paramedicine, are becoming increasingly common. Paramedics working in community paramedicine roles represent a distinct departure away from the traditional emergency paradigm of paramedic services. Despite this, little research has addressed how community paramedics are perceived by their clients. This study took an interpretivist qualitative approach to examine participants' perceptions of paramedics providing a community paramedicine program, named the Community Health Assessment Program through Emergency Medical Services (CHAP-EMS). Both participant observation and semi-structured interviews conducted with program participants were used to gain insight into the on-the-ground experiences of the program. Thematic analysis was employed to analyze all data. Three themes emerged: i) Caring and trusting relationships; ii) paramedics as health advocates; iii) the added value of EMS skills. Paramedics were perceived by residents as having dual identities: first in a novel role as health advocates and secondly in a traditional role as emergency experts despite lacking contextual features associated with emergency response. From this exploratory, qualitative study we present an emerging framework in which to conceptualize paramedic roles in community paramedicine settings. Future research should address the saliency of these roles in different contexts and how these roles relate to paramedic practice.

Evaluating and operationalizing an environmental auditing program: a pilot study.

PubMed

Gordon, Laura; Bruce, Natalie; Suh, Kathryn N; Roth, Virginia

2014-07-01

Environmental auditing is an important tool to ensure consistent and effective cleaning. Our pilot study compared an alcohol-based fluorescent marking product and an adenosine-5'-triphosphate bioluminescence product for use in an environmental auditing program to determine which product was more practical and acceptable to users. Both products were tested on 15 preselected high touch objects in randomly selected patient rooms, following regular daily cleaning. A room was considered a "pass" if ≥80% of surfaces were adequately cleaned as defined by manufacturers' guidelines. A qualitative survey assessed user preference and operational considerations. Using fluorescent marking, 9 of 37 patient rooms evaluated (24%) were considered a "pass" after daily cleaning. Using adenosine-5'-triphosphate bioluminescence, 21 of 37 patient rooms passed (57%). There was great variability in results between different high touch objects. Eighty percent of users preferred the alcohol-based fluorescent marking product because it provided an effective visual aid to coach staff on proper cleaning techniques and allowed simple and consistent application. Environmental auditing using translucent, alcohol-based fluorescent marking best met the requirements of our organization. Our results reinforce the importance of involving a multidisciplinary team in evaluating and operationalizing an environmental auditing program. Copyright © 2014 Association for Professionals in Infection Control and Epidemiology, Inc. Published by Mosby, Inc. All rights reserved.

The Design, Development, and Evaluation of a Qualitative Data Collection Application for Pregnant Women.

PubMed

Keedle, Hazel; Schmied, Virginia; Burns, Elaine; Dahlen, Hannah

2018-01-01

This article explores the development and evaluation of a smartphone mobile software application (app) to collect qualitative data. The app was specifically designed to capture real-time qualitative data from women planning a vaginal birth after caesarean delivery. This article outlines the design and development of the app to include funding, ethics, and the recruitment of an app developer, as well as the evaluation of using the app by seven participants. Data collection methods used in qualitative research include interviews and focus groups (either online, face-to-face, or by phone), participant diaries, or observations of interactions. This article identifies an alternative data collection methodology using a smartphone app to collect real-time data. The app provides real-time data and instant access to data alongside the ability to access participants from a variety of locations. This allows the researcher to gain insight into the experiences of participants through audio or video recordings in longitudinal studies without the need for constant interactions or interviews with participants. Using smartphone applications can allow researchers to access participants who are traditionally hard to reach and access their data in real time. Evaluating these apps before use in research is invaluable. © 2017 Sigma Theta Tau International.

Evaluating Federal Education Programs.

ERIC Educational Resources Information Center

Baker, Eva L., Ed.

A series of papers was developed by the authors as part of their deliberations as members of the National Research Council's Committee of Program Evaluation in Education. The papers provide a broad range of present evaluative thinking. The conflict between preferences in evaluation methodology comes through in these papers. The selections include:…

Women's experiences accessing a women-centered cardiac rehabilitation program: a qualitative study.

PubMed

Rolfe, Danielle E; Sutton, Erica J; Landry, Mireille; Sternberg, Len; Price, Jennifer A D

2010-01-01

The health benefits of cardiac rehabilitation (CR) for women living with heart disease are well documented, yet women remain underrepresented in traditionally structured CR programs. This health service delivery gap has been attributed to a number of sex-related factors experienced by women, including lower rates of physician referral, travel-related barriers, competing work and caregiving responsibilities, greater cardiovascular disease severity, and number of comorbid health conditions. Whether a program specifically designed for women is able to address these barriers and facilitate women's participation is a question that has seldom been explored in the CR literature. As part of a larger study exploring whether 6 predefined principles of women's health (empowerment of women, accessible programs, broad definition of health care, high-quality of care, collaborative planning, and innovative and creative approaches) are reflected in the practices of the Women's Cardiovascular Health Initiative (WCHI) (a comprehensive CR and primary prevention program designed for women), the objective of this analysis was to explore how the principle of "accessible programs" is experienced by women participating in the WCHI. Fourteen women previously enrolled in the WCHI program participated in a single, in-person qualitative interview. Transcripts were analyzed using a constant-comparative approach to identify relevant themes related to program accessibility. Key themes identified included participants' experiences with acquiring physician referral, negotiating transportation issues, and navigating program schedules. Women discussed how peer support and staff members' willingness to address their health-related concerns facilitated their participation. While a women-centered CR/primary prevention program may facilitate and encourage women's participation by providing flexible program schedules as well as peer and professional support, efforts are still required to address

What do practitioners think? A qualitative study of a shared care mental health and nutrition primary care program

PubMed Central

Paquette-Warren, Jann; Vingilis, Evelyn; Greenslade, Jaimi; Newnam, Sharon

2006-01-01

Abstract Objective To develop an in-depth understanding of a shared care model from primary mental health and nutrition care practitioners with a focus on program goals, strengths, challenges and target population benefits. Design Qualitative method of focus groups. Setting/Participants The study involved fifty-three practitioners from the Hamilton Health Service Organization Mental Health and Nutrition Program located in Hamilton, Ontario, Canada. Method Six focus groups were conducted to obtain the perspective of practitioners belonging to various disciplines or health care teams. A qualitative approach using both an editing and template organization styles was taken followed by a basic content analysis. Main findings Themes revealed accessibility, interdisciplinary care, and complex care as the main goals of the program. Major program strengths included flexibility, communication/collaboration, educational opportunities, access to patient information, continuity of care, and maintenance of practitioner and patient satisfaction. Shared care was described as highly dependent on communication style, skill and expertise, availability, and attitudes toward shared care. Time constraint with respect to collaboration was noted as the main challenge. Conclusion Despite some challenges and variability among practices, the program was perceived as providing better patient care by the most appropriate practitioner in an accessible and comfortable setting. PMID:17041680

A qualitative evaluation of leadership development workshops for mental health workers from four Pacific Island Countries.

PubMed

Fung, Paul; Montague, Ros

2015-06-01

This paper provides a qualitative evaluation of a series of leadership development workshops held at the New South Wales Institute of Psychiatry (NSWIOP) for mental health workers from Papua New Guinea, Federated States of Micronesia, Republic of Marshall Islands, and Palau. Fourteen mental health workers attended the week-long training focused on project management and partnership development skills. In-depth interviews were conducted with participants at the commencement and conclusion of the training, and questionnaires were completed. A focus group was conducted with the NSWIOP organisers. The data was analysed using qualitative techniques to identify emergent themes for both participants and NSWIOP project team. All Pacific Island participants responded positively to the training. All reported greater confidence in taking on formal or informal leadership roles in the workplace, developing project planning skills and interpersonal skills such as networking and partnerships. The NSWIOP organisers identified strengths and weaknesses in the delivery of this training. The strong partnerships developed between NSWIOP and the Ministry of Health in all four countries contributed to the success of the training. Leadership Development Programs are an important aspect of building capacity in the mental health services of Pacific Island Countries. © The Royal Australian and New Zealand College of Psychiatrists 2015.

Guidelines "PEP," Peer Evaluation Program. A Systematic Approach for Evaluating Educational Programs, 1973-1974.

ERIC Educational Resources Information Center

Grotsky, Jeffery N.; And Others

The Peer Evaluation Program (PEP) has been instituted by the Division of Special Education, Pennsylvania State Department of Education, to allow intermediate units an opportunity to continuously improve their programs. The advantages of the PEP system are: (1) it is a self-improvememt system of program development; (2) PEP allows local autonomy as…

Methadone maintenance in prison: evaluation of a pilot program in Puerto Rico.

PubMed

Heimer, Robert; Catania, Holly; Newman, Robert G; Zambrano, John; Brunet, Arlyn; Ortiz, Arturo Marti

2006-06-28

To describe and evaluate a pilot methadone maintenance program for heroin-dependent inmates of Las Malvinas men's prison in San Juan, Puerto Rico. Data from self-report of inmates' drug use before and during incarceration, attitudes about drug treatment in general and methadone maintenance in particular, and expectations about behaviors upon release from prison and from testing inmates' urine were analyzed comparing program patients (n=20) and inmates selected at random from the prison population (n=40). Qualitative data obtained by interviewing program staff, the correctional officers and superintendent, and commonwealth officials responsible for establishing and operating the program were analyzed to identify attitudes about methadone and program effectiveness. Heroin use among prisoners not in treatment was common; 58% reported any use while incarcerated and 38% reported use in past 30 days. All patients in the treatment program had used heroin in prison in the 30 days prior to enrolling in treatment. While in treatment, the percentage of patients not using heroin was reduced, according to both self-report and urine testing, to one in 18 (94% reduction) and one in 20 (95% reduction), respectively. Participation in treatment was associated with an increased acceptance of methadone maintenance. Prison personnel and commonwealth officials were supportive of the program. The program appears to be a success, and prison officials have begun an expansion from the current ceiling of 24 inmates to treat 300 or more inmates.

1999 commuter assistance program evaluation manual

DOT National Transportation Integrated Search

2001-01-01

This manual was developed to assist Florida's Commuter Assistance Programs (CAP) to measure and evaluate their performance. It provides information necessary for a CAP to create and implement its own evaluation program. It discusses performance measu...

Online evaluation programs: benefits and limitations.

PubMed

Burhansstipanov, Linda; Clark, Richard E; Watanabe-Galloway, Shinobu; Petereit, Daniel G; Eschiti, Valerie; Krebs, Linda U; Pingatore, Noel L

2012-04-01

Patient navigation programs are increasing throughout the USA, yet some evaluation measures are too vague to determine what and how navigation functions. Through collaborative efforts an online evaluation program was developed. The goal of this evaluation program is to make data entry accurate, simple, and efficient. This comprehensive program includes major components on staff, mentoring, committees, partnerships, grants/studies, products, dissemination, patient navigation, and reports. Pull down menus, radio buttons, and check boxes are incorporated whenever possible. Although the program has limitations, the benefits of having access to current, up-to-date program data 24/7 are worth overcoming the challenges. Of major benefit is the ability of the staff to tailor summary reports to provide anonymous feedback in a timely manner to community partners and participants. The tailored data are useful for the partners to generate summaries for inclusion in new grant applications.

Qualitative approaches to use of the RE-AIM framework: rationale and methods.

PubMed

Holtrop, Jodi Summers; Rabin, Borsika A; Glasgow, Russell E

2018-03-13

There have been over 430 publications using the RE-AIM model for planning and evaluation of health programs and policies, as well as numerous applications of the model in grant proposals and national programs. Full use of the model includes use of qualitative methods to understand why and how results were obtained on different RE-AIM dimensions, however, recent reviews have revealed that qualitative methods have been used infrequently. Having quantitative and qualitative methods and results iteratively inform each other should enhance understanding and lessons learned. Because there have been few published examples of qualitative approaches and methods using RE-AIM for planning or assessment and no guidance on how qualitative approaches can inform these processes, we provide guidance on qualitative methods to address the RE-AIM model and its various dimensions. The intended audience is researchers interested in applying RE-AIM or similar implementation models, but the methods discussed should also be relevant to those in community or clinical settings. We present directions for, examples of, and guidance on how qualitative methods can be used to address each of the five RE-AIM dimensions. Formative qualitative methods can be helpful in planning interventions and designing for dissemination. Summative qualitative methods are useful when used in an iterative, mixed methods approach for understanding how and why different patterns of results occur. In summary, qualitative and mixed methods approaches to RE-AIM help understand complex situations and results, why and how outcomes were obtained, and contextual factors not easily assessed using quantitative measures.

EVALUE : a computer program for evaluating investments in forest products industries

Treesearch

Peter J. Ince; Philip H. Steele

1980-01-01

EVALUE, a FORTRAN program, was developed to provide a framework for cash flow analysis of investment opportunities. EVALUE was designed to assist researchers in evaluating investment feasibility of new technology or new manufacturing processes. This report serves as user documentation for the EVALUE program. EVALUE is briefly described and notes on preparation of a...

Attitudes of affiliate faculty members toward medical student summative evaluation for clinical clerkships: a qualitative analysis.

PubMed

Wang, Karen E; Fitzpatrick, Caroline; George, David; Lane, Lindsey

2012-01-01

Summative evaluation of medical students is a critical component of the educational process. Despite extensive literature on evaluation, few studies have centered on affiliate faculty members' attitudes toward summative evaluation of students, though it has been suggested that these attitudes influence their effectiveness as evaluators. The objective is to examine affiliate faculty members' attitudes toward clinical clerkship evaluation using primarily qualitative research methods. The study used a nonexperimental research design and employed mixed methods. Data were collected through interviews, focus groups, and a questionnaire from 11 affiliate faculty members. Themes emerging from the data fell into three broad categories: (a) factors that influence grading, (b) consequences of negative evaluations, and (c) disconnections in the grading process. The quantitative portion of the study revealed important discrepancies supporting the use of qualitative methods. The study highlights faculty members' struggles with the evaluative process and emphasizes the need for improvements in evaluation tools and faculty development.

Evaluation of “The Space Place,” a NASA Integrated, Multi-mission Education and Public Outreach Program

NASA Astrophysics Data System (ADS)

Fisher, Diane K.; Leon, N. J.

2006-12-01

The Space Place is an integrated NASA education and public outreach program, so far representing over 40 different NASA missions. It combines Web-based, printed, and externally published media to reach underserved audiences across the nation. Its primary mission is to develop and provide a highly desirable suite of attractive and educational products designed to appeal to and immerse the general public in space exploration. Its primary target audience is elementary school age kids. The program has developed an extensive network of partnerships with museums and libraries in rural areas, English and Spanish language newspapers, astronomy societies, rocketry clubs, and national youth organizations. Materials are distributed monthly through all these channels. Originally a New Millennium Program (NMP) outreach effort only, it is open to all NASA missions. NMP (a NASA-level program managed out of the Jet Propulsion Laboratory) continues to provide the base of support to build and maintain the outreach program’s infrastructure. Obtaining independent evaluation and reporting of the effectiveness of the program is one of NASA’s requirements for education and public outreach efforts. The Program Evaluation and Research Group (PERG) at Lesley University, Cambridge, MA, was retained to perform this service for The Space Place. PERG is also evaluating education and public outreach programs for NASA’s Science Mission Directorate. PERG recently delivered a report evaluating The Space Place program. Using both qualitative and quantitative methods, PERG surveyed representative samples of Space Place partner museums, astronomy clubs, and newspapers. The survey included questions about all the products the program provides. The report concludes that The Space Place fills a niche by serving small institutions, giving them a personal alliance with NASA that they would otherwise not have. By providing free, quality materials, The Space Place program provides these under

A Qualitative Evaluation of Program Budgeting and Marginal Analysis in a Canadian Pediatric Tertiary Care Institution.

PubMed

Smith, Neale; Mitton, Craig; Hiltz, Mary-Ann; Campbell, Matthew; Dowling, Laura; Magee, J Fergall; Gujar, Shashi Ashok

2016-10-01

Hospitals in Canada are being asked by governments to improve efficiency and do more with fewer resources. Healthcare decision makers are thus driven to find better ways to manage budgets and deliver on their mission. Formal processes of priority setting and resource allocation (PSRA) are one means to this end. This paper reports an evaluation of one such approach, Program Budgeting and Marginal Analysis (PBMA), as applied at a children and women's tertiary care facility in Nova Scotia, Canada. A brief evaluation conducted immediately after the conclusion of the PBMA process was supplemented with a larger retrospective evaluation. The retrospective evaluation included 26 face-to-face individual interviews with senior and middle managers who took part in PBMA. Interview transcripts were analyzed against a template consisting of 19 elements of structure, process, attitudes, and outcomes associated with high performance in PSRA. Respondents had a good experience with the implementation of PBMA, and considered it an improvement over past practice. Success was attributed to effective leadership, and substantial efforts to engage staff members. Understanding of economic and ethical principles of decision making was reportedly increased. Areas for improvement included ensuring that everyone participated in good faith, better communication of final results, and stronger follow-through to determine if anticipated changes and benefits in fact occurred. The evaluation framework employed here proved useful in assessing the quality of this resource allocation exercise. The results are directly useful to local decision makers, and the identified strengths and weaknesses are broadly consistent with those reported in studies of other organizations.

Practical strategies for nursing education program evaluation.

PubMed

Lewallen, Lynne Porter

2015-01-01

Self-evaluation is required for institutions of higher learning and the nursing programs within them. The literature provides information on evaluation models and instruments, and descriptions of how specific nursing education programs are evaluated. However, there are few discussions in the nursing education literature of the practical aspects of nursing education program evaluation: how to get started, how to keep track of data, who to involve in data collection, and how to manage challenging criteria. This article discusses the importance of program evaluation in the academic setting and provides information on practical ways to organize the evaluation process and aggregate data, and strategies for gathering data from students, graduates, alumni, and employers of graduates. Copyright © 2015 Elsevier Inc. All rights reserved.

Adult Offenders' Perceptions of Rehabilitation Programs in Africa

ERIC Educational Resources Information Center

Ngozwana, Nomazulu

2017-01-01

This article reflects on adult offenders' perceptions of rehabilitation programs in Africa. It also evaluates whether offenders are consulted when planning rehabilitation programs. Adult education principles were used as a lens to understand offenders' perceptions of rehabilitation programs. Using an interpretive paradigm and qualitative approach,…

Accelerated Training of Skilled Birth Attendants in a Marginalized Population on the Thai-Myanmar Border: A Multiple Methods Program Evaluation

PubMed Central

White, Adrienne Lynne; Min, Thaw Htwe; Gross, Mechthild M.; Kajeechiwa, Ladda; Thwin, May Myo; Hanboonkunupakarn, Borimas; Than, Hla Hla; Zin, Thet Wai; Rijken, Marcus J.; Hoogenboom, Gabie; McGready, Rose

2016-01-01

Background To evaluate a skilled birth attendant (SBA) training program in a neglected population on the Thai-Myanmar border, we used multiple methods to show that refugee and migrant health workers can be given effective training in their own environment to become SBAs and teachers of SBAs. The loss of SBAs through resettlement to third countries necessitated urgent training of available workers to meet local needs. Methods and Findings All results were obtained from student records of theory grades and clinical log books. Qualitative evaluation of both the SBA and teacher programs was obtained using semi-structured interviews with supervisors and teachers. We also reviewed perinatal indicators over an eight-year period, starting prior to the first training program until after the graduation of the fourth cohort of SBAs. Results Four SBA training programs scheduled between 2009 and 2015 resulted in 79/88 (90%) of students successfully completing a training program of 250 theory hours and 625 supervised clinical hours. All 79 students were able to: achieve pass grades on theory examination (median 80%, range [70–89]); obtain the required clinical experience within twelve months; achieve clinical competence to provide safe care during childbirth. In 2010–2011, five experienced SBAs completed a train-the-trainer (TOT) program and went on to facilitate further training programs. Perinatal indicators within Shoklo Malaria Research Unit (SMRU), such as place of birth, maternal and newborn outcomes, showed no significant differences before and after introduction of training or following graduate deployment in the local maternity units. Confidence, competence and teamwork emerged from qualitative evaluation by senior SBAs working with and supervising students in the clinics. Conclusions We demonstrate that in resource-limited settings or in marginalized populations, it is possible to accelerate training of skilled birth attendants to provide safe maternity care

Effective Practices for Evaluating Education and Public Outreach Programs

NASA Astrophysics Data System (ADS)

Wilkerson, S.

2013-12-01

Stephanie Baird Wilkerson, PhD Carol Haden EdD Magnolia Consulting,LLC Education and public outreach (EPO) program developers and providers seeking insights regarding effective practices for evaluating EPO activities programs benefit from understanding why evaluation is critical to the success of EPO activities and programs, what data collection methods are appropriate, and how to effectively communicate and report findings. Based on our extensive experience evaluating EPO programs, we will share lessons learned and examples of how these practices play out in actual evaluation studies. EPO program developers, providers, and evaluators must consider several factors that influence which evaluation designs and data collection methods will be most appropriate, given the nature of EPO programs. Effective evaluation practices of EPO programs take into account a program's phase of development, duration, and budget as well as a program's intended outcomes. EPO programs that are just beginning development will have different evaluation needs and priorities than will well-established programs. Effective evaluation practices consider the 'life' of a program with an evaluation design that supports a program's growth through various phases including development, revision and refinement, and completion. It would be premature and inappropriate to expect the attainment of longer-term outcomes of activities during program development phases or early stages of implementation. During program development, EPO providers should clearly define program outcomes that are feasible and appropriate given a program's scope and expected reach. In many respects, this directly relates to the amount of time, or duration, intended audiences participate in EPO programs. As program duration increases so does the likelihood that the program can achieve longer-term outcomes. When choosing which outcomes are reasonable to impact and measure, program duration should be considered. Effective evaluation

A Qualitative Inquiry of Career Exploration in Highly Implemented Career and Technical Education Programs of Study

ERIC Educational Resources Information Center

Stipanovic, Natalie; Stringfield, Sam

2013-01-01

This qualitative study explores career counseling and guidance services as provided to students engaged in career and technical education programs at three sites in the United States. The sites, consisting of high schools and community colleges, were part of the National Research Center for Career and Technical Education's 5-year studies of…

Evaluation of a Community-Based Program That Integrates Joyful Movement Into Fall Prevention for Older Adults

PubMed Central

Carlucci, Celeste; Kardachi, Julie; Bradley, Sara M.; Prager, Jason; Wyka, Katarzyna

2018-01-01

Background: Despite the development of evidence-based fall-prevention programs, there remains a need for programming that will engage older adults in real-world settings. Objective: This study aimed to evaluate a community-based group program that integrates joyful movement into fall prevention. The curriculum emphasizes a positive experience of movement, cultivating a healthy body image, and retraining of biomechanics. Design: Program evaluation was conducted using a one-group pre–post test study design. Key outcomes were functional balance and confidence. Qualitative feedback was gathered at the final class sessions. Results: Two hundred fifteen older adults enrolled at four sites over the period from 2010 to 2014. Among 86 participants who provided feedback, most credited the program for an increased sense of optimism and/or confidence (70%), and better walking ability (50%). Among 102 participants who completed both initial and final assessments, there was evidence of significant improvements on the Functional Reach Test (d = .60, p < .001) and Modified Falls Efficacy Scale (d = .17, p < .001). Conclusion: A joyful movement curriculum is acceptable to older participants, and they show improvements in functional balance and confidence. Future research should examine whether the positive changes encouraged by joyful movement lead to lasting reductions in fall risk and additional health benefits. PMID:29796405

Qualitative evaluation of a mobile phone and web-based collaborative care intervention for patients with type 2 diabetes.

PubMed

Lyles, Courtney Rees; Harris, Lynne T; Le, Tung; Flowers, Jan; Tufano, James; Britt, Diane; Hoath, James; Hirsch, Irl B; Goldberg, Harold I; Ralston, James D

2011-05-01

Drawing on previous web-based diabetes management programs based on the Chronic Care Model, we expanded an intervention to include care management through mobile phones and a game console web browser. The pilot intervention enrolled eight diabetes patients from the University of Washington in Seattle into a collaborative care program: connecting them to a care provider specializing in diabetes, providing access to their full electronic medical record, allowing wireless glucose uploads and e-mail with providers, and connecting them to the program's web services through a game system. To evaluate the study, we conducted qualitative thematic analysis of semistructured interviews. Participants expressed frustrations with using the cell phones and the game system in their everyday lives, but liked the wireless system for collaborating with a provider on uploaded glucoses and receiving automatic feedback on their blood sugar trends. A majority of participants also expressed that their participation in the trial increased their health awareness. Mobile communication technologies showed promise within a web-based collaborative care program for type 2 diabetes. Future intervention design should focus on integrating easy-to-use applications within mobile technologies already familiar to patients and ensure the system allows for sufficient collaboration with a care provider.

Why do water quality monitoring programs succeed or fail? A qualitative comparative analysis of regulated testing systems in sub-Saharan Africa.

PubMed

Peletz, Rachel; Kisiangani, Joyce; Bonham, Mateyo; Ronoh, Patrick; Delaire, Caroline; Kumpel, Emily; Marks, Sara; Khush, Ranjiv

2018-05-31

Water quality testing is critical for guiding water safety management and ensuring public health. In many settings, however, water suppliers and surveillance agencies do not meet regulatory requirements for testing frequencies. This study examines the conditions that promote successful water quality monitoring in Africa, with the goal of providing evidence for strengthening regulated water quality testing programs. We compared monitoring programs among 26 regulated water suppliers and surveillance agencies across six African countries. These institutions submitted monthly water quality testing results over 18 months. We also collected qualitative data on the conditions that influenced testing performance via approximately 821 h of semi-structured interviews and observations. Based on our qualitative data, we developed the Water Capacity Rating Diagnostic (WaterCaRD) to establish a scoring framework for evaluating the effects of the following conditions on testing performance: accountability, staffing, program structure, finances, and equipment & services. We summarized the qualitative data into case studies for each of the 26 institutions and then used the case studies to score the institutions against the conditions captured in WaterCaRD. Subsequently, we applied fuzzy-set Qualitative Comparative Analysis (fsQCA) to compare these scores against performance outcomes for water quality testing. We defined the performance outcomes as the proportion of testing Targets Achieved (outcome 1) and Testing Consistency (outcome 2) based on the monthly number of microbial water quality tests conducted by each institution. Our analysis identified motivation & leadership, knowledge, staff retention, and transport as institutional conditions that were necessary for achieving monitoring targets. In addition, equipment, procurement, infrastructure, and enforcement contributed to the pathways that resulted in strong monitoring performance. Our identification of institutional

Qualitative computer aided evaluation of dental impressions in vivo.

PubMed

Luthardt, Ralph G; Koch, Rainer; Rudolph, Heike; Walter, Michael H

2006-01-01

Clinical investigations dealing with the precision of different impression techniques are rare. Objective of the present study was to develop and evaluate a procedure for the qualitative analysis of the three-dimensional impression precision based on an established in-vitro procedure. The zero hypothesis to be tested was that the precision of impressions does not differ depending on the impression technique used (single-step, monophase and two-step-techniques) and on clinical variables. Digital surface data of patient's teeth prepared for crowns were gathered from standardized manufactured master casts after impressions with three different techniques were taken in a randomized order. Data-sets were analyzed for each patient in comparison with the one-step impression chosen as the reference. The qualitative analysis was limited to data-points within the 99.5%-range. Based on the color-coded representation areas with maximum deviations were determined (preparation margin and the mantle and occlusal surface). To qualitatively analyze the precision of the impression techniques, the hypothesis was tested in linear models for repeated measures factors (p < 0.05). For the positive 99.5% deviations no variables with significant influence were determined in the statistical analysis. In contrast, the impression technique and the position of the preparation margin significantly influenced the negative 99.5% deviations. The influence of clinical parameter on the deviations between impression techniques can be determined reliably using the 99.5 percentile of the deviations. An analysis regarding the areas with maximum deviations showed high clinical relevance. The preparation margin was pointed out as the weak spot of impression taking.

LEA Title VII Program Evaluations. Panel Presentations.

ERIC Educational Resources Information Center

Balu, Raj

These panel presentations focus on LEA Title VII Program Evaluations. Raj Balu, an administrator of bilingual programs in Chicago presents information regarding the bilingual education program in the Chicago public schools, as well as information on Title VII programs and what kind of evaluation is being done. Jesus Salazar, who is currently…

Conceptual bases of Christian, faith-based substance abuse rehabilitation programs: qualitative analysis of staff interviews.

PubMed

McCoy, Lisa K; Hermos, John A; Bokhour, Barbara G; Frayne, Susan M

2004-09-01

Faith-based substance abuse rehabilitation programs provide residential treatment for many substance abusers. To determine key governing concepts of such programs, we conducted semi-structured interviews with sample of eleven clinical and administrative staff referred to us by program directors at six, Evangelical Christian, faith-based, residential rehabilitation programs representing two large, nationwide networks. Qualitative analysis using grounded theory methods examined how spirituality is incorporated into treatment and elicited key theories of addiction and recovery. Although containing comprehensive secular components, the core activities are strongly rooted in a Christian belief system that informs their understanding of addiction and recovery and drives the treatment format. These governing conceptions, that addiction stems from attempts to fill a spiritual void through substance use and recovery through salvation and a long-term relationship with God, provide an explicit, theory-driven model upon which they base their core treatment activities. Knowledge of these core concepts and practices should be helpful to clinicians in considering referrals to faith-based recovery programs.

Non cardiopatic and cardiopatic beta thalassemic patients: quantitative and qualitative cardiac iron deposition evaluation with MRI.

PubMed

Macarini, L; Marini, S; Pietrapertosa, A; Scardapane, A; Ettorre, G C

2005-01-01

Cardiomyopathy is one of the major complications of b thalassaemia major as a result of transfusional iron overload. The aim of our study is to evaluate with MR if there is any difference of iron deposition signal intensity (SI) or distribution between non-cardiopathic and cardiopathic thalassaemic patients in order to establish if there is a relationship between cardiopathy and iron deposition. We studied 20 patients affected by b thalassaemia major, of whom 10 cardiopathic and 10 non-cardiopathic, and 10 healthy volunteers as control group. Serum ferritin and left ventricular ejection fraction were calculated in thalassaemic patients. All patients were examined using a 1.5 MR unit with ECG-gated GE cine-MR T2*-weighted, SE T1-weighted and GE T2*-weighted sequences. In all cases, using an adequate ROI, the myocardial and skeletal muscle signal intensity (SI), the myocardial/skeletal muscle signal intensity ratio (SIR) and the SI average of the myocardium and skeletal muscle were calculated for every study group. The qualitative evaluation of iron deposition distribution was independently performed by three radiologists who analyzed the extension, the site and the morphology of iron deposition on the MR images and reported their observations on the basis of a four-level rating scale: 0 (absent), 1 (limited), 2 (partial), 3 (widespread deposition). The result of quantitative and qualitative evaluations were analysed with statistical tests. Cardiac iron deposition was found in 8/10 non-cardiopathic thalassaemic patients and in all cardiopathic thalassaemic patients. We noticed a significant SI difference (p>0.05) between the healthy volunteer control group and the thalassaemic patients with iron deposition, but no significant SI difference in iron deposition between non-cardiopathic and cardiopathic thalassaemic patients in the areas evaluated. The qualitative evaluation revealed a different distribution of iron deposition between the two thalassaemic groups, with

A mental health first aid training program for Australian Aboriginal and Torres Strait Islander peoples: description and initial evaluation.

PubMed

Kanowski, Len G; Jorm, Anthony F; Hart, Laura M

2009-06-03

Mental Health First Aid (MHFA) training was developed in Australia to teach members of the public how to give initial help to someone developing a mental health problem or in a mental health crisis situation. However, this type of training requires adaptation for specific cultural groups in the community. This paper describes the adaptation of the program to create an Australian Aboriginal and Torres Strait Islander Mental Health First Aid (AMHFA) course and presents an initial evaluation of its uptake and acceptability. To evaluate the program, two types of data were collected: (1) quantitative data on uptake of the course (number of Instructors trained and courses subsequently run by these Instructors); (2) qualitative data on strengths, weaknesses and recommendations for the future derived from interviews with program staff and focus groups with Instructors and community participants. 199 Aboriginal people were trained as Instructors in a five day Instructor Training Course. With sufficient time following training, the majority of these Instructors subsequently ran 14-hour AMHFA courses for Aboriginal people in their community. Instructors were more likely to run courses if they had prior teaching experience and if there was post-course contact with one of the Trainers of Instructors. Analysis of qualitative data indicated that the Instructor Training Course and the AMHFA course are culturally appropriate, empowering for Aboriginal people, and provided information that was seen as highly relevant and important in assisting Aboriginal people with a mental illness. There were a number of recommendations for improvements. The AMHFA program is culturally appropriate and acceptable to Aboriginal people. Further work is needed to refine the course and to evaluate its impact on help provided to Aboriginal people with mental health problems.

Qualitative and quantitative outcomes of audience response systems as an educational tool in a plastic surgery residency program.

PubMed

Arneja, Jugpal S; Narasimhan, Kailash; Bouwman, David; Bridge, Patrick D

2009-12-01

In-training evaluations in graduate medical education have typically been challenging. Although the majority of standardized examination delivery methods have become computer-based, in-training examinations generally remain pencil-paper-based, if they are performed at all. Audience response systems present a novel way to stimulate and evaluate the resident-learner. The purpose of this study was to assess the outcomes of audience response systems testing as compared with traditional testing in a plastic surgery residency program. A prospective 1-year pilot study of 10 plastic surgery residents was performed using audience response systems-delivered testing for the first half of the academic year and traditional pencil-paper testing for the second half. Examination content was based on monthly "Core Quest" curriculum conferences. Quantitative outcome measures included comparison of pretest and posttest and cumulative test scores of both formats. Qualitative outcomes from the individual participants were obtained by questionnaire. When using the audience response systems format, pretest and posttest mean scores were 67.5 and 82.5 percent, respectively; using traditional pencil-paper format, scores were 56.5 percent and 79.5 percent. A comparison of the cumulative mean audience response systems score (85.0 percent) and traditional pencil-paper score (75.0 percent) revealed statistically significantly higher scores with audience response systems (p = 0.01). Qualitative outcomes revealed increased conference enthusiasm, greater enjoyment of testing, and no user difficulties with the audience response systems technology. The audience response systems modality of in-training evaluation captures participant interest and reinforces material more effectively than traditional pencil-paper testing does. The advantages include a more interactive learning environment, stimulation of class participation, immediate feedback to residents, and immediate tabulation of results for the

The Cost-Income Compenent of Program Evaluation.

ERIC Educational Resources Information Center

Miner, Norris

Cost-income studies are designed to serve two functions in instructional program evaluation. First, they act as the indicator of the economic value of a program. This economic value in conjunction with the other educational values needed in program evaluation allow for the most realistic appraisal of program worth. Second, if the studies show a…

Evaluating a Development Program.

ERIC Educational Resources Information Center

Frantzreb, Arthur C.

Problems in evaluating the success of an institutional resource development program are examined and suggestions are offered for effective assessment. Such a program is seen as a sequence of events in the artful management of people, from the planning stage to obtaining a financial commitment in writing from persons who want to share their assets…

Challenges of teacher-based clinical evaluation from nursing students' point of view: Qualitative content analysis.

PubMed

Sadeghi, Tabandeh; Seyed Bagheri, Seyed Hamid

2017-01-01

Clinical evaluation is very important in the educational system of nursing. One of the most common methods of clinical evaluation is evaluation by the teacher, but the challenges that students would face in this evaluation method, have not been mentioned. Thus, this study aimed to explore the experiences and views of nursing students about the challenges of teacher-based clinical evaluation. This study was a descriptive qualitative study with a qualitative content analysis approach. Data were gathered through semi-structured focused group sessions with undergraduate nursing students who were passing their 8 th semester at Rafsanjan University of Medical Sciences. Date were analyzed using Graneheim and Lundman's proposed method. Data collection and analysis were concurrent. According to the findings, "factitious evaluation" was the main theme of study that consisted of three categories: "Personal preferences," "unfairness" and "shirking responsibility." These categories are explained using quotes derived from the data. According to the results of this study, teacher-based clinical evaluation would lead to factitious evaluation. Thus, changing this approach of evaluation toward modern methods of evaluation is suggested. The finding can help nursing instructors to get a better understanding of the nursing students' point of view toward this evaluation approach and as a result could be planning for changing of this approach.

Workforce Training Program Evaluations.

ERIC Educational Resources Information Center

Washington State Workforce Training and Education Coordinating Board, Olympia.

Three evaluations analyzed program characteristics and participant results in work force education and coordination in Washington State. The Employment Security Department's evaluation of Job Training Partnership Act Titles II and III looked for an association between participant characteristics and the type of training they receive and between…

Reflections on Empowerment Evaluation: Learning from Experience.

ERIC Educational Resources Information Center

Fetterman, David M.

1999-01-01

Reflects on empowerment evaluation, the use of evaluation to foster improvement and self-determination. Empowerment evaluation uses quantitative and qualitative methods, and usually focuses on program evaluation. Discusses the growth in empowerment evaluation as a result of interest in participatory evaluation. (SLD)

A Qualitative Study of Elementary Afterschool Teachers' Perceptions of the Impact of Afterschool Programs on Students Receiving Special Education Services

ERIC Educational Resources Information Center

Legaspi, Margareth

2013-01-01

The purpose of this study, "A Qualitative Study of Elementary Afterschool Teachers' Perceptions of the Impact of Afterschool Programs on Students Receiving Special Education Services," was to assess elementary afterschool teachers' perceptions of the impact of afterschool programs on students receiving special education…

Teaching qualitative research to BSW students through exposure to aging.

PubMed

Sidell, Nancy L

2007-01-01

This article describes one rural program's efforts to expose students to gerontology through teaching qualitative research methodology. A collaborative research pilot project was developed with a local nursing home. BSW students worked in two groups to conduct and present qualitative research projects by the course's completion. This article describes the research project and evaluates the project's success from student and instructor viewpoints. Significant differences were found in self-reported student knowledge of key concepts at the project's completion, compared with pre-test knowledge. Student comments revealed value in this teaching approach. Implications for further engaging students in gerontological research are discussed.

Criteria for Evaluating Advancement Programs.

ERIC Educational Resources Information Center

Heemann, Warren, Ed.

Criteria for evaluating college and university advancement programs are presented, based on the efforts of professional area trustees and advisory committees of the Council for Advancement and Support of Education (CASE). The criteria can be useful in three ways: as the basis of internal audits of advancement programs or program components; as the…

Evaluation of the Master’s curriculum for elderly nursing: a qualitative study

PubMed Central

Ghaffari, Fatemeh; Dehghan-Nayeri, Nahid; Navabi, Nasrin; Seylani, Khatereh

2016-01-01

Background Improving the quality of health care and rehabilitation for the elderly is one of the most important priorities of the health care system. Given the importance of evaluating the strengths and weaknesses of any program after its implementation, this study was conducted to identify the advantages and weaknesses of a geriatric nursing program at Tehran University of Medical Sciences. Methods This was a qualitative study, and the study population comprised students, graduates, and professors of geriatric nursing at the Master of Science level. Data were collected through face-to-face interviews and focus groups. Sixteen interviews were conducted. The interview guide was used as a research tool. Interviews continued until data saturation was reached. Conventional content analysis was used to analyze the data. Results Three main themes including “motivation to enter geriatric nursing”, “lack of employment groundwork”, and “lack of practical implementation of the curriculum” were the main findings of the study. Conclusion Efforts to restructure the administrative system and employment can deter geriatric nursing students from simply earning a degree and actually encourage them to learn the required content. Appraisal and improvement of education facilities for student recruitment can guarantee the practical implementation of the curriculum. Drafting policies to attract graduates in clinical environments, opening up employment opportunities, providing organizational positions for the recruitment of this group, as well as dedicating some wards for elderly special care and providing nursing care to elderly people only can increase students’ motivation to learn and their hopes of good job prospects. PMID:27713624

Qualitative evaluations and comparisons of six night-vision colorization methods

NASA Astrophysics Data System (ADS)

Zheng, Yufeng; Reese, Kristopher; Blasch, Erik; McManamon, Paul

2013-05-01

Current multispectral night vision (NV) colorization techniques can manipulate images to produce colorized images that closely resemble natural scenes. The colorized NV images can enhance human perception by improving observer object classification and reaction times especially for low light conditions. This paper focuses on the qualitative (subjective) evaluations and comparisons of six NV colorization methods. The multispectral images include visible (Red-Green- Blue), near infrared (NIR), and long wave infrared (LWIR) images. The six colorization methods are channel-based color fusion (CBCF), statistic matching (SM), histogram matching (HM), joint-histogram matching (JHM), statistic matching then joint-histogram matching (SM-JHM), and the lookup table (LUT). Four categries of quality measurements are used for the qualitative evaluations, which are contrast, detail, colorfulness, and overall quality. The score of each measurement is rated from 1 to 3 scale to represent low, average, and high quality, respectively. Specifically, high contrast (of rated score 3) means an adequate level of brightness and contrast. The high detail represents high clarity of detailed contents while maintaining low artifacts. The high colorfulness preserves more natural colors (i.e., closely resembles the daylight image). Overall quality is determined from the NV image compared to the reference image. Nine sets of multispectral NV images were used in our experiments. For each set, the six colorized NV images (produced from NIR and LWIR images) are concurrently presented to users along with the reference color (RGB) image (taken at daytime). A total of 67 subjects passed a screening test ("Ishihara Color Blindness Test") and were asked to evaluate the 9-set colorized images. The experimental results showed the quality order of colorization methods from the best to the worst: CBCF < SM < SM-JHM < LUT < JHM < HM. It is anticipated that this work will provide a benchmark for NV

[Qualitative evaluation of blood products records in a hospital].

PubMed

Lartigue, B; Catillon, E

2012-02-01

This study aimed at evaluating the qualitative performance of blood products traceability from paper and electronic medical records in a hospital. Quality of date/time documentation was assessed by detection, for 20minutes or more, of chronological errors and inter-source inconsistencies, in a random sample of 168 blood products transfused during 2009. A receipt date/time was confirmed in 52% of paper records; a data entry error was attested in 25% of paper records, and 21% of electronic records. A transfusion date/time was notified in 93% of paper records, with a data entry error in 26% of paper records and 25% of electronic records. The patient medical record held at least one date/time error in 18% and 17%, for receipt and transfusion respectively. Environmental factors (clinical setting, urgency, blood product category) did not contributed to data error rates. Although blood products traceability has good quantitative results, the recorded documentation is not qualitative. In our study, data entry errors are similar in electronic or paper records, but the global failure rate is lesser in electronic records because omissions are controlled. Copyright © 2011 Elsevier Masson SAS. All rights reserved.

Handbook for Evaluating Drug and Alcohol Prevention Programs: Staff/Team Evaluation of Prevention Programs (STEPP).

ERIC Educational Resources Information Center

Hawkins, J. David; Nederhood, Britt

This handbook was developed for the purpose of providing drug and alcohol prevention program managers with a comprehensive yet easy-to-use tool to help their evaluation efforts. The handbook emphasizes program staff members working together as a team. It provides instruments and activities for determining program effectiveness, as well as…

Perceptions of the Principal Evaluation Process and Performance Criteria: A Qualitative Study of the Challenge of Principal Evaluation

ERIC Educational Resources Information Center

Faginski-Stark, Erica; Casavant, Christopher; Collins, William; McCandless, Jason; Tencza, Marilyn

2012-01-01

Recent federal and state mandates have tasked school systems to move beyond principal evaluation as a bureaucratic function and to re-imagine it as a critical component to improve principal performance and compel school renewal. This qualitative study investigated the district leaders' and principals' perceptions of the performance evaluation…

Preparing pharmacists to deliver a targeted service in hypertension management: evaluation of an interprofessional training program.

PubMed

Bajorek, Beata V; Lemay, Kate S; Magin, Parker J; Roberts, Christopher; Krass, Ines; Armour, Carol L

2015-09-28

Non-adherence to medicines by patients and suboptimal prescribing by clinicians underpin poor blood pressure (BP) control in hypertension. In this study, a training program was designed to enable community pharmacists to deliver a service in hypertension management targeting therapeutic adjustments and medication adherence. A comprehensive evaluation of the training program was undertaken. Tailored training comprising a self-directed pre-work manual, practical workshop (using real patients), and practice scenarios, was developed and delivered by an inter-professional team (pharmacists, GPs). Supported by practical and written assessment, the training focused on the principles of BP management, BP measurement skills, and adherence strategies. Pharmacists' experience of the training (expectations, content, format, relevance) was evaluated quantitatively and qualitatively. Immediate feedback was obtained via a questionnaire comprising Likert scales (1 = "very well" to 7 = "poor") and open-ended questions. Further in-depth qualitative evaluation was undertaken via semi-structured interviews several months post-training (and post service implementation). Seventeen pharmacists were recruited, trained and assessed as competent. All were highly satisfied with the training; other than the 'amount of information provided' (median score = 5, "just right"), all aspects of training attained the most positive score of '1'. Pharmacists most valued the integrated team-based approach, GP involvement, and inclusion of real patients, as well as the pre-reading manual, BP measurement workshop, and case studies (simulation). Post-implementation the interviews highlighted that comprehensive training increased pharmacists' confidence in providing the service, however, training of other pharmacy staff and patient recruitment strategies were highlighted as a need in future. Structured, multi-modal training involving simulated and inter-professional learning is effective in preparing

Improving Culturally Appropriate Care Using a Community-Based Participatory Research Approach: Evaluation of a Multicomponent Cultural Competency Training Program, Arkansas, 2015–2016

PubMed Central

Long, Christopher R.; Rowland, Brett; Moore, Sarah; Wilmoth, Ralph; Ayers, Britni

2017-01-01

Introduction The United States continues to become more racially and ethnically diverse, and racial/ethnic minority communities encounter sociocultural barriers to quality health care, including implicit racial/ethnic bias among health care providers. In response, health care organizations are developing and implementing cultural competency curricula. Using a community-based participatory research (CBPR) approach, we developed and evaluated a cultural competency training program to improve the delivery of culturally appropriate care in Marshallese and Hispanic communities. Methods We used a mixed-methods evaluation approach based on the Kirkpatrick model of training evaluation. We collected quantitative evaluation data immediately after each training session (March 19, 2015–November 30, 2016) and qualitative data about implementation at 2 points: immediately after each session and 6 months after training. Individuals and organizational units provided qualitative data. Results We delivered 1,250 units of in-person training at 25 organizations. Participants reported high levels of changes in knowledge (91.2%), competence (86.6%), and performance (87.2%) as a result of the cultural competency training. Organizations reported making policy and environmental changes. Conclusion Initial outcomes demonstrate the value of developing and implementing cultural competency training programs using a CBPR approach. Additional research is needed to determine the effect on long-term patient outcomes. PMID:28771402

Building capacity for water, sanitation, and hygiene programming: Training evaluation theory applied to CLTS management training in Kenya.

PubMed

Crocker, Jonny; Shields, Katherine F; Venkataramanan, Vidya; Saywell, Darren; Bartram, Jamie

2016-10-01

Training and capacity building are long established critical components of global water, sanitation, and hygiene (WaSH) policies, strategies, and programs. Expanding capacity building support for WaSH in developing countries is one of the targets of the Sustainable Development Goals. There are many training evaluation methods and tools available. However, training evaluations in WaSH have been infrequent, have often not utilized these methods and tools, and have lacked rigor. We developed a conceptual framework for evaluating training in WaSH by reviewing and adapting concepts from literature. Our framework includes three target outcomes: learning, individual performance, and improved programming; and two sets of influences: trainee and context factors. We applied the framework to evaluate a seven-month community-led total sanitation (CLTS) management training program delivered to 42 government officials in Kenya from September 2013 to May 2014. Trainees were given a pre-training questionnaire and were interviewed at two weeks and seven months after initial training. We qualitatively analyzed the data using our conceptual framework. The training program resulted in trainees learning the CLTS process and new skills, and improving their individual performance through application of advocacy, partnership, and supervision soft skills. The link from trainees' performance to improved programming was constrained by resource limitations and pre-existing rigidity of trainees' organizations. Training-over-time enhanced outcomes and enabled trainees to overcome constraints in their work. Training in soft skills is relevant to managing public health programs beyond WaSH. We make recommendations on how training programs can be targeted and adapted to improve outcomes. Our conceptual framework can be used as a tool both for planning and evaluating training programs in WaSH. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

Program Evaluation: The Board Game--An Interactive Learning Tool for Evaluators

ERIC Educational Resources Information Center

Febey, Karen; Coyne, Molly

2007-01-01

The field of program evaluation lacks interactive teaching tools. To address this pedagogical issue, the authors developed a collaborative learning technique called Program Evaluation: The Board Game. The authors present the game and its development in this practitioner-oriented article. The evaluation board game is an adaptable teaching tool…

Methods for the evaluation of the Jamie Oliver Ministry of Food program, Australia.

PubMed

Flego, Anna; Herbert, Jessica; Gibbs, Lisa; Swinburn, Boyd; Keating, Catherine; Waters, Elizabeth; Moodie, Marj

2013-04-30

Community-based programs aimed at improving cooking skills, cooking confidence and individual eating behaviours have grown in number over the past two decades. Whilst some evidence exists to support their effectiveness, only small behavioural changes have been reported and limitations in study design may have impacted on results.This paper describes the first evaluation of the Jamie Oliver Ministry of Food Program (JMoF) Australia, in Ipswich, Queensland. JMoF Australia is a community-based cooking skills program open to the general public consisting of 1.5 hour classes weekly over a 10 week period, based on the program of the same name originating in the United Kingdom. A mixed methods study design is proposed. Given the programmatic implementation of JMoF in Ipswich, the quantitative study is a non-randomised, pre-post design comparing participants undergoing the program with a wait-list control group. There will be two primary outcome measures: (i) change in cooking confidence (self-efficacy) and (ii) change in self-reported mean vegetable intake (serves per day). Secondary outcome measures will include change in individual cooking and eating behaviours and psycho-social measures such as social connectedness and self-esteem. Repeated measures will be collected at baseline, program completion (10 weeks) and 6 months follow up from program completion. A sample of 250 participants per group will be recruited for the evaluation to detect a mean change of 0.5 serves a day of vegetables at 80% power (0.5% significance level). Data analysis will assess the magnitude of change of these variables both within and between groups and use sub group analysis to explore the relationships between socio-demographic characteristics and outcomes.The qualitative study will be a longitudinal design consisting of semi-structured interviews with approximately 10-15 participants conducted at successive time points. An inductive thematic analysis will be conducted to explore social

Methods for the evaluation of the Jamie Oliver Ministry of Food program, Australia

PubMed Central

2013-01-01

Background Community-based programs aimed at improving cooking skills, cooking confidence and individual eating behaviours have grown in number over the past two decades. Whilst some evidence exists to support their effectiveness, only small behavioural changes have been reported and limitations in study design may have impacted on results. This paper describes the first evaluation of the Jamie Oliver Ministry of Food Program (JMoF) Australia, in Ipswich, Queensland. JMoF Australia is a community-based cooking skills program open to the general public consisting of 1.5 hour classes weekly over a 10 week period, based on the program of the same name originating in the United Kingdom. Methods/Design A mixed methods study design is proposed. Given the programmatic implementation of JMoF in Ipswich, the quantitative study is a non-randomised, pre-post design comparing participants undergoing the program with a wait-list control group. There will be two primary outcome measures: (i) change in cooking confidence (self-efficacy) and (ii) change in self-reported mean vegetable intake (serves per day). Secondary outcome measures will include change in individual cooking and eating behaviours and psycho-social measures such as social connectedness and self-esteem. Repeated measures will be collected at baseline, program completion (10 weeks) and 6 months follow up from program completion. A sample of 250 participants per group will be recruited for the evaluation to detect a mean change of 0.5 serves a day of vegetables at 80% power (0.5% significance level). Data analysis will assess the magnitude of change of these variables both within and between groups and use sub group analysis to explore the relationships between socio-demographic characteristics and outcomes. The qualitative study will be a longitudinal design consisting of semi-structured interviews with approximately 10-15 participants conducted at successive time points. An inductive thematic analysis will be

Evaluation Project of a Postvention Program.

ERIC Educational Resources Information Center

Simon, Robert; And Others

A student suicide or parasuicide increases the risk that potentially suicidal teenagers see suicide as an enviable option. The "copycat effect" can be reduced by a postvention program. This proposed evaluative research project will provide an implementation and impact evaluation of a school's postvention program following a suicide or…

The Use of Multiple Evaluation Approaches in Program Evaluation

ERIC Educational Resources Information Center

Bledsoe, Katrina L.; Graham, James A.

2005-01-01

The authors discuss the use of multiple evaluation approaches in conducting program evaluations. Specifically, they illustrate four evaluation approaches (theory-driven, consumer-based, empowerment, and inclusive evaluation) and briefly discuss a fifth (use-focused evaluation) as a side effect of the use of the others. The authors also address the…

Evaluation of a Bullying Prevention Program

ERIC Educational Resources Information Center

Hallford, Abby; Borntrager, Cameo; Davis, Joanne L.

2006-01-01

In order to address the federal "No Child Left Behind Act", the state of Oklahoma required that all public schools address the problem of bullying. Although numerous anti-bullying programs exist, few have been evaluated to determine their effectiveness. The present study evaluated the effectiveness of one such program,…

Evaluation of the Brownfields Program

EPA Pesticide Factsheets

The Evaluation of 2003-2008 Brownfields Assessment, Revolving Loan Fund, and Cleanup Grants is the first national program evaluation of the outcomes, efficiencies, and economic benefits produced by Brownfields grants.

Putting program evaluation into practice: enhancing the Girls Just Wanna Have Fun program.

PubMed

Bean, Corliss N; Kendellen, Kelsey; Halsall, Tanya; Forneris, Tanya

2015-04-01

In recent years there has been a call for increased community physical activity and sport programs for female youth that are deliberately structured to foster positive developmental outcomes. In addition, researchers have recognized the need to empirically evaluate such programs to ensure that youth are provided with optimal opportunities to thrive. This study represents a utilization-focused evaluation of Girls Just Wanna Have Fun, a female-only physical activity-based life skills community program. A utilization-focused evaluation is particularly important when the evaluation is to help stakeholders utilize the findings in practice. The purpose of this study was twofold: (a) to gain an understanding of the ongoing successes and challenges after year two of program implementation and (b) to examine how the adaptations made based on feedback from the first year evaluation were perceived as impacting the program. From interviews with youth participants and program leaders, three main themes with eight sub-themes emerged. The main themes were: (a) applying lessons learned can make a significant difference, (b) continually implementing successful strategies, and (c) ongoing challenges. Overall, this evaluation represents an important step in understanding how to improve program delivery to better meet the needs of the participants in community-based programming. Copyright © 2014 Elsevier Ltd. All rights reserved.

Evaluating an mHealth App for Health and Well-Being at Work: Mixed-Method Qualitative Study

PubMed Central

Wiezer, Noortje; Janssen, Joris H; Vink, Peter; Kraaij, Wessel

2018-01-01

Background To improve workers’ health and well-being, workplace interventions have been developed, but utilization and reach are unsatisfactory, and effects are small. In recent years, new approaches such as mobile health (mHealth) apps are being developed, but the evidence base is poor. Research is needed to examine its potential and to assess when, where, and for whom mHealth is efficacious in the occupational setting. To develop interventions for workers that actually will be adopted, insight into user satisfaction and technology acceptance is necessary. For this purpose, various qualitative evaluation methods are available. Objective The objectives of this study were to gain insight into (1) the opinions and experiences of employees and experts on drivers and barriers using an mHealth app in the working context and (2) the added value of three different qualitative methods that are available to evaluate mHealth apps in a working context: interviews with employees, focus groups with employees, and a focus group with experts. Methods Employees of a high-tech company and experts were asked to use an mHealth app for at least 3 weeks before participating in a qualitative evaluation. Twenty-two employees participated in interviews, 15 employees participated in three focus groups, and 6 experts participated in one focus group. Two researchers independently coded, categorized, and analyzed all quotes yielded from these evaluation methods with a codebook using constructs from user satisfaction and technology acceptance theories. Results Interviewing employees yielded 785 quotes, focus groups with employees yielded 266 quotes, and the focus group with experts yielded 132 quotes. Overall, participants muted enthusiasm about the app. Combined results from the three evaluation methods showed drivers and barriers for technology, user characteristics, context, privacy, and autonomy. A comparison between the three qualitative methods showed that issues revealed by experts

The Value in Evaluating and Communicating Program Impact: The Ohio BR&E Program

ERIC Educational Resources Information Center

Daivs, Gregory

2012-01-01

Assessing program impact can provide useful program evaluation data. It also provides a basis for program development, marketing, and justification. This article discusses recent impact evaluation efforts and findings of a long-time Extension program; referred to as Business Retention and Expansion (BR&E). How such information can be…

Cost Analysis for Educational Program Evaluation.

ERIC Educational Resources Information Center

Haller, Emil J.

Concerned with the problem of determining program costs as part of the evaluation process, this article helps the practitioner arrive at useful conceptions of the term "cost" and procedures for assessing the costs of an educational program. Its purpose is to help design costing procedures for evaluation situations that are commonly encountered. An…

South Korean Family Caregiver Involvement in Delirium Care: A Qualitative Descriptive Study.

PubMed

Kang, Yun; Moyle, Wendy; Cooke, Marie; O'Dwyer, Siobhan

2017-12-01

The current study aimed to describe the effect of an educational program on RN-initiated efforts to involve family caregivers in delirium care. A descriptive qualitative study was performed. A purposive sample of 12 RNs who participated in a one group, pre-post evaluation of a delirium educational program, and a nominated sample of six family caregivers of patients who had been cared for by RNs in the program participated in individual, in-depth interviews. The qualitative findings indicated that the inclusion of a delirium brochure in ward orientation on admission, with reinforcement during shift changes, and RN-initiated engagement with family caregivers promoted family caregiver involvement in delirium prevention. Further studies in South Korea are needed to determine which family-centered nursing care interventions are culturally appropriate and most effective for RNs and family caregivers in delirium care. [Journal of Gerontological Nursing, 43(12), 44-51.]. Copyright 2017, SLACK Incorporated.

Program Evaluation Interest and Skills of School Counselors

ERIC Educational Resources Information Center

Astramovich, Randall L.

2017-01-01

School counselors participated in a study examining their program evaluation interest and skills. Findings suggest that school counselors understand the importance of program evaluation, yet they may lack the skills and confidence to successfully engage in program evaluation activities. Professional development training may be an important method…

Apollo Photograph Evaluation (APE) programming manual

NASA Technical Reports Server (NTRS)

Kim, I. J.

1974-01-01

This document describes the programming techniques used to implement the equations of the Apollo Photograph Evaluation (APE) program on the UNIVAC 1108 computer and contains detailed descriptions of the program structure, a User's Guide section to provide the necessary information for proper operation of the program, and information for the assessment of the program's adaptability to future problems.

Improving Healthy Living Youth Development Program Outreach in Extension: Lessons Learned from the 4-H Health Rocks! Program

ERIC Educational Resources Information Center

Kumaran, Muthusami; Fogarty, Kate; Fung, Whitney M.; Terminello, Amanda

2015-01-01

This article discusses a qualitative evaluation of the Florida 4-H Health Rocks! program aimed at youth alcohol, tobacco, and other drug use prevention. A questionnaire was distributed to Extension professionals across Florida to gain insight into the strengths and barriers they faced with programming. Programmatic strengths included targeting a…

Summary of Program Evaluation Results: 1985-1986 School Year Pre-Kindergarten Educational Program.

ERIC Educational Resources Information Center

Heath, Robert W.; And Others

Reported are findings of the 1985-86 program evaluation of the prenatal-to-preschool and preschool programs operating under the auspices of the Kamehameha Schools/Bishop Estate. Evaluation of the prenatal-to-preschool program (the Kupulani Program) included item analysis of the Questions about Pregnancy Test, development of a revised data…

A process evaluation of a social cognitive theory-based childhood obesity prevention intervention: the Comics for Health program.

PubMed

Branscum, Paul; Sharma, Manoj; Wang, Lihshing Leigh; Wilson, Bradley; Rojas-Guyler, Liliana

2013-03-01

Process evaluations are an often overlooked yet essential component of health promotion interventions. This study reports the results of a comprehensive process evaluation for the "Comics for Health" program, a childhood obesity prevention intervention implemented at 12 after-school programs. Qualitative and quantitative process data were collected using surveys, field notes, and open-item questionnaires, which assessed program fidelity, dose delivered, dose received, reach, recruitment, and context. Triangulation of methods was also employed to better understand how the program was implemented and received by the facilitator, staff members, and children in the program. Results indicated that program implementation had an almost perfect rate of fidelity with most lessons recording 100% tasks completed. Lessons were implemented in their intended order and lasted approximately 30 minutes as planned. After-school staff members reported that the program was well received by children, and this program should be replicated in the future. Attendance records showed that a majority of the children attended each lesson on the initial day of delivery (70.4%) and informal make-up lessons were implemented to compensate for the other children. Finally, several known sources of contamination were found such as past and concurrent exposure to similar health promotion interventions, which could potentially influence study outcomes. These findings will be used to help explain the results of this intervention and make recommendations for future intervention efforts.

Evaluating Student Success and Progress in the Maryland Sea Grant REU Program

NASA Astrophysics Data System (ADS)

Moser, F. C.; Allen, M. R.; Clark, J.

2012-12-01

The Maryland Sea Grant's Research Experiences for Undergraduate (REU) 12-week summer program is in its 24th year. This estuarine science-focused program has evolved, based in part on our use of assessment tools to measure the program's effectiveness. Our goal is to understand the REU program's effectiveness in such areas as improving student understanding of scientific research, scientific ethics and marine science careers. Initially, our assessment approach was limited to short surveys that used qualitative answers from students about their experience. However, in the last decade we have developed a more comprehensive approach to measure program effectiveness. Currently, we use paired pre- and post-survey questions to estimate student growth during the program. These matching questions evaluate the student's change in knowledge and perception of science research over the course of the summer program. Additionally, we administer several surveys during the 12 weeks of the program to measure immediate responses of students to program activities and to gauge the students' evolving attitudes to customize each year's program. Our 2011 cohort showed consistent improvement in numerous areas, including understanding the nature of science (pre: 4.35, post: 4.64 on a 5 point scale), what graduate school is like (3.71, 4.42), the job of a researcher (4.07, 4.50), and career options in science (3.86, 4.42). Student confidence also increased in numerous skills required for good scientists. To analyze the long-term impact of our program, we survey our alumni to assess graduate degrees earned and career choices. A large percentage (72%) of our tracked alumni have continued on to graduate school, with subsequent careers spanning the academic (51%), public (24%) and private (25%) sectors. These assessments demonstrate that our program is successful in meeting our key objectives of strengthening the training of undergraduates in the sciences and retaining them in marine science

Interviewer as instrument: accounting for human factors in evaluation research.

PubMed

Brown, Joel H

2006-04-01

This methodological study examines an original data collection model designed to incorporate human factors and enhance data richness in qualitative and evaluation research. Evidence supporting this model is drawn from in-depth youth and adult interviews in one of the largest policy/program evaluations undertaken in the United States, the Drug, Alcohol, and Tobacco Education evaluation (77 districts, 118 schools). When applying the explicit observation technique (EOT)--the strategic and nonjudgmental disclosure of nonverbal human factor cues by the interviewer to the respondent during interview--data revealed the observation disclosure pattern. Here, respondents linked perceptions with policy or program implementation or effectiveness evidence. Although more research is needed, it is concluded that the EOT yields richer data when compared with traditional semistructured interviews and, thus, holds promise to enhance qualitative and evaluation research methods. Validity and reliability as well as qualitative and evaluation research considerations are discussed.

The Role of Evaluation and Plans for Evaluating the Current Testing Program.

ERIC Educational Resources Information Center

Winters, Lynn

The Palos Verdes Peninsula Unified School District Office of Program Evaluation and Research is responsible for providing information for program development and improvement; providing test information to special programs coordinators; and acting as a clearinghouse for all information concerning tests, evaluation methodology, and educational…

Evaluation of an occupational rehabilitation program.

PubMed

Goodman, Glenn; Browning, Margaret; Campbell, Sims; Hudak, Huison

2005-01-01

The purpose of this study was to report the findings of a program evaluation for an occupational rehabilitation program in the Midwest. An ex-post facto chart review was performed on 50 charts using demographic data, results from tests of performance and pain measures, and data from patient satisfaction questionnaires to find evidence of excellence in the program, and to identify areas for improvement. Over 97% of the participants actually completed the program and 76% returned to work within 3 months after program completion. Ninety-two percent indicated overall satisfaction with the program, and all aspects of the program were rated with a mean score of 4.25 or above on a 1 to 5 Likert scale. Evaluation of perceived pain scales of the participants indicate no discernable relationship between intensity of pain and successful return to work. The program showed a high completion rate, a high return to work rate, and high levels of patient satisfaction. Suggestions for improvement include an increase in use of real and simulated work activities, better documentation of pain measures, better programs to address psychosocial issues, lengthening the program, and increased communication with case managers and professionals outside of the work program.

Evaluating programs that address ideological issues: ethical and practical considerations for practitioners and evaluators.

PubMed

Lieberman, Lisa D; Fagen, Michael C; Neiger, Brad L

2014-03-01

There are important practical and ethical considerations for organizations in conducting their own, or commissioning external, evaluations and for both practitioners and evaluators, when assessing programs built on strongly held ideological or philosophical approaches. Assessing whether programs "work" has strong political, financial, and/or moral implications, particularly when expending public dollars, and may challenge objectivity about a particular program or approach. Using a case study of the evaluation of a school-based abstinence-until-marriage program, this article discusses the challenges, lessons learned, and ethical responsibilities regarding decisions about evaluation, specifically associated with ideologically driven programs. Organizations should consider various stakeholders and views associated with their program to help identify potential pitfalls in evaluation. Once identified, the program or agency needs to carefully consider its answers to two key questions: Do they want the answer and are they willing to modify the program? Having decided to evaluate, the choice of evaluator is critical to assuring that ethical principles are maintained and potential skepticism or criticism of findings can be addressed appropriately. The relationship between program and evaluator, including agreements about ownership and eventual publication and/or promotion of data, should be addressed at the outset. Programs and organizations should consider, at the outset, their ethical responsibility when findings are not expected or desired. Ultimately, agencies, organizations, and programs have an ethical responsibility to use their data to provide health promotion programs, whether ideologically founded or not, that appropriately and effectively address the problems they seek to solve.

A Qualitative Evaluation of the Students of Service (SOS) Program for Sexual Abstinence in Louisiana

ERIC Educational Resources Information Center

Yoo, Seunghyun; Johnson, Carolyn C.; Rice, Janet; Manuel, Powlin

2004-01-01

Abstinence-only programs for preventing teen pregnancy are the only options in some states but are the programs of choice in others. Effectiveness data, however, are lacking. The SOS Adolescent Family Life Program (SOS), an abstinence-only teen pregnancy prevention program, was implemented in south central Louisiana. Peer mentoring with an…

Enhancing positive attitudes towards disability: evaluation of an integrated physiotherapy program.

PubMed

Morgan, Prue Elizabeth; Lo, Kristin

2013-02-01

This study explored whether attitudes towards disability in second year undergraduate physiotherapy students could be enhanced by an on-campus integrated curriculum program. A pre-post design was used. Year 2 (pre-clinical) students participated in a 12-week program focused on optimising attitudes towards people with acquired or developmental neurological disability. The Discomfort subscale of the Interaction with Disabled Persons scale, rated on a six-point Likert scale, was applied prior to and at completion of the 12-week program, and compared to year 4 students, just prior to graduation. Qualitative data from year 2 reflective narratives was also gathered. Forty-seven second year and 45 fourth year physiotherapy students participated. The difference in Discomfort subscale scores between weeks 1 and 12 of year 2 was statistically significant (p = 0.0016). The difference in Discomfort subscale scores between year 2 week 1 and year 4 students was also statistically significant (p = 0.040). There was no significant difference in attitudes between students at the end of year 2 and the end of year 4 (p = 0.703). Qualitative data supported the development of more positive attitudes towards neurological disability across the 12 week year 2 pre-clinical program. Student attitudes towards people with acquired and/or developmental neurological disabilities can be enhanced through an on campus integrated curriculum program.

Program Theory and Quality Matter: Changing the Course of Extension Program Evaluation

ERIC Educational Resources Information Center

Arnold, Mary E.; Cater, Melissa

2016-01-01

As internal evaluators for the 4-H program in two states, we simultaneously yet independently began to change the way we approached our evaluation practices, turning from evaluation capacity building (ECB) efforts that prepared educators to define and measure program outcomes to strategies that engage educators in defining and measuring program…

Strategies for Evaluating Complex Environmental Education Programs

NASA Astrophysics Data System (ADS)

Williams, V.

2011-12-01

Evidence for the effectiveness of environmental education programs has been difficult to establish for many reasons. Chief among them are the lack of clear program objectives and an inability to conceptualize how environmental education programs work. Both can lead to evaluations that make claims that are difficult to substantiate, such as significant changes in student achievement levels or behavioral changes based on acquisition of knowledge. Many of these challenges can be addressed by establishing the program theory and developing a logic model. However, claims of impact on larger societal outcomes are difficult to attribute solely to program activities. Contribution analysis may offer a promising method for addressing this challenge. Rather than attempt to definitively and causally link a program's activities to desired results, contribution analysis seeks to provide plausible evidence that can reduce uncertainty regarding the 'difference' a program is making to observed outcomes. It sets out to verify the theory of change behind a program and, at the same time, takes into consideration other influencing factors. Contribution analysis is useful in situations where the program is not experimental-there is little or no scope for varying how the program is implemented-and the program has been funded on the basis of a theory of change. In this paper, the author reviews the feasibility of using contribution analysis as a way of evaluating the impact of the GLOBE program, an environmental science and education program. Initially conceptualized by Al Gore in 1995, the program's implementation model is based on worldwide environmental monitoring by students and scientists around the globe. This paper will make a significant and timely contribution to the field of evaluation, and specifically environmental education evaluation by examining the usefulness of this analysis for developing evidence to assess the impact of environmental education programs.

Evaluating Federal Social Programs: An Uncertain Act.

ERIC Educational Resources Information Center

Levitan, Sar A.; Wurzburg, Gregory K.

This study of the federal government's evaluation of social programs indicates that it is virtually impossible to establish a bias-free, valid, and reliable system of inquiry to determine the effects of social programs. Divided into five chapters, the document examines the aspirations and limitations of evaluations, methodology, evaluation in the…

A Formative Evaluation of Two Evidence-Based Psychotherapies for PTSD In VA Residential Treatment Programs

PubMed Central

Cook, Joan M.; O’Donnell, Casey; Dinnen, Stephanie; Bernardy, Nancy; Rosenheck, Robert; Desai, Rani

2013-01-01

Thirty-eight U.S. Department of Veterans Affairs (VA) residential treatment programs for posttraumatic stress disorder (PTSD) participated in a formative evaluation of their programmatic services, including evidenced-based treatments (EBTs), between July 2008 and March 2011. Face-to-face qualitative interviews were conducted with over 250 staff by an independent psychologist along with on-site participant observations. This evaluation coincided with a national VA dissemination initiative to train providers in two EBTs for PTSD, Prolonged Exposure (PE) and Cognitive Processing Therapy (CPT). A substantial proportion of eligible (based on professional background) residential treatment providers received training in PE (37.4%) or CPT (64.2%), with 9.5% completing case consultation or becoming national trainers in each therapy respectively. In semi-structured interviews, providers reported that their clinical programs had adopted these EBTs at varying levels ranging from no adoption to every patient receiving the full protocol. Suggestions for improving the adoption of PE and CPT are noted, including distilling manualized treatments to essential common elements. PMID:23417875

Implementation and Outcomes of Forensic Housing First Programs.

PubMed

Kriegel, Liat S; Henwood, Benjamin F; Gilmer, Todd P

2016-01-01

This mixed-method study used administrative data from 68 supportive housing programs and evaluative and qualitative site visit data from a subset of four forensic programs to (a) compare fidelity to the Housing First model and residential client outcomes between forensic and nonforensic programs and (b) investigate whether and how providers working in forensic programs can navigate competing Housing First principles and criminal justice mandates. Quantitative findings suggested that forensic programs were less likely to follow a harm reduction approach to substance use and clients in those programs were more likely to live in congregate settings. Qualitative findings suggested that an interplay of court involvement, limited resources, and risk environments influenced staff decisions regarding housing and treatment. Existing mental health and criminal justice collaborations necessitate adaptation to the Housing First model to accommodate client needs.

Shifting Practices in Teacher Performance Evaluation: A Qualitative Examination of Administrator Change Readiness

ERIC Educational Resources Information Center

Spina, Nancy; Buckley, Phillip; Puchner, Laurel

2014-01-01

This study examines the perceptions, attitudes and beliefs of administrators and teachers in a Southwestern Illinois School District regarding the recent reforms in teacher performance evaluation. This study uses a qualitative approach and provides data from individual and focus group interviews to determine the extent to which the district is…

Using Radar Charts with Qualitative Evaluation: Techniques to Assess Change in Blended Learning

ERIC Educational Resources Information Center

Kaczynski, Dan; Wood, Leigh; Harding, Ansie

2008-01-01

When university academics implement changes in learning, such as introducing blended learning, it is conventional practice to examine and evaluate the impact of the resulting curriculum reform. Judging the worth and impact of an educational development is a complex task involving subtle differences in learning. Qualitative methods to explore these…

Suitability of a structured Fundamental Movement Skills program for long day care centres: a process evaluation.

PubMed

Petrunoff, Nick; Lloyd, Beverley; Watson, Natalie; Morrisey, David

2009-04-01

Early childhood presents an opportunity to encourage development of Fundamental Movement Skills (FMS). Implementation of a structured program in the Long Day Care (LDC) setting presents challenges. Implementation of a structured FMS program FunMoves was assessed in LDC in metropolitan New South Wales. LDC staff attended a training session conducted by trained Health Promotion Officers (HPOs) and completed an evaluation. During implementation HPOs completed lesson observations. De-identified attendance data was collected and director and staff feedback on the program including barriers to implementation was obtained via questionnaire. Qualitative information relevant to process evaluation was obtained via open questions on questionnaires, and a de-brief diary recording feedback from directors and staff. Knowledge of FMS and FunMoves and staff confidence to deliver the program were high after training. On average, staff stated they ran lessons more than the suggested twice weekly and the majority of children attended 1-3 lessons per week. However, lesson delivery was not as designed, and staff found FunMoves disruptive and time consuming. Six directors and the majority of staff thought that FunMoves could be improved. Structured program delivery was hampered by contextual issues including significant staff turnover and program length and structure being at odds with the setting. Implementation could be enhanced by guidelines for more flexible delivery options including less structured approaches, shorter and simpler lessons, ongoing conversations with the early childhood sector, in-centre engagement of staff and post-training support.

Multiple-Methods Needs Assessment of California 4-H Science Education Programming

ERIC Educational Resources Information Center

Worker, Steven M.; Schmitt-McQuitty, Lynn; Ambrose, Andrea; Brian, Kelley; Schoenfelder, Emily; Smith, Martin H.

2017-01-01

The California 4-H Science Leadership Team conducted a statewide assessment to evaluate the needs of county-based 4-H programs related to the key areas of the 4-H Science Initiative: program development and design, professional development, curricula, evaluation, partnerships, and fund development. The use of multiple qualitative data sources…

Strategies for Evaluating a Freshman Studies Program.

ERIC Educational Resources Information Center

Ketkar, Kusum; Bennett, Shelby D.

1989-01-01

The study developed an economic model for the evaluation of Seaton Hall University's freshman studies program. Two techniques used to evaluate the economic success of the program are break-even analysis and elasticity coefficient. (Author/MLW)

MRM Evaluation Research Program

NASA Technical Reports Server (NTRS)

Taylor, James C.

1998-01-01

This is an interim report on the current output of the MRM evaluation research program. During 1998 this research program has used new and existing data to create an important tool for the development and improvement of "maintenance resource management" (MRM). Thousands of surveys completed by participants in airline MRM training and/or behavior change programs have, for the first time, been consolidated into a panel of "MRM Attitudes and Opinion Profiles." These profiles can be used to compare the attitudes about decision making and communication in any given company at any stage in its MRM program with attitudes of a large sample of like employees during a similar period in their MRM involvement. This panel of comparison profiles for attitudes and opinions is a tool to help audit the effectiveness of a maintenance human factors program. The profile panel is the first of several tools envisioned for applying the information accumulating in MRM databases produced as one of the program's long range objectives.

Universal Free School Breakfast: A Qualitative Process Evaluation According to the Perspectives of Senior Stakeholders.

PubMed

Harvey-Golding, Louise; Donkin, Lynn Margaret; Defeyter, Margaret Anne

2016-01-01

In the last decade, the provision of school breakfast has increased significantly in the UK. However, there is an absence of knowledge regarding senior stakeholder views on the processes and potential outcomes on different groups, within the communities served by school breakfast programs. The purpose of this study was to examine the views and experiences of senior level stakeholders and thereby provide an original qualitative contribution to the research. A sample of senior level stakeholders was recruited, including senior officers, directors, and elected members, from within a Local Authority (LA) involved in the leadership, implementation and delivery of a council-wide universal free school breakfast (UFSB) program, and from the senior staff body of mainstream primary and special schools, participating in the program. A grounded theory analysis of the data collected identified issues encountered in the implementation and delivery, and views on the funding and future of a USFB program, in addition to perceived outcomes for children, parents, families, schools, and the wider community. The results refer to both positive and negative issues and implications associated with the program, according to the perspectives of senior level stakeholders. Perceived positive outcomes included benefits to children, families, schools, and the community. For instance, alleviating hunger, improving health outcomes, and conferring financial benefits, with the potential to cumulate in overall improvements in educational, social, and behavioral outcomes. Reported negative implications included the absence of an effective communication strategy in implementing the USFB program; in addition to concerns about the impacts of "double-breakfasting" on obesity levels among children, particularly in less deprived communities. Findings were validated using theoretical sampling and saturation, triangulation methods, member checks, and inter-rater reliability measures. In presenting these

Investigating the key factors in designing a communication skills program for medical students: A qualitative study.

PubMed

Mahdi Hazavehei, Seyyed M; Karimi Moonaghi, Hossein; Moeini, Babak; Moghimbeigi, Abbas; Emadzadeh, Ali

2015-11-01

Medical students have a serious need to acquire communication skills with others. In many medical schools, special curriculums are developed to improve such skills. Effective training of communication skills requires expert curriculum design. The aim of this study was to explore the experiences and views of experts and stakeholders in order to design a suitable training program in communication skills for medical students. The content analysis approach was used in this qualitative study. Forty-three participants were selected from the faculty, nurses, physicians, residents, and medical students at Mashhad University of Medical Sciences using purposive sampling. The data were collected through focus group discussions and semi-structured interviews. To ensure the accuracy of the data, the criteria of credibility, transferability, dependability, and conformability were met. The data were analyzed by MAXQDA software using the Graneheim & Lundman model. The findings of this study consisted of two main themes, i.e., "The vast nature of the present communication skills training" and "administrative requirements of the training program regarding communication skills." The first theme included the educational needs of students, the problems associated with training people to have good communication skills, the importance of good communication skills in performing professional duties, communication skills and job requirements, the learning environment of communication skills, and the status of existing training programs for communication skills. Strategies and suitable methods for teaching communication skills and methods of evaluating the students in this regard also were obtained. The findings of this study were the elements required to design a proper and local model to teach communication skills to medical students through analyzing the concepts of effective communication. The results of this study can be useful for medical faculties in designing a proper program for

Investigating the key factors in designing a communication skills program for medical students: A qualitative study

PubMed Central

Mahdi Hazavehei, Seyyed M.; Moonaghi, Hossein Karimi; Moeini, Babak; Moghimbeigi, Abbas; Emadzadeh, Ali

2015-01-01

Introduction Medical students have a serious need to acquire communication skills with others. In many medical schools, special curriculums are developed to improve such skills. Effective training of communication skills requires expert curriculum design. The aim of this study was to explore the experiences and views of experts and stakeholders in order to design a suitable training program in communication skills for medical students. Methods The content analysis approach was used in this qualitative study. Forty-three participants were selected from the faculty, nurses, physicians, residents, and medical students at Mashhad University of Medical Sciences using purposive sampling. The data were collected through focus group discussions and semi-structured interviews. To ensure the accuracy of the data, the criteria of credibility, transferability, dependability, and conformability were met. The data were analyzed by MAXQDA software using the Graneheim & Lundman model. Results The findings of this study consisted of two main themes, i.e., “The vast nature of the present communication skills training” and “administrative requirements of the training program regarding communication skills.” The first theme included the educational needs of students, the problems associated with training people to have good communication skills, the importance of good communication skills in performing professional duties, communication skills and job requirements, the learning environment of communication skills, and the status of existing training programs for communication skills. Strategies and suitable methods for teaching communication skills and methods of evaluating the students in this regard also were obtained. Conclusion The findings of this study were the elements required to design a proper and local model to teach communication skills to medical students through analyzing the concepts of effective communication. The results of this study can be useful for medical

Health systems research training enhances workplace research skills: a qualitative evaluation.

PubMed

Adams, Jolene; Schaffer, Angela; Lewin, Simon; Zwarenstein, Merrick; van der Walt, Hester

2003-01-01

In-service education is a widely used means of enhancing the skills of health service providers, for example, in undertaking research. However, the transfer of skills acquired during an education course to the workplace is seldom evaluated. The objectives of this study were to assess learner, teacher, and health service manager perceptions of the usefulness, in the work setting, of skills taught on a health systems research education course in South Africa and to assess the extent to which the course stimulated awareness and development of health systems research in the work setting. The education course was evaluated using a qualitative approach. Respondents were selected for interview using purposive sampling. Interviews were conducted with 39 respondents, including all of the major stakeholders. The interviews lasted between 20 and 60 minutes and were conducted either face to face or over the telephone. Thematic analysis was applied to the data, and key themes were identified. The course demystified health systems research and stimulated interest in reading and applying research findings. The course also changed participants' attitudes to routine data collection and was reported to have facilitated the application of informal research or problem-solving methods to everyday work situations. However, inadequate support within the workplace was a significant obstacle to applying the skills learned. A 2-week intensive, experiential course in health systems research methods can provide a mechanism for introducing basic research skills to a wide range of learners. Qualitative evaluation is a useful approach for assessing the impacts of education courses.

Qualitative evaluation of maternal milk and commercial infant formulas via LIBS.

PubMed

Abdel-Salam, Z; Al Sharnoubi, J; Harith, M A

2013-10-15

This study focuses on the use of laser-induced breakdown spectroscopy (LIBS) for the evaluation of the nutrients in maternal milk and some commercially available infant formulas. The results of such evaluation are vital for adequate and healthy feeding for babies during lactation period. Laser-induced breakdown spectroscopy offers special advantages in comparison to the other conventional analytical techniques. Specifically, LIBS is a straightforward technique that can be used in situ to provide qualitative analytical information in few minutes for the samples under investigation without preparation processes. The samples studied in the current work were maternal milk samples collected during the first 3 months of lactation (not colostrum milk) and samples from six different types of commercially available infant formulas. The samples' elemental composition has been compared with respect to the relative abundance of the elements of nutrition importance, namely Mg, Ca, Na, and Fe using their spectral emission lines in the relevant LIBS spectra. In addition, CN and C2 molecular emission bands in the same spectra have been studied as indicators of proteins content in the samples. The obtained analytical results demonstrate the higher elemental contents of the maternal milk compared with the commercial formulas samples. Similar results have been obtained as for the proteins content. It has been also shown that calcium and proteins have similar relative concentration trends in the studied samples. This work demonstrates the feasibility of adopting LIBS as a fast, safe, less costly technique evaluating qualitatively the nutrients content of both maternal and commercial milk samples. Copyright © 2013 Elsevier B.V. All rights reserved.

Implementation and Evaluation of a Smartphone-Based Telemonitoring Program for Patients With Heart Failure: Mixed-Methods Study Protocol

PubMed Central

Ross, Heather J; Cafazzo, Joseph A; Laporte, Audrey; Seto, Emily

2018-01-01

Background Meta-analyses of telemonitoring for patients with heart failure conclude that it can lower the utilization of health services and improve health outcomes compared with the standard of care. A smartphone-based telemonitoring program is being implemented as part of the standard of care at a specialty care clinic for patients with heart failure in Toronto, Canada. Objective The objectives of this study are to (1) evaluate the impact of the telemonitoring program on health service utilization, patient health outcomes, and their ability to self-care; (2) identify the contextual barriers and facilitators of implementation at the physician, clinic, and institutional level; (3) describe patient usage patterns to determine adherence and other behaviors in the telemonitoring program; and (4) evaluate the costs associated with implementation of the telemonitoring program from the perspective of the health care system (ie, public payer), hospital, and patient. Methods The evaluation will use a mixed-methods approach. The quantitative component will include a pragmatic pre- and posttest study design for the impact and cost analyses, which will make use of clinical data and questionnaires administered to at least 108 patients at baseline and 6 months. Furthermore, outcome data will be collected at 1, 12, and 24 months to explore the longitudinal impact of the program. In addition, quantitative data related to implementation outcomes and patient usage patterns of the telemonitoring system will be reported. The qualitative component involves an embedded single case study design to identify the contextual factors that influenced the implementation. The implementation evaluation will be completed using semistructured interviews with clinicians, and other program staff at baseline, 4 months, and 12 months after the program start date. Interviews conducted with patients will be triangulated with usage data to explain usage patterns and adherence to the system. Results The

Evaluating Voucher Programs: The Milwaukee Parental Choice Program

ERIC Educational Resources Information Center

Witte, John F.

2016-01-01

This paper is the first summary of two studies and 10 years of evaluating the Milwaukee Parental Choice (voucher) Program (MPCP). This paper discusses school voucher evaluations in general terms and how these studies are carried out. The paper outlines the types of studies completed in "Study I" and "Study II" and the results…

Handbook of evaluation of utility DSM programs

DOE Office of Scientific and Technical Information (OSTI.GOV)

Hirst, E.; Reed, J.; Bronfman, B.

Program evaluation has become a central issue in the world of utility integrated resource planning. The DSM programs that utilities were operating to meet federal requirements or to improve customer relations are now becoming big business. DSM is being considered an important resource in a utility`s portfolio of options. In the last five years, the amount of money that utilities have invested in DSM has grown exponentially in most regulatory jurisdictions. Market analysts are now talking about DSM being a $30 billion industry by the end of the decade. If the large volume of DSM-program investments was not enough tomore » highlight the importance of evaluation, then the introduction of regulatory incentives has really focused the spotlight. This handbook was developed through a process that involved many of those people who represent the diverse constituencies of DSM-program evaluation. We have come to recognize the many technical disciplines that must be employed to evaluate DSM programs. An analysis might start out based on the principles of utility load research to find out what happened, but a combination of engineering and statistical methods must be used to ``triangulate`` an estimate of what would have happened without the program. The difference, of course, is that elusive but prized result of evaluation: what happened as the direct result of the DSM program. Technical performance of DSM measures is not the sole determinant of the answer, either. We also recognize the importance of such behavioral attributes of DSM as persistence and free ridership. Finally, DSM evaluation is meaningless without attention to planning an approach, communicating results to relevant decision-makers, and focusing as much on the process as the impacts of the program. These topics are all covered in this handbook.« less

Evaluating an mHealth App for Health and Well-Being at Work: Mixed-Method Qualitative Study.

PubMed

de Korte, Elsbeth Marieke; Wiezer, Noortje; Janssen, Joris H; Vink, Peter; Kraaij, Wessel

2018-03-28

To improve workers' health and well-being, workplace interventions have been developed, but utilization and reach are unsatisfactory, and effects are small. In recent years, new approaches such as mobile health (mHealth) apps are being developed, but the evidence base is poor. Research is needed to examine its potential and to assess when, where, and for whom mHealth is efficacious in the occupational setting. To develop interventions for workers that actually will be adopted, insight into user satisfaction and technology acceptance is necessary. For this purpose, various qualitative evaluation methods are available. The objectives of this study were to gain insight into (1) the opinions and experiences of employees and experts on drivers and barriers using an mHealth app in the working context and (2) the added value of three different qualitative methods that are available to evaluate mHealth apps in a working context: interviews with employees, focus groups with employees, and a focus group with experts. Employees of a high-tech company and experts were asked to use an mHealth app for at least 3 weeks before participating in a qualitative evaluation. Twenty-two employees participated in interviews, 15 employees participated in three focus groups, and 6 experts participated in one focus group. Two researchers independently coded, categorized, and analyzed all quotes yielded from these evaluation methods with a codebook using constructs from user satisfaction and technology acceptance theories. Interviewing employees yielded 785 quotes, focus groups with employees yielded 266 quotes, and the focus group with experts yielded 132 quotes. Overall, participants muted enthusiasm about the app. Combined results from the three evaluation methods showed drivers and barriers for technology, user characteristics, context, privacy, and autonomy. A comparison between the three qualitative methods showed that issues revealed by experts only slightly overlapped with those

Computer-Assisted Analysis of Qualitative Gerontological Research.

ERIC Educational Resources Information Center

Hiemstra, Roger; And Others

1987-01-01

Asserts that qualitative research has great potential for use in gerontological research. Describes QUALOG, a computer-assisted, qualitative data analysis scheme using logic programming developed at Syracuse University. Reviews development of QUALOG and discusses how QUALOG was used to analyze data from a qualitative study of older adult learners.…

Human Services Program Evaluation: "How to Improve Your Accountability and Program Effectiveness"

ERIC Educational Resources Information Center

Barrett, Thomas; Sorensen, James

2015-01-01

The term "outcome evaluation" has become one of the most popular terms among human service providers and those whose job it is to evaluate the impact of human service programs. In the public sector alone, there are over a hundred instruments in use to evaluate the impact of state human service programs. Most states, many providers, and…

Improving public health evaluation: a qualitative investigation of practitioners' needs.

PubMed

Denford, Sarah; Lakshman, Rajalakshmi; Callaghan, Margaret; Abraham, Charles

2018-01-30

In 2011, the House of Lords published a report on Behaviour Change, in which they report that "a lot more could, and should, be done to improve the evaluation of interventions." This study aimed to undertake a needs assessment of what kind of evaluation training and materials would be of most use to UK public health practitioners by conducting interviews with practitioners about everyday evaluation practice and needed guidance and materials. Semi-structured interviews were conducted with 32 public health practitioners in two UK regions, Cambridgeshire and the South West. Participants included directors of public health, consultants in public health, health improvement advisors, public health intelligence, and public health research officers. A topic guide included questions designed to explore participants existing evaluation practice and their needs for further training and guidance. Data were analysed using thematic analyses. Practitioners highlighted the need for evaluation to defend the effectiveness of existing programs and protect funding provisions. However, practitioners often lacked training in evaluation, and felt unqualified to perform such a task. The majority of practitioners did not use, or were not aware of many existing evaluation guidance documents. They wanted quality-assured, practical guidance that relate to the real world settings in which they operate. Practitioners also mentioned the need for better links and support from academics in public health. Whilst numerous guidance documents supporting public health evaluation exist, these documents are currently underused by practitioners - either because they are not considered useful, or because practitioners are not aware of them. Integrating existing guides into a catalogue of guidance documents, and developing a new-quality assured, practical and useful document may support the evaluation of public health programs. This in turn has the potential to identify those programs that are effective

Testing of a Program Evaluation Model: Final Report.

ERIC Educational Resources Information Center

Nagler, Phyllis J.; Marson, Arthur A.

A program evaluation model developed by Moraine Park Technical Institute (MPTI) is described in this report. Following background material, the four main evaluation criteria employed in the model are identified as program quality, program relevance to community needs, program impact on MPTI, and the transition and growth of MPTI graduates in the…

Developmental Kindergarten Program Evaluation Report.

ERIC Educational Resources Information Center

Blois, George T.; Cushing, Katherine S.

The evaluation of the Developmental Kindergarten (DK) Program at the Harrison School District #2, Colorado Springs, Colorado, involved pre- and post-testing of student academic gains and interviewing of principals and teachers. The program aimed to provide developmentally appropriate activities for students believed to be "at risk" of…

Evaluating Educational Programs. ERIC Digest Series Number EA 54.

ERIC Educational Resources Information Center

Beswick, Richard

In this digest, readers are introduced to the scope of instructional program evaluation and evaluators' changing roles in school districts. A program evaluation measures outcomes based on student-attainment goals, implementation levels, and external factors such as budgetary restraints and community support. Instructional program evaluation may be…

The Nursing Leadership Institute program evaluation: a critique

PubMed Central

Havaei, Farinaz; MacPhee, Maura

2015-01-01

A theory-driven program evaluation was conducted for a nursing leadership program, as a collaborative project between university faculty, the nurses’ union, the provincial Ministry of Health, and its chief nursing officers. A collaborative logic model process was used to engage stakeholders, and mixed methods approaches were used to answer evaluation questions. Despite demonstrated, successful outcomes, the leadership program was not supported with continued funding. This paper examines what happened during the evaluation process: What factors failed to sustain this program? PMID:29355180