Blending Qualitative and Computational Linguistics Methods for Fidelity Assessment: Experience with the Familias Unidas Preventive Intervention

PubMed Central

Gallo, Carlos; Pantin, Hilda; Villamar, Juan; Prado, Guillermo; Tapia, Maria; Ogihara, Mitsunori; Cruden, Gracelyn; Brown, C Hendricks

2014-01-01

Careful fidelity monitoring and feedback are critical to implementing effective interventions. A wide range of procedures exist to assess fidelity; most are derived from observational assessments (Schoenwald et al, 2013). However, these fidelity measures are resource intensive for research teams in efficacy/effectiveness trials, and are often unattainable or unmanageable for the host organization to rate when the program is implemented on a large scale. We present a first step towards automated processing of linguistic patterns in fidelity monitoring of a behavioral intervention using an innovative mixed methods approach to fidelity assessment that uses rule-based, computational linguistics to overcome major resource burdens. Data come from an effectiveness trial of the Familias Unidas intervention, an evidence-based, family-centered preventive intervention found to be efficacious in reducing conduct problems, substance use and HIV sexual risk behaviors among Hispanic youth. This computational approach focuses on “joining,” which measures the quality of the working alliance of the facilitator with the family. Quantitative assessments of reliability are provided. Kappa scores between a human rater and a machine rater for the new method for measuring joining reached .83. Early findings suggest that this approach can reduce the high cost of fidelity measurement and the time delay between fidelity assessment and feedback to facilitators; it also has the potential for improving the quality of intervention fidelity ratings. PMID:24500022

Dismantling the Active Ingredients of an Intervention for Children with Autism.

PubMed

Pellecchia, Melanie; Connell, James E; Beidas, Rinad S; Xie, Ming; Marcus, Steven C; Mandell, David S

2015-09-01

This study evaluated the association of fidelity to each of the components of the Strategies for Teaching based on Autism Research (STAR) program, a comprehensive treatment package for children with autism that includes discrete trial training, pivotal response training, and teaching in functional routines, on outcomes for 191 students ages 5-8 years in a large public school district. Fidelity to all components was relatively low, despite considerable training and support, suggesting the need to develop new implementation strategies. Fidelity to pivotal response training, but not discrete trial training or functional routines, was positively associated with gains in cognitive ability despite low levels of fidelity, and may be an effective intervention choice in under-resourced settings.

Illustrating the multiple facets and levels of fidelity of implementation to a teacher classroom management intervention.

PubMed

Reinke, Wendy M; Herman, Keith C; Stormont, Melissa; Newcomer, Lori; David, Kimberly

2013-11-01

Many school-based interventions to promote student mental health rely on teachers as implementers. Thus, understanding the interplay between the multiple domains of fidelity to the intervention and intervention support systems such as coaching and teacher implementation of new skills is an important aspect of implementation science. This study describes a systematic process for assessing multiple domains of fidelity. Data from a larger efficacy trial of the Incredible Years Teacher Classroom Management (IY TCM) program are utilized. Data on fidelity to the IY TCM workshop training sessions and onsite weekly coaching indicate that workshop leaders and the IY TCM coach implemented the training and coaching model with adequate adherence. Further, workshop leaders' ratings of engagement were associated with teacher implementation of specific praise, following training on this content. Lastly, the IY TCM coach differentiation of teacher exposure to coaching was evaluated and found to be associated with teacher implementation of classroom management practices and student disruptive behavior.

TESTING AN INTEGRATED MODEL OF PROGRAM IMPLEMENTATION: THE FOOD, HEALTH & CHOICES SCHOOL-BASED CHILDHOOD OBESITY PREVENTION INTERVENTION PROCESS EVALUATION

PubMed Central

Gray, Heewon Lee; Tipton, Elizabeth; Contento, Isobel; Koch, Pamela

2016-01-01

Childhood obesity is a complex, worldwide problem. Significant resources are invested in its prevention, and high-quality evaluations of these efforts are important. Conducting trials in school settings is complicated, making process evaluations useful for explaining results. Intervention fidelity has been demonstrated to influence outcomes, but others have suggested that other aspects of implementation, including participant responsiveness, should be examined more systematically. During Food, Health & Choices (FHC), a school-based childhood obesity prevention trial designed to test a curriculum and wellness policy taught by trained FHC instructors to fifth grade students in 20 schools during 2012–2013, we assessed relationships among facilitator behaviors (i.e., fidelity and teacher interest), participant behaviors (i.e., student satisfaction and recall), and program outcomes (i.e., energy balance-related behaviors) using hierarchical linear models, controlling for student, class, and school characteristics. We found positive relationships between student satisfaction and recall and program outcomes, but not fidelity and program outcomes. We also found relationships between teacher interest and fidelity when teachers participated in implementation. Finally, we found a significant interaction between fidelity and satisfaction on behavioral outcomes. These findings suggest that individual students in the same class responded differently to the same intervention. They also suggest the importance of teacher buy-in for successful intervention implementation. Future studies should examine how facilitator and participant behaviors together are related to both outcomes and implementation. Assessing multiple aspects of implementation using models that account for contextual influences on behavioral outcomes is an important step forward for prevention intervention process evaluations. PMID:27921200

Testing an Integrated Model of Program Implementation: the Food, Health & Choices School-Based Childhood Obesity Prevention Intervention Process Evaluation.

PubMed

Burgermaster, Marissa; Gray, Heewon Lee; Tipton, Elizabeth; Contento, Isobel; Koch, Pamela

2017-01-01

Childhood obesity is a complex, worldwide problem. Significant resources are invested in its prevention, and high-quality evaluations of these efforts are important. Conducting trials in school settings is complicated, making process evaluations useful for explaining results. Intervention fidelity has been demonstrated to influence outcomes, but others have suggested that other aspects of implementation, including participant responsiveness, should be examined more systematically. During Food, Health & Choices (FHC), a school-based childhood obesity prevention trial designed to test a curriculum and wellness policy taught by trained FHC instructors to fifth grade students in 20 schools during 2012-2013, we assessed relationships among facilitator behaviors (i.e., fidelity and teacher interest); participant behaviors (i.e., student satisfaction and recall); and program outcomes (i.e., energy balance-related behaviors) using hierarchical linear models, controlling for student, class, and school characteristics. We found positive relationships between student satisfaction and recall and program outcomes, but not fidelity and program outcomes. We also found relationships between teacher interest and fidelity when teachers participated in implementation. Finally, we found a significant interaction between fidelity and satisfaction on behavioral outcomes. These findings suggest that individual students in the same class responded differently to the same intervention. They also suggest the importance of teacher buy-in for successful intervention implementation. Future studies should examine how facilitator and participant behaviors together are related to both outcomes and implementation. Assessing multiple aspects of implementation using models that account for contextual influences on behavioral outcomes is an important step forward for prevention intervention process evaluations.

The Project Towards No Drug Abuse (TND) Dissemination Trial: Implementation Fidelity and Immediate Outcomes

PubMed Central

Gunning, Melissa; Sun, Ping; Sussman, Steve

2009-01-01

One of the important research issues in the emerging area of research on dissemination of prevention programs relates to the type and extent of training needed by program providers to prepare them to implement effective programs with fidelity. The present paper describes the immediate outcomes of a dissemination and implementation trial of Project Toward No Drug Abuse, an evidence-based prevention program for high school students. A total of 65 high schools in 14 school districts across the USA were recruited and randomly assigned to one of three experimental conditions: comprehensive implementation support for teachers, regular workshop training only, or standard care control. The comprehensive intervention was comprised of on-site coaching, web-based support, and technical assistance, in addition to the regular workshop. Students (n=2,983) completed self-report surveys before and immediately after program implementation. Fidelity of implementation was assessed with a classroom observation procedure that focused on program process. Results indicated that relative to the controls, both intervention conditions produced effects on hypothesized program mediators, including greater gains in program-related knowledge; greater reductions in cigarette, marijuana and hard drug use intentions; and more positive changes in drug-related beliefs. There were stronger effects on implementation fidelity in the comprehensive, relative to the regular, training condition. However, seven of the ten immediate student outcome measures showed no significant differences between the two training conditions. The implications of these findings for dissemination research and practice are discussed. PMID:19757052

The Project Towards No Drug Abuse (TND) dissemination trial: implementation fidelity and immediate outcomes.

PubMed

Rohrbach, Louise Ann; Gunning, Melissa; Sun, Ping; Sussman, Steve

2010-03-01

One of the important research issues in the emerging area of research on dissemination of prevention programs relates to the type and extent of training needed by program providers to prepare them to implement effective programs with fidelity. The present paper describes the immediate outcomes of a dissemination and implementation trial of Project Toward No Drug Abuse, an evidence-based prevention program for high school students. A total of 65 high schools in 14 school districts across the USA were recruited and randomly assigned to one of three experimental conditions: comprehensive implementation support for teachers, regular workshop training only, or standard care control. The comprehensive intervention was comprised of on-site coaching, web-based support, and technical assistance, in addition to the regular workshop. Students (n = 2,983) completed self-report surveys before and immediately after program implementation. Fidelity of implementation was assessed with a classroom observation procedure that focused on program process. Results indicated that relative to the controls, both intervention conditions produced effects on hypothesized program mediators, including greater gains in program-related knowledge; greater reductions in cigarette, marijuana and hard drug use intentions; and more positive changes in drug-related beliefs. There were stronger effects on implementation fidelity in the comprehensive, relative to the regular, training condition. However, seven of the ten immediate student outcome measures showed no significant differences between the two training conditions. The implications of these findings for dissemination research and practice are discussed.

Assessing the link between implementation fidelity and health outcomes for a trial of intensive case management by community health workers: a mixed methods study protocol.

PubMed

Schmidt, Barbara; Watt, Kerrianne; McDermott, Robyn; Mills, Jane

2017-07-17

Better systems of care are required to address chronic disease in Indigenous people to ensure they can access all their care needs. Health research has produced evidence about effective models of care and chronic disease strategies to address Indigenous health, however the transfer of research findings into routine clinical practice has proven challenging. Complex interventions, such as those related to chronic disease, have many components that are often poorly implemented and hence rarely achieve implementation fidelity. Implementation fidelity is "the degree to which programs are implemented as intended by the program developer". Knowing if an intervention was implemented as planned is fundamental to knowing what has contributed to the success of an intervention. The aim of this study is to adapt the implementation fidelity framework developed by Keith et al. and apply it to the intervention implemented in phase 1 of the Getting Better at Chronic Care in North Queensland study. The objectives are to quantify the level of implementation fidelity achieved during phase 1 of the study, measure the association between implementation fidelity and health outcomes and to explore the features of the primary health care system that contributed to improved health outcomes. A convergent parallel mixed methods study design will be used to develop a process for assessing implementation fidelity. Information collected via a questionnaire and routine data generated during phase 1 of the study will be used to explain the context for the intervention in each site and develop an implementation fidelity score for each component of the intervention. A weighting will be applied to each component of the intervention to calculate the overall implementation score for each participating community. Statistical analysis will assess the level of association between implementation fidelity scores and health outcomes. Health services research seeks to find solutions to social and technical problems to improve health outcomes. The development of a tool and methodology for assessing implementation fidelity in the Indigenous primary health care context will help address some of the barriers to the translation of research into practice. ACTRN12610000812099 : 29.9.2010.

How to implement Illness Management and Recovery (IMR) in mental health service settings: evaluation of the implementation strategy.

PubMed

Egeland, Karina Myhren; Ruud, Torleif; Ogden, Terje; Färdig, Rickard; Lindstrøm, Jonas Christoffer; Heiervang, Kristin Sverdvik

2017-01-01

The purpose of this study was to evaluate the implementation strategy used in the first-phase of implementation of the Illness Management and Recovery (IMR) programme, an intervention for adults with severe mental illnesses, in nine mental health service settings in Norway. A total of 9 clinical leaders, 31 clinicians, and 44 consumers at 9 service settings participated in the implementation of IMR. Implementation was conducted by an external team of researchers and an experienced trainer. Data were gathered on fidelity to the intervention and implementation strategy, feasibility, and consumer outcomes. Although the majority of clinicians scored within the acceptable range of high intervention fidelity, their participation in the implementation strategy appeared to moderate anticipated future use of IMR. No service settings reached high intervention fidelity scores for organizational quality improvement after 12 months of implementation. IMR implementation seemed feasible, albeit with some challenges. Consumer outcomes indicated significant improvements in illness self-management, severity of problems, functioning, and hope. There were nonsignificant positive changes in symptoms and quality of life. The implementation strategy appeared adequate to build clinician competence over time, enabling clinicians to provide treatment that increased functioning and hope for consumers. Additional efficient strategies should be incorporated to facilitate organizational change and thus secure the sustainability of the implemented practice. Trial registration ClinicalTrials.gov NCT02077829. Registered 25 February 2014.

Implementation fidelity of a computer-assisted intervention for children with speech sound disorders.

PubMed

McCormack, Jane; Baker, Elise; Masso, Sarah; Crowe, Kathryn; McLeod, Sharynne; Wren, Yvonne; Roulstone, Sue

2017-06-01

Implementation fidelity refers to the degree to which an intervention or programme adheres to its original design. This paper examines implementation fidelity in the Sound Start Study, a clustered randomised controlled trial of computer-assisted support for children with speech sound disorders (SSD). Sixty-three children with SSD in 19 early childhood centres received computer-assisted support (Phoneme Factory Sound Sorter [PFSS] - Australian version). Educators facilitated the delivery of PFSS targeting phonological error patterns identified by a speech-language pathologist. Implementation data were gathered via (1) the computer software, which recorded when and how much intervention was completed over 9 weeks; (2) educators' records of practice sessions; and (3) scoring of fidelity (intervention procedure, competence and quality of delivery) from videos of intervention sessions. Less than one-third of children received the prescribed number of days of intervention, while approximately one-half participated in the prescribed number of intervention plays. Computer data differed from educators' data for total number of days and plays in which children participated; the degree of match was lower as data became more specific. Fidelity to intervention procedures, competency and quality of delivery was high. Implementation fidelity may impact intervention outcomes and so needs to be measured in intervention research; however, the way in which it is measured may impact on data.

Methodological quality in randomised controlled trials of transcutaneous electric nerve stimulation for pain: low fidelity may explain negative findings.

PubMed

Bennett, Michael I; Hughes, Nicola; Johnson, Mark I

2011-06-01

The benefits of transcutaneous electrical nerve stimulation (TENS) for pain relief have not been reliably established, as most systematic reviews find poor methodological quality in many studies. The paradox within the evidence base for TENS is that despite identified sources of bias that may lead to an overestimation of treatment effects, no benefits for TENS can be clearly demonstrated. Conventional assessments of quality assume a single direction of bias, and little work has been undertaken examining other directions of bias. Our hypothesis was that low fidelity in studies (bias leading to an underestimation of treatment effects) may account for inconclusive findings. We included 38 studies from 3 recently published Cochrane systematic reviews that examined TENS for acute, chronic, and cancer pain. We extracted data relating to treatment allocation, application of TENS and to the assessment of outcomes. We quantified these data and judged this against standardised assessment criteria using a "traffic light" approach based on the number of studies reaching the standard. We identified significant sources of potential bias in both directions in relation to study design and implementation fidelity that have not been quantified previously. Suboptimal dosing of TENS and inappropriate outcome assessment were particularly prevalent weaknesses indicating low fidelity. We propose criteria for judging directions of bias in future studies of TENS that may be adapted to assess other trials in which implementation fidelity is important, such as other nonpharmacological interventions for pain. Poor implementation fidelity was identified as a significant source of bias in systematic reviews of TENS studies and might explain lack of consistent treatment effects of TENS in pain. Here, criteria for assessing methodology are proposed for use in designing future clinical trials of TENS. Copyright © 2010 International Association for the Study of Pain. Published by Elsevier B.V. All rights reserved.

Fidelity to a behavioral intervention to improve goals of care decisions for nursing home residents with advanced dementia.

PubMed

Hanson, Laura C; Song, Mi-Kyung; Zimmerman, Sheryl; Gilliam, Robin; Rosemond, Cherie; Chisholm, Latarsha; Lin, Feng-Chang

2016-12-01

Ensuring fidelity to a behavioral intervention implemented in nursing homes requires awareness of the unique considerations of this setting for research. The purpose of this article is to describe the goals of care cluster-randomized trial and the methods used to monitor and promote fidelity to a goals of care decision aid intervention delivered in nursing homes. The cluster randomized trial tested whether a decision aid for goals of care in advanced dementia could improve (1) the quality of communication and decision-making, (2) the quality of palliative care, and (3) the quality of dying for nursing home residents with advanced dementia. In 11 intervention nursing homes, family decision-makers for residents with advanced dementia received a two-component intervention: viewing a video decision aid about goals of care choices and then participating in a structured decision-making discussion with the nursing home care plan team, ideally within 3 months after the decision aid was viewed. Following guidelines from the National Institutes of Health Behavior Change Consortium, fidelity was assessed in study design, in nursing home staff training for intervention implementation, and in monitoring and receipt of the intervention. We also monitored the content and timing of goals of care discussions. Investigators enrolled 151 family decision-maker/resident dyads in intervention sites; of those, 136 (90%) received both components of the intervention, and 92%-99% of discussions addressed each of four recommended content areas-health status, goals of care, choice of a goal, and treatment planning. A total of 94 (69%) of the discussions between family decision-makers and the nursing home care team were completed within 3 months. The methods we used for intervention fidelity allowed nursing home staff to implement a goals of care decision aid intervention for advanced dementia. Key supports for implementation included design features that aligned with nursing home practice, efficient staff training, and a structured guide for goals of care discussions between family decision-makers and staff. These approaches may be used to promote fidelity to behavioral interventions in future clinical trials. © The Author(s) 2016.

Assessing fidelity of cognitive behavioral therapy in rural VA clinics: design of a randomized implementation effectiveness (hybrid type III) trial.

PubMed

Cucciare, Michael A; Curran, Geoffrey M; Craske, Michelle G; Abraham, Traci; McCarthur, Michael B; Marchant-Miros, Kathy; Lindsay, Jan A; Kauth, Michael R; Landes, Sara J; Sullivan, Greer

2016-05-10

Broadly disseminating and implementing evidence-based psychotherapies with high fidelity, particularly cognitive behavioral therapy (CBT), has proved challenging for many health-care systems, including the Department of Veterans Affairs, especially in primary care settings such as small or remote clinics. A computer-based tool (based on the coordinated anxiety learning and management (CALM) program) was designed to support primary care-based mental health providers in delivering CBT. The objectives of this study are to modify the CALM tool to meet the needs of mental health clinicians in veterans affairs (VA) community-based outpatient clinics (CBOCs) and rural "veterans", use external facilitation to implement CBT and determine the effect of the CALM tool versus a manualized version of CALM to improve fidelity to the CBT treatment model, and conduct a needs assessment to understand how best to support future implementation of the CALM tool in routine care. Focus groups will inform the redesign of the CALM tool. Mental health providers at regional VA CBOCs; CBT experts; VA experts in implementation of evidence-based mental health practices; and veterans with generalized anxiety disorder, panic disorder, social anxiety disorder, posttraumatic stress disorder, "with or without" depression will be recruited. A hybrid type III design will be used to examine the effect of receiving CBT training plus either the CALM tool or a manual version of CALM on treatment fidelity. External facilitation will be used as the overarching strategy to implement both CBT delivery methods. Data will also be collected on symptoms of the targeted disorders. To help prepare for the future implementation of the CALM tool in VA CBOCs, we will perform an implementation need assessment with mental health providers participating in the clinical trial and their CBOC directors. This project will help inform strategies for delivering CBT with high fidelity in VA CBOCs to veterans with anxiety disorders and PTSD with or without depression. If successful, results of this study could be used to inform a national rollout of the CALM tool in VA CBOCs including providing recommendations for optimizing the adoption and sustained use of the computerized CALM tool among mental health providers in this setting. ClinicalTrials.gov, NCT02488551.

The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: evidence for a protective effect.

PubMed

Aarons, Gregory A; Sommerfeld, David H; Hecht, Debra B; Silovsky, Jane F; Chaffin, Mark J

2009-04-01

Staff retention is an ongoing challenge in mental health and community-based service organizations. Little is known about the impact of evidence-based practice implementation on the mental health and social service workforce. The present study examined the effect of evidence-based practice implementation and ongoing fidelity monitoring on staff retention in a children's services system. The study took place in the context of a statewide, regionally randomized effectiveness trial of an evidence-based intervention designed to reduce child neglect. In the study 21 teams consisting of 153 home-based service providers were followed over a 29-month period. Survival analyses revealed greater staff retention in the condition where the evidence-based practice was implemented along with ongoing fidelity monitoring presented to staff as supportive consultation. These results should help to allay concerns about staff retention when implementing evidence-based practices where there is good values-innovation fit and when fidelity monitoring is designed as an aid and support to service providers in providing a high standard of care for children and families.

The Impact of Evidence-Based Practice Implementation and Fidelity Monitoring on Staff Turnover: Evidence for a Protective Effect

PubMed Central

Aarons, Gregory A.; Sommerfeld, David H.; Hecht, Debra B.; Silovsky, Jane F.; Chaffin, Mark J.

2009-01-01

Staff retention is an ongoing challenge in mental health and community-based service organizations. Little is known about the impact of evidence-based practice implementation on the mental health and social service workforce. The present study examined the effect of evidence-based practice implementation and ongoing fidelity monitoring on staff retention in a children’s services system. The study took place in the context of a statewide regionally randomized effectiveness trial of an evidence-based intervention designed to reduce child neglect. Twenty-one teams consisting of 153 home-based service providers were followed over a 29 month period. Survival analyses revealed greater staff retention in the condition where the evidence-based practice was implemented along with ongoing fidelity monitoring presented to staff as supportive consultation. These results should help to allay concerns about staff retention when implementing evidence-based practices where there is good values-innovation fit and when fidelity monitoring is designed as an aid and support to service providers in providing a high standard of care for children and families. PMID:19309186

Efficacy of Individualized Clinical Coaching in a Virtual Reality Classroom for Increasing Teachers' Fidelity of Implementation of Discrete Trial Teaching

ERIC Educational Resources Information Center

Garland, Krista Vince; Vasquez, Eleazar, III; Pearl, Cynthia

2012-01-01

Discrete-trials teaching (DTT) is an evidence-based practice used in educational programs for children with autism spectrum disorders (ASD). Although there is strong demand for preparing teachers to effectively implement DTT, there is a scarcity of published research on such studies. A multiple baseline across participants design was utilized to…

Beyond Binary: Using Propensity Scores to Account for Varying Levels of Program Participation in Randomized Controlled Trials

ERIC Educational Resources Information Center

Stuart, Elizabeth A.; Warkentien, Siri; Jo, Booil

2011-01-01

The purpose of the current project is to explore the use of propensity scores to estimate the effects of interventions within randomized control trials, accounting for varying levels of implementation or fidelity. This work extends that of Jo and Stuart (2009) to settings with multiple or continuous measures of implementation. Rather than focus…

The role of treatment fidelity on outcomes during a randomized field trial of an autism intervention

PubMed Central

Mandell, David S; Stahmer, Aubyn C; Shin, Sujie; Xie, Ming; Reisinger, Erica; Marcus, Steven C

2013-01-01

This randomized field trial comparing Strategies for Teaching based on Autism Research and Structured Teaching enrolled educators in 33 kindergarten-through-second-grade autism support classrooms and 119 students, aged 5–8 years in the School District of Philadelphia. Students were assessed at the beginning and end of the academic year using the Differential Ability Scales. Program fidelity was measured through video coding and use of a checklist. Outcomes were assessed using linear regression with random effects for classroom and student. Average fidelity was 57% in Strategies for Teaching based on Autism Research classrooms and 48% in Structured Teaching classrooms. There was a 9.2-point (standard deviation = 9.6) increase in Differential Ability Scales score over the 8-month study period, but no main effect of program. There was a significant interaction between fidelity and group. In classrooms with either low or high program fidelity, students in Strategies for Teaching based on Autism Research experienced a greater gain in Differential Ability Scales score than students in Structured Teaching (11.2 vs 5.5 points and 11.3 vs 8.9 points, respectively). In classrooms with moderate fidelity, students in Structured Teaching experienced a greater gain than students in Strategies for Teaching based on Autism Research (10.1 vs 4.4 points). The results suggest significant variability in implementation of evidence-based practices, even with supports, and also suggest the need to address challenging issues related to implementation measurement in community settings. PMID:23592849

Fidelity considerations in translational research: Eating As Treatment - a stepped wedge, randomised controlled trial of a dietitian delivered behaviour change counselling intervention for head and neck cancer patients undergoing radiotherapy.

PubMed

Beck, Alison Kate; Baker, Amanda; Britton, Ben; Wratten, Chris; Bauer, Judith; Wolfenden, Luke; Carter, Gregory

2015-10-15

The confidence with which researchers can comment on intervention efficacy relies on evaluation and consideration of intervention fidelity. Accordingly, there have been calls to increase the transparency with which fidelity methodology is reported. Despite this, consideration and/or reporting of fidelity methods remains poor. We seek to address this gap by describing the methodology for promoting and facilitating the evaluation of intervention fidelity in The EAT (Eating As Treatment) project: a multi-site stepped wedge randomised controlled trial of a dietitian delivered behaviour change counselling intervention to improve nutrition (primary outcome) in head and neck cancer patients undergoing radiotherapy. In accordance with recommendations from the National Institutes of Health Behaviour Change Consortium Treatment Fidelity Workgroup, we sought to maximise fidelity in this stepped wedge randomised controlled trial via strategies implemented from study design through to provider training, intervention delivery and receipt. As the EAT intervention is designed to be incorporated into standard dietetic consultations, we also address unique challenges for translational research. We offer a strong model for improving the quality of translational findings via real world application of National Institutes of Health Behaviour Change Consortium recommendations. Greater transparency in the reporting of behaviour change research is an important step in improving the progress and quality of behaviour change research. ACTRN12613000320752 (Date of registration 21 March 2013).

Applying New Methods to the Measurement of Fidelity of Implementation: Examining the Critical Ingredients of the Responsive Classroom Approach in Relation to Mathematics Achievement

ERIC Educational Resources Information Center

Abry, Tashia D. S.; Rimm-Kaufman, Sara E.; Larsen, Ross A.; Brewer, Alix J.

2011-01-01

The present study examines data collected during the second year of a three-year longitudinal cluster randomized controlled trial, the Responsive Classroom Efficacy Study (RCES). In the context of and RCT, the research questions address naturally occurring variability in the independent variables of interest (i.e., teachers' (fidelity of…

Implementing a knowledge translation intervention in long-term care: feasibility results from the Vitamin D and Osteoporosis Study (ViDOS).

PubMed

Kennedy, Courtney C; Thabane, Lehana; Ioannidis, George; Adachi, Jonathan D; Papaioannou, Alexandra

2014-12-01

To evaluate the feasibility of implementing an interdisciplinary, multifaceted knowledge translation intervention within long-term care (LTC) and to identify any challenges that should be considered in designing future studies. Cluster randomized controlled trial. Forty LTC homes across the province of Ontario, Canada. LTC teams composed of physicians, nurses, pharmacists, and other staff. Cluster-level feasibility measures, including recruitment, retention, data completion, and participation in the intervention. A process evaluation was completed by directors of care indicating which process/policy changes had been implemented. Recruitment and retention rates were 22% and 63%, respectively. Good fidelity with the intervention was achieved, including attendance at educational meetings. After ViDOS, 7 process indicators were being newly implemented by more than 50% of active intervention homes. Despite recruitment and retention challenges, the multifaceted intervention produced a number of policy/process changes and had good intervention fidelity. This study is registered at ClinicalTrials.gov NCT01398527. Copyright © 2014. Published by Elsevier Inc.

Protocol for the mixed-methods process and context evaluation of the TB & Tobacco randomised controlled trial in Bangladesh and Pakistan: a hybrid effectiveness–implementation study

PubMed Central

Nohavova, Iveta; Dogar, Omara; Kralikova, Eva; Pankova, Alexandra; Zvolska, Kamila; Huque, Rumana; Fatima, Razia; Noor, Maryam; Elsey, Helen; Sheikh, Aziz; Siddiqi, Kamran; Kotz, Daniel

2018-01-01

Introduction Tuberculosis (TB) remains a significant public health problem in South Asia. Tobacco use increases the risks of TB infection and TB progression. The TB& Tobacco placebo-controlled randomised trial aims to (1) assess the effectiveness of the tobacco cessation medication cytisine versus placebo when combined with behavioural support and (2) implement tobacco cessation medication and behavioural support as part of general TB care in Bangladesh and Pakistan. This paper summarises the process and context evaluation protocol embedded in the effectiveness–implementation hybrid design. Methods and analysis We are conducting a mixed-methods process and context evaluation informed by an intervention logic model that draws on the UK Medical Research Council’s Process Evaluation Guidance. Our approach includes quantitative and qualitative data collection on context, recruitment, reach, dose delivered, dose received and fidelity. Quantitative data include patient characteristics, reach of recruitment among eligible patients, routine trial data on dose delivered and dose received, and a COM-B (‘capability’, ‘opportunity’, ‘motivation’ and ‘behaviour’) questionnaire filled in by participating health workers. Qualitative data include semistructured interviews with TB health workers and patients, and with policy-makers at district and central levels in each country. Interviews will be analysed using the framework approach. The behavioural intervention delivery is audio recorded and assessed using a predefined fidelity coding index based on behavioural change technique taxonomy. Ethics and dissemination The study complies with the guidelines of the Declaration of Helsinki. Ethics approval for the study and process evaluation was granted by the University of Leeds (qualitative components), University of York (trial data and fidelity assessment), Bangladesh Medical Research Council and Bangladesh Drug Administration (trial data and qualitative components) and Pakistan Medical Research Council (trial data and qualitative components). Results of this research will be disseminated through reports to stakeholders and peer-reviewed publications and conference presentations. Trial registration number ISRCTN43811467; Pre-results. PMID:29602847

Effect of an Automated Training Presentation on Pre-Service Behavior Analysts' Implementation of Trial-Based Functional Analysis

ERIC Educational Resources Information Center

Lambert, Joseph M.; Lloyd, Blair P.; Staubitz, Johanna L.; Weaver, Emily S.; Jennings, Chelsea M.

2014-01-01

The trial-based functional analysis (FA) is a useful alternative to the traditional FA in contexts in which it is challenging to establish environmental control for extended periods of time. Previous researchers have demonstrated that others can be trained to conduct trial-based FAs with high procedural fidelity by providing a didactic…

Implementation of evidence-based supported employment in regional Australia.

PubMed

Morris, Adrienne; Waghorn, Geoffrey; Robson, Emma; Moore, Lyndell; Edwards, Emma

2014-06-01

To implement the Individual Placement and Support (IPS) approach at 4 locations in regional New South Wales, Australia. Outcomes attained were compared with a national non-IPS program and with international trials of IPS within and outside the United States. Four IPS programs were established through formal partnerships between mental health services and disability employment services. Ninety-five mental health service clients commenced employment assistance and were tracked for a minimum of 12 months. Two sites achieved good fidelity to IPS principles, and 2 sites achieved fair fidelity. IPS clients had 3.5 times greater odds of attaining 13 weeks' employment than those receiving assistance in the national network of disability employment services. Implementing IPS is challenging in the Australian service delivery context. Factors other than program fidelity appear to contribute to excellent employment outcomes. Further research is needed to identify these factors.

A multidimensional approach to assessing intervention fidelity in a process evaluation of audit and feedback interventions to reduce unnecessary blood transfusions: a study protocol.

PubMed

Lorencatto, Fabiana; Gould, Natalie J; McIntyre, Stephen A; During, Camilla; Bird, Jon; Walwyn, Rebecca; Cicero, Robert; Glidewell, Liz; Hartley, Suzanne; Stanworth, Simon J; Foy, Robbie; Grimshaw, Jeremy M; Michie, Susan; Francis, Jill J

2016-12-12

In England, NHS Blood and Transplant conducts national audits of transfusion and provides feedback to hospitals to promote evidence-based practice. Audits demonstrate 20% of transfusions fall outside guidelines. The AFFINITIE programme (Development & Evaluation of Audit and Feedback INterventions to Increase evidence-based Transfusion practIcE) involves two linked, 2×2 factorial, cluster-randomised trials, each evaluating two theoretically-enhanced audit and feedback interventions to reduce unnecessary blood transfusions in UK hospitals. The first intervention concerns the content/format of feedback reports. The second aims to support hospital transfusion staff to plan their response to feedback and includes a web-based toolkit and telephone support. Interpretation of trials is enhanced by comprehensively assessing intervention fidelity. However, reviews demonstrate fidelity evaluations are often limited, typically only assessing whether interventions were delivered as intended. This protocol presents methods for assessing fidelity across five dimensions proposed by the Behaviour Change Consortium fidelity framework, including intervention designer-, provider- and recipient-levels. (1) Design: Intervention content will be specified in intervention manuals in terms of component behaviour change techniques (BCTs). Treatment differentiation will be examined by comparing BCTs across intervention/standard practice, noting the proportion of unique/convergent BCTs. (2) Training: draft feedback reports and audio-recorded role-play telephone support scenarios will be content analysed to assess intervention providers' competence to deliver manual-specified BCTs. (3) Delivery: intervention materials (feedback reports, toolkit) and audio-recorded telephone support session transcripts will be content analysed to assess actual delivery of manual-specified BCTs during the intervention period. (4) Receipt and (5) enactment: questionnaires, semi-structured interviews based on the Theoretical Domains Framework, and objective web-analytics data (report downloads, toolkit usage patterns) will be analysed to assess hospital transfusion staff exposure to, understanding and enactment of the interventions, and to identify contextual barriers/enablers to implementation. Associations between observed fidelity and trial outcomes (% unnecessary transfusions) will be examined using mediation analyses. If the interventions have acceptable fidelity, then results of the AFFINITIE trials can be attributed to effectiveness, or lack of effectiveness, of the interventions. Hence, this comprehensive assessment of fidelity will be used to interpret trial findings. These methods may inform fidelity assessments in future trials. ISRCTN 15490813 . Registered 11/03/2015.

Moving from rhetoric to reality: adapting Housing First for homeless individuals with mental illness from ethno-racial groups

PubMed Central

2012-01-01

Background The literature on interventions addressing the intersection of homelessness, mental illness and race is scant. The At Home/Chez Soi research demonstration project is a pragmatic field trial investigating a Housing First intervention for homeless individuals with mental illness in five cities across Canada. A unique focus at the Toronto site has been the development and implementation of a Housing First Ethno-Racial Intensive Case Management (HF ER-ICM) arm of the trial serving 100 homeless individuals with mental illness from ethno-racial groups. The HF ER-ICM program combines the Housing First approach with an anti-racism/anti-oppression framework of practice. This paper presents the findings of an early implementation and fidelity evaluation of the HF ER-ICM program, supplemented by participant narrative interviews to inform our understanding of the HF ER-ICM program theory. Methods Descriptive statistics are used to describe HF ER-ICM participant characteristics. Focus group interviews, key informant interviews and fidelity assessments were conducted between November 2010 and January 2011, as part of the program implementation evaluation. In-depth qualitative interviews with HF ER-ICM participants and control group members were conducted between March 2010 and June 2011. All qualitative data were analysed using grounded theory methodology. Results The target population had complex health and social service needs. The HF ER-ICM program enjoyed a high degree of fidelity to principles of both anti-racism/anti-oppression practice and Housing First and comprehensively addressed the housing, health and sociocultural needs of participants. Program providers reported congruence of these philosophies of practice, and program participants valued the program and its components. Conclusions Adapting Housing First with anti-racism/anti-oppression principles offers a promising approach to serving the diverse needs of homeless people from ethno-racial groups and strengthening the service systems developed to support them. The use of fidelity and implementation evaluations can be helpful in supporting successful adaptations of programs and services. Trial registration Current Controlled Trials ISRCTN42520374 PMID:23031406

Treatment Fidelity Among Family Health Promoters Delivering a Physical Activity and Nutrition Intervention to Immigrant and Refugee Families.

PubMed

Bronars, Carrie A; Hanza, Marcelo M; Meiers, Sonja J; Patten, Christi A; Clark, Matthew M; Nigon, Julie A; Weis, Jennifer A; Wieland, Mark L; Sia, Irene G

2017-04-01

Lack of treatment fidelity can be an important source of variation affecting the credibility and utility of outcomes from behavioral intervention research. Development and implementation of a well-designed treatment fidelity plan, especially with research involving underserved populations, requires careful conceptualization of study needs in conjunction with what is feasible in the population. The purpose of this article is to review a fidelity-monitoring plan consistent with the National Institutes of Health Behavior Change Consortium guidelines (e.g., design, training, delivery, receipt, and enactment) for an intervention trial designed to improve physical activity and nutrition among immigrant and refugee families. Description of the fidelity monitoring plan is provided and challenges related to monitoring treatment fidelity in a community-based participatory intervention for immigrant and refugee families are discussed.

Fidelity of Implementation in Project Towards No Drug Abuse (TND): A Comparison of Classroom Teachers and Program Specialists

PubMed Central

Dent, Clyde W.; Skara, Silvana; Sun, Ping; Sussman, Steve

2011-01-01

This paper presents the results of an effectiveness trial of Project Towards No Drug Abuse [TND], in which we compared program delivery by regular classroom teachers and program specialists within the same high schools. Within 18 schools that were randomly assigned to the program or control conditions, health classrooms were assigned to program delivery by teachers or (outside) specialists. Classroom sessions were observed by pairs of observers to assess three domains of implementation fidelity: adherence, classroom process, and perceived student acceptance of the program. Pre- and immediate posttest survey data were collected from 2331 students. Of the four composite indexes of implementation fidelity that were examined, only one (quality of delivery) showed a difference between specialists and teachers, with marginally higher ratings of specialists (p < .10). Both teachers and program specialists achieved effects on three of the five immediate outcome measures, including program-specific knowledge, addiction concern, and social self-control. Students’ posttest ratings of the program overall and the quality of program delivery failed to reveal differences between the teacher- and specialist-led classrooms. These results suggest that motivated, trained classroom teachers can implement evidence-based prevention programs with fidelity and achieve immediate effects. PMID:17180722

Observational Measures of Implementer Fidelity for a School-based Preventive Intervention: Development, Reliability and Validity

PubMed Central

Cross, Wendi; West, Jennifer; Wyman, Peter A.; Schmeelk-Cone, Karen; Xia, Yinglin; Tu, Xin; Teisl, Michael; Brown, C. Hendricks; Forgatch, Marion

2014-01-01

Current measures of implementer fidelity often fail to adequately measure core constructs of adherence and competence, and their relationship to outcomes can be mixed. To address these limitations, we used observational methods to assess these constructs and their relationships to proximal outcomes in a randomized trial of a school-based preventive intervention (Rochester Resilience Project) designed to strengthen emotion self-regulation skills in 1st–3rd graders with elevated aggressive-disruptive behaviors. Within the intervention group (n = 203), a subsample (n = 76) of students was selected to reflect the overall sample. Implementers were 10 paraprofessionals. Videotaped observations of three lessons from Year 1 of the intervention (14 lessons) were coded for each implementer-child dyad on Adherence (content) and Competence (quality). Using multi-level modeling we examined how much of the variance in the fidelity measures was attributed to implementer and to the child within implementer. Both measures had large and significant variance accounted for by implementer (Competence, 68%; Adherence, 41%); child within implementer did not account for significant variance indicating that ratings reflected stable qualities of the implementer rather than the child. Raw Adherence and Competence scores shared 46% of variance (r = .68). Controlling for baseline differences and age, the amount (Adherence) and quality (Competence) of program delivered predicted children’s enhanced response to the intervention on both child and parent reports after six months, but not on teacher report of externalizing behavior. Our findings support the use of multiple observations for measuring fidelity and that adherence and competence are important components of fidelity which could be assessed by many programs using these methods. PMID:24736951

Development and Implementation: B’More Healthy Communities for Kid’s Store and Wholesaler Intervention

PubMed Central

Schwendler, Teresa; Shipley, Cara; Budd, Nadine; Trude, Angela; Surkan, Pamela J.; Steeves, Elizabeth Anderson; de Morais Sato, Priscila; Eckmann, Thomas; Loh, Hong; Gittelsohn, Joel

2017-01-01

Higher rates of obesity and obesity-related chronic disease are prevalent in communities where there is limited access to affordable, healthy food. The B’More Healthy Communities for Kids (BHCK) trial worked at multiple levels of the food environment including food wholesalers and corner stores to improve the surrounding community’s access to healthy food. The objective of this article is to describe the development and implementation of BHCK’s corner store and wholesaler interventions through formal process evaluation. Researchers evaluated each level of the intervention to assess reach, dose delivered, and fidelity. Corner store and wholesaler reach, dose delivered, and fidelity were measured by number of interactions, promotional materials distributed, and maintenance of study materials, respectively. Overall, the corner store implementation showed moderate reach, dose delivered, and high fidelity. The wholesaler intervention was implemented with high reach, dose, and fidelity. The program held 355 corner store interactive sessions and had 9,347 community member interactions, 21% of which were with children between the ages of 10 and 14 years. There was a 15% increase in corner store promoted food stocking during Wave 1 and a 17% increase during Wave 2. These findings demonstrate a successfully implemented food retailer intervention in a low-income urban setting. PMID:28343413

Inside the Black Box: Examining Mediators and Moderators of a Middle School Science Intervention

ERIC Educational Resources Information Center

Desimone, Laura M.; Hill, Kirsten Lee

2017-01-01

We use data from a randomized controlled trial of a middle school science intervention to explore the causal mechanisms by which the intervention produced previously documented gains in student achievement. Our study finds that implementation fidelity, operationalized as a measure of the frequency of implementation of the cognitive science…

Treatment fidelity of brief motivational interviewing and health education in a randomized clinical trial to promote dental attendance of low-income mothers and children: Community-Based Intergenerational Oral Health Study "Baby Smiles".

PubMed

Weinstein, Philip; Milgrom, Peter; Riedy, Christine A; Mancl, Lloyd A; Garson, Gayle; Huebner, Colleen E; Smolen, Darlene; Sutherland, Marilynn; Nykamp, Ann

2014-02-24

Fidelity assessments are integral to intervention research but few published trials report these processes in detail. We included plans for fidelity monitoring in the design of a community-based intervention trial. The study design was a randomized clinical trial of an intervention provided to low-income women to increase utilization of dental care during pregnancy (mother) or the postpartum (child) period. Group assignment followed a 2 × 2 factorial design in which participants were randomly assigned to receive either brief Motivational Interviewing (MI) or Health Education (HE) during pregnancy (prenatal) and then randomly reassigned to one of these groups for the postpartum intervention. The study setting was four county health departments in rural Oregon State, USA. Counseling was standardized using a step-by-step manual. Counselors were trained to criteria prior to delivering the intervention and fidelity monitoring continued throughout the implementation period based on audio recordings of counselor-participant sessions. The Yale Adherence and Competence Scale (YACS), modified for this study, was used to code the audio recordings of the counselors' delivery of both the MI and HE interventions. Using Interclass Correlation Coefficients totaling the occurrences of specific MI counseling behaviors, ICC for prenatal was .93, for postpartum the ICC was .75. Participants provided a second source of fidelity data. As a second source of fidelity data, the participants completed the Feedback Questionnaire that included ratings of their satisfaction with the counselors at the completion of the prenatal and post-partum interventions. Coding indicated counselor adherence to MI protocol and variation among counselors in the use of MI skills in the MI condition. Almost no MI behaviors were found in the HE condition. Differences in the length of time to deliver intervention were found; as expected, the HE intervention took less time. There were no differences between the overall participants' satisfaction ratings of the HE and MI sessions by individual counselor or overall (p > .05). Trial design, protocol specification, training, and continuous supervision led to a high degree of treatment fidelity for the counseling interventions in this randomized clinical trial and will increase confidence in the interpretation of the trial findings.

Translating staff experience into organisational improvement: the HEADS-UP stepped wedge, cluster controlled, non-randomised trial

PubMed Central

Athanasiou, Thanos; Long, Susannah J; Beveridge, Iain; Sevdalis, Nick

2017-01-01

Objectives Frontline insights into care delivery correlate with patients’ clinical outcomes. These outcomes might be improved through near-real time identification and mitigation of staff concerns. We evaluated the effects of a prospective frontline surveillance system on patient and team outcomes. Design Prospective, stepped wedge, non-randomised, cluster controlled trial; prespecified per protocol analysis for high-fidelity intervention delivery. Participants Seven interdisciplinary medical ward teams from two hospitals in the UK. Intervention Prospective clinical team surveillance (PCTS): structured daily interdisciplinary briefings to capture staff concerns, with organisational facilitation and feedback. Main measures The primary outcome was excess length of stay (eLOS): an admission more than 24 hours above the local average for comparable patients. Secondary outcomes included safety and teamwork climates, and incident reporting. Mixed-effects models adjusted for time effects, age, comorbidity, palliation status and ward admissions. Safety and teamwork climates were measured with the Safety Attitudes Questionnaire. High-fidelity PCTS delivery comprised high engagement and high briefing frequency. Results Implementation fidelity was variable, both in briefing frequency (median 80% working days/month, IQR 65%–90%) and engagement (median 70 issues/ward/month, IQR 34–113). 1714/6518 (26.3%) intervention admissions had eLOS versus 1279/4927 (26.0%) control admissions, an absolute risk increase of 0.3%. PCTS increased eLOS in the adjusted intention-to-treat model (OR 1.32, 95% CI 1.10 to 1.58, p=0.003). Conversely, high-fidelity PCTS reduced eLOS (OR 0.79, 95% CI 0.67 to 0.94, p=0.006). High-fidelity PCTS also increased total, high-yield and non-nurse incident reports (incidence rate ratios 1.28–1.79, all p<0.002). Sustained PCTS significantly improved safety and teamwork climates over time. Conclusions This study highlighted the potential benefits and pitfalls of ward-level interdisciplinary interventions. While these interventions can improve care delivery in complex, fluid environments, the manner of their implementation is paramount. Suboptimal implementation may have an unexpectedly negative impact on performance. Trial registration number ISRCTN 34806867 (http://www.isrctn.com/ISRCTN34806867). PMID:28720612

Evaluation of a novel vital sign device to reduce maternal mortality and morbidity in low-resource settings: a mixed method feasibility study for the CRADLE-3 trial.

PubMed

Vousden, Nicola; Lawley, Elodie; Nathan, Hannah L; Seed, Paul T; Brown, Adrian; Muchengwa, Tafadzwa; Charantimath, Umesh; Bellad, Mrutyunjaya; Gidiri, Muchabayiwa Francis; Goudar, Shivaprasad; Chappell, Lucy C; Sandall, Jane; Shennan, Andrew H

2018-04-27

The CRADLE-3 trial is a stepped-wedge randomised controlled trial aiming to reduce maternal mortality and morbidity by implementing a novel vital sign device (CRADLE Vital Sign Alert) and training package into routine maternity care in 10 low-income sites. The MRC Guidance on complex interventions proposes that interventions and implementation strategies be shaped by early phase piloting and development work. We present the findings of a three-month mixed-methodology feasibility study for this trial, describe how this was informed by the MRC guidance and the study design was refined. The fidelity, dose, feasibility and acceptability of implementation and training materials were assessed in three representative non-trial sites (Zimbabwe, Ethiopia, India) using multiple-choice questionnaires, evaluation of clinical management (action log), healthcare provider (HCP) semi-structured interviews and focus groups 4-10 weeks after implementation. Simultaneously, the 10 sites included in the main trial (eight countries) collected primary outcome data to inform the power calculation and randomisation allocation and assess the feasibility of data collection. The package was implemented with high fidelity (85% of HCP trained, n = 204). The questionnaires indicated a good understanding of device use with 75% of participants scoring > 75% (n = 97; 90% of those distributed). Action logs were inconsistently completed but indicated that the majority of HCP responded appropriately to abnormal results. From 18 HCP interviews and two focus groups it was widely reported that the intervention improved capacity to make clinical decisions, escalate care and make appropriate referrals. Nine of the ten main trial sites achieved ethical approval for pilot data collection. Intensive care was an inconsistent marker of morbidity and stroke an infrequent outcome and therefore they were removed from the main trial composite outcome. Tools and methods of data collection were optimized and event rates used to inform randomisation. This feasibility study demonstrates that the components of the intervention were acceptable, methods of implementing were successful and the main trial design would be feasible. Qualitative work identified key moderators that informed the main trial process evaluation. Changes to the training package, implementation strategy, study design and processes were identified to refine the implementation in the main trial. ISRCTN41244132 ; Registered 24/11/2015.

Improving and sustaining delivery of CPT for PTSD in mental health systems: a cluster randomized trial.

PubMed

Wiltsey Stirman, Shannon; Finley, Erin P; Shields, Norman; Cook, Joan; Haine-Schlagel, Rachel; Burgess, James F; Dimeff, Linda; Koerner, Kelly; Suvak, Michael; Gutner, Cassidy A; Gagnon, David; Masina, Tasoula; Beristianos, Matthew; Mallard, Kera; Ramirez, Vanessa; Monson, Candice

2017-03-06

Large-scale implementation of evidence-based psychotherapies (EBPs) such as cognitive processing therapy (CPT) for posttraumatic stress disorder can have a tremendous impact on mental and physical health, healthcare utilization, and quality of life. While many mental health systems (MHS) have invested heavily in programs to implement EBPs, few eligible patients receive EBPs in routine care settings, and clinicians do not appear to deliver the full treatment protocol to many of their patients. Emerging evidence suggests that when CPT and other EBPs are delivered at low levels of fidelity, clinical outcomes are negatively impacted. Thus, identifying strategies to improve and sustain the delivery of CPT and other EBPs is critical. Existing literature has suggested two competing strategies to promote sustainability. One emphasizes fidelity to the treatment protocol through ongoing consultation and fidelity monitoring. The other focuses on improving the fit and effectiveness of these treatments through appropriate adaptations to the treatment or the clinical setting through a process of data-driven, continuous quality improvement. Neither has been evaluated in terms of impact on sustained implementation. To compare these approaches on the key sustainability outcomes and provide initial guidance on sustainability strategies, we propose a cluster randomized trial with mental health clinics (n = 32) in three diverse MHSs that have implemented CPT. Cohorts of clinicians and clinical managers will participate in 1 year of a fidelity oriented learning collaborative or 1 year of a continuous quality improvement-oriented learning collaborative. Patient-level PTSD symptom change, CPT fidelity and adaptation, penetration, and clinics' capacity to deliver EBP will be examined. Survey and interview data will also be collected to investigate multilevel influences on the success of the two learning collaborative strategies. This research will be conducted by a team of investigators with expertise in CPT implementation, mixed method research strategies, quality improvement, and implementation science, with input from stakeholders in each participating MHS. It will have broad implications for supporting ongoing delivery of EBPs in mental health and healthcare systems and settings. The resulting products have the potential to significantly improve efforts to ensure ongoing high quality implementation and consumer access to EBPs. NCT02449421 . Registered 02/09/2015.

An exploratory randomised controlled trial of a premises-level intervention to reduce alcohol-related harm including violence in the United Kingdom

PubMed Central

2012-01-01

Background To assess the feasibility of a randomised controlled trial of a licensed premises intervention to reduce severe intoxication and disorder; to establish effect sizes and identify appropriate approaches to the development and maintenance of a rigorous research design and intervention implementation. Methods An exploratory two-armed parallel randomised controlled trial with a nested process evaluation. An audit of risk factors and a tailored action plan for high risk premises, with three month follow up audit and feedback. Thirty-two premises that had experienced at least one assault in the year prior to the intervention were recruited, match paired and randomly allocated to control or intervention group. Police violence data and data from a street survey of study premises’ customers, including measures of breath alcohol concentration and surveyor rated customer intoxication, were used to assess effect sizes for a future definitive trial. A nested process evaluation explored implementation barriers and the fidelity of the intervention with key stakeholders and senior staff in intervention premises using semi-structured interviews. Results The process evaluation indicated implementation barriers and low fidelity, with a reluctance to implement the intervention and to submit to a formal risk audit. Power calculations suggest the intervention effect on violence and subjective intoxication would be raised to significance with a study size of 517 premises. Conclusions It is methodologically feasible to conduct randomised controlled trials where licensed premises are the unit of allocation. However, lack of enthusiasm in senior premises staff indicates the need for intervention enforcement, rather than voluntary agreements, and on-going strategies to promote sustainability. Trial registration UKCRN 7090; ISRCTN: 80875696 PMID:22676069

A mixed methods protocol for developing and testing implementation strategies for evidence-based obesity prevention in childcare: a cluster randomized hybrid type III trial.

PubMed

Swindle, Taren; Johnson, Susan L; Whiteside-Mansell, Leanne; Curran, Geoffrey M

2017-07-18

Despite the potential to reach at-risk children in childcare, there is a significant gap between current practices and evidence-based obesity prevention in this setting. There are few investigations of the impact of implementation strategies on the uptake of evidence-based practices (EBPs) for obesity prevention and nutrition promotion. This study protocol describes a three-phase approach to developing and testing implementation strategies to support uptake of EBPs for obesity prevention practices in childcare (i.e., key components of the WISE intervention). Informed by the i-PARIHS framework, we will use a stakeholder-driven evidence-based quality improvement (EBQI) process to apply information gathered in qualitative interviews on barriers and facilitators to practice to inform the design of implementation strategies. Then, a Hybrid Type III cluster randomized trial will compare a basic implementation strategy (i.e., intervention as usual) with an enhanced implementation strategy informed by stakeholders. All Head Start centers (N = 12) within one agency in an urban area in a southern state in the USA will be randomized to receive the basic or enhanced implementation with approximately 20 classrooms per group (40 educators, 400 children per group). The educators involved in the study, the data collectors, and the biostastician will be blinded to the study condition. The basic and enhanced implementation strategies will be compared on outcomes specified by the RE-AIM model (e.g., Reach to families, Effectiveness of impact on child diet and health indicators, Adoption commitment of agency, Implementation fidelity and acceptability, and Maintenance after 6 months). Principles of formative evaluation will be used throughout the hybrid trial. This study will test a stakeholder-driven approach to improve implementation, fidelity, and maintenance of EBPs for obesity prevention in childcare. Further, this study provides an example of a systematic process to develop and test a tailored, enhanced implementation strategy. ClinicalTrials.gov, NCT03075085.

The Cascading Effects of Multiple Dimensions of Implementation on Program Outcomes: a Test of a Theoretical Model.

PubMed

Berkel, Cady; Mauricio, Anne M; Sandler, Irwin N; Wolchik, Sharlene A; Gallo, Carlos G; Brown, C Hendricks

2017-12-14

This study tests a theoretical cascade model in which multiple dimensions of facilitator delivery predict indicators of participant responsiveness, which in turn lead to improvements in targeted program outcomes. An effectiveness trial of the 10-session New Beginnings Program for divorcing families was implemented in partnership with four county-level family courts. This study included 366 families assigned to the intervention condition who attended at least one session. Independent observers provided ratings of program delivery (i.e., fidelity to the curriculum and process quality). Facilitators reported on parent attendance and parents' competence in home practice of program skills. At pretest and posttest, children reported on parenting and parents reported child mental health. We hypothesized effects of quality on attendance, fidelity and attendance on home practice, and home practice on improvements in parenting and child mental health. Structural Equation Modeling with mediation and moderation analyses were used to test these associations. Results indicated quality was significantly associated with attendance, and attendance moderated the effect of fidelity on home practice. Home practice was a significant mediator of the links between fidelity and improvements in parent-child relationship quality and child externalizing and internalizing problems. Findings provide support for fidelity to the curriculum, process quality, attendance, and home practice as valid predictors of program outcomes for mothers and fathers. Future directions for assessing implementation in community settings are discussed.

Characterizing implementation strategies using a systems engineering survey and interview tool: a comparison across 10 prevention programs for drug abuse and HIV sexual risk behavior.

PubMed

Czaja, Sara J; Valente, Thomas W; Nair, Sankaran N; Villamar, Juan A; Brown, C Hendricks

2016-05-17

Although many behavioral interventions have proven to be efficacious, new methodologies are required beyond efficacy trials to understand how to adopt, implement with fidelity, and sustain behavioral interventions in community settings. In this paper, we present a new approach, based on systems engineering concepts and methods, for characterizing implementation strategies that are used to deliver evidence-based behavioral interventions in health and social service settings. We demonstrate the use of this approach with implementation strategies, used or being used for broader dissemination of 10 evidence-based prevention program projects focused on the prevention of drug or HIV sex risk behaviors. The results indicate that there are wide variations in intervention approaches and that there are challenges in program implementation including maintaining program fidelity, serving community needs, and adequate resources. The results also indicate that implementation requires a committed partnership between the program developers, implementation researchers, and community partners. In addition, there is a need for adaptability within programs to meet community needs, resources, and priorities while maintaining program fidelity. Our methodological approach enabled us to highlight challenges associated with the community implementation of health risk prevention interventions. We also demonstrate how comprehensive descriptions of interventions facilitate understanding of the requirements of program implementation and decisions about the feasibility of implementing a program in community settings.

Fidelity Failures in Brief Strategic Family Therapy for Adolescent Drug Abuse: A Clinical Analysis.

PubMed

Lebensohn-Chialvo, Florencia; Rohrbaugh, Michael J; Hasler, Brant P

2018-04-30

As evidence-based family treatments for adolescent substance use and conduct problems gain traction, cutting edge research moves beyond randomized efficacy trials to address questions such as how these treatments work and how best to disseminate them to community settings. A key factor in effective dissemination is treatment fidelity, which refers to implementing an intervention in a manner consistent with an established manual. While most fidelity research is quantitative, this study offers a qualitative clinical analysis of fidelity failures in a large, multisite effectiveness trial of Brief Strategic Family Therapy (BSFT) for adolescent drug abuse, where BSFT developers trained community therapists to administer this intervention in their own agencies. Using case notes and video recordings of therapy sessions, an independent expert panel first rated 103 cases on quantitative fidelity scales grounded in the BSFT manual and the broader structural-strategic framework that informs BSFT intervention. Because fidelity was generally low, the panel reviewed all cases qualitatively to identify emergent types or categories of fidelity failure. Ten categories of failures emerged, characterized by therapist omissions (e.g., failure to engage key family members, failure to think in threes) and commissions (e.g., off-model, nonsystemic formulations/interventions). Of these, "failure to think in threes" appeared basic and particularly problematic, reflecting the central place of this idea in structural theory and therapy. Although subject to possible bias, our observations highlight likely stumbling blocks in exporting a complex family treatment like BSFT to community settings. These findings also underscore the importance of treatment fidelity in family therapy research. © 2018 Family Process Institute.

Examining the Impact of ABRACADABRA on Early Literacy in Northern Australia: An Implementation Fidelity Analysis

ERIC Educational Resources Information Center

Wolgemuth, Jennifer R.; Abrami, Philip C.; Helmer, Janet; Savage, Robert; Harper, Helen; Lea, Tess

2014-01-01

To address students' poor literacy outcomes, an intervention using a computer-based literacy tool, ABRACADABRA, was implemented in 6 Northern Australia primary schools. A pretest, posttest parallel group, single blind multisite randomized controlled trial was conducted with 308 students between the ages of 4 and 8 years old (M age = 5.8 years, SD…

Formative process evaluation for implementing a social marketing intervention to increase walking among African Americans in the Positive Action for Today's Health trial.

PubMed

Coulon, Sandra M; Wilson, Dawn K; Griffin, Sarah; St George, Sara M; Alia, Kassandra A; Trumpeter, Nevelyn N; Wandersman, Abraham K; Forthofer, Melinda; Robinson, Shamika; Gadson, Barney

2012-12-01

Evaluating programs targeting physical activity may help to reduce disparate rates of obesity among African Americans. We report formative process evaluation methods and implementation dose, fidelity, and reach in the Positive Action for Today's Health trial. We applied evaluation methods based on an ecological framework in 2 community-based police-patrolled walking programs targeting access and safety in underserved African American communities. One program also targeted social connectedness and motivation to walk using a social marketing approach. Process data were systematically collected from baseline to 12 months. Adequate implementation dose was achieved, with fidelity achieved but less stable in both programs. Monthly walkers increased to 424 in the walking-plus-social marketing program, indicating expanding program reach, in contrast to no increase in the walking-only program. Increased reach was correlated with peer-led Pride Strides (r = .92; P < .001), a key social marketing component, and program social interaction was the primary reason for which walkers reported participating. Formative process evaluation demonstrated that the walking programs were effectively implemented and that social marketing increased walking and perceived social connectedness in African American communities.

Moving Triadic Gaze Intervention Into Practice: Measuring Clinician Attitude and Implementation Fidelity

PubMed Central

Olswang, Lesley B.; Greenslade, Kathryn; Pinder, Gay Lloyd; Dowden, Patricia; Madden, Jodi

2017-01-01

Purpose This research investigated a first step in implementing the dynamic assessment (DA) component of Triadic Gaze Intervention (Olswang, Feuerstein, Pinder, & Dowden, 2013; Olswang et al., 2014), an evidence-based protocol for teaching early signals of communication to young children with physical disabilities. Clinician attitudes about adopting external evidence into practice and implementation fidelity in DA protocol delivery were examined following training. Method Seven early intervention clinicians from multiple disciplines were trained to deliver the four essential elements of the DA protocol: (a) provide communication opportunity, (b) recognize child's potentially communicative signal, (c) shape child's signal toward triadic gaze, and (d) reinforce with play. Clinician attitude regarding adopting evidence into practice was measured at baseline and follow-up, with the Evidence-Based Practice Attitude Scale (Aarons, 2004). Implementation fidelity in delivering the protocol was measured for adherence (accuracy) and competence (quality) during trial implementation. Results Clinicians' attitudes about trying new evidence that at first was perceived as incongruent with their practice improved over the course of the research. Clinicians demonstrated strong adherence to the DA protocol; however, competence varied across clinicians and appeared related to child performance. Conclusions The results provided insight into moving Triadic Gaze Intervention into practice and yielded valuable information regarding the implementation process, with implications for future research. PMID:28525577

Examination of an interventionist-led HIV intervention among criminal justice-involved female prisoners

PubMed Central

Havens, Jennifer R.; Leukefeld, Carl G.; Oser, Carrie B.; Staton-Tindall, Michele; Knudsen, Hannah K.; Mooney, Jennifer; Duvall, Jamieson L.; Clarke, Jennifer G.; Frisman, Linda; Surratt, Hilary L.; Inciardi, James A.

2009-01-01

The purpose of this study was to examine the implementation, adherence and protocol fidelity for the Reducing Risky Relationships for HIV (RRR-HIV) study. The RRR-HIV study is a phase III trial of a randomized intervention to reduce human immunodeficiency virus (HIV) risk behaviors among incarcerated women in four US states: Connecticut, Delaware, Kentucky and Rhode Island. The intervention consists of five interventionist-led prison-based group sessions and a sixth individual community-based session. Data on adherence, implementation, acceptability and fidelity of the intervention were obtained from forms completed after the five prison-based sessions by both the interventionist and participant. Data from the sixth session were collected by the interventionist. Of the 363 women recruited to date, 173 (47.6%) have been randomly allocated to the experimental RRR intervention, of which implementation measures were available for 162 (93.6%). Almost three-quarters of women attended all five sessions, each of which lasted a median of 90 minutes, indicating successful implementation of the protocol across multiple study sites. Interventionists and participants alike reported that all of the topics for each session were discussed, suggesting adherence to the protocol. In addition, protocol interventionists indicated that more than 95% of the women were engaged/involved, interested, and understood the materials presented, indicating high levels of acceptability among the participants and fidelity to the intervention protocols. The majority of participants also answered all of the post-test questions correctly, which is another strong indicator of the fidelity to the intervention. Results suggest that the RRR-HIV study has been successfully implemented across multiple study sites. Adherence to the protocol, as well as protocol fidelity and acceptability, were also strong, which is essential to establish prior to examining outcome data. PMID:20090928

Moving from rhetoric to reality: adapting Housing First for homeless individuals with mental illness from ethno-racial groups.

PubMed

Stergiopoulos, Vicky; O'Campo, Patricia; Gozdzik, Agnes; Jeyaratnam, Jeyagobi; Corneau, Simon; Sarang, Aseefa; Hwang, Stephen W

2012-10-02

The literature on interventions addressing the intersection of homelessness, mental illness and race is scant. The At Home/Chez Soi research demonstration project is a pragmatic field trial investigating a Housing First intervention for homeless individuals with mental illness in five cities across Canada. A unique focus at the Toronto site has been the development and implementation of a Housing First Ethno-Racial Intensive Case Management (HF ER-ICM) arm of the trial serving 100 homeless individuals with mental illness from ethno-racial groups. The HF ER-ICM program combines the Housing First approach with an anti-racism/anti-oppression framework of practice. This paper presents the findings of an early implementation and fidelity evaluation of the HF ER-ICM program, supplemented by participant narrative interviews to inform our understanding of the HF ER-ICM program theory. Descriptive statistics are used to describe HF ER-ICM participant characteristics. Focus group interviews, key informant interviews and fidelity assessments were conducted between November 2010 and January 2011, as part of the program implementation evaluation. In-depth qualitative interviews with HF ER-ICM participants and control group members were conducted between March 2010 and June 2011. All qualitative data were analysed using grounded theory methodology. The target population had complex health and social service needs. The HF ER-ICM program enjoyed a high degree of fidelity to principles of both anti-racism/anti-oppression practice and Housing First and comprehensively addressed the housing, health and sociocultural needs of participants. Program providers reported congruence of these philosophies of practice, and program participants valued the program and its components. Adapting Housing First with anti-racism/anti-oppression principles offers a promising approach to serving the diverse needs of homeless people from ethno-racial groups and strengthening the service systems developed to support them. The use of fidelity and implementation evaluations can be helpful in supporting successful adaptations of programs and services.

Implementation of Teacher Consultation and Coaching in Urban Schools: A Mixed Method Study

PubMed Central

Cappella, Elise; Jackson, Daisy R.; Kim, Ha Yeon; Bilal, Caroline; Holland, Sibyl; Atkins, Marc S.

2015-01-01

Guided by implementation science scholarship and school mental health research, the current study uses qualitative and quantitative data to illuminate the barriers, opportunities, and processes underlying the implementation of a teacher consultation and coaching model (BRIDGE) in urban elementary schools. Data come from five public elementary schools, 12 school mental health staff (BRIDGE consultants), and 18 teachers participating in a classroom-randomized trial of BRIDGE. Findings from directed content analysis of teacher focus group and interview data suggest that aspects of the BRIDGE intervention model, school organization and classroom contexts, and teachers/consultants and their relationship were relevant as implementation facilitators or barriers. In addition, case study analysis of intervention materials and fidelity tools from classrooms with moderate-to-high dosage and adherence suggest variation in consultation and coaching by initial level of observed classroom need. Results illuminate the need for implementation research to extend beyond simple indicators of fidelity to the multiple systems and variation in processes at play across levels of the implementation context. PMID:27293490

The Feedback Intervention Trial (FIT) — Improving Hand-Hygiene Compliance in UK Healthcare Workers: A Stepped Wedge Cluster Randomised Controlled Trial

PubMed Central

Fuller, Christopher; Michie, Susan; Savage, Joanne; McAteer, John; Besser, Sarah; Charlett, Andre; Hayward, Andrew; Cookson, Barry D.; Cooper, Ben S.; Duckworth, Georgia; Jeanes, Annette; Roberts, Jenny; Teare, Louise; Stone, Sheldon

2012-01-01

Introduction Achieving a sustained improvement in hand-hygiene compliance is the WHO’s first global patient safety challenge. There is no RCT evidence showing how to do this. Systematic reviews suggest feedback is most effective and call for long term well designed RCTs, applying behavioural theory to intervention design to optimise effectiveness. Methods Three year stepped wedge cluster RCT of a feedback intervention testing hypothesis that the intervention was more effective than routine practice in 16 English/Welsh Hospitals (16 Intensive Therapy Units [ITU]; 44 Acute Care of the Elderly [ACE] wards) routinely implementing a national cleanyourhands campaign). Intervention-based on Goal & Control theories. Repeating 4 week cycle (20 mins/week) of observation, feedback and personalised action planning, recorded on forms. Computer-generated stepwise entry of all hospitals to intervention. Hospitals aware only of own allocation. Primary outcome: direct blinded hand hygiene compliance (%). Results All 16 trusts (60 wards) randomised, 33 wards implemented intervention (11 ITU, 22 ACE). Mixed effects regression analysis (all wards) accounting for confounders, temporal trends, ward type and fidelity to intervention (forms/month used). Intention to Treat Analysis Estimated odds ratio (OR) for hand hygiene compliance rose post randomisation (1.44; 95% CI 1.18, 1.76;p<0.001) in ITUs but not ACE wards, equivalent to 7–9% absolute increase in compliance. Per-Protocol Analysis for Implementing Wards OR for compliance rose for both ACE (1.67 [1.28–2.22]; p<0.001) & ITUs (2.09 [1.55–2.81];p<0.001) equating to absolute increases of 10–13% and 13–18% respectively. Fidelity to intervention closely related to compliance on ITUs (OR 1.12 [1.04, 1.20];p = 0.003 per completed form) but not ACE wards. Conclusion Despite difficulties in implementation, intention-to-treat, per-protocol and fidelity to intervention, analyses showed an intervention coupling feedback to personalised action planning produced moderate but significant sustained improvements in hand-hygiene compliance, in wards implementing a national hand-hygiene campaign. Further implementation studies are needed to maximise the intervention’s effect in different settings. Trial Registration Controlled-Trials.com ISRCTN65246961 PMID:23110040

Consumer Feedback following Participation in a Family-Based Intervention for Youth Mental Health

PubMed Central

Lewis, Andrew J.; Bertino, Melanie D.; Robertson, Narelle; Knight, Tess; Toumbourou, John W.

2012-01-01

Background. This paper presents findings derived from consumer feedback, following a multicentre randomised controlled trial for adolescent mental health problems and substance misuse. The paper focuses on the implementation of a family-based intervention, including fidelity of delivery, family members' experiences, and their suggestions for program improvements. Methods. Qualitative and quantitative data (n = 21) were drawn from the Deakin Family Options trial consumer focus groups, which occurred six months after the completion of the trial. Consumer focus groups were held in both metropolitan and regional locations in Victoria, Australia. Findings. Overall reductions in parental isolation, increases in parental self-care, and increased separation/individuation were the key therapeutic features of the intervention. Sharing family experiences with other parents was a key supportive factor, which improved parenting confidence and efficacy and potentially reduced family conflict. Consumer feedback also led to further development of the intervention, with a greater focus on aiding parents to engage adolescents in services and addressing family factors related to adolescent's mood and anxiety symptoms. Conclusions. Participant feedback provides valuable qualitative data, to monitor the fidelity of treatment implementation within a trial, to confirm predictions about the effective mechanisms of an intervention, and to inform the development of new interventions. PMID:22988494

Implementation of Text-Messaging and Social Media Strategies in a Multilevel Childhood Obesity Prevention Intervention: Process Evaluation Results.

PubMed

Loh, Ivory H; Schwendler, Teresa; Trude, Angela C B; Anderson Steeves, Elizabeth T; Cheskin, Lawrence J; Lange, Sarah; Gittelsohn, Joel

2018-01-01

Social media and text messaging show promise as public health interventions, but little evaluation of implementation exists. The B'more Healthy Communities for Kids (BHCK) was a multilevel, multicomponent (wholesalers, food stores, recreation centers) childhood obesity prevention trial that included social media and text-messaging components. The BHCK was implemented in 28 low-income areas of Baltimore City, Maryland, in 2 waves. The texting intervention targeted 241 low-income African American caregivers (of 283), who received 3 texts/week reinforcing key messages, providing nutrition information, and weekly goals. Regular posting on social media platforms (Facebook, Instagram, Twitter) targeted community members and local stakeholders. High implementation standards were set a priori (57 for social media, 11 for texting), with low implementation defined as <50%, medium as 50% to 99%, high as ≥100% of the high standard for each measure. Reach, dose delivered, and fidelity were assessed via web-based analytic tools. Between waves, social media implementation improved from low-moderate to high reach, dose delivered, and fidelity. Text messaging increased from moderate to high in reach and dose delivered, fidelity decreased from high to moderate. Data were used to monitor and revise the BHCK intervention throughout implementation. Our model for evaluating text messaging-based and social media-based interventions may be applicable to other settings.

What Tension between Fidelity and Cultural Adaptation? A Reaction to Marsiglia and Booth

ERIC Educational Resources Information Center

Sampson, McClain; Torres, Luis R.

2015-01-01

This paper is a reaction to Marsiglia and Booths' paper, "Cultural Adaptation of Interventions in Real Practice Settings." In their paper, Marsiglia and Booth present the difficulty of implementing and replicating evidence-supported treatments, such as randomized clinical trials, among culturally diverse clients. Practitioners working in…

Formative Process Evaluation for Implementing a Social Marketing Intervention to Increase Walking Among African Americans in the Positive Action for Today’s Health Trial

PubMed Central

Wilson, Dawn K.; Griffin, Sarah; St. George, Sara M.; Alia, Kassandra A.; Trumpeter, Nevelyn N.; Wandersman, Abraham K.; Forthofer, Melinda; Robinson, Shamika; Gadson, Barney

2012-01-01

Objectives. Evaluating programs targeting physical activity may help to reduce disparate rates of obesity among African Americans. We report formative process evaluation methods and implementation dose, fidelity, and reach in the Positive Action for Today’s Health trial. Methods: We applied evaluation methods based on an ecological framework in 2 community-based police-patrolled walking programs targeting access and safety in underserved African American communities. One program also targeted social connectedness and motivation to walk using a social marketing approach. Process data were systematically collected from baseline to 12 months. Results: Adequate implementation dose was achieved, with fidelity achieved but less stable in both programs. Monthly walkers increased to 424 in the walking-plus-social marketing program, indicating expanding program reach, in contrast to no increase in the walking-only program. Increased reach was correlated with peer-led Pride Strides (r = .92; P < .001), a key social marketing component, and program social interaction was the primary reason for which walkers reported participating. Conclusions: Formative process evaluation demonstrated that the walking programs were effectively implemented and that social marketing increased walking and perceived social connectedness in African American communities. PMID:23078486

Intervention fidelity in primary care complex intervention trials: qualitative study using telephone interviews of patients and practitioners.

PubMed

Dyas, Jane V; Togher, Fiona; Siriwardena, A Niroshan

2014-01-01

Treatment fidelity has previously been defined as the degree to which a treatment or intervention is delivered to participants as intended. Underreporting of fidelity in primary care randomised controlled trials (RCTs) of complex interventions reduces our confidence that findings are due to the treatment or intervention being investigated, rather than unknown confounders. We aimed to investigate treatment fidelity (for the purpose of this paper, hereafter referred to as intervention fidelity), of an educational intervention delivered to general practice teams and designed to improve the primary care management of insomnia. We conducted telephone interviews with patients and practitioners participating in the intervention arm of the trial to explore trial fidelity. Qualitative analysis was undertaken using constant comparison and a priori themes (categories): 'adherence to the delivery of the intervention', 'patients received and understood intervention' and 'patient enactment'. If the intervention protocol was not adhered to by the practitioner then patient receipt, understanding and enactment levels were reduced. Recruitment difficulties in terms of the gap between initially being recruited into the study and attending an intervention consultation also reduced the effectiveness of the intervention. Patient attributes such as motivation to learn and engage contributed to the success of the uptake of the intervention. Qualitative methods using brief telephone interviews are an effective way of collecting the depth of data required to assess intervention fidelity. Intervention fidelity monitoring should be an important element of definitive trial design. ClinicalTrials. gov id isrctn 55001433 - www.controlled-trials.com/isrctn55001433.

Effects of school-wide positive behavioral interventions and supports and fidelity of implementation on problem behavior in high schools.

PubMed

Flannery, K B; Fenning, P; Kato, M McGrath; McIntosh, K

2014-06-01

High school is an important time in the educational career of students. It is also a time when adolescents face many behavioral, academic, and social-emotional challenges. Current statistics about the behavioral, academic, and social-emotional challenges faced by adolescents, and the impact on society through incarceration and dropout, have prompted high schools to direct their attention toward keeping students engaged and reducing high-risk behavioral challenges. The purpose of the study was to examine the effects of School-Wide Positive Behavioral Interventions and Supports (SW-PBIS) on the levels of individual student problem behaviors during a 3-year effectiveness trial without random assignment to condition. Participants were 36,653 students in 12 high schools. Eight schools implemented SW-PBIS, and four schools served as comparison schools. Results of a multilevel latent growth model showed statistically significant decreases in student office discipline referrals in SW-PBIS schools, with increases in comparison schools, when controlling for enrollment and percent of students receiving free or reduced price meals. In addition, as fidelity of implementation increased, office discipline referrals significantly decreased. Results are discussed in terms of effectiveness of a SW-PBIS approach in high schools and considerations to enhance fidelity of implementation. PsycINFO Database Record (c) 2014 APA, all rights reserved.

Evaluating the implementation of a school-based emotional well-being programme: a cluster randomized controlled trial of Zippy's Friends for children in disadvantaged primary schools.

PubMed

Clarke, Aleisha M; Bunting, Brendan; Barry, Margaret M

2014-10-01

Schools are recognized as one of the most important settings for promoting social and emotional well-being among children and adolescents. This clustered randomized controlled trial evaluated Zippy's Friends, an international school-based emotional well-being programme, with 766 children from designated disadvantaged schools. The purpose of this study was to evaluate the immediate and long term impact of the programme and to determine the impact of implementation fidelity on programme outcomes. Teachers reported emotional literacy outcomes using the Emotional Literacy Checklist, and emotional and behavioural outcomes using the Strengths and Difficulties Questionnaire. Controlling for the hierarchical structure of the data, path analysis using structural equation modelling revealed that the programme had a significant positive impact on the children's emotional literacy scores including significant improvements in the subscale scores of self-awareness (P < 0.001), self-regulation (P < 0.01), motivation (P < 0.001) and social skills (P < 0.001) at post-intervention. These results were maintained at 12-month follow-up (P < 0.01). The programme, however, did not have a significant impact on children's emotional and behavioural problems. Analysis of programme fidelity indicated that high fidelity was directly related to improved emotional literacy scores at post-intervention. © The Author 2014. Published by Oxford University Press. All rights reserved. For permissions, please email: journals.permissions@oup.com.

Taking Brief Strategic Family Therapy from Bench to Trench: Evidence Generation Across Translational Phases.

PubMed

Horigian, Viviana E; Anderson, Austen R; Szapocznik, José

2016-09-01

In this article, we review the research evidence generated over 40 years on Brief Strategic Family Therapy illustrating the NIH stages of intervention development and highlighting the translational process. Basic research (Stage 0) led to the discovery of the characteristics of the population and the nature of the problems that needed to be addressed. This step informed the selection of an intervention model that addressed the problems presented by the population, but in a fashion that was congruent with the population's culture, defined in terms of its value orientations. From this basic research, an intervention that integrated structural and strategic elements was selected and refined through testing (Stage I). The second stage of translation (Stage II) included efficacy trials of a specialized engagement module that responded to challenges to the provision of services. It also included several other efficacy trials that documented the effects of the intervention, mostly in research settings or with research therapists. Stages III/IV in the translational process led to the testing of the effectiveness of the intervention in real-world settings with community therapists and some oversight from the developer. This work revealed that an implementation/organizational intervention was required to achieve fidelity and sustainability of the intervention in real-world settings. The work is currently in Stage V in which new model development led to an implementation intervention that can ensure fidelity and sustainability. Future research will evaluate the effectiveness of the current implementation model in increasing adoption, fidelity, and long-term sustainability in real-world settings. © 2016 Family Process Institute.

Implementation fidelity of a self-management course for epilepsy: method and assessment.

PubMed

Wojewodka, G; Hurley, S; Taylor, S J C; Noble, A J; Ridsdale, L; Goldstein, L H

2017-07-11

Complex interventions such as self-management courses are difficult to evaluate due to the many interacting components. The way complex interventions are delivered can influence the effect they have for patients, and can impact the interpretation of outcomes of clinical trials. Implementation fidelity evaluates whether complex interventions are delivered according to protocol. Such assessments have been used for one-to-one psychological interventions; however, the science is still developing for group interventions. We developed and tested an instrument to measure implementation fidelity of a two-day self-management course for people with epilepsy, SMILE(UK). Using audio recordings, we looked at adherence and competence of course facilitators. Adherence was assessed by checklists. Competence was measured by scoring group interaction, an overall impression score and facilitator "didacticism". To measure "didacticism", we developed a novel way to calculate facilitator speech using computer software. Using this new instrument, implementation fidelity of SMILE(UK) was assessed on three modules of the course, for 28% of all courses delivered. Using the instrument for adherence, scores from two independent raters showed substantial agreement with weighted Kappa of 0.67 and high percent agreement of 81.2%. For didacticism, the results from both raters were highly correlated with an intraclass coefficient of 0.97 (p < 0.0001). We found that the courses were delivered with a good level of adherence (> 50% of scored items received the maximum of 2 points) and high competence. Groups were interactive (mean score: 1.9-2.0 out of 2) and the overall impression was on average assessed as "good". Didacticism varied from 42% to 93% of total module time and was not associated with the other competence scores. The instrument devised to measure implementation fidelity was reproducible and easy to use. The courses for the SMILE(UK) study were delivered with a good level of adherence to protocol while not compromising facilitator competence. ISRCTN57937389 .

Sustaining program effectiveness after implementation: The case of the self-management of well-being group intervention for older adults.

PubMed

Goedendorp, Martine M; Kuiper, Daphne; Reijneveld, Sijmen A; Sanderman, Robbert; Steverink, Nardi

2017-06-01

The Self-Management of Well-being (SMW) group intervention for older women was implemented in health and social care. Our aim was to assess whether effects of the SMW intervention were comparable with the original randomized controlled trial (RCT). Furthermore, we investigated threats to effectiveness, such as participant adherence, group reached, and program fidelity. In the implementation study (IMP) 287 and RCT 142 women participated. We compared scores on self-management ability and well-being of the IMP and RCT. For adherence, drop-out rates and session attendance were compared. Regarding reach, we compared participants' baseline characteristics. Professionals completed questions regarding program fidelity. No significant differences were found on effect outcomes and adherence between IMP and RCT (all p≥0.135). Intervention effect sizes were equal (0.47-0.59). IMP participants were significantly less lonely and more likely to be married, but had lower well-being. Most professionals followed the protocol, with only minimal deviations. The effectiveness of the SMW group intervention was reproduced after implementation, with similar participant adherence, minimal changes in the group reached, and high program fidelity. The SMW group intervention can be transferred to health and social care without loss of effectiveness. Implementation at a larger scale is warranted. Copyright © 2017 Elsevier B.V. All rights reserved.

Using conversation analytic methods to assess fidelity to a talk-based healthcare intervention for frequently attending patients.

PubMed

Barnes, Rebecca K; Jepson, Marcus; Thomas, Clare; Jackson, Sue; Metcalfe, Chris; Kessler, David; Cramer, Helen

2018-06-01

The study aim was to assess implementation fidelity (i.e., adherence) to a talk-based primary care intervention using Conversation Analytic (CA) methods. The context was a UK feasibility trial where General Practitioners (GPs) were trained to use "BATHE" (Background,Affect,Trouble,Handling,Empathy) - a technique to screen for psychosocial issues during consultations - with frequently attending patients. 35 GPs received BATHE training between July-October 2015. 15 GPs across six practices self-selected to record a sample of their consultations with study patients at three and six months. 31 consultations were recorded. 21/26 patients in four intervention practices gave permission for analysis. The recordings were transcribed and initially coded for the presence or absence of the five BATHE components. CA methods were applied to assess delivery, focusing on position and composition of each component, and patients' responses. Initial coding showed most of the BATHE components to be present in most contacts. However the CA analysis revealed unplanned deviations in position and adaptations in composition. Frequently the intervention was initiated too early in the consultation, and the BATHE questions misunderstood by patients as pertaining to their presenting problems rather than the psychosocial context for their problems. Often these deviations resulted in reducing theoretical fidelity of the intervention as a whole. A CA approach enabled a dynamic assessment of the delivery and receipt of BATHE in situ revealing common pitfalls in delivery and provided valuable examples of more and less efficacious implementations. During the trial this evidence was used in top-up trainings to address problems in delivery and to improve GP engagement. Using CA methods enabled a more accurate assessment of implementation fidelity, a fuller description of the intervention itself, and enhanced resources for future training. When positioned appropriately, BATHE can be a useful tool for eliciting information about the wider context of the medical visit. Copyright © 2018 Elsevier Ltd. All rights reserved.

Process evaluation of the Enabling Mothers toPrevent Pediatric Obesity Through Web-Based Learning and Reciprocal Determinism (EMPOWER) randomized control trial.

PubMed

Knowlden, Adam P; Sharma, Manoj

2014-09-01

Family-and-home-based interventions are an important vehicle for preventing childhood obesity. Systematic process evaluations have not been routinely conducted in assessment of these interventions. The purpose of this study was to plan and conduct a process evaluation of the Enabling Mothers to Prevent Pediatric Obesity Through Web-Based Learning and Reciprocal Determinism (EMPOWER) randomized control trial. The trial was composed of two web-based, mother-centered interventions for prevention of obesity in children between 4 and 6 years of age. Process evaluation used the components of program fidelity, dose delivered, dose received, context, reach, and recruitment. Categorical process evaluation data (program fidelity, dose delivered, dose exposure, and context) were assessed using Program Implementation Index (PII) values. Continuous process evaluation variables (dose satisfaction and recruitment) were assessed using ANOVA tests to evaluate mean differences between groups (experimental and control) and sessions (sessions 1 through 5). Process evaluation results found that both groups (experimental and control) were equivalent, and interventions were administered as planned. Analysis of web-based intervention process objectives requires tailoring of process evaluation models for online delivery. Dissemination of process evaluation results can advance best practices for implementing effective online health promotion programs. © 2014 Society for Public Health Education.

Testing the Relation between Fidelity of Implementation and Student Outcomes in Math

ERIC Educational Resources Information Center

Crawford, Lindy; Carpenter, Dick M., II; Wilson, Mary T.; Schmeister, Megan; McDonald, Marilee

2012-01-01

The relation between fidelity of implementation and student outcomes in a computer-based middle school mathematics curriculum was measured empirically. Participants included 485 students and 23 teachers from 11 public middle schools across seven states. Implementation fidelity was defined using two constructs: fidelity to structure and fidelity to…

An application of Extended Normalisation Process Theory in a randomised controlled trial of a complex social intervention: Process evaluation of the Strengthening Families Programme (10-14) in Wales, UK.

PubMed

Segrott, Jeremy; Murphy, Simon; Rothwell, Heather; Scourfield, Jonathan; Foxcroft, David; Gillespie, David; Holliday, Jo; Hood, Kerenza; Hurlow, Claire; Morgan-Trimmer, Sarah; Phillips, Ceri; Reed, Hayley; Roberts, Zoe; Moore, Laurence

2017-12-01

Process evaluations generate important data on the extent to which interventions are delivered as intended. However, the tendency to focus only on assessment of pre-specified structural aspects of fidelity has been criticised for paying insufficient attention to implementation processes and how intervention-context interactions influence programme delivery. This paper reports findings from a process evaluation nested within a randomised controlled trial of the Strengthening Families Programme 10-14 (SFP 10-14) in Wales, UK. It uses Extended Normalisation Process Theory to theorise how interaction between SFP 10-14 and local delivery systems - particularly practitioner commitment/capability and organisational capacity - influenced delivery of intended programme activities: fidelity (adherence to SFP 10-14 content and implementation requirements); dose delivered; dose received (participant engagement); participant recruitment and reach (intervention attendance). A mixed methods design was utilised. Fidelity assessment sheets (completed by practitioners), structured observation by researchers, and routine data were used to assess: adherence to programme content; staffing numbers and consistency; recruitment/retention; and group size and composition. Interviews with practitioners explored implementation processes and context. Adherence to programme content was high - with some variation, linked to practitioner commitment to, and understanding of, the intervention's content and mechanisms. Variation in adherence rates was associated with the extent to which multi-agency delivery team planning meetings were held. Recruitment challenges meant that targets for group size/composition were not always met, but did not affect adherence levels or family engagement. Targets for staffing numbers and consistency were achieved, though capacity within multi-agency networks reduced over time. Extended Normalisation Process Theory provided a useful framework for assessing implementation and explaining variation by examining intervention-context interactions. Findings highlight the need for process evaluations to consider both the structural and process components of implementation to explain whether programme activities are delivered as intended and why.

Assessing the Implementation Fidelity of Check-In Check-Out Behavioral Interventions in Elementary and Middle Schools

ERIC Educational Resources Information Center

Ruiz, María Isolina; Smith, Traci N.; Naquin, Gale M.; Morgan-D'Atrio, Cynthia; Dellinger, Amy B.

2014-01-01

Implementation fidelity is crucial to the success of behavioral interventions. However, measuring and maintaining intervention implementation fidelity in schools' natural settings can be challenging. This article reports findings from a study examining the implementation fidelity of check-in check-out interventions at an urban school district…

The Development of the DSM-5 Cultural Formulation Interview-Fidelity Instrument (CFI-FI): A Pilot Study

PubMed Central

Aggarwal, Neil Krishan; Glass, Andrew; Tirado, Amilcar; Boiler, Marit; Nicasio, Andel; Alegría, Margarita; Wall, Melanie; Lewis-Fernández, Roberto

2015-01-01

This paper reports on the development of the Cultural Formulation Interview-Fidelity Instrument (CFI-FI) which assesses clinician fidelity to the DSM-5 Cultural Formulation Interview (CFI). The CFI consists of a manualized set of standard questions that can precede every psychiatric evaluation. It is based on the DSM-IV Outline for Cultural Formulation, the cross-cultural assessment with the most evidence in psychiatric training. Using the New York sample of the DSM-5 CFI field trial, two independent raters created and finalized items for the CFI-FI based on six audio-taped and transcribed interviews. The raters then used the final CFI-FI to rate the remaining 23 interviews. Inter-rater reliability ranged from .73 to 1 for adherence items and .52 to 1 for competence items. The development of the CFI-FI can help researchers and administrators determine whether the CFI has been implemented with fidelity, permitting future intervention research. PMID:25130248

Implementing school nursing strategies to reduce LGBTQ adolescent suicide: a randomized cluster trial study protocol.

PubMed

Willging, Cathleen E; Green, Amy E; Ramos, Mary M

2016-10-22

Reducing youth suicide in the United States (U.S.) is a national public health priority, and lesbian, gay, bisexual, transgender, and queer or questioning (LGBTQ) youth are at elevated risk. The Centers for Disease Control and Prevention (CDC) endorses six evidence-based (EB) strategies that center on meeting the needs of LGBTQ youth in schools; however, fewer than 6 % of U.S. schools implement all of them. The proposed intervention model, "RLAS" (Implementing School Nursing Strategies to Reduce LGBTQ Adolescent Suicide), builds on the Exploration, Preparation, Implementation, and Sustainment (EPIS) conceptual framework and the Dynamic Adaptation Process (DAP) to implement EB strategies in U.S. high schools. The DAP accounts for the multilevel context of school settings and uses Implementation Resource Teams (IRTs) to facilitate appropriate expertise, advise on acceptable adaptations, and provide data feedback to make schools implementation ready and prepared to sustain changes. Mixed methods will be used to examine individual, school, and community factors influencing both implementation process and youth outcomes. A cluster randomized controlled trial will assess whether LGBTQ students and their peers in RLAS intervention schools (n = 20) report reductions in suicidality, depression, substance use, bullying, and truancy related to safety concerns compared to those in usual care schools (n = 20). Implementation progress and fidelity for each EB strategy in RLAS intervention schools will be examined using a modified version of the Stages of Implementation Completion checklist. During the implementation and sustainment phases, annual focus groups will be conducted with the 20 IRTs to document their experiences identifying and advancing adaptation supports to facilitate use of EB strategies and their perceptions of the DAP. The DAP represents a data-informed, collaborative, multiple stakeholder approach to progress from exploration to sustainment and obtain fidelity during the implementation of EB strategies in school settings. This study is designed to address the real-world implications of enabling the use of EB strategies by school nurses with the goal of decreasing suicide and youth risk behaviors among LGBTQ youth. Through its participatory processes to refine and sustain EB strategies in high schools, the RLAS represents a novel contribution to implementation science. ClinicalTrials.gov, NCT02875535.

Investigating variations in implementation fidelity of an organizational-level occupational health intervention.

PubMed

Augustsson, Hanna; von Thiele Schwarz, Ulrica; Stenfors-Hayes, Terese; Hasson, Henna

2015-06-01

The workplace has been suggested as an important arena for health promotion, but little is known about how the organizational setting influences the implementation of interventions. The aims of this study are to evaluate implementation fidelity in an organizational-level occupational health intervention and to investigate possible explanations for variations in fidelity between intervention units. The intervention consisted of an integration of health promotion, occupational health and safety, and a system for continuous improvements (Kaizen) and was conducted in a quasi-experimental design at a Swedish hospital. Implementation fidelity was evaluated with the Conceptual Framework for Implementation Fidelity and implementation factors used to investigate variations in fidelity with the Framework for Evaluating Organizational-level Interventions. A multi-method approach including interviews, Kaizen notes, and questionnaires was applied. Implementation fidelity differed between units even though the intervention was introduced and supported in the same way. Important differences in all elements proposed in the model for evaluating organizational-level interventions, i.e., context, intervention, and mental models, were found to explain the differences in fidelity. Implementation strategies may need to be adapted depending on the local context. Implementation fidelity, as well as pre-intervention implementation elements, is likely to affect the implementation success and needs to be assessed in intervention research. The high variation in fidelity across the units indicates the need for adjustments to the type of designs used to assess the effects of interventions. Thus, rather than using designs that aim to control variation, it may be necessary to use those that aim at exploring and explaining variation, such as adapted study designs.

Impact of a Teacher-as-Coach Model: Improving Paraprofessionals Fidelity of Implementation of Discrete Trial Training for Students with Moderate-to-Severe Developmental Disabilities

ERIC Educational Resources Information Center

Mason, Rose A.; Schnitz, Alana G.; Wills, Howard P.; Rosenbloom, Raia; Kamps, Debra M.; Bast, Darcey

2017-01-01

Ensuring educational progress for students with moderate-to-severe developmental disabilities requires exposure to well executed evidence-based practices. This necessitates that the special education workforce, including paraprofessionals, be well-trained. Yet evidence regarding effective training mechanisms for paraprofessionals is limited. A…

Pairing New Science Curriculum with Professional Learning Increases Student Achievement. Lessons from Research

ERIC Educational Resources Information Center

Killion, Joellen

2016-01-01

A randomized trial study, conducted over two school years in 18 high schools in Washington, finds that "An Inquiry Approach," a three-year, educative curriculum for high school science, has a positive impact on student achievement, teacher practice, and fidelity of implementation of the curriculum when the curriculum is paired with…

An Experimental Study of Procedures to Enhance Ratings of Fidelity to an Evidence-Based Family Intervention.

PubMed

Smith, Justin D; Dishion, Thomas J; Brown, Kimbree; Ramos, Karina; Knoble, Naomi B; Shaw, Daniel S; Wilson, Melvin N

2016-01-01

The valid and reliable assessment of fidelity is critical at all stages of intervention research and is particularly germane to interpreting the results of efficacy and implementation trials. Ratings of protocol adherence typically are reliable, but ratings of therapist competence are plagued by low reliability. Because family context and case conceptualization guide the therapist's delivery of interventions, the reliability of fidelity ratings might be improved if the coder is privy to client context in the form of an ecological assessment. We conducted a randomized experiment to test this hypothesis. A subsample of 46 families with 5-year-old children from a multisite randomized trial who participated in the feedback session of the Family Check-Up (FCU) intervention were selected. We randomly assigned FCU feedback sessions to be rated for fidelity to the protocol using the COACH rating system either after the coder reviewed the results of a recent ecological assessment or had not. Inter-rater reliability estimates of fidelity ratings were meaningfully higher for the assessment information condition compared to the no-information condition. Importantly, the reliability of the COACH mean score was found to be statistically significantly higher in the information condition. These findings suggest that the reliability of observational ratings of fidelity, particularly when the competence or quality of delivery is considered, could be improved by providing assessment data to the coders. Our findings might be most applicable to assessment-driven interventions, where assessment data explicitly guides therapist's selection of intervention strategies tailored to the family's context and needs, but they could also apply to other intervention programs and observational coding of context-dependent therapy processes, such as the working alliance.

An Experimental Study of Procedures to Enhance Ratings of Fidelity to an Evidence-Based Family Intervention

PubMed Central

Smith, Justin D.; Dishion, Thomas J.; Brown, Kimbree; Ramos, Karina; Knoble, Naomi B.; Shaw, Daniel S.; Wilson, Melvin N.

2015-01-01

The valid and reliable assessment of fidelity is critical at all stages of intervention research and is particularly germane to interpreting the results of efficacy and implementation trials. Ratings of protocol adherence typically are reliable, but ratings of therapist competence are plagued by low reliability. Because family context and case conceptualization guide the therapist's delivery of interventions, the reliability of fidelity ratings might be improved if the coder is privy to client context in the form of an ecological assessment. We conducted a randomized experiment to test this hypothesis. A subsample of 46 families with 5-year-old children from a multisite randomized trial who participated in the feedback session of the Family Check-Up (FCU) intervention were selected. We randomly assigned FCU feedback sessions to be rated for fidelity to the protocol using the COACH rating system either after the coder reviewed the results of a recent ecological assessment or had not. Inter-rater reliability estimates of fidelity ratings were meaningfully higher for the assessment information condition compared to the no-information condition. Importantly, the reliability of the COACH mean score was found to be statistically significantly higher in the information condition. These findings suggest that the reliability of observational ratings of fidelity, particularly when the competence or quality of delivery is considered, could be improved by providing assessment data to the coders. Our findings might be most applicable to assessment-driven interventions, where assessment data explicitly guides therapist's selection of intervention strategies tailored to the family's context and needs, but they could also apply to other intervention programs and observational coding of context-dependent therapy processes, such as the working alliance. PMID:26271300

Promoting Afterschool Quality and Positive Youth Development: Cluster Randomized Trial of the Pax Good Behavior Game.

PubMed

Smith, Emilie Phillips; Osgood, D Wayne; Oh, Yoonkyung; Caldwell, Linda C

2018-02-01

This randomized trial tested a strategy originally developed for school settings, the Pax Good Behavior Game (PAX GBG), in the new context of afterschool programs. We examined this approach in afterschool since 70% of all juvenile crime occurs between the hours of 3-6 pm, making afterschool an important setting for prevention and promotion. Dual-career and working families need monitoring and supervision for their children in quality settings that are safe and appropriately structured. While substantial work has identified important features of afterschool programs, increasing attention is being given to how to foster quality. PAX GBG, with its focus on shared norms, cooperative teams, contingent activity rewards, and liberal praise, could potentially enhance not only appropriate structure and supportive relationships, but also youth self-regulation, co-regulation, and socio-emotional development. This study examined the PAX GBG among 76 afterschool programs, serving 811 youth ages 5-12, who were diverse in race-ethnicity, socio-economic status, and geographic locale. Demographically matched pairs of afterschool programs were randomized to PAX GBG or treatment-as-usual. Independent observers conducted ratings of implementation fidelity and program quality across time; along with surveys of children's problem and prosocial behavior. Interaction effects were found using hierarchical linear models such that experimental programs evidencing higher implementation fidelity demonstrated better program quality than controls, (i.e., less harshness, increased appropriate structure, support, and engagement), as well as reduced child-reported hyperactivity and intent-to-treat effects on prosocial behavior. This study demonstrates that best practices fostered by PAX GBG and implemented with fidelity in afterschool result in higher quality contexts for positive youth development.

The Relation of the Response Distribution to Self-Report Questionnaire Items and Cognitive Ability among School-Aged Children

ERIC Educational Resources Information Center

Stapleton, Laura M.; Reiner, Laura S.; Aluvathingal, Anu J.

2010-01-01

This proposed research is part of an on-going line of research of developing questionnaire instruments for use at the elementary school level. Because field trials often use child self-report as outcome measures and sometimes determine implementation fidelity using such measures, evaluation of the validity of the use of such measures with…

A randomized trial to identify accurate and cost-effective fidelity measurement methods for cognitive-behavioral therapy: project FACTS study protocol.

PubMed

Beidas, Rinad S; Maclean, Johanna Catherine; Fishman, Jessica; Dorsey, Shannon; Schoenwald, Sonja K; Mandell, David S; Shea, Judy A; McLeod, Bryce D; French, Michael T; Hogue, Aaron; Adams, Danielle R; Lieberman, Adina; Becker-Haimes, Emily M; Marcus, Steven C

2016-09-15

This randomized trial will compare three methods of assessing fidelity to cognitive-behavioral therapy (CBT) for youth to identify the most accurate and cost-effective method. The three methods include self-report (i.e., therapist completes a self-report measure on the CBT interventions used in session while circumventing some of the typical barriers to self-report), chart-stimulated recall (i.e., therapist reports on the CBT interventions used in session via an interview with a trained rater, and with the chart to assist him/her) and behavioral rehearsal (i.e., therapist demonstrates the CBT interventions used in session via a role-play with a trained rater). Direct observation will be used as the gold-standard comparison for each of the three methods. This trial will recruit 135 therapists in approximately 12 community agencies in the City of Philadelphia. Therapists will be randomized to one of the three conditions. Each therapist will provide data from three unique sessions, for a total of 405 sessions. All sessions will be audio-recorded and coded using the Therapy Process Observational Coding System for Child Psychotherapy-Revised Strategies scale. This will enable comparison of each measurement approach to direct observation of therapist session behavior to determine which most accurately assesses fidelity. Cost data associated with each method will be gathered. To gather stakeholder perspectives of each measurement method, we will use purposive sampling to recruit 12 therapists from each condition (total of 36 therapists) and 12 supervisors to participate in semi-structured qualitative interviews. Results will provide needed information on how to accurately and cost-effectively measure therapist fidelity to CBT for youth, as well as important information about stakeholder perspectives with regard to each measurement method. Findings will inform fidelity measurement practices in future implementation studies as well as in clinical practice. NCT02820623 , June 3rd, 2016.

Implementing trials of complex interventions in community settings: the USC-Rancho Los Amigos pressure ulcer prevention study (PUPS).

PubMed

Clark, Florence; Pyatak, Elizabeth A; Carlson, Mike; Blanche, Erna Imperatore; Vigen, Cheryl; Hay, Joel; Mallinson, Trudy; Blanchard, Jeanine; Unger, Jennifer B; Garber, Susan L; Diaz, Jesus; Florindez, Lucia I; Atkins, Michal; Rubayi, Salah; Azen, Stanley Paul

2014-04-01

Randomized trials of complex, non-pharmacologic interventions implemented in home and community settings, such as the University of Southern California (USC)-Rancho Los Amigos National Rehabilitation Center (RLANRC) Pressure Ulcer Prevention Study (PUPS), present unique challenges with respect to (1) participant recruitment and retention, (2) intervention delivery and fidelity, (3) randomization and assessment, and (4) potential inadvertent treatment effects. We describe the methods employed to address the challenges confronted in implementing PUPS. In this randomized controlled trial, we are assessing the efficacy of a complex, preventive intervention in reducing the incidence of, and costs associated with, the development of medically serious pressure ulcers in people with spinal cord injury. Individuals with spinal cord injury recruited from RLANRC were assigned to either a 12-month preventive intervention group or a standard care control group. The primary outcome is the incidence of serious pressure ulcers with secondary endpoints including ulcer-related surgeries, medical treatment costs, and quality of life. These outcomes are assessed at 12 and 24 months after randomization. Additionally, we are studying the mediating mechanisms that account for intervention outcomes. PUPS has been successfully implemented, including recruitment of the target sample size of 170 participants, assurance of the integrity of intervention protocol delivery with an average 90% treatment adherence rate, and enactment of the assessment plan. However, implementation has been replete with challenges. To meet recruitment goals, we instituted a five-pronged approach customized for an underserved, ethnically diverse population. In intervention delivery, we increased staff time to overcome economic and cultural barriers to retention and adherence. To ensure treatment fidelity and replicability, we monitored intervention protocol delivery in accordance with a rigorous plan. Finally, we have overcome unanticipated assessment and design concerns related to (1) determining pressure ulcer incidence/severity, (2) randomization imbalance, and (3) inadvertent potential control group contamination. We have addressed the most daunting challenges encountered in the recruitment, assessment, and intervention phases of PUPS. Some challenges and solutions may not apply to trials conducted in other settings. Overcoming challenges has required a multifaceted approach incorporating individualization, flexibility, and persistence, as well as the ability to implement needed mid-course corrections.

Comparison of registered and published intervention fidelity assessment in cluster randomised trials of public health interventions in low- and middle-income countries: systematic review protocol.

PubMed

Pérez, Myriam Cielo; Minoyan, Nanor; Ridde, Valéry; Sylvestre, Marie-Pierre; Johri, Mira

2016-10-19

Cluster randomised trials (CRTs) are a key instrument to evaluate public health interventions, particularly in low- and middle-income countries (LMICs). Fidelity assessment examines study processes to gauge whether an intervention was delivered as initially planned. Evaluation of implementation fidelity (IF) is required to establish whether the measured effects of a trial are due to the intervention itself and may be particularly important for CRTs of complex interventions. Current CRT reporting guidelines offer no guidance on IF assessment. We will systematically review the scientific literature to study current practices concerning the assessment of IF in CRTs of public health interventions in LMICs. We will include CRTs of public health interventions in LMICs that planned or assessed IF in either the trial protocol or the main trial report (or an associated document). Search strategies use Medical Subject Headings (MESH) and text words related to CRTs, developing countries, and public health interventions. The electronic database search was developed first for MEDLINE and adapted for the following databases: EMBASE, CINAHL, PubMed, and EMB Reviews, to identify CRT reports in English, Spanish, or French published on or after January 1, 2012. To ensure availability of a study protocol, we will include CRTs reporting a registration number in the abstract. For each included study, we will compare planned versus reported assessment of IF, and consider the dimensions of IF studied, and data collection methods used to evaluate each dimension. Data will be synthesised using quantitative and narrative techniques. Risk of bias for individual studies will be assessed using the Cochrane Collaboration Risk of Bias Tool criteria and additional criteria related to CRT methods. We will investigate possible sources of heterogeneity by performing subgroup analysis. This review was not eligible for inclusion in the PROSPERO registry. Fidelity assessment may be a key tool for making studies more reliable, internally valid, and externally generalizable. This review will provide a portrait of current practices related to the assessment of intervention fidelity in CRTs and offer suggestions for improvement. Results will be relevant to researchers, those who finance health interventions, and for decision-makers who seek the best evidence on public health interventions.

A conceptual framework for implementation fidelity

PubMed Central

Carroll, Christopher; Patterson, Malcolm; Wood, Stephen; Booth, Andrew; Rick, Jo; Balain, Shashi

2007-01-01

Background Implementation fidelity refers to the degree to which an intervention or programme is delivered as intended. Only by understanding and measuring whether an intervention has been implemented with fidelity can researchers and practitioners gain a better understanding of how and why an intervention works, and the extent to which outcomes can be improved. Discussion The authors undertook a critical review of existing conceptualisations of implementation fidelity and developed a new conceptual framework for understanding and measuring the process. The resulting theoretical framework requires testing by empirical research. Summary Implementation fidelity is an important source of variation affecting the credibility and utility of research. The conceptual framework presented here offers a means for measuring this variable and understanding its place in the process of intervention implementation. PMID:18053122

A modified theoretical framework to assess implementation fidelity of adaptive public health interventions.

PubMed

Pérez, Dennis; Van der Stuyft, Patrick; Zabala, Maríadel Carmen; Castro, Marta; Lefèvre, Pierre

2016-07-08

One of the major debates in implementation research turns around fidelity and adaptation. Fidelity is the degree to which an intervention is implemented as intended by its developers. It is meant to ensure that the intervention maintains its intended effects. Adaptation is the process of implementers or users bringing changes to the original design of an intervention. Depending on the nature of the modifications brought, adaptation could either be potentially positive or could carry the risk of threatening the theoretical basis of the intervention, resulting in a negative effect on expected outcomes. Adaptive interventions are those for which adaptation is allowed or even encouraged. Classical fidelity dimensions and conceptual frameworks do not address the issue of how to adapt an intervention while still maintaining its effectiveness. We support the idea that fidelity and adaptation co-exist and that adaptations can impact either positively or negatively on the intervention's effectiveness. For adaptive interventions, research should answer the question how an adequate fidelity-adaptation balance can be reached. One way to address this issue is by looking systematically at the aspects of an intervention that are being adapted. We conducted fidelity research on the implementation of an empowerment strategy for dengue prevention in Cuba. In view of the adaptive nature of the strategy, we anticipated that the classical fidelity dimensions would be of limited use for assessing adaptations. The typology we used in the assessment-implemented, not-implemented, modified, or added components of the strategy-also had limitations. It did not allow us to answer the question which of the modifications introduced in the strategy contributed to or distracted from outcomes. We confronted our empirical research with existing literature on fidelity, and as a result, considered that the framework for implementation fidelity proposed by Carroll et al. in 2007 could potentially meet our concerns. We propose modifications to the framework to assess both fidelity and adaptation. The modified Carroll et al.'s framework we propose may permit a comprehensive assessment of the implementation fidelity-adaptation balance required when implementing adaptive interventions, but more empirical research is needed to validate it.

Process evaluation results from the HEALTHY physical education intervention

PubMed Central

Hall, William J.; Zeveloff, Abigail; Steckler, Allan; Schneider, Margaret; Thompson, Deborah; Pham, Trang; Volpe, Stella L.; Hindes, Katie; Sleigh, Adriana; McMurray, Robert G.

2012-01-01

Process evaluation is an assessment of the implementation of an intervention. A process evaluation component was embedded in the HEALTHY study, a primary prevention trial for Type 2 diabetes implemented over 3 years in 21 middle schools across the United States. The HEALTHY physical education (PE) intervention aimed at maximizing student engagement in moderate-to-vigorous physical activity through delivery of structured lesson plans by PE teachers. Process evaluation data collected via class observations and interventionist interviews assessed fidelity, dose delivered, implementor participation, dose received and barriers. Process evaluation results indicate a high level of fidelity in implementing HEALTHY PE activities and offering 225 min of PE every 10 school days. Concerning dose delivered, students were active for approximately 33 min of class, representing an average of 61% of the class time. Results also indicate that PE teachers were generally engaged in implementing the HEALTHY PE curriculum. Data on dose received showed that students were highly engaged with the PE intervention; however, student misbehavior was the most common barrier observed during classes. Other barriers included teacher disengagement, large classes, limited gym space and poor classroom management. Findings suggest that the PE intervention was generally implemented and received as intended despite several barriers. PMID:22156231

Process evaluation results from the HEALTHY physical education intervention.

PubMed

Hall, William J; Zeveloff, Abigail; Steckler, Allan; Schneider, Margaret; Thompson, Deborah; Pham, Trang; Volpe, Stella L; Hindes, Katie; Sleigh, Adriana; McMurray, Robert G

2012-04-01

Process evaluation is an assessment of the implementation of an intervention. A process evaluation component was embedded in the HEALTHY study, a primary prevention trial for Type 2 diabetes implemented over 3 years in 21 middle schools across the United States. The HEALTHY physical education (PE) intervention aimed at maximizing student engagement in moderate-to-vigorous physical activity through delivery of structured lesson plans by PE teachers. Process evaluation data collected via class observations and interventionist interviews assessed fidelity, dose delivered, implementor participation, dose received and barriers. Process evaluation results indicate a high level of fidelity in implementing HEALTHY PE activities and offering 225 min of PE every 10 school days. Concerning dose delivered, students were active for approximately 33 min of class, representing an average of 61% of the class time. Results also indicate that PE teachers were generally engaged in implementing the HEALTHY PE curriculum. Data on dose received showed that students were highly engaged with the PE intervention; however, student misbehavior was the most common barrier observed during classes. Other barriers included teacher disengagement, large classes, limited gym space and poor classroom management. Findings suggest that the PE intervention was generally implemented and received as intended despite several barriers.

Evaluation of Intervention Fidelity in a Multisite Clinical Trial in Persons With Multiple Sclerosis.

PubMed

Morrison, Janet D; Becker, Heather; Stuifbergen, Alexa K

2017-12-01

Careful consideration of intervention fidelity is critical to establishing the validity and reliability of research findings, yet such reports are often lacking in the research literature. It is imperative that intervention fidelity be methodically evaluated and reported to promote the translation of effective interventions into sound evidence-based practice. The purpose of this article is to explore strategies used to promote intervention fidelity, incorporating examples from a multisite clinical trial, that illustrate the National Institutes of Health Behavior Change Consortium's 5 domains for recommended treatment practices: (1) study design, (2) facilitator training, (3) intervention delivery, (4) intervention receipt, and (5) intervention enactment. A multisite randomized clinical trial testing the efficacy of a computer-assisted cognitive rehabilitation intervention for adults with multiple sclerosis is used to illustrate strategies promoting intervention fidelity. Data derived from audiotapes of intervention classes, audits of computer exercises completed by participants, participant class attendance, and goal attainment scaling suggested relatively high fidelity to the intervention protocol. This study illustrates how to report intervention fidelity in the literature guided by best practice strategies, which may serve to promote fidelity monitoring and reporting in future studies.

Sobriety Treatment and Recovery Teams: Implementation Fidelity and Related Outcomes.

PubMed

Huebner, Ruth A; Posze, Lynn; Willauer, Tina M; Hall, Martin T

2015-01-01

Although integrated programs between child welfare and substance abuse treatment are recommended for families with co-occurring child maltreatment and substance use disorders, implementing integrated service delivery strategies with fidelity is a challenging process. This study of the first five years of the Sobriety Treatment and Recovery Team (START) program examines implementation fidelity using a model proposed by Carroll et al. (2007). The study describes the process of strengthening moderators of implementation fidelity, trends in adherence to START service delivery standards, and trends in parent and child outcomes. Qualitative and quantitative measures were used to prospectively study three START sites serving 341 families with 550 parents and 717 children. To achieve implementation fidelity to service delivery standards required a pre-service year and two full years of operation, persistent leadership, and facilitative actions that challenged the existing paradigm. Over four years of service delivery, the time from the child protective services report to completion of five drug treatment sessions was reduced by an average of 75 days. This trend was associated with an increase in parent retention, parental sobriety, and parent retention of child custody. Conclusions/Importance: Understanding the implementation processes necessary to establish complex integrated programs may support realistic allocation of resources. Although implementation fidelity is a moderator of program outcome, complex inter-agency interventions may benefit from innovative measures of fidelity that promote improvement without extensive cost and data collection burden. The implementation framework applied in this study was useful in examining implementation processes, fidelity, and related outcomes.

A Randomized Controlled Trial of a Response-to-Intervention (RTI) Tier 2 Literacy Program: Leveled Literacy Intervention (LLI)

ERIC Educational Resources Information Center

Ransford-Kaldon, Carolyn; Flynt, E. Sutton; Ross, Cristin

2011-01-01

The purpose of this study was twofold: (1) to determine the efficacy of the Leveled Literacy Intervention program (LLI) in increasing reading achievement for K-2 students and (2) to examine LLI program implementation fidelity. This study evaluated LLI in two U.S. school districts and used a mixed-method design to address the following key research…

Altering school climate through school-wide Positive Behavioral Interventions and Supports: findings from a group-randomized effectiveness trial.

PubMed

Bradshaw, Catherine P; Koth, Christine W; Thornton, Leslie A; Leaf, Philip J

2009-06-01

Positive Behavioral Interventions and Supports (PBIS) is a universal, school-wide prevention strategy that is currently implemented in over 7,500 schools to reduce disruptive behavior problems. The present study examines the impact of PBIS on staff reports of school organizational health using data from a group-randomized controlled effectiveness trial of PBIS conducted in 37 elementary schools. Longitudinal multilevel analyses on data from 2,596 staff revealed a significant effect of PBIS on the schools' overall organizational health, resource influence, staff affiliation, and academic emphasis over the 5-year trial; the effects on collegial leadership and institutional integrity were significant when implementation fidelity was included in the model. Trained schools that adopted PBIS the fastest tended to have higher levels of organizational health at baseline, but the later-implementing schools tended to experience the greatest improvements in organizational health after implementing PBIS. This study indicated that changes in school organizational health are important consequences of the PBIS whole-school prevention model, and may in turn be a potential contextual mediator of the effect of PBIS on student performance.

Implementing an early childhood school-based mental health promotion intervention in low-resource Ugandan schools: study protocol for a cluster randomized controlled trial.

PubMed

Huang, Keng-Yen; Nakigudde, Janet; Calzada, Esther; Boivin, Michael J; Ogedegbe, Gbenga; Brotman, Laurie Miller

2014-12-01

Children in Sub-Saharan Africa (SSA) are burdened by significant unmet mental health needs, but this region has limited access to mental health workers and resources to address these needs. Despite the successes of numerous school-based interventions for promoting child mental health, most evidence-based interventions are not available in SSA. This study will investigate the transportability of an evidence-based program from a developed country (United States) to a SSA country (Uganda). The approach includes task-shifting to early childhood teachers and consists of professional development (five days) to introduce strategies for effective behavior management and positive teacher-student interactions, and group-based consultation (14 sessions) to support adoption of effective practices and tailoring to meet the needs of individual students. The design of this study is guided by two implementation frameworks, the Consolidated Framework for Implementation Research and the Teacher Training Implementation Model, that consider multidimensional aspects of intervention fidelity and contextual predictors that may influence implementation and teacher outcomes. Using a cluster randomized design, 10 schools in Uganda will be randomized to either the intervention group (five schools) or the waitlist control group (five schools). A total of 80 to 100 early childhood teachers will be enrolled in the study. Teacher utilization of evidence-based strategies and practices will be assessed at baseline, immediate post-intervention (six months after baseline), and at seven months post-intervention (during a new academic year). Fidelity measures will be assessed throughout the program implementation period (during professional development and consultation sessions). Individual teacher and contextual factors will be assessed at baseline. Data will be collected from multiple sources. Linear mixed-effect modeling, adjusting for school nesting, will be applied to address study questions. The study will produce important information regarding the value of an evidence-based early intervention, and a theory-guided implementation process and tools designed for use in implementing early childhood evidence-based programs in SSA countries or resource-constrained community settings. This trial was registered with ClinicalTrials.gov (registration number: NCT097115) on 15 May 2013.

Translation of an Effective Tai Chi Intervention Into a Community-Based Falls-Prevention Program

PubMed Central

Li, Fuzhong; Harmer, Peter; Glasgow, Russell; Mack, Karin A.; Sleet, David; Fisher, K. John; Kohn, Melvin A.; Millet, Lisa M.; Mead, Jennifer; Xu, Junheng; Lin, Mei-Li; Yang, Tingzhong; Sutton, Beth; Tompkins, Yvaughn

2008-01-01

Tai Chi—Moving for Better Balance, a falls-prevention program developed from a randomized controlled trial for community-based use, was evaluated with the RE-AIM framework in 6 community centers. The program had a 100% adoption rate and 87% reach into the target older adult population. All centers implemented the intervention with good fidelity, and participants showed significant improvements in health-related outcome measures. This evidence-based tai chi program is practical to disseminate and can be effectively implemented and maintained in community settings. PMID:18511723

Study protocol for a controlled trial of Strengths Model Case Management in mental health services in Hong Kong.

PubMed

Tsoi, Wing-See Emily; Tse, Samson; Fukui, Sadaaki; Jones, Steven

2015-10-06

Although strengths-based models are popular within recovery-oriented approaches, there is still a lack of conclusive research to guide how they should be implemented. A recent meta-analysis confirmed the lack of clarity in how this perspective is operationalised and that fidelity monitoring during the implementation process is lacking. Hence, there is a clear need to evaluate the feasibility of delivering and evaluating a clearly operationalised strengths-based intervention that incorporates fidelity checks to inform more definitive research. This protocol therefore describes a controlled trial of Strengths Model Case Management (SMCM), a complex intervention, for people with severe mental illnesses in Hong Kong. This trial follows the guidelines of the Medical Research Council as a phase 2 trial. Hence, it is a pilot study that tests the feasibility and effectiveness of the model. This is a 9-month controlled trial that uses the Kansas Model. Participants and a matched control group are recruited on a voluntary basis, after screening for eligibility. Effectiveness of the SMCM will be measured through outcome measures taken at baseline, the mid-point and at the end of the trial. Outcomes for service users include personal recovery, hope, subjective well-being, psychiatric symptoms, perceived level of recovery features within the organisation, therapeutic alliance and achievement of recovery goals. Outcomes for care workers will include job burnout, organisational features of recovery and perceived supervisory support. With a 2×3 analysis of variance design and a moderate intervention effect (Cohen's d=0.50), a total of 86 participants will be needed for a statistical power of 0.80. Ethical approval has been obtained from the Human Research Ethics Committee for Non-Clinical Faculties at The University of Hong Kong (HRECNCF: EA140913). Australian New Zealand Clinical Trial Registry (ACTRN)12613001120763. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Do Program Implementation Factors or Fidelity Affect Chronic Disease Self-Management Education Programs' Outcomes?

PubMed

Brady, Teresa J; Murphy, Louise B; O'Colmain, Benita J; Hobson, Reeti Desai

2017-09-01

To evaluate whether implementation factors or fidelity moderate chronic disease self-management education program outcomes. Meta-analysis of 34 Arthritis Self-Management Program and Chronic Disease Self-Management Program studies. Community. N = 10 792. Twelve implementation factors: program delivery fidelity and setting and leader and participant characteristics. Eighteen program outcomes: self-reported health behaviors, physical health status, psychological health status, and health-care utilization. Meta-analysis using pooled effect sizes. Modest to moderate statistically significant differences for 4 of 6 implementation factors; these findings were counterintuitive with better outcomes when leaders and participants were unpaid, leaders had less than minimum training, and implementation did not meet fidelity requirements. Exploratory study findings suggest that these interventions tolerate some variability in implementation factors. Further work is needed to identify key elements where fidelity is essential for intervention effectiveness.

Challenges in Patient Recruitment, Implementation, and Fidelity in a Mobile Telehealth Study.

PubMed

Baron, Justine; Hirani, Shashivadan; Newman, Stanton

2016-05-01

Mobile telehealth (MTH) evaluations in diabetes have been conducted, but few report details and issues related to recruitment, implementation (intervention delivery), fidelity, and context. These have important implications on the interpretation of the findings and effectiveness of the intervention. This article reports these data from an MTH study and describes the challenges experienced in running an intervention such as this in an active clinical environment. We conducted a mixed-methods MTH study that included a 9-month randomized controlled trial in people with poorly controlled diabetes. Detailed recruitment data were recorded during the study. Data on contacts between MTH participants and the MTH team were collected and used to report on intervention delivery and fidelity. Meeting and field notes, as well as communications between research team members during the study, were used to report on the contextual factors that affected recruitment, implementation, and fidelity. The recruited sample size represented 6% of the total clinic population (n = 1,360) and 10.7% of the number of potentially eligible people at the clinic (n = 802) identified at the beginning of the study. Contextual factors related to patients, healthcare providers, the institution, or the recruitment protocol contributed to influence access to study participants and the number of participants randomized (n = 81). Technical and device-related aspects of the MTH intervention were delivered successfully, but the expected education and clinical feedback by the MTH nurse were not delivered according to the protocol. Although 92.5% of introductory calls were made by the MTH nurses, only 13.3% of expected educational calls were performed. Changes to the MTH nursing staff affected intervention participants differently and contributed to the low fidelity of intervention delivery. The current article presents data on the influence of contextual factors on the conduct of this MTH study and underlines the need for these processes to be assessed and reported adequately in future MTH research.

Investigating Predictors of Fidelity of Implementation for a Preschool Vocabulary and Language Curriculum

ERIC Educational Resources Information Center

Phillips, Beth M.; Ingrole, Smriti A.; Burris, Pamela W.; Tabulda, Galiya

2017-01-01

This study describes initial development, feasibility, and fidelity of implementation of a vocabulary and language curriculum supplement that combined explicit instruction and educational television programming to enhance children's vocabulary acquisition. Efficacy of an intervention relies in part on its fidelity of implementation, but little is…

Implementation of an educational intervention to improve hand washing in primary schools: process evaluation within a randomised controlled trial.

PubMed

Chittleborough, Catherine R; Nicholson, Alexandra L; Young, Elaine; Bell, Sarah; Campbell, Rona

2013-08-15

Process evaluations are useful for understanding how interventions are implemented in trial settings. This is important for interpreting main trial results and indicating how the intervention might function beyond the trial. The purpose of this study was to examine the reach, dose, fidelity, acceptability, and sustainability of the implementation of an educational hand washing intervention in primary schools, and to explore views regarding acceptability and sustainability of the intervention. Process evaluation within a cluster randomised controlled trial, including focus groups with pupils aged 6 to 11, semi-structured interviews with teachers and external staff who coordinated the intervention delivery, and school reports and direct observations of the intervention delivery. The educational package was delivered in 61.4% of schools (85.2% of intervention schools, 37.8% of control schools following completion of the trial). Teachers and pupils reacted positively to the intervention, although concerns were raised about the age-appropriateness of the resources. Teachers adapted the resources to suit their school setting and pupils. Staff coordinating the intervention delivery had limited capacity to follow up and respond to schools. The hand washing intervention was acceptable to schools, but its reach outside of a randomised trial, evidenced in the low proportion of schools in the control arm who received it after the trial had ended, suggests that the model of delivery may not be sustainable. ISRCTN: ISRCTN93576146.

A Computerized Decision Support System for Depression in Primary Care

PubMed Central

Kurian, Benji T.; Trivedi, Madhukar H.; Grannemann, Bruce D.; Claassen, Cynthia A.; Daly, Ella J.; Sunderajan, Prabha

2009-01-01

Objective: In 2004, results from The Texas Medication Algorithm Project (TMAP) showed better clinical outcomes for patients whose physicians adhered to a paper-and-pencil algorithm compared to patients who received standard clinical treatment for major depressive disorder (MDD). However, implementation of and fidelity to the treatment algorithm among various providers was observed to be inadequate. A computerized decision support system (CDSS) for the implementation of the TMAP algorithm for depression has since been developed to improve fidelity and adherence to the algorithm. Method: This was a 2-group, parallel design, clinical trial (one patient group receiving MDD treatment from physicians using the CDSS and the other patient group receiving usual care) conducted at 2 separate primary care clinics in Texas from March 2005 through June 2006. Fifty-five patients with MDD (DSM-IV criteria) with no significant difference in disease characteristics were enrolled, 32 of whom were treated by physicians using CDSS and 23 were treated by physicians using usual care. The study's objective was to evaluate the feasibility and efficacy of implementing a CDSS to assist physicians acutely treating patients with MDD compared to usual care in primary care. Primary efficacy outcomes for depression symptom severity were based on the 17-item Hamilton Depression Rating Scale (HDRS17) evaluated by an independent rater. Results: Patients treated by physicians employing CDSS had significantly greater symptom reduction, based on the HDRS17, than patients treated with usual care (P < .001). Conclusions: The CDSS algorithm, utilizing measurement-based care, was superior to usual care for patients with MDD in primary care settings. Larger randomized controlled trials are needed to confirm these findings. Trial Registration: clinicaltrials.gov Identifier: NCT00551083 PMID:19750065

Impact of an interprofessional shared decision-making and goal-setting decision aid for patients with diabetes on decisional conflict--study protocol for a randomized controlled trial.

PubMed

Yu, Catherine H; Ivers, Noah M; Stacey, Dawn; Rezmovitz, Jeremy; Telner, Deanna; Thorpe, Kevin; Hall, Susan; Settino, Marc; Kaplan, David M; Coons, Michael; Sodhi, Sumeet; Sale, Joanna; Straus, Sharon E

2015-06-27

Competing health concerns present real obstacles to people living with diabetes and other chronic diseases as well as to their primary care providers. Guideline implementation interventions rarely acknowledge this, leaving both patients and providers feeling overwhelmed by the volume of recommended actions. Interprofessional (IP) shared decision-making (SDM) with the use of decision aids may help to set treatment priorities. We developed an evidence-based SDM intervention for patients with diabetes and other conditions that was framed by the IP-SDM model and followed a user-centered approach. Our objective in the present study is to pilot an IP-SDM and goal-setting toolkit following the Knowledge-to-Action Framework to assess (1) intervention fidelity and the feasibility of conducting a larger trial and (2) impact on decisional conflict, diabetes distress, health-related quality of life and patient assessment of chronic illness care. A two-step, parallel-group, clustered randomized controlled trial (RCT) will be conducted, with the primary goal being to assess intervention fidelity and the feasibility of conducting a larger RCT. The first step is a provider-directed implementation only; the second (after a 6-month delay) involves both provider- and patient-directed implementation. Half of the clusters will be assigned to receive the IP-SDM toolkit, and the other will be assigned to be mailed a diabetes guidelines summary. Individual interviews with patients, their family members and health care providers will be conducted upon trial completion to explore toolkit use. A secondary purpose of this trial is to gather estimates of the toolkit's impact on decisional conflict. Secondary outcomes include diabetes distress, quality of life and chronic illness care, which will be assessed on the basis of patient-completed questionnaires of validated scales at baseline and at 6 and 12 months. Multilevel hierarchical regression models will be used to account for the clustered nature of the data. An individualized approach to patients with multiple chronic conditions using SDM and goal setting is a desirable strategy for achieving guideline-concordant treatment in a patient-centered fashion. Our pilot trial will provide insights regarding strategies for the routine implementation of such interventions in clinical practice, and it will offer an assessment of the impact of this approach. Clinicaltrials.gov Identifier: NCT02379078. Date of Registration: 11 February 2015.

Using Family-Focused Garden, Nutrition, and Physical Activity Programs To Reduce Childhood Obesity: The Texas! Go! Eat! Grow! Pilot Study.

PubMed

Spears-Lanoix, Erica C; McKyer, E Lisako J; Evans, Alexandra; McIntosh, William Alex; Ory, Marcia; Whittlesey, Lisa; Kirk, Alice; Hoelscher, Deanna M; Warren, Judith L

2015-12-01

The TEXAS! GROW! EAT! GO! (TGEG) randomized, control trial is a 5-year study to measure the impact of a nutrition and gardening intervention and/or physical activity (PA) intervention on the weight status of third-grade students. This article describes the results of the pilot study to test the feasibility of two interventions and test the measures to be used in the main trial. The pilot study was conducted in one school with third-grade students and their parents or guardians. The Junior Master Gardner (JMG) and Walk Across Texas (WAT) interventions were implemented over a 5-month period in three third-grade classrooms during spring 2012. The respective interventions focused on improving healthy eating and PA behaviors of children and their families. Baseline and immediate post-test data were collected from students and parents/guardians to measure four child, four parent, and four parent-child interaction behaviors. Process data regarding implementation were also collected from teachers and school administration. Forty-four students and 34 parents or guardians provided both pre- and post-test data. Paired-sample t-tests showed statistically significant changes in student knowledge, vegetable preferences, vegetable consumption, and home food availability (all p < 0.05). At baseline, participants' weight status categories included 57% obese, 10% overweight, and 31% normal weight. Postintervention, weight status categories included 39% obese, 16% overweight, and normal 45%. Data collected from teachers indicated high levels of implementation fidelity. Implementation of both interventions occurred at a very high fidelity level, which led to positive changes in BMI status, and several dietary and PA behaviors. Although the pilot study indicated feasibility of the two interventions for school implementation, results guided revisions to the TGEG program and its survey instruments.

A preliminary analysis of the effects of coaching feedback on teacher implementation fidelity of first step to success.

PubMed

Rodriguez, Billie Jo; Loman, Sheldon L; Horner, Robert H

2009-01-01

First Step to Success (FSS) is a "manualized" intervention with documented effectiveness in reducing problem behaviors for young children at risk for school failure. This study is a preliminary analysis focusing on the role of performance feedback from FSS coaches on the fidelity with which teachers implemented FSS. Three typically developing students (2 kindergarten and 1 first grade) and their respective teachers served as participants. Student behavior and teacher implementation fidelity were assessed using a nonconcurrent multiple baseline design across student/teacher dyads, in which a no coaching baseline was followed by a performance feedback condition. The baseline phase showed modest levels of implementation fidelity and initial low levels of problem behavior followed by gradually increasing trends. A relation was demonstrated between coaching feedback and improved implementation fidelity. In addition, improved fidelity was associated with improvements in student problem behavior. Implications are drawn for clinical application of FSS and other "manualized" interventions.

Assessing Implementation Fidelity: Challenges as Seen through the Lens of Two Experimental Studies

ERIC Educational Resources Information Center

Vig, Rozy; Taylor, Megan W.; Star, Jon R.; Chao, Theodore

2014-01-01

The concept of "implementation fidelity" is broadly used to capture the extent to which an intervention is executed as intended by the designers of the intervention (Century, Rudnick, & Freeman, 2010; Huntley, 2005, McNaught, Tarr, & Sears, 2010, Munter, 2010). Though implementation fidelity instruments are often used to assess…

Movement as Medicine for Type 2 Diabetes: protocol for an open pilot study and external pilot clustered randomised controlled trial to assess acceptability, feasibility and fidelity of a multifaceted behavioural intervention targeting physical activity in primary care.

PubMed

Avery, Leah; Sniehotta, Falko F; Denton, Sarah J; Steen, Nick; McColl, Elaine; Taylor, Roy; Trenell, Michael I

2014-02-03

Physical activity (PA) and nutrition are the cornerstones of diabetes management. Several reviews and meta-analyses report that PA independently produces clinically important improvements in glucose control in people with Type 2 diabetes. However, it remains unclear what the optimal strategies are to increase PA behaviour in people with Type 2 diabetes in routine primary care. This study will determine whether an evidence-informed multifaceted behaviour change intervention (Movement as Medicine for Type 2 Diabetes) targeting both consultation behaviour of primary healthcare professionals and PA behaviour in adults with Type 2 diabetes is both acceptable and feasible in the primary care setting. An open pilot study conducted in two primary care practices (phase one) will assess acceptability, feasibility and fidelity. Ongoing feedback from participating primary healthcare professionals and patients will provide opportunities for systematic adaptation and refinement of the intervention and study procedures. A two-arm parallel group clustered pilot randomised controlled trial with patients from participating primary care practices in North East England will assess acceptability, feasibility, and fidelity of the intervention (versus usual clinical care) and trial processes over a 12-month period. Consultation behaviour involving fidelity of intervention delivery, diabetes and PA related knowledge, attitudes/beliefs, intentions and self-efficacy for delivering a behaviour change intervention targeting PA behaviour will be assessed in primary healthcare professionals. We will rehearse the collection of outcome data (with the focus on data yield and quality) for a future definitive trial, through outcome assessment at baseline, one, six and twelve months. An embedded qualitative process evaluation and treatment fidelity assessment will explore issues around intervention implementation and assess whether intervention components can be reliably and faithfully delivered in routine primary care. Movement as Medicine for Type 2 Diabetes will address an important gap in the evidence-base, that is, the need for interventions to increase free-living PA behaviour in adults with Type 2 diabetes. The multifaceted intervention incorporates an online accredited training programme for primary healthcare professionals and represents, to the best of our knowledge, the first of its kind in the United Kingdom. This study will establish whether the multifaceted behavioural intervention is acceptable and feasible in routine primary care. Movement as Medicine for Type 2 Diabetes (MaMT2D) was registered with Current Controlled Trials on the 14th January 2012: ISRCTN67997502. The first primary care practice was randomised on the 5th October 2012.

A Comprehensive Examination of Preschool Teachers' Implementation Fidelity When Using a Supplemental Language and Literacy Curriculum

ERIC Educational Resources Information Center

Piasta, Shayne B.; Justice, Laura M.; McGinty, Anita; Mashburn, Andrew; Slocum, Laura

2015-01-01

Background: Recent curriculum studies raise a number of questions concerning teachers' implementation fidelity, including the extent to which fidelity to multiple curriculum components is achieved and measured and the extent to which fidelity serves as a mechanism for impacting children's learning. Objective: Within the context of a language and…

Scale-out of a community-based behavioral intervention for childhood obesity: pilot implementation evaluation.

PubMed

Heerman, William J; Schludnt, David; Harris, Dawn; Teeters, Leah; Apple, Rachel; Barkin, Shari L

2018-04-13

Expanding the use of evidence-based behavioral interventions in community settings has met with limited success in various health outcomes as fidelity and dose of clinical interventions are often diluted when translated to communities. We conducted a pilot implementation study to examine adoption of the rigorously evaluated Healthier Families Program by Parks and Recreation centers in 3 cities across the country (MI, GA, NV) with diverse socio-cultural environments. Using the RE-AIM framework, we evaluated the program both quantitatively (pre/post surveys of health behavior change; attendance & fidelity) and qualitatively (interviews with Parks and Recreation staff and participants following the program). The 3 partner sites recruited a total of 26 parent-child pairs. Among the 24 participants who completed pre/post surveys, 62.5% were 25-34 years old, and average child age was 3.6 (SD 0.7) years. The distribution of self-reported race/ethnicity was 54% non-Hispanic White, 38% non-Hispanic Black, and 8% Latino. Qualitative interviews with participants demonstrated increased use of the built environment for physical activity and continued use of key strategies for health behavior change. Three of five (60%) collaborating sites proceeded with implementation of the program. The average attendance for the 12-week program was 7.6 (SD 3.9) sessions, with 71% attending > 50% of sessions. Average fidelity for the 12 weekly sessions was 25.2 (SD 1.2; possible range 9-27). All 3 partner sites continued offering the program after grant funding was complete. This pilot is among the first attempts to scale-out an evidence-based childhood obesity intervention in community Parks and Recreation centers. While this pilot was not intended to confirm the efficacy of the original trial on Body Mass Index (BMI) reduction, the effective and sustained behavior change among a geographically and ethnically diverse population with high attendance and fidelity demonstrates an effective approach on which to base future large-scale implementation efforts to reduce childhood obesity in community settings.

Implementing trials of complex interventions in community settings: The USC – Rancho Los Amigos Pressure Ulcer Prevention Study (PUPS)

PubMed Central

Clark, Florence; Pyatak, Elizabeth A.; Carlson, Mike; Blanche, Erna Imperatore; Vigen, Cheryl; Hay, Joel; Mallinson, Trudy; Blanchard, Jeanine; Unger, Jennifer B.; Garber, Susan L.; Diaz, Jesus; Florindez, Lucia I.; Atkins, Michal; Rubayi, Salah; Azen, Stanley Paul

2014-01-01

Background Randomized trials of complex, non-pharmacologic interventions implemented in home and community settings, such as the University of Southern California (USC)–Rancho Los Amigos National Rehabilitation Center (RLANRC) Pressure Ulcer Prevention Study (PUPS), present unique challenges with respect to: (a) participant recruitment and retention, (b) intervention delivery and fidelity, (c) randomization and assessment, and (d) potential inadvertent treatment effects. Purpose We describe the methods employed to address the challenges confronted in implementing PUPS. In this randomized controlled trial, we are assessing the efficacy of a complex, preventive intervention in reducing the incidence of, and costs associated with, the development of medically serious pressure ulcers in people with spinal cord injury. Method Individuals with spinal cord injury recruited from RLANRC were assigned to either a 12-month preventive intervention group or a standard care control group. The primary outcome is the incidence of serious pressure ulcers with secondary endpoints including ulcer-related surgeries, medical treatment costs, and quality of life. These outcomes are assessed at 12 and 24 months after randomization. Additionally, we are studying the mediating mechanisms that account for intervention outcomes. Results PUPS has been successfully implemented, including recruitment of the target sample size of 170 participants, assurance of the integrity of intervention protocol delivery with an average 90% treatment adherence rate, and enactment of the assessment plan. However, implementation has been replete with challenges. To meet recruitment goals, we instituted a five-pronged approach customized for an underserved, ethnically diverse population. In intervention delivery, we increased staff time to overcome economic and cultural barriers to retention and adherence. To ensure treatment fidelity and replicability, we monitored intervention protocol delivery in accord with a rigorous plan. Finally, we have overcome unanticipated assessment and design concerns related to: (a) determining pressure ulcer incidence/severity, (b) randomization imbalance, and (c) inadvertent potential control group contamination. Limitations We have addressed the most daunting challenges encountered in the recruitment, assessment, and intervention phases of PUPS. Some challenges and solutions may not apply to trials conducted in other settings. Conclusions Overcoming challenges has required a multifaceted approach incorporating individualization, flexibility, and persistence as well as the ability to implement needed mid-course corrections. PMID:24577972

The tug-of-war: fidelity versus adaptation throughout the health promotion program life cycle.

PubMed

Bopp, Melissa; Saunders, Ruth P; Lattimore, Diana

2013-06-01

Researchers across multiple fields have described the iterative and nonlinear phases of the translational research process from program development to dissemination. This process can be conceptualized within a "program life cycle" framework that includes overlapping and nonlinear phases: development, adoption, implementation, maintenance, sustainability or termination, and dissemination or diffusion, characterized by tensions between fidelity to the original plan and adaptation for the setting and population. In this article, we describe the life cycle (phases) for research-based health promotion programs, the key influences at each phase, and the issues related to the tug-of-war between fidelity and adaptation throughout the process using a fictionalized case study based on our previous research. This article suggests the importance of reconceptualizing intervention design, involving stakeholders, and monitoring fidelity and adaptation throughout all phases to maintain implementation fidelity and completeness. Intervention fidelity should be based on causal mechanisms to ensure effectiveness, while allowing for appropriate adaption to ensure maximum implementation and sustainability. Recommendations for future interventions include considering the determinants of implementation including contextual factors at each phase, the roles of stakeholders, and the importance of developing a rigorous, adaptive, and flexible definition of implementation fidelity and completeness.

Response to Intervention: An Investigation of Training, Perceptions, and Fidelity of Implementation

ERIC Educational Resources Information Center

Latacha, Kathryn N.

2013-01-01

The authors investigated the extent to which the amount of training in Response to Intervention (RtI) impacts staff members' perceptions of RtI, how staff members' perceptions of RtI relate to their fidelity of implementation, and to what degree staff members' involvement in training influences their fidelity of implementation. A convenience…

Measuring Fidelity of Implementation--Methodological and Conceptual Issues and Challenges. CRESST Report 811

ERIC Educational Resources Information Center

Osmundson, Ellen; Herman, Joan; Ringstaff, Cathy; Dai, Yunyun; Timms, Mike

2012-01-01

A central challenge in efficacy studies centers on the issue of "fidelity of implementation," that is, the extent to which participants use the curriculum specified by curriculum developers. In this study, we describe and discuss a "fidelity of implementation" model using multiple methods and instruments to compare two versions of a science…

Applying the Five-Step Model of Fidelity Assessment to a Randomized Experiment of a High School STEM Intervention

ERIC Educational Resources Information Center

Kopp, Jason P.; Hulleman, Chris S.; Harackiewicz, Judith M.; Rozek, Chris

2012-01-01

Assessing fidelity of implementation is becoming increasingly important in education research, in particular as a tool for understanding variations in treatment effectiveness. Fidelity of implementation is defined as "the determination of how well an intervention is implemented in comparison with the original program design during an efficacy…

Relationship between Systems Coaching and Problem-Solving Implementation Fidelity in a Response-to-Intervention Model

ERIC Educational Resources Information Center

March, Amanda L.; Castillo, Jose M.; Batsche, George M.; Kincaid, Donald

2016-01-01

The literature on RTI has indicated that professional development and coaching are critical to facilitating problem-solving implementation with fidelity. This study examined the extent to which systems coaching related to the fidelity of problem-solving implementation in 31 schools from six districts. Schools participated in three years of a…

Electronic medication complete communication strategy for opioid prescriptions in the emergency department: Rationale and design for a three-arm provider randomized trial.

PubMed

McCarthy, Danielle M; Courtney, D Mark; Lank, Patrick M; Cameron, Kenzie A; Russell, Andrea M; Curtis, Laura M; Kim, Kwang-Youn A; Walton, Surrey M; Montague, Enid; Lyden, Abbie L; Gravenor, Stephanie J; Wolf, Michael S

2017-08-01

Thousands of people die annually from prescription opioid overdoses; however there are few strategies to ensure patients receive medication risk information at the time of prescribing. To compare the effectiveness of the Emergency Department (ED) Electronic Medication Complete Communication (EMC 2 ) Opioid Strategy (with and without text messaging) to promote safe medication use and improved patient knowledge as compared to usual care. The ED EMC 2 Opioid Strategy consists of 5 automated components to promote safe medication use: 1) physician reminder to counsel, 2) inbox message sent on to the patient's primary care physician, 3) pharmacist message on the prescription to counsel, 4) MedSheet supporting prescription information, and 5) patient-centered Take-Wait-Stop wording of prescription instructions. This strategy will be assessed both with and without the addition of text messages via a three-arm randomized trial. The study will take place at an urban academic ED (annual volume>85,000) in Chicago, IL. Patients being discharged with a new prescription for hydrocodone-acetaminophen will be enrolled and randomized (based on their prescribing physician). The primary outcome of the study is medication safe use as measured by a demonstrated dosing task. Additionally actual safe use, patient knowledge and provider counseling will be measured. Implementation fidelity as well as costs will be reported. The ED EMC 2 Opioid Strategy embeds a risk communication strategy into the electronic health record and promotes medication counseling with minimal workflow disruption. This trial will evaluate the strategy's effectiveness and implementation fidelity as compared to usual care. This trial is registered on clinicaltrials.gov with identifier NCT02431793. Copyright © 2017 Elsevier Inc. All rights reserved.

Movement as Medicine for Type 2 Diabetes: protocol for an open pilot study and external pilot clustered randomised controlled trial to assess acceptability, feasibility and fidelity of a multifaceted behavioural intervention targeting physical activity in primary care

PubMed Central

2014-01-01

Background Physical activity (PA) and nutrition are the cornerstones of diabetes management. Several reviews and meta-analyses report that PA independently produces clinically important improvements in glucose control in people with Type 2 diabetes. However, it remains unclear what the optimal strategies are to increase PA behaviour in people with Type 2 diabetes in routine primary care. Methods This study will determine whether an evidence-informed multifaceted behaviour change intervention (Movement as Medicine for Type 2 Diabetes) targeting both consultation behaviour of primary healthcare professionals and PA behaviour in adults with Type 2 diabetes is both acceptable and feasible in the primary care setting. An open pilot study conducted in two primary care practices (phase one) will assess acceptability, feasibility and fidelity. Ongoing feedback from participating primary healthcare professionals and patients will provide opportunities for systematic adaptation and refinement of the intervention and study procedures. A two-arm parallel group clustered pilot randomised controlled trial with patients from participating primary care practices in North East England will assess acceptability, feasibility, and fidelity of the intervention (versus usual clinical care) and trial processes over a 12-month period. Consultation behaviour involving fidelity of intervention delivery, diabetes and PA related knowledge, attitudes/beliefs, intentions and self-efficacy for delivering a behaviour change intervention targeting PA behaviour will be assessed in primary healthcare professionals. We will rehearse the collection of outcome data (with the focus on data yield and quality) for a future definitive trial, through outcome assessment at baseline, one, six and twelve months. An embedded qualitative process evaluation and treatment fidelity assessment will explore issues around intervention implementation and assess whether intervention components can be reliably and faithfully delivered in routine primary care. Discussion Movement as Medicine for Type 2 Diabetes will address an important gap in the evidence-base, that is, the need for interventions to increase free-living PA behaviour in adults with Type 2 diabetes. The multifaceted intervention incorporates an online accredited training programme for primary healthcare professionals and represents, to the best of our knowledge, the first of its kind in the United Kingdom. This study will establish whether the multifaceted behavioural intervention is acceptable and feasible in routine primary care. Trial registration Movement as Medicine for Type 2 Diabetes (MaMT2D) was registered with Current Controlled Trials on the 14th January 2012: ISRCTN67997502. The first primary care practice was randomised on the 5th October 2012. PMID:24491134

Implementation of an educational intervention to improve hand washing in primary schools: process evaluation within a randomised controlled trial

PubMed Central

2013-01-01

Background Process evaluations are useful for understanding how interventions are implemented in trial settings. This is important for interpreting main trial results and indicating how the intervention might function beyond the trial. The purpose of this study was to examine the reach, dose, fidelity, acceptability, and sustainability of the implementation of an educational hand washing intervention in primary schools, and to explore views regarding acceptability and sustainability of the intervention. Methods Process evaluation within a cluster randomised controlled trial, including focus groups with pupils aged 6 to 11, semi-structured interviews with teachers and external staff who coordinated the intervention delivery, and school reports and direct observations of the intervention delivery. Results The educational package was delivered in 61.4% of schools (85.2% of intervention schools, 37.8% of control schools following completion of the trial). Teachers and pupils reacted positively to the intervention, although concerns were raised about the age-appropriateness of the resources. Teachers adapted the resources to suit their school setting and pupils. Staff coordinating the intervention delivery had limited capacity to follow up and respond to schools. Conclusions The hand washing intervention was acceptable to schools, but its reach outside of a randomised trial, evidenced in the low proportion of schools in the control arm who received it after the trial had ended, suggests that the model of delivery may not be sustainable. Trial registration ISRCTN: ISRCTN93576146 PMID:23947388

Juvenile Justice-Translational Research on Interventions for Adolescents in the Legal System (JJ-TRIALS): a cluster randomized trial targeting system-wide improvement in substance use services.

PubMed

Knight, Danica K; Belenko, Steven; Wiley, Tisha; Robertson, Angela A; Arrigona, Nancy; Dennis, Michael; Bartkowski, John P; McReynolds, Larkin S; Becan, Jennifer E; Knudsen, Hannah K; Wasserman, Gail A; Rose, Eve; DiClemente, Ralph; Leukefeld, Carl

2016-04-29

The purpose of this paper is to describe the Juvenile Justice-Translational Research on Interventions for Adolescents in the Legal System (JJ-TRIALS) study, a cooperative implementation science initiative involving the National Institute on Drug Abuse, six research centers, a coordinating center, and Juvenile Justice Partners representing seven US states. While the pooling of resources across centers enables a robust implementation study design involving 36 juvenile justice agencies and their behavioral health partner agencies, co-producing a study protocol that has potential to advance implementation science, meets the needs of all constituencies (funding agency, researchers, partners, study sites), and can be implemented with fidelity across the cooperative can be challenging. This paper describes (a) the study background and rationale, including the juvenile justice context and best practices for substance use disorders, (b) the selection and use of an implementation science framework to guide study design and inform selection of implementation components, and (c) the specific study design elements, including research questions, implementation interventions, measurement, and analytic plan. The JJ-TRIALS primary study uses a head-to-head cluster randomized trial with a phased rollout to evaluate the differential effectiveness of two conditions (Core and Enhanced) in 36 sites located in seven states. A Core strategy for promoting change is compared to an Enhanced strategy that incorporates all core strategies plus active facilitation. Target outcomes include improvements in evidence-based screening, assessment, and linkage to substance use treatment. Contributions to implementation science are discussed as well as challenges associated with designing and deploying a complex, collaborative project. NCT02672150 .

How to Measure Motivational Interviewing Fidelity in Randomized Controlled Trials: Practical Recommendations.

PubMed

Jelsma, Judith G M; Mertens, Vera-Christina; Forsberg, Lisa; Forsberg, Lars

2015-07-01

Many randomized controlled trials in which motivational interviewing (MI) is a key intervention make no provision for the assessment of treatment fidelity. This methodological shortcoming makes it impossible to distinguish between high- and low-quality MI interventions, and, consequently, to know whether MI provision has contributed to any intervention effects. This article makes some practical recommendations for the collection, selection, coding and reporting of MI fidelity data, as measured using the Motivational Interviewing Treatment Integrity Code. We hope that researchers will consider these recommendations and include MI fidelity measures in future studies. Copyright © 2015 Elsevier Inc. All rights reserved.

Process evaluation results from the HEALTHY nutrition intervention to modify the total school food environment

PubMed Central

Volpe, S. L.; Hall, W. J.; Steckler, A.; Schneider, M.; Thompson, D.; Mobley, C.; Pham, T.; El ghormli, L.

2013-01-01

The process evaluation of HEALTHY, a large multi-center trial to decrease type 2 diabetes mellitus in middle school children, monitored the implementation of the intervention to ascertain the extent that components were delivered and received as intended. The purpose of this article is to report the process evaluation findings concerning the extent to which the HEALTHY nutrition intervention was implemented during the HEALTHY trial. Overall, the observed fidelity of implementing nutrition strategies improved from baseline to the end of the study. By the last semester, all but two nutrition process evaluation goals were met. The most challenging goal to implement was serving high fiber foods, including grain-based foods and legumes. The easiest goals to implement were lowering the fat content of foods offered and offering healthier beverages. The most challenging barriers experienced by research dietitians and food service staff were costs, availability of foods and student acceptance. Forming strong relationships between the research dietitians and food service staff was identified as a key strategy to meet HEALTHY nutrition goals. PMID:24107856

Ethnography, fidelity, and the evidence that anthropology adds: supplementing the fidelity process in a clinical trial of supported employment.

PubMed

Smith-Morris, Carolyn; Lopez, Gilberto; Ottomanelli, Lisa; Goetz, Lance; Dixon-Lawson, Kimberly

2014-06-01

This discussion considers the role and findings of ethnographic research within a clinical trial of supported employment for veterans with spinal cord injury. Contributing to qualitative evaluation research and to debates over anthropological evidence vis-à-vis clinical trials, we demonstrate how enactors of a randomized controlled trial can simultaneously attend to both the trial's evidentiary and procedural requirements and to the lived experiences and needs of patients and clinicians. Three major findings are described: (1) contextual information essential to fidelity efforts within the trial; (2) the role of human interrelationships and idiosyncratic networks in the trial's success; and (3) a mapping of the power and authority structures relevant to the staff's ability to perform the protocol. We emphasize strengths of anthropological ethnography in clinical trials that include the provision of complementary, qualitative data, the capture of otherwise unmeasured parts of the trial, and the realization of important information for the translation of the clinical findings into new settings. © 2014 by the American Anthropological Association.

Long Live Love+: evaluation of the implementation of an online school-based sexuality education program in the Netherlands.

PubMed

van Lieshout, Sanne; Mevissen, Fraukje; de Waal, Esri; Kok, Gerjo

2017-06-01

Schools are a common setting for adolescents to receive health education, but implementation of these programs with high levels of completeness and fidelity is not self-evident. Programs that are only partially implemented (completeness) or not implemented as instructed (fidelity) are unlikely to be effective. Therefore, it is important to identify which determinants affect completeness and fidelity of program implementation. As part of the launch of Long Live Love+ (LLL+), an online school-based sexuality education program for adolescents aged 15-17, we performed a process evaluation among teachers and students to measure the levels of completeness and fidelity, identify factors influencing teachers' implementation, and to evaluate the students' response. Sixteen Biology teachers from nine secondary schools throughout the Netherlands who implemented LLL+ were interviewed and 60 students participated in 13 focus group discussions. Results showed that teachers' completeness ranged between 22-100% (M = 75%). Fidelity was high, but many teachers added elements. Teachers and students enjoyed LLL+, particularly the diversity in the exercises and its interactive character. The most important factors that influenced implementation were time and organizational constraints, lack of awareness on the impact of completeness and fidelity, and student response. These factors should be taken into account when developing school-based prevention programs. © The Author 2017. Published by Oxford University Press. All rights reserved. For permissions, please email: journals.permissions@oup.com.

Sustaining Fidelity Following the Nationwide PMTO™ Implementation in Norway

PubMed Central

Forgatch, Marion S.; DeGarmo, David S.

2011-01-01

This report describes three studies from the nationwide Norwegian implementation of Parent Management Training – Oregon Model (PMTO™), an empirically supported treatment for families of children with behavior problems (Forgatch and Patterson 2010). Separate stages of the implementation were evaluated using a fidelity measure based on direct observation of intervention sessions. Study 1 assessed growth in fidelity observed early, mid, and late in the training of a group of practitioners. We hypothesized increased fidelity and decreased variability in practice. Study 2 evaluated method fidelity over the course of three generations of practitioners trained in PMTO. Generation 1 (G1) was trained by the PMTO developer/purveyors; Generation 2 (G2) was trained by selected G1 Norwegian trainers; and Generation 3 (G3) was trained by G1 and G2 trainers. We hypothesized decrease in fidelity with each generation. Study 3 tested the predictive validity of fidelity in a cross-cultural replication, hypothesizing that higher fidelity scores would correlate with improved parenting practices observed in parent-child interactions before and after treatment. In Study 1, trainees' performance improved and became more homogeneous as predicted. In Study 2, a small decline in fidelity followed the transfer from the purveyor trainers to Norwegian trainers in G2, but G3 scores were equivalent to those attained by G1. Thus, the hypothesis was not fully supported. Finally, the FIMP validity model replicated; PMTO fidelity significantly contributed to improvements in parenting practices from pre- to post-treatment. The data indicate that PMTO was transferred successfully to Norwegian implementation with sustained fidelity and cross-cultural generalization. PMID:21671090

Actionable Steps for Engaging Assessment Practitioners and Faculty in Implementation Fidelity Research

ERIC Educational Resources Information Center

Smith, Kristen L.; Finney, Sara J.; Fulcher, Keston H.

2017-01-01

Implementation fidelity data indicate to what extent the delivered educational intervention (e.g., pedagogies, curricula) differs from the designed intervention (Gerstner & Finney, 2013; O'Donnell, 2008). Fidelity data help practitioners make more accurate inferences regarding program effectiveness (Dumas, Lynch, Laughlin, Smith, & Prinz,…

Implementation and Randomized Controlled Trial Evaluation of Universal Postnatal Nurse Home Visiting

PubMed Central

Goodman, W. Benjamin; Murphy, Robert A.; O’Donnell, Karen; Sato, Jeannine; Guptill, Susan

2014-01-01

Objectives. We evaluated whether a brief, universal, postnatal nurse home-visiting intervention can be implemented with high penetration and fidelity, prevent emergency health care services, and promote positive parenting by infant age 6 months. Methods. Durham Connects is a manualized 4- to 7-session program to assess family needs and connect parents with community resources to improve infant health and well-being. All 4777 resident births in Durham, North Carolina, between July 1, 2009, and December 31, 2010, were randomly assigned to intervention and control conditions. A random, representative subset of 549 families received blinded interviews for impact evaluation. Results. Of all families, 80% initiated participation; adherence was 84%. Hospital records indicated that Durham Connects infants had 59% fewer infant emergency medical care episodes than did control infants. Durham Connects mothers reported fewer infant emergency care episodes and more community connections, more positive parenting behaviors, participation in higher quality out-of-home child care, and lower rates of anxiety than control mothers. Blinded observers reported higher quality home environments for Durham Connects than for control families. Conclusions. A brief universal home-visiting program implemented with high penetration and fidelity can lower costly emergency medical care and improve family outcomes. PMID:24354833

Measures of fidelity of delivery of, and engagement with, complex, face-to-face health behaviour change interventions: A systematic review of measure quality.

PubMed

Walton, Holly; Spector, Aimee; Tombor, Ildiko; Michie, Susan

2017-11-01

Understanding the effectiveness of complex, face-to-face health behaviour change interventions requires high-quality measures to assess fidelity of delivery and engagement. This systematic review aimed to (1) identify the types of measures used to monitor fidelity of delivery of, and engagement with, complex, face-to-face health behaviour change interventions and (2) describe the reporting of psychometric and implementation qualities. Electronic databases were searched, systematic reviews and reference lists were hand-searched, and 21 experts were contacted to identify articles. Studies that quantitatively measured fidelity of delivery of, and/or engagement with, a complex, face-to-face health behaviour change intervention for adults were included. Data on interventions, measures, and psychometric and implementation qualities were extracted and synthesized using narrative analysis. Sixty-six studies were included: 24 measured both fidelity of delivery and engagement, 20 measured fidelity of delivery, and 22 measured engagement. Measures of fidelity of delivery included observation (n = 17; 38.6%), self-report (n = 15; 34%), quantitatively rated qualitative interviews (n = 1; 2.3%), or multiple measures (n = 11; 25%). Measures of engagement included self-report (n = 18; 39.1%), intervention records (n = 11; 24%), or multiple measures (n = 17; 37%). Fifty-one studies (77%) reported at least one psychometric or implementation quality; 49 studies (74.2%) reported at least one psychometric quality, and 17 studies (25.8%) reported at least one implementation quality. Fewer than half of the reviewed studies measured both fidelity of delivery of, and engagement with complex, face-to-face health behaviour change interventions. More studies reported psychometric qualities than implementation qualities. Interpretation of intervention outcomes from fidelity of delivery and engagement measurements may be limited due to a lack of reporting of psychometric and implementation qualities. Statement of contribution What is already known on this subject? Evidence of fidelity and engagement is needed to understand effectiveness of complex interventions Evidence of fidelity and engagement are rarely reported High-quality measures are needed to measure fidelity and engagement What does this study add? Evidence that indicators of quality of measures are reported in some studies Evidence that psychometric qualities are reported more frequently than implementation qualities A recommendation for intervention evaluations to report indicators of quality of fidelity and engagement measures. © 2017 The Authors. British Journal of Health Psychology published by John Wiley & Sons Ltd on behalf of British Psychological Society.

Beyond teacher training: the critical role of professional development in maintaining curriculum fidelity.

PubMed

LaChausse, Robert G; Clark, Kim R; Chapple, Sabrina

2014-03-01

To examine how teacher characteristics affected program fidelity in an impact evaluation study of the Positive Prevention PLUS program, and to propose a comprehensive teacher training and professional development structure to increase program fidelity. Curriculum fidelity logs, lesson observations, and teacher surveys were used to measure teacher characteristics and implementation fidelity including adherence, adaptation, and lesson quality. Compared with non-health credentialed teachers, credential health education teachers had greater comfort and self-efficacy regarding sex-related instruction. Teacher self-efficacy and comfort were significant predictors of adherence. Implementation fidelity may be linked to teacher characteristics that can be enhanced during curriculum training. A 2-day teacher training may not adequately address teacher facilitation skills or the maintenance of institutional supports for implementing a program with fidelity and quality. A new model of comprehensive teacher training and support is offered. This new training infrastructure is intended to contribute to the school district's institutionalization of higher-quality comprehensive sexual health education and increase program fidelity. Copyright © 2014 Society for Adolescent Health and Medicine. All rights reserved.

Commentary: Learning from Variations in Fidelity of Implementation

ERIC Educational Resources Information Center

Balu, Rekha; Doolittle, Fred

2016-01-01

The articles in this special issue discuss efforts to improve academic reading outcomes for students and ways to achieve high implementation fidelity of promising strategies. At times the authors discuss if--and how--strong fidelity is associated with strong outcomes and potentially even impacts (the difference between program and control group…

Preschool Children's Development in Classic Montessori, Supplemented Montessori, and Conventional Programs

ERIC Educational Resources Information Center

Lillard, Angeline S.

2012-01-01

Research on the outcomes of Montessori education is scarce and results are inconsistent. One possible reason for the inconsistency is variations in Montessori implementation fidelity. To test whether outcomes vary according to implementation fidelity, we examined preschool children enrolled in high fidelity classic Montessori programs, lower…

Illness Management & Recovery (IMR) in the Netherlands; a naturalistic pilot study to explore the feasibility of a randomized controlled trial.

PubMed

Roosenschoon, Bert-Jan; van Weeghel, Jaap; Bogaards, Moniek; Deen, Mathijs L; Mulder, Cornelis L

2016-11-09

Illness Management & Recovery (IMR) is a curriculum-based program for people with severe and persistent mental illness. To date, four randomized controlled trials (RCTs) have been published on it. As these produced mixed results, we conducted a pilot study to test the feasibility of conducting a new RCT in a Dutch psychiatric institute. Because our primary objective was to evaluate support for implementing IMR on a broader scale, we examined participant recruitment, client outcomes, and clients' and clinicians' satisfaction. Secondary objectives were to evaluate fidelity, trainers' training and supervision, and to explore program duration, dropout, and client characteristics related to dropout. For reporting, we used the checklist for pilot studies adopted from the CONSORT Statement. This program evaluation included a process-evaluation and an outcome evaluation with a One Group Pre-Posttest Design (N = 81). Interviews and internal reports were used to monitor participant numbers, program duration, dropout, and completers' characteristics. Clients' and clinicians' satisfaction and provision of trainers' training and supervision were assessed through interviews. Fidelity was assessed on the IMR Fidelity Scale; client outcomes were assessed on the IMR scale (client and clinician versions) and the Recovery Markers Questionnaire (RMQ). Eighty-one participants were recruited of 167 people who were assessed for eligibility. Completers and clinicians were satisfied, and scores for completers improved significantly on the IMR scale (clinician version) (d = 0.84) and RMQ (d = 0.52), and not significantly on the IMR scale client version (d = 0.41). Mean fidelity was good, but three groups had only moderate fidelity. Our feasibility criterion for trainers' education and supervision was partly attained. Dropout from treatment was 51 %; female participants and people who scored higher on both IMR-scales at baseline had a significantly lower chance of dropping out. The duration of IMR varied (M = 12.7 months, SD = 2.87). Results suggested that feasibility of conducting an RCT on IMR was good. Special attention is required to fidelity, IMR duration, trainers' education and supervision, and dropout, especially of men. One study limitation was our inability to conduct follow-up measurements of non-completers.

An empirical study of multidimensional fidelity of COMPASS consultation.

PubMed

Wong, Venus; Ruble, Lisa A; McGrew, John H; Yu, Yue

2018-06-01

Consultation is essential to the daily practice of school psychologists (National Association of School Psychologist, 2010). Successful consultation requires fidelity at both the consultant (implementation) and consultee (intervention) levels. We applied a multidimensional, multilevel conception of fidelity (Dunst, Trivette, & Raab, 2013) to a consultative intervention called the Collaborative Model for Promoting Competence and Success (COMPASS) for students with autism. The study provided 3 main findings. First, multidimensional, multilevel fidelity is a stable construct and increases over time with consultation support. Second, mediation analyses revealed that implementation-level fidelity components had distant, indirect effects on student Individualized Education Program (IEP) outcomes. Third, 3 fidelity components correlated with IEP outcomes: teacher coaching responsiveness at the implementation level, and teacher quality of delivery and student responsiveness at the intervention levels. Implications and future directions are discussed. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Rationale, design and methods for process evaluation in the HEALTHY study.

PubMed

Schneider, M; Hall, W J; Hernandez, A E; Hindes, K; Montez, G; Pham, T; Rosen, L; Sleigh, A; Thompson, D; Volpe, S L; Zeveloff, A; Steckler, A

2009-08-01

The HEALTHY study was a multi-site randomized trial designed to determine whether a 3-year school-based intervention targeting nutrition and physical activity behaviors could effectively reduce risk factors associated with type 2 diabetes in middle school children. Pilot and formative studies were conducted to inform the development of the intervention components and the process evaluation methods for the main trial. During the main trial, both qualitative and quantitative assessments monitored the fidelity of the intervention and motivated modifications to improve intervention delivery. Structured observations of physical education classes, total school food environments, classroom-based educational modules, and communications and promotional campaigns provided verification that the intervention was delivered as intended. Interviews and focus groups yielded a multidimensional assessment of how the intervention was delivered and received, as well as identifying the barriers to and facilitators of the intervention across and within participating schools. Interim summaries of process evaluation data were presented to the study group as a means of ensuring standardization and quality of the intervention across the seven participating centers. Process evaluation methods and procedures documented the fidelity with which the HEALTHY study was implemented across 21 intervention schools and identified ways in which the intervention delivery might be enhanced throughout the study.

Evaluating the Training of Chinese-Speaking Community Health Workers to Implement a Small-Group Intervention Promoting Mammography.

PubMed

Gu, Jiayan; Maxwell, Annette E; Ma, Grace X; Qian, Xiaokun; Tan, Yin; Hsieh, Hsing-Chuan; Tu, Shin-Ping; Wang, Judy Huei-Yu

2018-04-14

This study evaluated the training of Chinese American Community Health Workers (CHWs) to implement a small-group mammography video and discussion program as part of a randomized controlled trial that had the goal to increase adherence to mammography screening guidelines among Chinese American women. A total of 26 Chinese American CHWs in the metropolitan Washington DC area, Southern California, and New York City participated in a 4-h training workshop and completed surveys before and after the workshop to assess their knowledge regarding mammography screening guidelines and human subjects protection rules. The results showed significantly increased knowledge of mammography screening guidelines and human subjects protection rules (both p < 0.01) after the training. CHWs were also trained to lead a discussion of the video, including screening benefits and misconceptions. Forty-three audio recordings of discussions led by 13 active CHWs were transcribed and qualitatively analyzed to assess implementation fidelity. Ten out of 13 active CHWs fully addressed about 3 of the 5 benefit items, and 11 out of 13 CHWs fully addressed more than 5 of the 9 misconception items. Chinese CHWs can be trained to implement research-based intervention programs. However, a one-time training resulted in moderate adherence to the discussion protocol. Ongoing or repeat trainings throughout the intervention period may be needed to enhance implementation fidelity.

Improving practice in community-based settings: a randomized trial of supervision – study protocol

PubMed Central

2013-01-01

Background Evidence-based treatments for child mental health problems are not consistently available in public mental health settings. Expanding availability requires workforce training. However, research has demonstrated that training alone is not sufficient for changing provider behavior, suggesting that ongoing intervention-specific supervision or consultation is required. Supervision is notably under-investigated, particularly as provided in public mental health. The degree to which supervision in this setting includes ‘gold standard’ supervision elements from efficacy trials (e.g., session review, model fidelity, outcome monitoring, skill-building) is unknown. The current federally-funded investigation leverages the Washington State Trauma-focused Cognitive Behavioral Therapy Initiative to describe usual supervision practices and test the impact of systematic implementation of gold standard supervision strategies on treatment fidelity and clinical outcomes. Methods/Design The study has two phases. We will conduct an initial descriptive study (Phase I) of supervision practices within public mental health in Washington State followed by a randomized controlled trial of gold standard supervision strategies (Phase II), with randomization at the clinician level (i.e., supervisors provide both conditions). Study participants will be 35 supervisors and 130 clinicians in community mental health centers. We will enroll one child per clinician in Phase I (N = 130) and three children per clinician in Phase II (N = 390). We use a multi-level mixed within- and between-subjects longitudinal design. Audio recordings of supervision and therapy sessions will be collected and coded throughout both phases. Child outcome data will be collected at the beginning of treatment and at three and six months into treatment. Discussion This study will provide insight into how supervisors can optimally support clinicians delivering evidence-based treatments. Phase I will provide descriptive information, currently unavailable in the literature, about commonly used supervision strategies in community mental health. The Phase II randomized controlled trial of gold standard supervision strategies is, to our knowledge, the first experimental study of gold standard supervision strategies in community mental health and will yield needed information about how to leverage supervision to improve clinician fidelity and client outcomes. Trial registration ClinicalTrials.gov NCT01800266 PMID:23937766

A cluster randomized Hybrid Type III trial testing an implementation support strategy to facilitate the use of an evidence-based practice in VA homeless programs.

PubMed

Smelson, David A; Chinman, Matthew; McCarthy, Sharon; Hannah, Gordon; Sawh, Leon; Glickman, Mark

2015-05-28

The Housing and Urban Development-Veterans Affairs Supportive Housing (HUD-VASH) program is one of the largest initiatives to end Veteran homelessness. However, mental health and substance use disorders continue to reduce client stability and impede program success. HUD-VASH programs do not consistently employ evidence-based practices that address co-occurring mental health and substance use disorders. This paper presents a study protocol to evaluate the implementation of an evidence-based, co-occurring disorder treatment called Maintaining Independence and Sobriety Through Systems Integration, Outreach, and Networking-Veterans Edition (MISSION-Vet) in HUD-VASH using an implementation strategy called Getting To Outcomes (GTO). In three large VA Medical Centers, this Hybrid Type III trial will randomize case managers and their clients by HUD-VASH sub-teams to receive either MISSION-Vet Implementation as Usual (IU-standard training and access to the MISSION-Vet treatment manuals) or MISSION-Vet implementation augmented by GTO. In addition to testing GTO, effectiveness of the treatment (MISSION-Vet) will be assessed using existing Veteran-level data from the HUD-VASH data monitoring system. This project will compare GTO and IU case managers and their clients on the following variables: (1) fidelity to the MISSION-Vet intervention; (2) proportion of time the Veteran is housed; (3) mental health, substance use, and functional outcomes among Veterans; and (4) factors key to the successful deployment of a new treatment as specified by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) model. This project is an important step for developing an implementation strategy to increase adoption of evidence-based practice use in VA homeless programs, and to further examine efficacy of MISSION-Vet in HUD-VASH. This project has important implications for program managers, policy makers, and researchers within the homelessness field. VA Central IRB approval for this study was granted in October 2011. The three sites were trained on MISSION-Vet and GTO in the first half of 2013. The first GTO planning meetings began after training occurred, between January 2013 and November 2013, across the three sites. The data collection-via a fidelity measure embedded into the VA Computerized Patient Record System-began as each site initiated MISSION-Vet, between April 2013 and January 2014. ClinicalTrials.gov: NCT01430741.

Focus on Implementation: Assessing and Promoting Treatment Fidelity

ERIC Educational Resources Information Center

Collier-Meek, Melissa A.; Fallon, Lindsay M.; Sanetti, Lisa M. H.; Maggin, Daniel M.

2013-01-01

Implementation, or treatment fidelity, is the link between evidence-based interventions and changes in student outcomes. Although higher levels of implementation are generally associated with more efficient improvements in student outcomes, research indicates that many adults struggle to consistently implement student interventions. As school…

Implementation Fidelity of a Voluntary Sector-Led Diabetes Education Programme

ERIC Educational Resources Information Center

Kok, Michele S. Y.; Jones, Mat; Solomon-Moore, Emma; Smith, Jane R.

2018-01-01

Purpose: The quality of voluntary sector-led community health programmes is an important concern for service users, providers and commissioners. Research on the fidelity of programme implementation offers a basis for assessing and further enhancing practice. The purpose of this paper is to report on the fidelity assessment of Living Well Taking…

Assessing Fidelity of Implementation of an Unprescribed, Diagnostic Mathematics Intervention

ERIC Educational Resources Information Center

Munter, Charles; Wilhelm, Anne Garrison; Cobb, Paul; Cordray, David S.

2014-01-01

This article draws on previously employed methods for conducting fidelity studies and applies them to an evaluation of an unprescribed intervention. We document the process of assessing the fidelity of implementation of the Math Recovery first-grade tutoring program, an unprescribed, diagnostic intervention. We describe how we drew on recent…

Differences in Fidelity of Implementation Measures: What Videos and Surveys Reveal about Algebra Instruction

ERIC Educational Resources Information Center

Durkin, Kelley; Pollack, Courtney; Star, Jon R.; Rittle-Johnson, Bethany

2012-01-01

The current paper investigated the following research questions regarding measures of fidelity: (1) Is there a significant relationship between two different measures of fidelity of implementation: a survey of instructional practices and coded videos of classroom lessons? Does the strength of this relationship differ between treatment and control…

Program Fidelity Measures Associated With an Effective Child Restraint Program: Buckle-Up Safely

PubMed Central

Keay, Lisa; Simpson, Judy M.; Brown, Julie; Bilston, Lynne E.; Fegan, Maureen; Cosgrove, Louise; Stevenson, Mark; Ivers, Rebecca Q.

2015-01-01

Objectives. We sought to identify the program fidelity factors associated with successful implementation of the Buckle-Up Safely program, targeting correct use of age-appropriate child car restraints. Methods. In 2010, we conducted a cluster randomized controlled trial of 830 families with children attending preschools and long day care centers in South West Sydney, New South Wales, Australia. Families received the Buckle-Up Safely program in the intervention arm of the study (13 services). Independent observers assessed the type of restraint and whether it was used correctly. Results. This detailed process evaluation showed that the multifaceted program was implemented with high fidelity. Program protocols were adhered to and messaging was consistently delivered. Results from multilevel and logistic regression analyses show that age-appropriate restraint use was associated with attendance at a parent information session hosted at the center (adjusted odd ratio [AOR] = 3.66; 95% confidence interval [CI] = 1.61, 8.29) and adversely affected by the child being aged 2 to 3 years (AOR = 0.14; 95% CI = 0.07, 0.30) or being from a family with more than 2 children (AOR = 0.34; 95% CI = 0.17, 0.67). Conclusions. Findings highlight the importance of parents receiving hands-on education regarding the proper use of age-appropriate child restraints. PMID:25602901

Development of a measure of model fidelity for mental health Crisis Resolution Teams.

PubMed

Lloyd-Evans, Brynmor; Bond, Gary R; Ruud, Torleif; Ivanecka, Ada; Gray, Richard; Osborn, David; Nolan, Fiona; Henderson, Claire; Mason, Oliver; Goater, Nicky; Kelly, Kathleen; Ambler, Gareth; Morant, Nicola; Onyett, Steve; Lamb, Danielle; Fahmy, Sarah; Brown, Ellie; Paterson, Beth; Sweeney, Angela; Hindle, David; Fullarton, Kate; Frerichs, Johanna; Johnson, Sonia

2016-12-01

Crisis Resolution Teams (CRTs) provide short-term intensive home treatment to people experiencing mental health crisis. Trial evidence suggests CRTs can be effective at reducing hospital admissions and increasing satisfaction with acute care. When scaled up to national level however, CRT implementation and outcomes have been variable. We aimed to develop and test a fidelity scale to assess adherence to a model of best practice for CRTs, based on best available evidence. A concept mapping process was used to develop a CRT fidelity scale. Participants (n = 68) from a range of stakeholder groups prioritised and grouped statements (n = 72) about important components of the CRT model, generated from a literature review, national survey and qualitative interviews. These data were analysed using Ariadne software and the resultant cluster solution informed item selection for a CRT fidelity scale. Operational criteria and scoring anchor points were developed for each item. The CORE CRT fidelity scale was then piloted in 75 CRTs in the UK to assess the range of scores achieved and feasibility for use in a 1-day fidelity review process. Trained reviewers (n = 16) rated CRT service fidelity in a vignette exercise to test the scale's inter-rater reliability. There were high levels of agreement within and between stakeholder groups regarding the most important components of the CRT model. A 39-item measure of CRT model fidelity was developed. Piloting indicated that the scale was feasible for use to assess CRT model fidelity and had good face validity. The wide range of item scores and total scores across CRT services in the pilot demonstrate the measure can distinguish lower and higher fidelity services. Moderately good inter-rater reliability was found, with an estimated correlation between individual ratings of 0.65 (95% CI: 0.54 to 0.76). The CORE CRT Fidelity Scale has been developed through a rigorous and systematic process. Promising initial testing indicates its value in assessing adherence to a model of CRT best practice and to support service improvement monitoring and planning. Further research is required to establish its psychometric properties and international applicability.

Teachers Teaching Differently: A Qualitative Study of Implementation Fidelity to Professional Development

ERIC Educational Resources Information Center

Woolley, Michael E.; Rose, Roderick A.; Mercado, Micaela; Orthner, Dennis K.

2013-01-01

Intervention researchers in school settings often implement interventions that involve professional development to schoolteachers or other professional staff to implement school-based interventions. In terms of classroom interventions, teachers are the primary implementation agents; therefore, the fidelity of such interventions depends on teacher…

Feasibility of group intervention for bereaved siblings after pediatric cancer death.

PubMed

Greenwald, Naomi; Barrera, Maru; Neville, Alexandra; Hancock, Kelly

2017-01-01

This study evaluated the feasibility (acceptability, recruitment, retention rates, treatment fidelity, and outcome measures) of implementing a manualized group intervention for bereaved siblings after pediatric cancer death. A convenience sample of 10 siblings participated. The intervention consisted of eight 2-hour sessions that focused on strategies for coping with grief, relationships, and emotional growth. Positive outcomes were obtained with respect to acceptability, recruitment, retention rates, and treatment fidelity. Preliminary outcomes were mixed. Parent pre- and post-intervention outcomes suggested improvements in siblings' overall emotional and social quality of life. Siblings' self-reports reflected no improvements. These data support the feasibility of conducting this intervention and suggest beneficial outcomes based on parental reports. Further research is recommended to evaluate the group intervention efficacy with a large sample using a randomized controlled trial to address the needs of bereaved siblings.

Commentary: Learning from Variations in Fidelity of Implementation.

PubMed

Balu, Rekha; Doolittle, Fred

2016-12-01

The articles in this special issue discuss efforts to improve academic reading outcomes for students and ways to achieve high implementation fidelity of promising strategies. At times the authors discuss if-and how-strong fidelity is associated with strong outcomes and potentially even impacts (the difference between program and control group outcomes). We want to explore this theme in two ways: (a) learning from the variation in fidelity to think about potential points of entry and levers for improvement in implementation, and (b) broadening the evaluation focus to include "service contrast" as a factor driving impacts on student outcomes. We conclude with suggestions for future research. © 2016 Wiley Periodicals, Inc.

Implementer-Initiated Adaptation of Evidence-Based Interventions: Kids Remember the Blue Wig

ERIC Educational Resources Information Center

Gibbs, D. A.; Krieger, K. E.; Cutbush, S. L.; Clinton-Sherrod, A. M.; Miller, S.

2016-01-01

Adaptation of evidence-based interventions by implementers is widespread. Although frequently viewed as departures from fidelity, adaptations may be positive in impact and consistent with fidelity. Research typically catalogs adaptations but rarely includes the implementers' perspectives on adaptation. We report data on individuals implementing an…

The Impact of Implementation Fidelity on Mortality Under a CD4-Stratified Timing Strategy for Antiretroviral Therapy in Patients With Tuberculosis.

PubMed

Patel, Monita R; Westreich, Daniel; Yotebieng, Marcel; Nana, Mbonze; Eron, Joseph J; Behets, Frieda; Van Rie, Annelies

2015-05-01

Among patients with tuberculosis and human immunodeficiency virus type 1, CD4-stratified initiation of antiretroviral therapy (ART) is recommended, with earlier ART in those with low CD4 counts. However, the impact of implementation fidelity to this recommendation is unknown. We examined a prospective cohort study of 395 adult patients diagnosed with tuberculosis and human immunodeficiency virus between August 2007 and November 2009 in Kinshasa, Democratic Republic of the Congo. ART was to be initiated after 1 month of tuberculosis treatment at a CD4 count of <100 cells/mm(3) or World Health Organization stage 4 (other than extrapulmonary tuberculosis) and after 2 months of tuberculosis treatment at a CD4 count of 100-350 cells/mm(3). We used the parametric g-formula to estimate the impact of implementation fidelity on 6-month mortality. Observed implementation fidelity was low (46%); 54% of patients either experienced delays in ART initiation or did not initiate ART, which could be avoided under perfect implementation fidelity. The observed mortality risk was 12.0% (95% confidence interval (CI): 8.2, 15.7); under complete (counterfactual) implementation fidelity, the mortality risk was 7.8% (95% CI: 2.4, 12.3), corresponding to a risk reduction of 4.2% (95% CI: 0.3, 8.1) and a preventable fraction of 35.1% (95% CI: 2.9, 67.9). Strategies to achieve high implementation fidelity to CD4-stratified ART timing are needed to maximize survival benefit. © The Author 2015. Published by Oxford University Press on behalf of the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Supervision of Facilitators in a Multisite Study: Goals, Process, and Outcomes

PubMed Central

2010-01-01

Objective To describe the aims, implementation, and desired outcomes of facilitator supervision for both interventions (treatment and control) in Project Eban and to present the Eban Theoretical Framework for Supervision that guided the facilitators’ supervision. The qualifications and training of supervisors and facilitators are also described. Design This article provides a detailed description of supervision in a multisite behavioral intervention trial. The Eban Theoretical Framework for Supervision is guided by 3 theories: cognitive behavior therapy, the Life-long Model of Supervision, and “Empowering supervisees to empower others: a culturally responsive supervision model.” Methods Supervision is based on the Eban Theoretical Framework for Supervision, which provides guidelines for implementing both interventions using goals, process, and outcomes. Results Because of effective supervision, the interventions were implemented with fidelity to the protocol and were standard across the multiple sites. Conclusions Supervision of facilitators is a crucial aspect of multisite intervention research quality assurance. It provides them with expert advice, optimizes the effectiveness of facilitators, and increases adherence to the protocol across multiple sites. Based on the experience in this trial, some of the challenges that arise when conducting a multisite randomized control trial and how they can be handled by implementing the Eban Theoretical Framework for Supervision are described. PMID:18724192

Validating a Fidelity Scale to Understand Intervention Effects in Classroom-Based Studies

ERIC Educational Resources Information Center

Buckley, Pamela; Moore, Brooke; Boardman, Alison G.; Arya, Diana J.; Maul, Andrew

2017-01-01

K-12 intervention studies often include fidelity of implementation (FOI) as a mediating variable, though most do not report the validity of fidelity measures. This article discusses the critical need for validated FOI scales. To illustrate our point, we describe the development and validation of the Implementation Validity Checklist (IVC-R), an…

The Relationship between the Fidelity of Project-Based Curriculum Implementation and Foreign Language Teachers' Beliefs in Teaching and Learning

ERIC Educational Resources Information Center

de Fur, Karen

2009-01-01

The purpose of this correlational study was to examine the relationship between fidelity of project-based curriculum implementation and foreign language teachers' beliefs in teaching and learning as constructivist or behaviorist. Quantitative data on teachers' beliefs was collected using a self-administered survey. Quantitative data on fidelity of…

Leveraging routine clinical materials and mobile technology to assess CBT fidelity: the Innovative Methods to Assess Psychotherapy Practices (imAPP) study.

PubMed

Wiltsey Stirman, Shannon; Marques, Luana; Creed, Torrey A; Gutner, Cassidy A; DeRubeis, Robert; Barnett, Paul G; Kuhn, Eric; Suvak, Michael; Owen, Jason; Vogt, Dawne; Jo, Booil; Schoenwald, Sonja; Johnson, Clara; Mallard, Kera; Beristianos, Matthew; La Bash, Heidi

2018-05-22

Identifying scalable strategies for assessing fidelity is a key challenge in implementation science. However, for psychosocial interventions, the existing, reliable ways to test treatment fidelity quality are often labor intensive, and less burdensome strategies may not reflect actual clinical practice. Cognitive behavioral therapies (CBTs) provide clinicians with a set of effective core elements to help treat a multitude of disorders, which, evidence suggests, need to be delivered with fidelity to maximize potential client impact. The current "gold standard" for rating CBTs is rating recordings of therapy sessions, which is extremely time-consuming and requires a substantial amount of initial training. Although CBTs can vary based on the target disorder, one common element employed in most CBTs is the use of worksheets to identify specific behaviors and thoughts that affect a client's ability to recover. The present study will develop and evaluate an innovative new approach to rate CBT fidelity, by developing a universal CBT scoring system based on worksheets completed in therapy sessions. To develop a scoring system for CBT worksheets, we will compile common CBT elements from a variety of CBT worksheets for a range of psychiatric disorders and create adherence and competence measures. We will collect archival worksheets from past studies to test the scoring system and assess test-retest reliability. To evaluate whether CBT worksheet scoring accurately reflects clinician fidelity, we will recruit clinicians who are engaged in a CBT for depression, anxiety, and/or posttraumatic stress disorder. Clinicians and clients will transmit routine therapy materials produced in session (e.g., worksheets, clinical notes, session recordings) to the study team after each session. We will compare observer-rated fidelity, clinical notes, and fidelity-rated worksheets to identify the most effective and efficient method to assess clinician fidelity. Clients will also be randomly assigned to either complete the CBT worksheets on paper forms or on a mobile application (app) to learn if worksheet format influences clinician and client experience or differs in terms of reflecting fidelity. Scoring fidelity using CBT worksheets may allow clinics to test fidelity in a short and effective manner, enhancing continuous quality improvement in the workplace. Clinicians and clinics can use such data to improve clinician fidelity in real time, leading to improved patient outcomes. ClinicalTrials.gov NCT03479398 . Retrospectively registered March 20, 2018.

Measuring Implementation Fidelity in a Community-Based Parenting Intervention

PubMed Central

Breitenstein, Susan M.; Fogg, Louis; Garvey, Christine; Hill, Carri; Resnick, Barbara; Gross, Deborah

2012-01-01

Background Establishing the feasibility and validity of implementation fidelity monitoring strategies is an important methodological step in implementing evidence-based interventions on a large scale. Objectives The objective of the study was to examine the reliability and validity of the Fidelity Checklist, a measure designed to assess group leader adherence and competence delivering a parent training intervention (the Chicago Parent Program) in child care centers serving low-income families. Method The sample included 9 parent groups (12 group sessions each), 12 group leaders, and 103 parents. Independent raters reviewed 106 audiotaped parent group sessions and coded group leaders’ fidelity on the Adherence and Competence Scales of the Fidelity Checklist. Group leaders completed self-report adherence checklists and a measure of parent engagement in the intervention. Parents completed measures of consumer satisfaction and child behavior. Results High interrater agreement (Adherence Scale = 94%, Competence Scale = 85%) and adequate intraclass correlation coefficients (Adherence Scale = .69, Competence Scale = .91) were achieved for the Fidelity Checklist. Group leader adherence changed over time, but competence remained stable. Agreement between group leader self-report and independent ratings on the Adherence Scale was 85%; disagreements were more frequently due to positive bias in group leader self-report. Positive correlations were found between group leader adherence and parent attendance and engagement in the intervention and between group leader competence and parent satisfaction. Although child behavior problems improved, improvements were not related to fidelity. Discussion The results suggest that the Fidelity Checklist is a feasible, reliable, and valid measure of group leader implementation fidelity in a group-based parenting intervention. Future research will be focused on testing the Fidelity Checklist with diverse and larger samples and generalizing to other group-based interventions using a similar intervention model. PMID:20404777

Development and implementation of the Baltimore healthy carry-outs feasibility trial: process evaluation results

PubMed Central

2013-01-01

Background Prepared food sources, including fast food restaurants and carry-outs, are common in low-income urban areas. These establishments provide foods high in calories, sugar, fat, and sodium. The aims of the study were to (1) describe the development and implementation of a carry-out intervention to provide and promote healthy food choices in prepared food sources, and (2) to assess its feasibility through a process evaluation. Methods To promote healthy eating in this setting, a culturally appropriate intervention was developed based on formative research from direct observation, interviews and focus groups. We implemented a 7-month feasibility trial in 8 carry-outs (4 intervention and 4 comparison) in low-income neighborhoods in Baltimore, MD. The trial included three phases: 1) Improving menu boards and labeling to promote healthier items; 2) Promoting healthy sides and beverages and introducing new items; and 3) Introducing affordable healthier combo meals and improving food preparation methods. A process evaluation was conducted to assess intervention reach, dose received, and fidelity using sales receipts, carry-out visit observations, and an intervention exposure assessment. Results On average, Baltimore Healthy Carry-outs (BHC) increased customer reach at intervention carry-outs; purchases increased by 36.8% at the end of the study compared to baseline. Additionally, menu boards and labels were seen by 100.0% and 84.2% of individuals (n = 101), respectively, at study completion compared to baseline. Customers reported purchasing specific foods due to the presence of a photo on the menu board (65.3%) or menu labeling (42.6%), suggesting moderate to high dose received. Promoted entrée availability and revised menu and poster presence all demonstrated high fidelity and feasibility. Conclusions The results suggest that BHC is a culturally acceptable intervention. The program was also immediately adopted by the Baltimore City Food Policy Initiative as a city-wide intervention in its public markets. PMID:23837722

Development and implementation of the Baltimore healthy carry-outs feasibility trial: process evaluation results.

PubMed

Lee-Kwan, Seung Hee; Goedkoop, Sonja; Yong, Rachel; Batorsky, Benjamin; Hoffman, Vanessa; Jeffries, Jayne; Hamouda, Mohamed; Gittelsohn, Joel

2013-07-09

Prepared food sources, including fast food restaurants and carry-outs, are common in low-income urban areas. These establishments provide foods high in calories, sugar, fat, and sodium. The aims of the study were to (1) describe the development and implementation of a carry-out intervention to provide and promote healthy food choices in prepared food sources, and (2) to assess its feasibility through a process evaluation. To promote healthy eating in this setting, a culturally appropriate intervention was developed based on formative research from direct observation, interviews and focus groups. We implemented a 7-month feasibility trial in 8 carry-outs (4 intervention and 4 comparison) in low-income neighborhoods in Baltimore, MD. The trial included three phases: 1) Improving menu boards and labeling to promote healthier items; 2) Promoting healthy sides and beverages and introducing new items; and 3) Introducing affordable healthier combo meals and improving food preparation methods. A process evaluation was conducted to assess intervention reach, dose received, and fidelity using sales receipts, carry-out visit observations, and an intervention exposure assessment. On average, Baltimore Healthy Carry-outs (BHC) increased customer reach at intervention carry-outs; purchases increased by 36.8% at the end of the study compared to baseline. Additionally, menu boards and labels were seen by 100.0% and 84.2% of individuals (n = 101), respectively, at study completion compared to baseline. Customers reported purchasing specific foods due to the presence of a photo on the menu board (65.3%) or menu labeling (42.6%), suggesting moderate to high dose received. Promoted entrée availability and revised menu and poster presence all demonstrated high fidelity and feasibility. The results suggest that BHC is a culturally acceptable intervention. The program was also immediately adopted by the Baltimore City Food Policy Initiative as a city-wide intervention in its public markets.

Implementation of a Text-Based Content Intervention in Secondary Social Studies Classes.

PubMed

Wanzek, Jeanne; Vaughn, Sharon

2016-12-01

We describe teacher fidelity (adherence to the components of the treatment as specified by the research team) based on a series of studies of a multicomponent intervention, Promoting Acceleration of Comprehension and Content Through Text (PACT), with middle and high school social studies teachers and their students. Findings reveal that even with highly specified materials and implementing practices that are aligned with effective reading comprehension and content instruction, teachers' fidelity was consistently low for some components and high for others. Teachers demonstrated consistently high implementation fidelity and quality for the instructional components of building background knowledge (comprehension canopy) and teaching key content vocabulary (essential words), whereas we recorded consistently lower fidelity and quality of implementation for the instructional components of critical reading and knowledge application. © 2016 Wiley Periodicals, Inc.

The Camino Verde intervention in Nicaragua, 2004-2012.

PubMed

Arosteguí, Jorge; Ledogar, Robert J; Coloma, Josefina; Hernández-Alvarez, Carlos; Suazo-Laguna, Harold; Cárcamo, Alvaro; Reyes, Rosa María; Belli, Alejandro; Andersson, Neil; Harris, Eva

2017-05-30

Camino Verde (the Green Way) is an evidence-based community mobilisation tool for prevention of dengue and other mosquito-borne viral diseases. Its effectiveness was demonstrated in a cluster-randomised controlled trial conducted in 2010-2013 in Nicaragua and Mexico. The Nicaraguan arm of the trial was preceded, from 2004 to 2008, by a feasibility study that provided valuable lessons and trained facilitators for the trial itself. Here, guided by the Template for Intervention Description and Replication (TIDieR), we describe the Camino Verde intervention in Nicaragua, presenting its rationale, its time and location, activities, materials used, the main actors, modes of delivery, how it was tailored to encourage community engagement, modifications made from the feasibility study to the trial itself, and how fidelity to the process originally designed was maintained. We also present information on costs and discuss the place of this study within the literature on implementation science. ISRCTN27581154 .

A Taxonomy of Delivery and Documentation Deviations During Delivery of High-Fidelity Simulations.

PubMed

McIvor, William R; Banerjee, Arna; Boulet, John R; Bekhuis, Tanja; Tseytlin, Eugene; Torsher, Laurence; DeMaria, Samuel; Rask, John P; Shotwell, Matthew S; Burden, Amanda; Cooper, Jeffrey B; Gaba, David M; Levine, Adam; Park, Christine; Sinz, Elizabeth; Steadman, Randolph H; Weinger, Matthew B

2017-02-01

We developed a taxonomy of simulation delivery and documentation deviations noted during a multicenter, high-fidelity simulation trial that was conducted to assess practicing physicians' performance. Eight simulation centers sought to implement standardized scenarios over 2 years. Rules, guidelines, and detailed scenario scripts were established to facilitate reproducible scenario delivery; however, pilot trials revealed deviations from those rubrics. A taxonomy with hierarchically arranged terms that define a lack of standardization of simulation scenario delivery was then created to aid educators and researchers in assessing and describing their ability to reproducibly conduct simulations. Thirty-six types of delivery or documentation deviations were identified from the scenario scripts and study rules. Using a Delphi technique and open card sorting, simulation experts formulated a taxonomy of high-fidelity simulation execution and documentation deviations. The taxonomy was iteratively refined and then tested by 2 investigators not involved with its development. The taxonomy has 2 main classes, simulation center deviation and participant deviation, which are further subdivided into as many as 6 subclasses. Inter-rater classification agreement using the taxonomy was 74% or greater for each of the 7 levels of its hierarchy. Cohen kappa calculations confirmed substantial agreement beyond that expected by chance. All deviations were classified within the taxonomy. This is a useful taxonomy that standardizes terms for simulation delivery and documentation deviations, facilitates quality assurance in scenario delivery, and enables quantification of the impact of deviations upon simulation-based performance assessment.

Preschool children's development in classic Montessori, supplemented Montessori, and conventional programs.

PubMed

Lillard, Angeline S

2012-06-01

Research on the outcomes of Montessori education is scarce and results are inconsistent. One possible reason for the inconsistency is variations in Montessori implementation fidelity. To test whether outcomes vary according to implementation fidelity, we examined preschool children enrolled in high fidelity classic Montessori programs, lower fidelity Montessori programs that supplemented the program with conventional school activities, and, for comparison, conventional programs. Children were tested at the start and end of the school year on a range of social and academic skills. Although they performed no better in the fall, children in Classic Montessori programs, as compared with children in Supplemented Montessori and Conventional programs, showed significantly greater school-year gains on outcome measures of executive function, reading, math, vocabulary, and social problem-solving, suggesting that high fidelity Montessori implementation is associated with better outcomes than lower fidelity Montessori programs or conventional programs. Copyright © 2012 Society for the Study of School Psychology. Published by Elsevier Ltd. All rights reserved.

Improving occupational health care for construction workers: a process evaluation

PubMed Central

2013-01-01

Background To evaluate the process of a job-specific workers’ health surveillance (WHS) in improving occupational health care for construction workers. Methods From January to July 2012 were 899 bricklayers and supervisors invited for the job-specific WHS at three locations of one occupational health service throughout the Netherlands. The intervention aimed at detecting signs of work-related health problems, reduced work capacity and/or reduced work functioning. Measurements were obtained using a recruitment record and questionnaires at baseline and follow-up. The process evaluation included the following: reach (attendance rate), intervention dose delivered (provision of written recommendations and follow-up appointments), intervention dose received (intention to follow-up on advice directly after WHS and remembrance of advice three months later), and fidelity (protocol adherence). The workers scored their increase in knowledge from 0–10 with regard to health status and work ability, their satisfaction with the intervention and the perceived (future) effect of such an intervention. Program implementation was defined as the mean score of reach, fidelity, and intervention dose delivered and received. Results Reach was 9% (77 workers participated), fidelity was 67%, the intervention dose delivered was 92 and 63%, and the intervention dose received was 68 and 49%. The total programme implementation was 58%. The increases in knowledge regarding the health status and work ability of the workers after the WHS were graded as 7.0 and 5.9, respectively. The satisfaction of the workers with the entire intervention was graded as 7.5. The perceived (future) effects on health status were graded as 6.3, and the effects on work ability were graded with a 5.2. The economic recession affected the workers as well as the occupational health service that enacted the implementation. Conclusions Programme implementation was acceptable. Low reach, limited protocol adherence and modest engagement of the workers with respect to the intervention were the most prominent aspects that influenced the intervention process. The increase in the workers’ knowledge about their health status and work ability was substantial, and the workers’ satisfaction with the intervention was good. The perceived effect of the advised preventive actions on health status was sufficient. Trial registration Netherlands Trial Register: http://NTR3012 PMID:23497119

Testing the Feasibility of Fidelity Evaluation in a Multisite, Multiprogram Initiative

ERIC Educational Resources Information Center

Cornish, Disa Lubker; Losch, Mary E.; Avery, Mitchell

2016-01-01

Monitoring fidelity of implementation is a critical task when initiating evidence-based programs. This pilot study sought to identify best practices in a fidelity monitoring process and determine the feasibility of continuing a fidelity monitoring process with a multisite, multiprogram initiative. A fidelity log was created for each of 11…

What Is Essential in Developmental Evaluation? On Integrity, Fidelity, Adultery, Abstinence, Impotence, Long-Term Commitment, Integrity, and Sensitivity in Implementing Evaluation Models

ERIC Educational Resources Information Center

Patton, Michael Quinn

2016-01-01

Fidelity concerns the extent to which a specific evaluation sufficiently incorporates the core characteristics of the overall approach to justify labeling that evaluation by its designated name. Fidelity has traditionally meant implementing a model in exactly the same way each time following the prescribed steps and procedures. The essential…

Peer support to improve diabetes care: an implementation evaluation of the Australasian Peers for Progress Diabetes Program.

PubMed

Aziz, Zahra; Riddell, Michaela A; Absetz, Pilvikki; Brand, Margaret; Oldenburg, Brian

2018-02-17

Several studies have now demonstrated the benefits of peer support in promoting diabetes control. The aim of this study is to evaluate the implementation of a cluster randomised controlled trial of a group-based, peer support program to improve diabetes self-management and thereby, diabetes control in people with Type 2 Diabetes in Victoria, Australia. The intervention program was designed to address four key peer support functions i.e. 1) assistance in daily management, 2) social and emotional support, 3) regular linkage to clinical care, and 4) ongoing and sustained support to assist with the lifelong needs of diabetes self-care management. The intervention participants attended monthly group meetings facilitated by a trained peer leader for 12 months. Data was collected on the intervention's reach, participation, implementation fidelity, groups' effectiveness and participants' perceived support and satisfaction with the intervention. The RE-AIM and PIPE frameworks were used to guide this evaluation. The trial reached a high proportion (79%) of its target population through mailed invitations. Out of a total of 441 eligible individuals, 273 (61.9%) were willing to participate. The intervention fidelity was high (92.7%). The proportion of successful participants who demonstrated a reduction in 5 years cardiovascular disease risk score was 65.1 and 44.8% in the intervention and control arm respectively. Ninety-four percent (94%) of the intervention participants stated that the program helped them manage their diabetes on a day to day basis. Overall, attending monthly group meetings provided 'a lot of support' to 57% and 'moderate' support to 34% of the participants. Peer support programs are feasible, acceptable and can be used to supplement treatment for patients motivated to improve behaviours related to diabetes. However, program planners need to focus on the participation component in designing future programs. The use of two evaluation frameworks allowed a comprehensive evaluation of the trial from the provider-, participant- and public health perspective. The learnings gained from this evaluation will guide and improve future implementation by improving program feasibility for adoption and acceptability among participants, and will ultimately increase the likelihood of program effectiveness for the participants. Australian New Zealand Clinical Trials Registry (ANZCTR) ACTRN12609000469213 . Registered 16 June 2009.

MENOS4 trial: a multicentre randomised controlled trial (RCT) of a breast care nurse delivered cognitive behavioural therapy (CBT) intervention to reduce the impact of hot flushes in women with breast cancer: Study Protocol.

PubMed

Fenlon, Deborah; Nuttall, Jacqueline; May, Carl; Raftery, James; Fields, Jo; Kirkpatrick, Emma; Abab, Julia; Ellis, Mary; Rose, Taylor; Khambhaita, Priya; Galanopoulou, Angeliki; Maishman, Tom; Haviland, Jo; Griffiths, Gareth; Turner, Lesley; Hunter, Myra

2018-05-08

Women who have been treated for breast cancer may identify vasomotor symptoms, such as hot flushes and night sweats (HFNS), as a serious problem. HFNS are unpleasant to experience and can have a significant impact on daily life, potentially leading to reduced adherence to life saving adjuvant hormonal therapy. It is known that Cognitive Behavioural Therapy (CBT) is effective for the alleviation of hot flushes in both well women and women who have had breast cancer. Most women with breast cancer will see a breast care nurse and there is evidence that nurses can be trained to deliver psychological treatments to a satisfactory level, whilst also maintaining treatment fidelity. The research team will assess whether breast care nurses can effectively deliver a CBT intervention to alleviate hot flushes in women with breast cancer. This study is a multi-centre phase III individually randomised controlled trial of group CBT versus usual care to reduce the impact of hot flushes in women with breast cancer. 120-160 women with primary breast cancer experiencing seven or more problematic HFNS a week will be randomised to receive either treatment as usual (TAU) or participation in the group CBT intervention plus TAU (CBT Group). A process evaluation using May's Normalisation Process Theory will be conducted, as well as practical and organisational issues relating to the implementation of the intervention. Fidelity of implementation of the intervention will be conducted by expert assessment. The cost effectiveness of the intervention will also be assessed. There is a need for studies that enable effective interventions to be implemented in practice. There is good evidence that CBT is helpful for women with breast cancer who experience HFNS, yet it is not widely available. It is not yet known whether the intervention can be effectively delivered by breast care nurses or implemented in practice. This study will provide information on both whether the intervention can effectively help women with hot flushes and whether and how it can be translated into routine clinical practice. ISRCTN 12824632 . Registered 25-01-2017.

Implementation of integrated dual disorders treatment: a qualitative analysis of facilitators and barriers.

PubMed

Brunette, Mary F; Asher, Dianne; Whitley, Rob; Lutz, Wilma J; Wieder, Barbara L; Jones, Amanda M; McHugo, Gregory J

2008-09-01

Approximately half of the people who have serious mental illnesses experience a co-occurring substance use disorder at some point in their lifetime. Integrated dual disorders treatment, a program to treat persons with co-occurring disorders, improves outcomes but is not widely available in public mental health settings. This report describes the extent to which this intervention was implemented by 11 community mental health centers participating in a large study of practice implementation. Facilitators and barriers to implementation are described. Trained implementation monitors conducted regular site visits over two years. During visits, monitors interviewed key informants, conducted ethnographic observations of implementation efforts, and assessed fidelity to the practice model. These data were coded and used as a basis for detailed site reports summarizing implementation processes. The authors reviewed the reports and distilled the three top facilitators and barriers for each site. The most prominent cross-site facilitators and barriers were identified. Two sites reached high fidelity, six sites reached moderate fidelity, and three sites remained at low fidelity over the two years. Prominent facilitators and barriers to implementation with moderate to high fidelity were administrative leadership, consultation and training, supervisor mastery and supervision, chronic staff turnover, and finances. Common facilitators and barriers to implementation of integrated dual disorders treatment emerged across sites. The results confirmed the importance of the use of the consultant-trainer in the model of implementation, as well as the need for intensive activities at multiple levels to facilitate implementation. Further research on service implementation is needed, including but not limited to clarifying strategies to overcome barriers.

Improving practice in community-based settings: a randomized trial of supervision - study protocol.

PubMed

Dorsey, Shannon; Pullmann, Michael D; Deblinger, Esther; Berliner, Lucy; Kerns, Suzanne E; Thompson, Kelly; Unützer, Jürgen; Weisz, John R; Garland, Ann F

2013-08-10

Evidence-based treatments for child mental health problems are not consistently available in public mental health settings. Expanding availability requires workforce training. However, research has demonstrated that training alone is not sufficient for changing provider behavior, suggesting that ongoing intervention-specific supervision or consultation is required. Supervision is notably under-investigated, particularly as provided in public mental health. The degree to which supervision in this setting includes 'gold standard' supervision elements from efficacy trials (e.g., session review, model fidelity, outcome monitoring, skill-building) is unknown. The current federally-funded investigation leverages the Washington State Trauma-focused Cognitive Behavioral Therapy Initiative to describe usual supervision practices and test the impact of systematic implementation of gold standard supervision strategies on treatment fidelity and clinical outcomes. The study has two phases. We will conduct an initial descriptive study (Phase I) of supervision practices within public mental health in Washington State followed by a randomized controlled trial of gold standard supervision strategies (Phase II), with randomization at the clinician level (i.e., supervisors provide both conditions). Study participants will be 35 supervisors and 130 clinicians in community mental health centers. We will enroll one child per clinician in Phase I (N = 130) and three children per clinician in Phase II (N = 390). We use a multi-level mixed within- and between-subjects longitudinal design. Audio recordings of supervision and therapy sessions will be collected and coded throughout both phases. Child outcome data will be collected at the beginning of treatment and at three and six months into treatment. This study will provide insight into how supervisors can optimally support clinicians delivering evidence-based treatments. Phase I will provide descriptive information, currently unavailable in the literature, about commonly used supervision strategies in community mental health. The Phase II randomized controlled trial of gold standard supervision strategies is, to our knowledge, the first experimental study of gold standard supervision strategies in community mental health and will yield needed information about how to leverage supervision to improve clinician fidelity and client outcomes. ClinicalTrials.gov NCT01800266.

Process evaluation of an environmental health risk audit and action plan intervention to reduce alcohol related violence in licensed premises.

PubMed

Williams, Annie; Moore, Simon C; Shovelton, Claire; Moore, Laurence; Murphy, Simon

2016-05-28

Alcohol-related violence is associated with licensed premise environments and their management. There is a lack of evidence for effective interventions to address these, and there are significant barriers to implementation. This study aims to understand how development and implementation processes can facilitate intervention reach, fidelity and receipt and therefore provides key process data necessary to interpret the results of the randomised controlled trial conducted in parallel. A process evaluation, embedded within a randomised controlled trial. Intervention development and implementation were assessed via focus groups (n = 2) and semi-structured interviews (n = 22) with Environmental Health Practitioners (EHPs). Reach and fidelity were assessed via routinely collected intervention data, which was was collected from 276 licenced premises across Wales, UK. Case study semi-structured interviews with licensed premises proprietors (n = 30) explored intervention receipt. Intervention co-production with senior EHPs facilitated organisational adoption and implementation. Training events for EHPs played an important role in addressing wider organisational concerns regarding partnership working and the contextual integration of the intervention. EHPs delivered the intervention to 98 % of intervention premises; 35 % of premises should have received a follow up enforcement visit, however EHP confidence in dealing with alcohol risk factors meant only 7 % of premises received one. Premises therefore received a similar intervention dose regardless of baseline risk. Intervention receipt appeared to be greatest in premises with an existing commitment to prevention and those in urban environments. The study suggests that a collaborative approach to the development and diffusion of interventions is associated with high levels of organisational adoption, implementation and reach. However, the lack of enforcement visits represents implementation failure for a key mechanism of action that is likely to influence intervention effectiveness. To be effective, any future intervention may require a longer implementation period to develop EHP confidence in using enforcement approaches in this area and multiagency enforcement support, which includes the police, to deliver an adequate intervention dose.

A computerized decision support system for depression in primary care.

PubMed

Kurian, Benji T; Trivedi, Madhukar H; Grannemann, Bruce D; Claassen, Cynthia A; Daly, Ella J; Sunderajan, Prabha

2009-01-01

In 2004, results from The Texas Medication Algorithm Project (TMAP) showed better clinical outcomes for patients whose physicians adhered to a paper-and-pencil algorithm compared to patients who received standard clinical treatment for major depressive disorder (MDD). However, implementation of and fidelity to the treatment algorithm among various providers was observed to be inadequate. A computerized decision support system (CDSS) for the implementation of the TMAP algorithm for depression has since been developed to improve fidelity and adherence to the algorithm. This was a 2-group, parallel design, clinical trial (one patient group receiving MDD treatment from physicians using the CDSS and the other patient group receiving usual care) conducted at 2 separate primary care clinics in Texas from March 2005 through June 2006. Fifty-five patients with MDD (DSM-IV criteria) with no significant difference in disease characteristics were enrolled, 32 of whom were treated by physicians using CDSS and 23 were treated by physicians using usual care. The study's objective was to evaluate the feasibility and efficacy of implementing a CDSS to assist physicians acutely treating patients with MDD compared to usual care in primary care. Primary efficacy outcomes for depression symptom severity were based on the 17-item Hamilton Depression Rating Scale (HDRS(17)) evaluated by an independent rater. Patients treated by physicians employing CDSS had significantly greater symptom reduction, based on the HDRS(17), than patients treated with usual care (P < .001). The CDSS algorithm, utilizing measurement-based care, was superior to usual care for patients with MDD in primary care settings. Larger randomized controlled trials are needed to confirm these findings. clinicaltrials.gov Identifier: NCT00551083.

Measuring the Implementation Fidelity of Student Affairs Programs: A Critical Component of the Outcomes Assessment Cycle

ERIC Educational Resources Information Center

Gerstner, Jerusha J.; Finney, Sara J.

2013-01-01

Implementation fidelity assessment provides a means of measuring the alignment between the planned program and the implemented program. Unfortunately, the implemented program can differ from the planned program, resulting in ambiguous inferences about the planned program's effectiveness (i.e., it is uncertain if poor results are due to an…

Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: a cluster randomized trial study protocol.

PubMed

Aarons, Gregory A; Ehrhart, Mark G; Moullin, Joanna C; Torres, Elisa M; Green, Amy E

2017-03-03

Evidence-based practice (EBP) implementation represents a strategic change in organizations that requires effective leadership and alignment of leadership and organizational support across organizational levels. As such, there is a need for combining leadership development with organizational strategies to support organizational climate conducive to EBP implementation. The leadership and organizational change for implementation (LOCI) intervention includes leadership training for workgroup leaders, ongoing implementation leadership coaching, 360° assessment, and strategic planning with top and middle management regarding how they can support workgroup leaders in developing a positive EBP implementation climate. This test of the LOCI intervention will take place in conjunction with the implementation of motivational interviewing (MI) in 60 substance use disorder treatment programs in California, USA. Participants will include agency executives, 60 program leaders, and approximately 360 treatment staff. LOCI will be tested using a multiple cohort, cluster randomized trial that randomizes workgroups (i.e., programs) within agency to either LOCI or a webinar leadership training control condition in three consecutive cohorts. The LOCI intervention is 12 months, and the webinar control intervention takes place in months 1, 5, and 8, for each cohort. Web-based surveys of staff and supervisors will be used to collect data on leadership, implementation climate, provider attitudes, and citizenship. Audio recordings of counseling sessions will be coded for MI fidelity. The unit of analysis will be the workgroup, randomized by site within agency and with care taken that co-located workgroups are assigned to the same condition to avoid contamination. Hierarchical linear modeling (HLM) will be used to analyze the data to account for the nested data structure. LOCI has been developed to be a feasible and effective approach for organizations to create a positive climate and fertile context for EBP implementation. The approach seeks to cultivate and sustain both effective general and implementation leadership as well as organizational strategies and support that will remain after the study has ended. Development of a positive implementation climate for MI should result in more positive service provider attitudes and behaviors related to the use of MI and, ultimately, higher fidelity in the use of MI. This study is registered with Clinicaltrials.gov ( NCT03042832 ), 2 February 2017, retrospectively registered.

Evaluating the Implementation of the Empowering Coaching™ Programme: Balancing Fidelity and Adaptation

ERIC Educational Resources Information Center

Søvik, Margaret L.; Larsen, Torill; Tjomsland, Hege; Samdal, Oddrun

2016-01-01

Purpose: The purpose of this paper is to explore the implementation of a theoretically grounded coach education training programme for youth football coaches in Norway, through observational methods. In particular, it focuses on implementation fidelity and programme adaptation, and possible differences between the coach educators (CEs) according…

A "Common Factors" Approach to Developing Culturally Tailored HIV Prevention Interventions

ERIC Educational Resources Information Center

Owczarzak, Jill; Phillips, Sarah D.; Filippova, Olga; Alpatova, Polina; Mazhnaya, Alyona; Zub, Tatyana; Aleksanyan, Ruzanna

2016-01-01

The current dominant model of HIV prevention intervention dissemination involves packaging interventions developed in one context, training providers to implement that specific intervention, and evaluating the extent to which providers implement it with fidelity. Research shows that providers rarely implement these programs with fidelity due to…

Implementation Challenges for a Constructivist Physical Education Curriculum

ERIC Educational Resources Information Center

Zhu, Xihe; Ennis, Catherine D.; Chen, Ang

2011-01-01

Background: Curriculum fidelity describes the extent to which a curriculum is implemented faithfully as planned. Curriculum fidelity issues may arise when teachers implement the curriculum inconsistently due to differences in philosophy, barriers in the setting, or other local concerns. Purpose: The study examined challenges that a teacher faced…

Implementing a high-fidelity simulation program in a community college setting.

PubMed

Tuoriniemi, Pamela; Schott-Baer, Darlene

2008-01-01

Despite their relatively high cost, there is heightened interest by faculty in undergraduate nursing programs to implement high-fidelity simulation (HFS) programs. High-fidelity simulators are appealing because they allow students to experience high-risk, low-volume patient problems in a realistic setting. The decision to purchase a simulator is the first step in the process of implementing and maintaining an HFS lab. Knowledge, technical skill, commitment, and considerable time are needed to develop a successful program. The process, as experienced by one community college nursing program, is described.

Implementation of The World Starts With Me, a comprehensive rights-based sex education programme in Uganda.

PubMed

Rijsdijk, Liesbeth E; Bos, Arjan E R; Lie, Rico; Leerlooijer, Joanne N; Eiling, Ellen; Atema, Vera; Gebhardt, Winifred A; Ruiter, Robert A C

2014-04-01

This article presents a process evaluation of the implementation of the sex education programme the World Starts With Me (WSWM) for secondary school students in Uganda. The purpose of this mixed-methods study was to examine factors associated with dose delivered (number of lessons implemented) and fidelity of implementation (implementation according to the manual), as well as to identify the main barriers and facilitators of implementation. Teachers' confidence in teaching WSWM was negatively associated with dose delivered. Confidence in educating and discussing sexuality issues in class was positively associated with fidelity of implementation, whereas the importance teachers attached to open sex education showed a negative association with fidelity. Main barriers for implementing WSWM were lack of time, unavailability of computers, lack of student manuals and lack of financial support and rewards. Other barriers for successful implementation were related to high turnover of staff and insufficient training and guidance of teachers. Teachers' beliefs/attitudes towards sexuality of adolescents, condom use and sex education were found to be important socio-cognitive factors intervening with full fidelity of implementation. These findings can be used to improve the intervention implementation and to better plan for large-scale dissemination of school-based sex education programmes in sub-Saharan Africa.

Enhancing Therapeutic Gains: Examination of Fidelity to the Model for the Intensive Mental Health Program

ERIC Educational Resources Information Center

Randall, Camille J.; Biggs, Bridget K.

2008-01-01

Given that the development of treatment fidelity assessment protocol is an integral but too frequently ignored aspect of clinical trials for psychological treatments, the Intensive Mental Health Program (IMHP) sought to build fidelity activities into training, program evaluation, and clinical recordkeeping from the outset of a 3 year study period.…

High versus Low Theoretical Fidelity Pedometer Intervention Using Social-Cognitive Theory on Steps and Self-Efficacy

ERIC Educational Resources Information Center

Raedeke, Thomas D.; Dlugonski, Deirdre

2017-01-01

Purpose: This study was designed to compare a low versus high theoretical fidelity pedometer intervention applying social-cognitive theory on step counts and self-efficacy. Method: Fifty-six public university employees participated in a 10-week randomized controlled trial with 2 conditions that varied in theoretical fidelity. Participants in the…

Value-based payment in implementing evidence-based care: the Mental Health Integration Program in Washington state.

PubMed

Bao, Yuhua; McGuire, Thomas G; Chan, Ya-Fen; Eggman, Ashley A; Ryan, Andrew M; Bruce, Martha L; Pincus, Harold Alan; Hafer, Erin; Unützer, Jürgen

2017-01-01

To assess the role of value-based payment (VBP) in improving fidelity and patient outcomes in community implementation of an evidence-based mental health intervention, the Collaborative Care Model (CCM). Retrospective study based on a natural experiment. We used the clinical tracking data of 1806 adult patients enrolled in a large implementation of the CCM in community health clinics in Washington state. VBP was initiated in year 2 of the program, creating a natural experiment. We compared implementation fidelity (measured by 3 process-of-care elements of the CCM) between patient-months exposed to VBP and patient-months not exposed to VBP. A series of regressions were estimated to check robustness of findings. We estimated a Cox proportional hazard model to assess the effect of VBP on time to achieving clinically significant improvement in depression (measured based on changes in depression symptom scores over time). Estimated marginal effects of VBP on fidelity ranged from 9% to 30% of the level of fidelity had there been no exposure to VBP (P <.05 for every fidelity measure). Improvement in fidelity in response to VBP was greater among providers with a larger patient panel and among providers with a lower level of fidelity at baseline. Exposure to VBP was associated with an adjusted hazard ratio of 1.45 (95% confidence interval, 1.04-2.03) for achieving clinically significant improvement in depression. VBP improved fidelity to key elements of the CCM, both directly incentivized and not explicitly incentivized by the VBP, and improved patient depression outcomes.

A Hybrid Process Fidelity Assessment in a Home-based Randomized Clinical Trial

PubMed Central

WILDE, MARY H.; LIEBEL, DIANNE; FAIRBANKS, EILEEN; WILSON, PAULA; LASH, MARGARET; SHAH, SHIVANI; McDONALD, MARGARET V.; BRASCH, JUDITH; ZHANG, FENG; SCHEID, EILEEN; McMAHON, JAMES M.

2016-01-01

A process fidelity assessment was conducted as a nested study within a home-based randomized clinical trial teaching self-management to 101 long-term indwelling urinary catheter users in the treatment group. Our hybrid model combined external assessments (outside observations and tape recordings) with internal evaluation methods (through study nurse forms and notes) for a comprehensive process fidelity assessment. Barriers, patient-related issues, and nurse perspectives were identified demonstrating the complexity in home care intervention research. The complementary and synergistic approaches provided in depth information about the context of the delivery and the impact of the intervention on study outcomes. PMID:25894688

Defining, Designing, Implementing, and Evaluating Phase 4 HIV Prevention Effectiveness Trials for Vulnerable Populations

PubMed Central

Kelly, Jeffrey A.; Spielberg, Freya; McAuliffe, Timothy L.

2008-01-01

Summary The efficacy of behavioral HIV prevention interventions has been convincingly demonstrated in a large number of randomized controlled phase 3 research outcome trials. Little research attention has been directed toward studying the effectiveness of the same interventions when delivered by providers to their own clients or community members, however. This article argues for the need to conduct phase 4 effectiveness trials of HIV prevention interventions that have been found efficacious in the research arena. Such trials can provide important information concerning the impact of interventions when applied in heterogeneous “real-world” circumstances. This article raises design issues and methodologic questions that need to be addressed in the conduct of phase 4 trials of behavioral interventions. These issues include the selection and training of service providers engaged in such trials, maintenance of fidelity to intervention protocol in provider-delivered interventions, determination of intervention core elements versus aspects that require tailoring, selection of relevant phase 4 study outcomes, interpretation of findings indicative of field effectiveness, sustainability, and other aspects of phase 4 trial design. PMID:18301131

Long Live Love+: Evaluation of the Implementation of an Online School-Based Sexuality Education Program in the Netherlands

ERIC Educational Resources Information Center

van Lieshout, Sanne; Mevissen, Fraukje; de Waal, Esri; Kok, Gerjo

2017-01-01

Schools are a common setting for adolescents to receive health education, but implementation of these programs with high levels of completeness and fidelity is not self-evident. Programs that are only partially implemented (completeness) or not implemented as instructed (fidelity) are unlikely to be effective. Therefore, it is important to…

Effectiveness of a multi-level implementation strategy for ASD interventions: study protocol for two linked cluster randomized trials.

PubMed

Brookman-Frazee, Lauren; Stahmer, Aubyn C

2018-05-09

The Centers for Disease Control (2018) estimates that 1 in 59 children has autism spectrum disorder, and the annual cost of ASD in the U.S. is estimated to be $236 billion. Evidence-based interventions have been developed and demonstrate effectiveness in improving child outcomes. However, research on generalizable methods to scale up these practices in the multiple service systems caring for these children has been limited and is critical to meet this growing public health need. This project includes two, coordinated studies testing the effectiveness of the Translating Evidence-based Interventions (EBI) for ASD: Multi-Level Implementation Strategy (TEAMS) model. TEAMS focuses on improving implementation leadership, organizational climate, and provider attitudes and motivation in order to improve two key implementation outcomes-provider training completion and intervention fidelity and subsequent child outcomes. The TEAMS Leadership Institute applies implementation leadership strategies and TEAMS Individualized Provider Strategies for training applies motivational interviewing strategies to facilitate provider and organizational behavior change. A cluster randomized implementation/effectiveness Hybrid, type 3, trial with a dismantling design will be used to understand the effectiveness of TEAMS and the mechanisms of change across settings and participants. Study #1 will test the TEAMS model with AIM HI (An Individualized Mental Health Intervention for ASD) in publicly funded mental health services. Study #2 will test TEAMS with CPRT (Classroom Pivotal Response Teaching) in education settings. Thirty-seven mental health programs and 37 school districts will be randomized, stratified by county and study, to one of four groups (Standard Provider Training Only, Standard Provider Training + Leader Training, Enhanced Provider Training, Enhanced Provider Training + Leader Training) to test the effectiveness of combining standard, EBI-specific training with the two TEAMS modules individually and together on multiple implementation outcomes. Implementation outcomes including provider training completion, fidelity (coded by observers blind to group assignment) and child behavior change will be examined for 295 mental health providers, 295 teachers, and 590 children. This implementation intervention has the potential to increase quality of care for ASD in publicly funded settings by improving effectiveness of intervention implementation. The process and modules will be generalizable to multiple service systems, providers, and interventions, providing broad impact in community services. This study is registered with Clinicaltrials.gov ( NCT03380078 ). Registered 20 December 2017, retrospectively registered.

Study protocol: Hybrid Type I cost-effectiveness and implementation study of interpersonal psychotherapy (IPT) for men and women prisoners with major depression

PubMed Central

Miller, Ted R.; Stout, Robert L.; Zlotnick, Caron; Cerbo, Louis A.; Andrade, Joel T.; Wiltsey-Stirman, Shannon

2016-01-01

Purpose This article describes the protocol for a Hybrid Type I cost-effectiveness and implementation study of interpersonal psychotherapy (IPT) for men and women prisoners with major depressive disorder (MDD). The goal is to promote uptake of evidence-based treatments in criminal justice settings by conducting a randomized effectiveness study that collects implementation data, including a full cost-effectiveness analysis. Background More than 2.3 million people are incarcerated in the United States on any given day. MDD is the most common severe mental illness among incarcerated individuals. Despite the prevalence and consequences of MDD among incarcerated populations, this study will be the first fully-powered randomized trial of any treatment for MDD in an incarcerated population. Design Given the politically charged nature of the justice system, advantageous health outcomes are often not enough to get an intervention implemented in prisons. To increase the policy impact of this trial, we sought advice from prison providers and administrators about outcomes that would be persuasive to policy-makers and defensible to the public. In this trial, effectiveness questions will be answered using a randomized clinical trial design comparing IPT plus prison treatment as usual (TAU) to TAU alone, with outcomes including depressive symptoms (primary), suicidality, and in prison functioning (enrollment and completion of correctional programs; disciplinary and incident reports; aggression/victimization; social support). Implementation outcomes will include cost-effectiveness; feasibility and acceptability of IPT to clients, providers, and administrators; prison provider intervention fidelity, attitudes, and competencies; and barriers and facilitators of implementation assessed through surveys, interviews, and process notes. PMID:26845030

The relationship between school-level characteristics and implementation fidelity of a coordinated school health childhood obesity prevention intervention.

PubMed

Lederer, Alyssa M; King, Mindy H; Sovinski, Danielle; Seo, Dong-Chul; Kim, Nayoung

2015-01-01

Curtailing childhood obesity is a public health imperative. Although multicomponent school-based programs reduce obesity among children, less is known about the implementation fidelity of these interventions. This study examines process evaluation findings for the Healthy, Energetic Ready, Outstanding, Enthusiastic, Schools (HEROES) Initiative, a tri-state school-based childhood obesity prevention intervention based on the coordinated school health (CSH) model. Site visits were conducted that included key stakeholder interviews, observation, and document review. Scores were given for 8 domains, and a total implementation score was calculated. Two-way analyses of variance were conducted to examine the relationship of 4 school-level characteristics: elementary vs. middle/high schools, public vs. private schools, district vs. building level implementation, and socioeconomic status on each implementation area. Overall, schools had high fidelity scores, although some domains were implemented more successfully than others. Three school-level characteristics were associated with 1 or more domains, with elementary schools and schools implementing at the building level consistently having higher implementation scores than their counterparts. Process evaluation findings provide insight into successes and challenges schools implementing the CSH approach may encounter. Although preliminary, these findings on school-level characteristics establish a new area of research related to school-based childhood obesity prevention programs' implementation fidelity. © 2014, American School Health Association.

Fidelity of Implementation: Bringing Written Curriculum Materials into the Equation

ERIC Educational Resources Information Center

Superfine, Alison Castro; Marshall, Anne Marie; Kelso, Cathy

2015-01-01

Fidelity of curriculum implementation (FOI) is an important area of research because of the critical role it plays in understanding how and why curriculum materials work and how they can be improved. This analysis explores written features within the Math Trailblazers curriculum that may influence the ways teachers implement mathematics curriculum…

Administrator Strategies that Support High Fidelity Implementation of the Pyramid Model for Promoting Social-Emotional Competence & Addressing Challenging Behavior

ERIC Educational Resources Information Center

Mincic, Melissa; Smith, Barbara J.; Strain, Phil

2009-01-01

Implementing the Pyramid Model with fidelity and achieving positive outcomes for children and their families requires that administrators understand their roles in the implementation process. Every administrative decision impacts program quality and sustainability. This Policy Brief underscores the importance of facilitative administrative…

Within-Year Fidelity Growth of SWPBIS during Installation and Initial Implementation

ERIC Educational Resources Information Center

Schaper, Andrew; McIntosh, Kent; Hoselton, Robert

2016-01-01

The purpose of this study was to document within-year fidelity growth during installation and initial implementation of School-Wide Positive Behavioral Interventions and Supports (SWPBIS). Participants included school teams from schools throughout the United States that were in years 1 to 4 of SWPBIS implementation and routinely evaluated their…

Within-Year Fidelity Growth of SWPBIS during Installation and Initial Implementation

ERIC Educational Resources Information Center

Schaper, Andrew; McIntosh, Kent; Hoselton, Robert

2016-01-01

The purpose of this study was to document within-year fidelity growth during installation and initial implementation of School-Wide Positive Behavioral Interventions and Supports (SWPBIS). Participants included school teams from schools throughout the United States that were in years one to four of SWPBIS implementation and routinely evaluated…

Factors contributing to intervention fidelity in a multi-site chronic disease self-management program.

PubMed

Perrin, Karen M; Burke, Somer Goad; O'Connor, Danielle; Walby, Gary; Shippey, Claire; Pitt, Seraphine; McDermott, Robert J; Forthofer, Melinda S

2006-10-26

Disease self-management programs have been a popular approach to reducing morbidity and mortality from chronic disease. Replicating an evidence-based disease management program successfully requires practitioners to ensure fidelity to the original program design. The Florida Health Literacy Study (FHLS) was conducted to investigate the implementation impact of the Pfizer, Inc. Diabetes Mellitus and Hypertension Disease Self-Management Program based on health literacy principles in 14 community health centers in Florida. The intervention components discussed include health educator recruitment and training, patient recruitment, class sessions, utilization of program materials, translation of program manuals, patient retention and follow-up, and technical assistance. This report describes challenges associated with achieving a balance between adaptation for cultural relevance and fidelity when implementing the health education program across clinic sites. This balance was necessary to achieve effectiveness of the disease self-management program. The FHLS program was implemented with a high degree of fidelity to the original design and used original program materials. Adaptations identified as advantageous to program participation are discussed, such as implementing alternate methods for recruiting patients and developing staff incentives for participation. Effective program implementation depends on the talent, skill and willing participation of clinic staff. Program adaptations that conserve staff time and resources and recognize their contribution can increase program effectiveness without jeopardizing its fidelity.

Measuring the Implementation Fidelity of the Response to Intervention Framework in Milwaukee Public Schools. Stated Briefly. REL 2017-192

ERIC Educational Resources Information Center

Ruffini, Steffen J.; Lindsay, Jim; Miskell, Ryan; Proger, Amy

2016-01-01

Regional Educational Laboratory Midwest assisted Milwaukee Public Schools in developing a fidelity monitoring system for measuring schools' progress in implementing Response to Intervention (RTI). The study examined the ratings produced by that system to determine the system's reliability, schools' progress in implementing RTI, and whether ratings…

A Preliminary Investigation of Using School-Based Coaches to Support Intervention Fidelity of a Classwide Behavior Management Program

ERIC Educational Resources Information Center

Gilmour, Allison F.; Wehby, Joseph H.; McGuire, Terrell M.

2017-01-01

Many schools are beginning to implement universal behavior management interventions as part of multitiered systems of support. Past research suggests that teachers need support to implement new practices with fidelity. Coaching is one method for supporting implementation, but little is understood about how coaching takes place when relying on…

An Explanatory Mixed-Methods Approach to Tracing "Career Pathways" Policy in Virginia: How School Counselors and Student Demographics Influence Implementation Fidelity

ERIC Educational Resources Information Center

Ormsmith, Michael Isaac

2014-01-01

This explanatory mixed-methods policy analysis describes how school counselors' thoughts and attitudes contribute to the implementation fidelity of the Academic and Career Plan (ACP) policy in a suburban Virginia school division. A quantitative survey investigated counselor thoughts about the policy, implementation behaviors, and counselor ideas…

Implementation Fidelity of MyTeachingPartner Literacy and Language Activities: Association with Preschoolers' Language and Literacy Growth

ERIC Educational Resources Information Center

Hamre, Bridget K.; Justice, Laura M.; Pianta, Robert C.; Kilday, Carolyn; Sweeney, Beverly; Downer, Jason T.; Leach, Allison

2010-01-01

There is surprisingly little empirical research examining issues of fidelity of implementation within the early childhood education literature. In the MyTeachingPartner project, 154 teachers were provided with materials to implement a supplemental classroom curriculum addressing six aspects of literacy and language development. The present study…

Teacher Agency and Professional Learning: Rethinking Fidelity of Implementation as Multiplicities of Enactment

ERIC Educational Resources Information Center

Buxton, Cory A.; Allexsaht-Snider, Martha; Kayumova, Shakhnoza; Aghasaleh, Rouhollah; Choi, Youn-Jeng; Cohen, Allan

2015-01-01

In this paper we use practice theory, with its focus on the interplay of structure and agency, to theorize about teacher engagement in professional learning and teacher enactment of pedagogical practices as an alternative to framing implementation research in terms of program adherence and fidelity of implementation. Practice theory allowed us to…

Classroom characteristics and implementation of a substance use prevention curriculum in European countries.

PubMed

Caria, Maria Paola; Faggiano, Fabrizio; Bellocco, Rino; Galanti, Maria Rosaria

2013-12-01

Partial implementation may explain modest effectiveness of many school-based preventive programmes against substance use. We studied whether specific characteristics of the class could predict the level of implementation of a curriculum delivered by class teachers in schools from some European countries. Secondary analysis of data from an evaluation trial. In seven European countries, 78 schools (173 classes) were randomly assigned to a 12-unit, interactive, standardized programme based on the comprehensive social influence model. Curriculum completeness, application fidelity, average unit duration and use of role-play were monitored using structured report forms. Predictors of implementation were measured by aggregating at class level information from the baseline student survey. Class size, gender composition, mean age, factors related to substance use and to affection to school were analysed, with associations estimated by multilevel regression models. Implementation was not significantly predicted by mean age, proportion of students with positive academic expectation or liking school. Proportion of boys was associated with a shorter time devoted to each unit [β = -0.19, 95% confidence intervals (CI) -0.32 to -0.06]. Class size was inversely related to application fidelity [Odds ratio (OR) 0.92, 95% CI 0.85 to 0.99]. Prevalence of substance use was associated with a decreased odds of implementing all the curriculum units (OR 0.81, 95% CI 0.65 to 0.99). Students' connectedness to their class was associated with increased odds of teachers using role-play (OR 1.52, 95% CI 1.03 to 2.29). Teachers' implementation of preventive programmes may be affected by structural and social characteristics of classes and therefore benefit from organizational strategies and teachers' training in class management techniques.

The Role of School Context in Implementing a Statewide Anti-Bullying Policy and Protecting Students

PubMed Central

Hall, William J.; Chapman, Mimi V.

2016-01-01

Bullying is a significant problem in U.S. schools. Policies have been developed to reduce bullying, yet policy implementation by educators is an essential yet difficult and complex process. Few studies have investigated factors that act as barriers to or facilitators of bullying policy implementation and teacher protection of students. This study examined the influence of school context on educators’ capacity to implement a statewide bullying law and protect students from bullying following the enactment of the policy. Data were collected from 505 educators in 324 schools. School administrators tended to rate fidelity of policy implementation and teacher protection of students higher than teachers, education support professionals, and student service professionals. Policy implementation fidelity scores were higher in high schools than elementary schools. School size and the prevalence of student suspensions were inversely related to implementation fidelity. Higher levels of teacher protection were reported in elementary schools.

The Role of School Context in Implementing a Statewide Anti-Bullying Policy and Protecting Students.

PubMed

Hall, William J; Chapman, Mimi V

2018-06-01

Bullying is a significant problem in U.S. schools. Policies have been developed to reduce bullying, yet policy implementation by educators is an essential yet difficult and complex process. Few studies have investigated factors that act as barriers to or facilitators of bullying policy implementation and teacher protection of students. This study examined the influence of school context on educators' capacity to implement a statewide bullying law and protect students from bullying following the enactment of the policy. Data were collected from 505 educators in 324 schools. School administrators tended to rate fidelity of policy implementation and teacher protection of students higher than teachers, education support professionals, and student service professionals. Policy implementation fidelity scores were higher in high schools than elementary schools. School size and the prevalence of student suspensions were inversely related to implementation fidelity. Higher levels of teacher protection were reported in elementary schools.

Protocol for process evaluation of a randomised controlled trial of family-led rehabilitation post stroke (ATTEND) in India

PubMed Central

Liu, Hueiming; Lindley, Richard; Alim, Mohammed; Felix, Cynthia; Gandhi, Dorcas B C; Verma, Shweta J; Tugnawat, Deepak Kumar; Syrigapu, Anuradha; Ramamurthy, Ramaprabhu Krishnappa; Pandian, Jeyaraj D; Walker, Marion; Forster, Anne; Anderson, Craig S; Langhorne, Peter; Murthy, Gudlavalleti Venkata Satyanarayana; Shamanna, Bindiganavale Ramaswamy; Hackett, Maree L; Maulik, Pallab K; Harvey, Lisa A; Jan, Stephen

2016-01-01

Introduction We are undertaking a randomised controlled trial (fAmily led rehabiliTaTion aftEr stroke in INDia, ATTEND) evaluating training a family carer to enable maximal rehabilitation of patients with stroke-related disability; as a potentially affordable, culturally acceptable and effective intervention for use in India. A process evaluation is needed to understand how and why this complex intervention may be effective, and to capture important barriers and facilitators to its implementation. We describe the protocol for our process evaluation to encourage the development of in-process evaluation methodology and transparency in reporting. Methods and analysis The realist and RE-AIM (Reach, Effectiveness, Adoption, Implementation and Maintenance) frameworks informed the design. Mixed methods include semistructured interviews with health providers, patients and their carers, analysis of quantitative process data describing fidelity and dose of intervention, observations of trial set up and implementation, and the analysis of the cost data from the patients and their families perspective and programme budgets. These qualitative and quantitative data will be analysed iteratively prior to knowing the quantitative outcomes of the trial, and then triangulated with the results from the primary outcome evaluation. Ethics and dissemination The process evaluation has received ethical approval for all sites in India. In low-income and middle-income countries, the available human capital can form an approach to reducing the evidence practice gap, compared with the high cost alternatives available in established market economies. This process evaluation will provide insights into how such a programme can be implemented in practice and brought to scale. Through local stakeholder engagement and dissemination of findings globally we hope to build on patient-centred, cost-effective and sustainable models of stroke rehabilitation. Trial registration number CTRI/2013/04/003557. PMID:27633636

Evaluating the Accuracy of Results for Teacher Implemented Trial-Based Functional Analyses.

PubMed

Rispoli, Mandy; Ninci, Jennifer; Burke, Mack D; Zaini, Samar; Hatton, Heather; Sanchez, Lisa

2015-09-01

Trial-based functional analysis (TBFA) allows for the systematic and experimental assessment of challenging behavior in applied settings. The purposes of this study were to evaluate a professional development package focused on training three Head Start teachers to conduct TBFAs with fidelity during ongoing classroom routines. To assess the accuracy of the TBFA results, the effects of a function-based intervention derived from the TBFA were compared with the effects of a non-function-based intervention. Data were collected on child challenging behavior and appropriate communication. An A-B-A-C-D design was utilized in which A represented baseline, and B and C consisted of either function-based or non-function-based interventions counterbalanced across participants, and D represented teacher implementation of the most effective intervention. Results showed that the function-based intervention produced greater decreases in challenging behavior and greater increases in appropriate communication than the non-function-based intervention for all three children. © The Author(s) 2015.

Designing, testing, and implementing a sustainable nurse home visiting program: right@home.

PubMed

Goldfeld, Sharon; Price, Anna; Kemp, Lynn

2018-05-01

Nurse home visiting (NHV) offers a potential platform to both address the factors that limit access to services for families experiencing adversity and provide effective interventions. Currently, the ability to examine program implementation is hampered by a lack of detailed description of actual, rather than expected, program development and delivery in published studies. Home visiting implementation remains a black box in relation to quality and sustainability. However, previous literature would suggest that efforts to both report and improve program implementation are vital for NHV to have population impact and policy sustainability. In this paper, we provide a case study of the design, testing, and implementation of the right@home program, an Australian NHV program and randomized controlled trial. We address existing gaps related to implementation of NHV programs by describing the processes used to develop the program to be trialed, summarizing its effectiveness, and detailing the quality processes and implementation evaluation. The weight of our evidence suggests that NHV can be a powerful and sustainable platform for addressing inequitable outcomes, particularly when the program focuses on parent engagement and partnership, delivers evidence-based strategies shown to improve outcomes, includes fidelity monitoring, and is adapted to and embedded within existing service delivery systems. © 2018 The Authors. Annals of the New York Academy of Sciences published by Wiley Periodicals, Inc. on behalf of The New York Academy of Sciences.

A qualitative analysis of the concepts of fidelity and adaptation in the implementation of an evidence-based HIV prevention intervention

PubMed Central

Owczarzak, Jill; Broaddus, Michelle; Pinkerton, Steven

2016-01-01

Continued debate about the relative value of fidelity versus adaptation, and lack of clarity about the meaning of fidelity, raise concerns about how frontline service providers resolve similar issues in their daily practice. We use SISTA (‘Sisters Informing Sisters on Topics about acquired immune deficiency syndrome’), an evidence-based human immunodeficiency virus (HIV) prevention intervention for African American women, to understand how facilitators and program directors interpret and enact implementation fidelity with the need for adaptation in real-world program delivery. We conducted 22 in-depth, semi-structured interviews with service providers from four agencies implementing SISTA. Facilitators valued their skills as group leaders and ability to emotionally engage participants as more critical to program effectiveness than delivering the intervention with strict fidelity. Consequently, they saw program manuals as guides rather than static texts that should never be changed and, moreover, viewed the prescriptive nature of manuals as undermining their efforts to fully engage with participants. Our findings suggest that greater consideration should be given to understanding the role of facilitators in program effectiveness over and above the question of whether they implement the program with fidelity. Moreover, training curricula should provide facilitators with transferable skills through general facilitator training rather than only program-specific or manual-specific training. PMID:26944867

EBT Fidelity Trajectories Across Training Cohorts Using the Interagency Collaborative Team Strategy

PubMed Central

Hecht, Debra; Aarons, Greg; Fettes, Danielle; Hurlburt, Michael; Ledesma, Karla

2015-01-01

The Interdisciplinary Collaborative Team (ICT) strategy uses front-line providers as adaptation, training and quality control agents for multi-agency EBT implementation. This study tests whether an ICT transmits fidelity to subsequent provider cohorts. SafeCare was implemented by home visitors from multiple community-based agencies contracting with child welfare. Client-reported fidelity trajectories for 5,769 visits, 957 clients and 45 providers were compared using three-level growth models. Provider cohorts trained and live-coached by the ICT attained benchmark fidelity after 12 weeks, and this was sustained. Hispanic clients reported high cultural competency, supporting a cultural adaptation crafted by the ICT. PMID:25586878

EBT Fidelity Trajectories Across Training Cohorts Using the Interagency Collaborative Team Strategy.

PubMed

Chaffin, Mark; Hecht, Debra; Aarons, Greg; Fettes, Danielle; Hurlburt, Michael; Ledesma, Karla

2016-03-01

The Interdisciplinary Collaborative Team (ICT) strategy uses front-line providers as adaptation, training and quality control agents for multi-agency EBT implementation. This study tests whether an ICT transmits fidelity to subsequent provider cohorts. SafeCare was implemented by home visitors from multiple community-based agencies contracting with child welfare. Client-reported fidelity trajectories for 5,769 visits, 957 clients and 45 providers were compared using three-level growth models. Provider cohorts trained and live-coached by the ICT attained benchmark fidelity after 12 weeks, and this was sustained. Hispanic clients reported high cultural competency, supporting a cultural adaptation crafted by the ICT.

The effect of simulator fidelity on acquiring non-technical skills: a randomized non-inferiority trial.

PubMed

Gu, Yuqi; Witter, Tobias; Livingston, Patty; Rao, Purnima; Varshney, Terry; Kuca, Tom; Dylan Bould, M

2017-12-01

As simulator fidelity (i.e., realism) increases from low to high, the simulator more closely resembles the real environment, but it also becomes more expensive. It is generally assumed that the use of high-fidelity simulators results in better learning; however, the effect of fidelity on learning non-technical skills (NTS) is unknown. This was a non-inferiority trial comparing the efficacy of high- vs low-fidelity simulators on learning NTS. Thirty-six postgraduate medical trainees were recruited for the trial. During the pre-test phase, the trainees were randomly assigned to manage a scenario using either a high-fidelity simulator (HFS) or a low-fidelity simulator (LFS), followed by expert debriefing. All trainees then underwent a video recorded post-test scenario on a HFS, and the NTS were assessed between the two groups. The primary outcome was the overall post-test Ottawa Global Rating Scale (OGRS), while controlling for overall pre-test OGRS scores. Non-inferiority between the LFS and HFS was based on a non-inferiority margin of greater than 1. For our primary outcome, the mean (SD) post-test overall OGRS score was not significantly different between the HFS and LFS groups after controlling for pre-test overall OGRS scores [3.8 (0.9) vs 4.0 (0.9), respectively; mean difference, 0.2; 95% confidence interval, -0.4 to 0.8; P = 0.48]. For our secondary outcomes, the post-test total OGRS score was not significantly different between the HFS and LFS groups after controlling for pre-test total OGRS scores (P = 0.33). There were significant improvements in mean overall (P = 0.01) and total (P = 0.003) OGRS scores from pre-test to post-test. There were no significant associations between postgraduate year (P = 0.82) and specialty (P = 0.67) on overall OGRS performance. This study suggests that low-fidelity simulators are non-inferior to the more costly high-fidelity simulators for teaching NTS to postgraduate medical trainees.

Understanding Leisure-related Program Effects by Using Process Data in the HealthWise South Africa Project.

PubMed

Caldwell, Linda L; Younker, Anita S; Wegner, Lisa; Patrick, Megan E; Vergnani, Tania; Smith, Edward A; Flisher, Alan J

2008-01-01

As the push for evidence-based programming gathers momentum, many human services programs and interventions are under increased scrutiny to justify their effectiveness across different conditions and populations. Government agencies and the public want to be assured that their resources are being put to good use on programs that are effective and efficient (Guskey, 2000). Thus, programs are increasingly based on theory and evaluated through randomized control trials using longitudinal data. Despite this progress, hypothesized outcomes are often not detected and/or their effect sizes are small (Gingiss, Roberts-Gray, Boerm, 2006). Moreover, findings may go against intuition or "gut feelings" on the part of project staff. Given the need to understand how program implementation issues relate to outcomes, this study focuses on whether process measures that focus on program implementation and fidelity can shed light on associated outcomes. In particular, we linked the process evaluation of the HealthWise motivation lesson with outcomes across four waves of data collection. We hypothesized that HealthWise would increase learners' intrinsic and identified forms of motivation, and decrease amotivation and extrinsic motivation. We did not hypothesize a direction of effects on introjected motivation due to its conceptual ambiguity. Data came from youth in four intervention schools (n = 902, 41.1%) and five control schools (n = 1291, 58.9%) who were participating in a multi-cohort, longitudinal study. The schools were in a township near Cape Town, South Africa. For each cohort, baseline data are collected on learners as they begin Grade 8. We currently have four waves of data collected on the first cohort, which is the focus of this paper. The mean age of the sample at Wave 3 was 15.0 years (SD = .86) and 51% of students were female. Results suggested that there was evidence of an overall program effect of the curriculum on amotivation regardless of fidelity of implementation. Compared to the control schools, all treatment school learners reported lower levels of amotivation in Wave 4 compared to Wave 3, as hypothesized. Using process evaluation data to monitor implementation fidelity, however, we also conclude that the school with better trained teachers who also reported higher levels of program fidelity had better outcomes than the other schools. We discuss the implications of linking process data with outcome data and associated methodological challenges in linking these data.

Disentangling the effects of a multiple behaviour change intervention for diarrhoea control in Zambia: a theory-based process evaluation.

PubMed

Greenland, Katie; Chipungu, Jenala; Chilekwa, Joyce; Chilengi, Roma; Curtis, Val

2017-10-17

Diarrhoea is a leading cause of child death in Zambia. As elsewhere, the disease burden could be greatly reduced through caregiver uptake of existing prevention and treatment strategies. We recently reported the results of the Komboni Housewives intervention which tested a novel strategy employing motives including affiliation and disgust to improve caregiver practice of four diarrhoea control behaviours: exclusive breastfeeding; handwashing with soap; and correct preparation and use of oral rehydration salts (ORS) and zinc. The intervention was delivered via community events (women's forums and road shows), at health clinics (group session) and via radio. A cluster randomised trial revealed that the intervention resulted in a small improvement in exclusive breastfeeding practices, but was only associated with small changes in the other behaviours in areas with greater intervention exposure. This paper reports the findings of the process evaluation that was conducted alongside the trial to investigate how factors associated with intervention delivery and receipt influenced caregiver uptake of the target behaviours. Process data were collected from the eight peri-urban and rural intervention areas throughout the six-month implementation period and in all 16 clusters 4-6 weeks afterwards. Intervention implementation (fidelity, reach, dose delivered and recruitment strategies) and receipt (participant engagement and responses, and mediators) were explored through review of intervention activity logs, unannounced observation of intervention events, semi-structured interviews, focus groups with implementers and intervention recipients, and household surveys. Evaluation methods and analyses were guided by the intervention's theory of change and the evaluation framework of Linnan and Steckler. Intervention reach was lower than intended: 39% of the surveyed population reported attending one or more face-to-face intervention event, of whom only 11% attended two or more intervention events. The intervention was not equally feasible to deliver in all settings: fewer events took place in remote rural areas, and the intervention did not adequately penetrate communities in several peri-urban sites where the population density was high, the population was slightly higher socio-economic status, recruitment was challenging, and numerous alternative sources of entertainment existed. Adaptations made by the implementers affected the fidelity of implementation of messages for all target behaviours. Incorrect messages were consequently recalled by intervention recipients. Participants were most receptive to the novel disgust and skills-based interactive demonstrations targeting exclusive breastfeeding and ORS preparation respectively. However, initial disgust elicitation was not followed by a change in associated psychological mediators, and social norms were not measurably changed. The lack of measured behaviour change was likely due to issues with both the intervention's content and its delivery. Achieving high reach and intensity in community interventions delivered in diverse settings is challenging. Achieving high fidelity is also challenging when multiple behaviours are targeted for change. Further work using improved tools is needed to explore the use of subconscious motives in behaviour change interventions. To better uncover how and why interventions achieve their measured effects, process evaluations of complex interventions should develop and employ frameworks for investigation and interpretation that are structured around the intervention's theory of change and the local context. The study was registered as part of the larger trial on 5 March 2014 with ClinicalTrials.gov: NCT02081521 .

The Effects of Bug-in-Ear Coaching on Pre-Service Behavior Analysts' Use of Functional Communication Training.

PubMed

Artman-Meeker, Kathleen; Rosenberg, Nancy; Badgett, Natalie; Yang, Xueyan; Penney, Ashley

2017-09-01

Behavior analysts play an important role in supporting the behavior and learning of young children with disabilities in natural settings. However, there is very little research related specifically to developing the skills and competencies needed by pre-service behavior analysts. This study examined the effects of "bug-in-ear" (BIE) coaching on pre-service behavior analysts' implementation of functional communication training with pre-school children with autism in their classrooms. BIE coaching was associated with increases in the rate of functional communication training trials each intern initiated per session and in the fidelity with which interns implemented functional communication training. Adults created more intentional opportunities for children to communicate, and adults provided more systematic instruction around those opportunities.

Using mixed methods to assess fidelity of delivery and its influencing factors in a complex self-management intervention for people with osteoarthritis and low back pain

PubMed Central

Toomey, Elaine; Matthews, James; Hurley, Deirdre A

2017-01-01

Objectives and design Despite an increasing awareness of the importance of fidelity of delivery within complex behaviour change interventions, it is often poorly assessed. This mixed methods study aimed to establish the fidelity of delivery of a complex self-management intervention and explore the reasons for these findings using a convergent/triangulation design. Setting Feasibility trial of the Self-management of Osteoarthritis and Low back pain through Activity and Skills (SOLAS) intervention (ISRCTN49875385), delivered in primary care physiotherapy. Methods and outcomes 60 SOLAS sessions were delivered across seven sites by nine physiotherapists. Fidelity of delivery of prespecified intervention components was evaluated using (1) audio-recordings (n=60), direct observations (n=24) and self-report checklists (n=60) and (2) individual interviews with physiotherapists (n=9). Quantitatively, fidelity scores were calculated using percentage means and SD of components delivered. Associations between fidelity scores and physiotherapist variables were analysed using Spearman’s correlations. Interviews were analysed using thematic analysis to explore potential reasons for fidelity scores. Integration of quantitative and qualitative data occurred at an interpretation level using triangulation. Results Quantitatively, fidelity scores were high for all assessment methods; with self-report (92.7%) consistently higher than direct observations (82.7%) or audio-recordings (81.7%). There was significant variation between physiotherapists’ individual scores (69.8% - 100%). Both qualitative and quantitative data (from physiotherapist variables) found that physiotherapists’ knowledge (Spearman’s association at p=0.003) and previous experience (p=0.008) were factors that influenced their fidelity. The qualitative data also postulated participant-level (eg, individual needs) and programme-level factors (eg, resources) as additional elements that influenced fidelity. Conclusion The intervention was delivered with high fidelity. This study contributes to the limited evidence regarding fidelity assessment methods within complex behaviour change interventions. The findings suggest a combination of quantitative methods is suitable for the assessment of fidelity of delivery. A mixed methods approach provided a more insightful understanding of fidelity and its influencing factors. Trial registration number ISRCTN49875385; Pre-results. PMID:28780544

The Impact of Evidence-Based Practice Implementation and Fidelity Monitoring on Staff Turnover: Evidence for a Protective Effect

ERIC Educational Resources Information Center

Aarons, Gregory A.; Sommerfeld, David H.; Hecht, Debra B.; Silovsky, Jane F.; Chaffin, Mark J.

2009-01-01

Staff retention is an ongoing challenge in mental health and community-based service organizations. Little is known about the impact of evidence-based practice implementation on the mental health and social service workforce. The present study examined the effect of evidence-based practice implementation and ongoing fidelity monitoring on staff…

Multifamily Group Psychoeducation in New York State: Implementation and Fidelity Outcomes.

PubMed

Kealey, Edith M; Leckman-Westin, Emily; Jewell, Thomas C; Finnerty, Molly T

2015-11-01

The study examined implementation outcomes from a large state initiative to support dissemination of multifamily group (MFG) psychoeducation in outpatient mental health settings. Thirty-one sites participated in the project. Baseline training in the MFG model was followed by monthly expert consultation delivered in either a group (16 sites) or individual format (15 sites). Research staff assessed fidelity to the MFG model by telephone at baseline and 12, 18, and 24 months and documented time to completion of three key milestones: holding a family joining session, a family educational workshop, and an MFG meeting. Intent-to-train analyses found that 12 sites (39%) achieved high fidelity to the MFG model, and 20 (65%) achieved moderate or high fidelity. Mean scores on the Family Psychoeducation Fidelity Assessment Scale increased over time. Twenty-five sites (81%) conducted at least one joining session, and 20 (65%) conducted at least one MFG. Mean±SD time from baseline to the first group was 11.75±4.78 months. Programs that held the first joining session within four to 12 months after training were significantly more likely than programs that did not to conduct a group (p<.05). No significant differences were found by consultation format. Implementation of moderate- to high-fidelity MFG programs in routine outpatient mental health settings is feasible. Sites that moved very quickly or very slowly in early implementation stages were less likely to be successful in conducting an MFG. More research on the efficiency and effectiveness of consultation formats is needed to guide future implementation efforts.

Integrating technology into complex intervention trial processes: a case study.

PubMed

Drew, Cheney J G; Poile, Vincent; Trubey, Rob; Watson, Gareth; Kelson, Mark; Townson, Julia; Rosser, Anne; Hood, Kerenza; Quinn, Lori; Busse, Monica

2016-11-17

Trials of complex interventions are associated with high costs and burdens in terms of paperwork, management, data collection, validation, and intervention fidelity assessment occurring across multiple sites. Traditional data collection methods rely on paper-based forms, where processing can be time-consuming and error rates high. Electronic source data collection can potentially address many of these inefficiencies, but has not routinely been used in complex intervention trials. Here we present the use of an on-line system for managing all aspects of data handling and for the monitoring of trial processes in a multicentre trial of a complex intervention. We custom built a web-accessible software application for the delivery of ENGAGE-HD, a multicentre trial of a complex physical therapy intervention. The software incorporated functionality for participant randomisation, data collection and assessment of intervention fidelity. It was accessible to multiple users with differing levels of access depending on required usage or to maintain blinding. Each site was supplied with a 4G-enabled iPad for accessing the system. The impact of this system was quantified through review of data quality and collation of feedback from site coordinators and assessors through structured process interviews. The custom-built system was an efficient tool for collecting data and managing trial processes. Although the set-up time required was significant, using the system resulted in an overall data completion rate of 98.5% with a data query rate of 0.1%, the majority of which were resolved in under a week. Feedback from research staff indicated that the system was highly acceptable for use in a research environment. This was a reflection of the portability and accessibility of the system when using the iPad and its usefulness in aiding accurate data collection, intervention fidelity and general administration. A combination of commercially available hardware and a bespoke online database designed to support data collection, intervention fidelity and trial progress provides a viable option for streamlining trial processes in a multicentre complex intervention trial. There is scope to further extend the system to cater for larger trials and add further functionality such as automatic reporting facilities and participant management support. ISRCTN65378754 , registered on 13 March 2014.

Technical Adequacy of the SWPBIS Tiered Fidelity Inventory

ERIC Educational Resources Information Center

McIntosh, Kent; Massar, Michelle M.; Algozzine, Robert F.; George, Heather Peshak; Horner, Robert H.; Lewis, Timothy J.; Swain-Bradway, Jessica

2017-01-01

Full and durable implementation of school-based interventions is supported by regular evaluation of fidelity of implementation. Multiple assessments have been developed to evaluate the extent to which schools are applying the core features of school-wide positive behavioral interventions and supports (SWPBIS). The "SWPBIS Tiered Fidelity…

Factors contributing to intervention fidelity in a multi-site chronic disease self-management program

PubMed Central

Perrin, Karen M; Burke, Somer Goad; O'Connor, Danielle; Walby, Gary; Shippey, Claire; Pitt, Seraphine; McDermott, Robert J; Forthofer, Melinda S

2006-01-01

Background and objectives Disease self-management programs have been a popular approach to reducing morbidity and mortality from chronic disease. Replicating an evidence-based disease management program successfully requires practitioners to ensure fidelity to the original program design. Methods The Florida Health Literacy Study (FHLS) was conducted to investigate the implementation impact of the Pfizer, Inc. Diabetes Mellitus and Hypertension Disease Self-Management Program based on health literacy principles in 14 community health centers in Florida. The intervention components discussed include health educator recruitment and training, patient recruitment, class sessions, utilization of program materials, translation of program manuals, patient retention and follow-up, and technical assistance. Results This report describes challenges associated with achieving a balance between adaptation for cultural relevance and fidelity when implementing the health education program across clinic sites. This balance was necessary to achieve effectiveness of the disease self-management program. The FHLS program was implemented with a high degree of fidelity to the original design and used original program materials. Adaptations identified as advantageous to program participation are discussed, such as implementing alternate methods for recruiting patients and developing staff incentives for participation. Conclusion Effective program implementation depends on the talent, skill and willing participation of clinic staff. Program adaptations that conserve staff time and resources and recognize their contribution can increase program effectiveness without jeopardizing its fidelity. PMID:17067388

Tool for evaluating research implementation challenges: A sense-making protocol for addressing implementation challenges in complex research settings

PubMed Central

2013-01-01

Background Many challenges arise in complex organizational interventions that threaten research integrity. This article describes a Tool for Evaluating Research Implementation Challenges (TECH), developed using a complexity science framework to assist research teams in assessing and managing these challenges. Methods During the implementation of a multi-site, randomized controlled trial (RCT) of organizational interventions to reduce resident falls in eight nursing homes, we inductively developed, and later codified the TECH. The TECH was developed through processes that emerged from interactions among research team members and nursing home staff participants, including a purposive use of complexity science principles. Results The TECH provided a structure to assess challenges systematically, consider their potential impact on intervention feasibility and fidelity, and determine actions to take. We codified the process into an algorithm that can be adopted or adapted for other research projects. We present selected examples of the use of the TECH that are relevant to many complex interventions. Conclusions Complexity theory provides a useful lens through which research procedures can be developed to address implementation challenges that emerge from complex organizations and research designs. Sense-making is a group process in which diverse members interpret challenges when available information is ambiguous; the groups’ interpretations provide cues for taking action. Sense-making facilitates the creation of safe environments for generating innovative solutions that balance research integrity and practical issues. The challenges encountered during implementation of complex interventions are often unpredictable; however, adoption of a systematic process will allow investigators to address them in a consistent yet flexible manner, protecting fidelity. Research integrity is also protected by allowing for appropriate adaptations to intervention protocols that preserve the feasibility of ‘real world’ interventions. PMID:23281623

Staff turnover in statewide implementation of ACT: relationship with ACT fidelity and other team characteristics

PubMed Central

Rollins, Angela L.; Salyers, Michelle P.; Tsai, Jack; Lydick, Jennifer M.

2010-01-01

Staff turnover on assertive community treatment (ACT) teams is a poorly understood phenomenon. This study examined annual turnover and fidelity data collected in a statewide implementation of ACT over a 5-year period. Mean annual staff turnover across all observations was 30.0%. Turnover was negatively correlated with overall fidelity at Year 1 and 3. The team approach fidelity item was negatively correlated with staff turnover at Year 3. For 13 teams with 3 years of follow-up data, turnover rates did not change over time. Most ACT staff turnover rates were comparable or better than other turnover rates reported in the mental health and substance abuse literature. PMID:20012481

Staff turnover in statewide implementation of ACT: relationship with ACT fidelity and other team characteristics.

PubMed

Rollins, Angela L; Salyers, Michelle P; Tsai, Jack; Lydick, Jennifer M

2010-09-01

Staff turnover on assertive community treatment (ACT) teams is a poorly understood phenomenon. This study examined annual turnover and fidelity data collected in a statewide implementation of ACT over a 5-year period. Mean annual staff turnover across all observations was 30.0%. Turnover was negatively correlated with overall fidelity at Year 1 and 3. The team approach fidelity item was negatively correlated with staff turnover at Year 3. For 13 teams with 3 years of follow-up data, turnover rates did not change over time. Most ACT staff turnover rates were comparable or better than other turnover rates reported in the mental health and substance abuse literature.

An intervention fidelity framework for technology-based behavioral interventions.

PubMed

Devito Dabbs, Annette; Song, Mi-Kyung; Hawkins, Robert; Aubrecht, Jill; Kovach, Karen; Terhorst, Lauren; Connolly, Mary; McNulty, Mary; Callan, Judith

2011-01-01

Despite the proliferation of health technologies, descriptions of the unique considerations and practical guidance for evaluating the intervention fidelity of technology-based behavioral interventions are lacking. The aims of this study were to (a) discuss how technology-based behavioral interventions challenge conventions about how intervention fidelity is conceptualized and evaluated, (b) propose an intervention fidelity framework that may be more appropriate for technology-based behavioral interventions, and (c) present a plan for operationalizing each concept in the framework using the intervention fidelity monitoring plan for Pocket PATH (Personal Assistant for Tracking Health), a mobile health technology designed to promote self-care behaviors after lung transplantation, as an exemplar. The literature related to intervention fidelity and technology acceptance was used to identify the issues that are unique to the fidelity of technology-based behavioral interventions and thus important to include in a proposed intervention fidelity framework. An intervention fidelity monitoring plan for technology-based behavioral interventions was developed as an example. The intervention fidelity monitoring plan was deemed feasible and practical to implement and showed utility in operationalizing the concepts such as assessing interventionists' delivery and participants' acceptance of the technology-based behavioral intervention. The framework has the potential to guide the development of implementation fidelity monitoring tools for other technology-based behavioral interventions. Further application and testing of this framework will allow for a better understanding of the role that technology acceptance plays in the adoption and enactment of the behaviors that technology-based behavioral interventions are intended to promote.

A Systematic Review of Fidelity of Implementation in Parent-Mediated Early Communication Intervention

ERIC Educational Resources Information Center

Lieberman-Betz, Rebecca G.

2015-01-01

This article examined the reporting of four elements of fidelity of implementation (FOI) in parent-mediated early communication treatment studies. Thirty-five studies were reviewed to extract information regarding reporting of dosage, adherence, quality, and participant responsiveness for both practitioners and parents involved in parent-delivered…

Developing a Fidelity of Implementation Measure for the "Responsive Classroom" Approach

ERIC Educational Resources Information Center

Nathanson, Lori

2009-01-01

This study examines the psychometric properties, reliability, and validity of a measure designed to assess fidelity of implementation of the "Responsive Classroom"[R] ("RC") approach. The Classroom Practices Teacher Survey (CPTS) assesses teachers' use of the "RC" approach, a social and emotional learning (SEL) intervention currently under…

Evaluating Math Recovery: Measuring Fidelity of Implementation

ERIC Educational Resources Information Center

Munter, Charles; Garrison, Anne; Cobb, Paul; Cordray, David

2010-01-01

In this paper, the authors describe a case of measuring implementation fidelity within an evaluation study of Math Recovery (MR), a pullout tutoring program aimed at increasing the mathematics achievement of low-performing first graders, thereby closing the school-entry achievement gap by enabling them to achieve at the level of their…

A qualitative analysis of the concepts of fidelity and adaptation in the implementation of an evidence-based HIV prevention intervention.

PubMed

Owczarzak, Jill; Broaddus, Michelle; Pinkerton, Steven

2016-04-01

Continued debate about the relative value of fidelity versus adaptation, and lack of clarity about the meaning of fidelity, raise concerns about how frontline service providers resolve similar issues in their daily practice. We use SISTA ('Sisters Informing Sisters on Topics about acquired immune deficiency syndrome'), an evidence-based human immunodeficiency virus (HIV) prevention intervention for African American women, to understand how facilitators and program directors interpret and enact implementation fidelity with the need for adaptation in real-world program delivery. We conducted 22 in-depth, semi-structured interviews with service providers from four agencies implementing SISTA. Facilitators valued their skills as group leaders and ability to emotionally engage participants as more critical to program effectiveness than delivering the intervention with strict fidelity. Consequently, they saw program manuals as guides rather than static texts that should never be changed and, moreover, viewed the prescriptive nature of manuals as undermining their efforts to fully engage with participants. Our findings suggest that greater consideration should be given to understanding the role of facilitators in program effectiveness over and above the question of whether they implement the program with fidelity. Moreover, training curricula should provide facilitators with transferable skills through general facilitator training rather than only program-specific or manual-specific training. © The Author 2016. Published by Oxford University Press. All rights reserved. For permissions, please email: journals.permissions@oup.com.

Implementation of a Personalized, Cost-Effective Physical Therapy Approach (Coach2Move) for Older Adults: Barriers and Facilitators.

PubMed

van de Sant, Arjan J W; de Vries, Nienke M; Hoogeboom, Thomas J; Nijhuis-van der Sanden, Maria W G

2017-07-27

This article reports on a recent randomized clinical trial that showed a personalized approach to physical therapy (Coach2Move) by a physical therapist specialized in geriatrics (PTG) to be more cost-effective than usual physical therapy care in people with mobility problems (n = 130, mean age = 78 years). We used an explanatory mixed-methods sequential design alongside the randomized clinical trial to gain insight into (a) the contrast between the 2 interventions, (b) the fidelity of the Coach2Move delivery; (c) PTGs' experiences of Coach2Move; and (d) possible barriers and facilitators for future implementation. The study included 13 PTGs educated in the strategy and 13 physical therapists with expertise in geriatrics delivering the usual care. In total, 106 medical records were available for assessment: 57 (85%) Coach2Move, 49 (75%) usual care. Quantitative process indicators were used to analyze electronic medical records to determine contrasts in the phases of clinical reasoning. The fidelity of the delivery was tested using indicator scores focusing on 4 key elements of Coach2Move. In-depth interviews with Coach2Move therapists were thematically analyzed to explore experiences and facilitators/barriers related to implementation. Indicator scores showed significant and clinically relevant contrasts in all phases of clinical reasoning, with consistently higher scores among PTGs, except for the treatment plan. Moreover, the fidelity of Coach2Move delivery was more than 70% in all phases, except the evaluation phase (53%). Experiences of Coach2Move were positive. In particular, extended intake allowing motivational interviewing, physical examination and an in-depth problem analysis, and shared goal setting were considered valuable. Facilitators for implementation were the addition of a Coach2Move medical record, frequent coaching by the researcher, and readiness to change in the therapist. Barriers were (1) having to use 2 parallel electronic medical record systems, (2) having to clear the calendar to schedule an intake of 90 minutes, (3) fear of losing income, (4) the sense that patients do not want to change their lifestyle, and (5) not acknowledging that increasing physical activity is an important goal for older adults with mobility problems. Physical therapy based on the Coach2Move strategy is substantially different from usual care. Future implementation should focus on increasing regular evaluation and feedback, taking into account individuals' contextual factors, and improving organizational facilities while mitigating income loss.This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.

Implementation of an efficacious intervention for high risk women in Mexico: protocol for a multi-site randomized trial with a parallel study of organizational factors

PubMed Central

2012-01-01

Background Studies of implementation of efficacious human immunodeficiency virus (HIV) prevention interventions are rare, especially in resource-poor settings, but important, because they have the potential to increase the impact of interventions by improving uptake and sustainability. Few studies have focused on provider and organizational factors that may influence uptake and fidelity to core intervention components. Using a hybrid design, we will study the implementation of an efficacious intervention to reduce sexually transmitted infections (STIs) among female sex workers (FSWs) in 12 cities across Mexico. Our protocol will test a ‘train-the-trainer’ implementation model for transporting the Mujer Segura (Healthy Woman) intervention into community-based organizations (CBOs). Methods We have partnered with Mexican Foundation for Family Planning (Mexfam), a non-governmental organization that has CBOs throughout Mexico. At each CBO, trained ethnographers will survey CBO staff on characteristics of their organization and on their attitudes toward their CBO and toward the implementation of evidence-based interventions (EBIs). Then, after CBO staff recruit a sample of 80 eligible FSWs and deliver a standard-care, didactic intervention to 40 women randomly selected from that pool, a Mexfam staff person will be trained in the Mujer Segura intervention and will then train other counselors to deliver Mujer Segura to the 40 remaining participating FSWs. FSW participants will receive a baseline behavioral assessment and be tested for HIV and STIs (syphilis, gonorrhea, and chlamydia); they will be reassessed at six months post-intervention to measure for possible intervention effects. At the same time, both qualitative and quantitative data will be collected on the implementation process, including measures of counselors’ fidelity to the intervention model. After data collection at each CBO is complete, the relative efficacy of the Mujer Segura intervention will be analyzed, and across CBOs, correlations will be examined between individual and organizational provider characteristics and intervention efficacy. Discussion This cooperative, bi-national research study will provide critical insights into barriers and facilitating factors associated with implementing interventions in CBOs using the ‘train the trainer’ model. Our work builds on similar scale-up strategies that have been effective in the United States. This study has the potential to increase our knowledge of the generalizability of such strategies across health issues, national contexts, and organizational contexts. Trial registration NCT01465607 PMID:23107285

Implementation-effectiveness trial of an ecological intervention for physical activity in ethnically diverse low income senior centers.

PubMed

Rich, Porchia; Aarons, Gregory A; Takemoto, Michelle; Cardenas, Veronica; Crist, Katie; Bolling, Khalisa; Lewars, Brittany; Sweet, Cynthia Castro; Natarajan, Loki; Shi, Yuyan; Full, Kelsie M; Johnson, Eileen; Rosenberg, Dori E; Whitt-Glover, Melicia; Marcus, Bess; Kerr, Jacqueline

2017-07-18

As the US population ages, there is an increasing need for evidence based, peer-led physical activity programs, particularly in ethnically diverse, low income senior centers where access is limited. The Peer Empowerment Program 4 Physical Activity' (PEP4PA) is a hybrid Type II implementation-effectiveness trial that is a peer-led physical activity (PA) intervention based on the ecological model of behavior change. The initial phase is a cluster randomized control trial randomized to either a peer-led PA intervention or usual center programming. After 18 months, the intervention sites are further randomized to continued support or no support for another 6 months. This study will be conducted at twelve senior centers in San Diego County in low income, diverse communities. In the intervention sites, 24 peer health coaches and 408 adults, aged 50 years and older, are invited to participate. Peer health coaches receive training and support and utilize a tablet computer for delivery and tracking. There are several levels of intervention. Individual components include pedometers, step goals, counseling, and feedback charts. Interpersonal components include group walks, group sharing and health tips, and monthly celebrations. Community components include review of PA resources, walkability audit, sustainability plan, and streetscape improvements. The primary outcome of interest is intensity and location of PA minutes per day, measured every 6 months by wrist and hip accelerometers and GPS devices. Secondary outcomes include blood pressure, physical, cognitive, and emotional functioning. Implementation measures include appropriateness & acceptability (perceived and actual fit), adoption & penetration (reach), fidelity (quantity & quality of intervention delivered), acceptability (satisfaction), costs, and sustainability. Using a peer led implementation strategy to deliver a multi-level community based PA program can enhance program adoption, implementation, and sustainment. ClinicalTrials.gov, USA ( NCT02405325 ). Date of registration, March 20, 2015. This website also contains all items from the World Health Organization Trial Registration Data Set.

A Supervisor-Targeted Implementation Approach to Promote System Change: The R3 Model.

PubMed

Saldana, Lisa; Chamberlain, Patricia; Chapman, Jason

2016-11-01

Opportunities to evaluate strategies to create system-wide change in the child welfare system (CWS) and the resulting public health impact are rare. Leveraging a real-world, system-initiated effort to infuse the use of evidence-based principles throughout a CWS workforce, a pilot of the R 3 model and supervisor-targeted implementation approach is described. The development of R 3 and its associated fidelity monitoring was a collaboration between the CWS and model developers. Outcomes demonstrate implementation feasibility, strong fidelity scale measurement properties, improved supervisor fidelity over time, and the acceptability and perception of positive change by agency leadership. The value of system-initiated collaborations is discussed.

Implementing a Primary Prevention Social Skills Intervention in Urban Preschools: Factors Associated with Quality and Fidelity

PubMed Central

Wenz-Gross, Melodie; Upshur, Carole

2012-01-01

Research Findings Preschool behavior problems are of increasing concern to early childhood educators. Preventive interventions are being developed, but implementation in under-resourced child care programs is challenging. This study describes the implementation of an adapted Second Step curriculum to increase children’s social skills and decrease behavior problems in preschool classrooms. Teacher training/coaching, organizational supports, and procedures for sustainability are described. Measures included baseline classroom characteristics, lesson completion and fidelity, and teacher satisfaction. Teachers completed 88% of the lessons across two years. Follow-up monitoring in Year 3, once all training was terminated, found 95% of lessons were delivered with higher fidelity than the first two years. Teachers were satisfied with training, felt the curriculum improved children’s social skills, and stated they would continue to use the curriculum. Lesson completion and fidelity were associated with various teacher characteristics, training attendance, classroom characteristics, teacher and child turnover, and the percent of children with developmental delays in the classroom. Practice Results support the implementation strategy and suggest that paying attention to teacher morale, general skill development, and a period of on-going support for training is important and will lead to sustainable and high implementation rates. PMID:22791957

A Cluster Randomized Trial to Promote Healthy Menu Items for Children: The Kids’ Choice Restaurant Program

PubMed Central

Ayala, Guadalupe X.; Castro, Iana A.; Pickrel, Julie L.; Lin, Shih-Fan; Williams, Christine B.; Madanat, Hala; Jun, Hee-Jin; Zive, Michelle

2017-01-01

Evidence indicates that restaurant-based interventions have the potential to promote healthier purchasing and improve the nutrients consumed. This study adds to this body of research by reporting the results of a trial focused on promoting the sale of healthy child menu items in independently owned restaurants. Eight pair-matched restaurants that met the eligibility criteria were randomized to a menu-only versus a menu-plus intervention condition. Both of the conditions implemented new healthy child menu items and received support for implementation for eight weeks. The menu-plus condition also conducted a marketing campaign involving employee trainings and promotional materials. Process evaluation data captured intervention implementation. Sales of new and existing child menu items were tracked for 16 weeks. Results indicated that the interventions were implemented with moderate to high fidelity depending on the component. Sales of new healthy child menu items occurred immediately, but decreased during the post-intervention period in both conditions. Sales of existing child menu items demonstrated a time by condition effect with restaurants in the menu-plus condition observing significant decreases and menu-only restaurants observing significant increases in sales of existing child menu items. Additional efforts are needed to inform sustainable methods for improving access to healthy foods and beverages in restaurants. PMID:29194392

A Cluster Randomized Trial to Promote Healthy Menu Items for Children: The Kids' Choice Restaurant Program.

PubMed

Ayala, Guadalupe X; Castro, Iana A; Pickrel, Julie L; Lin, Shih-Fan; Williams, Christine B; Madanat, Hala; Jun, Hee-Jin; Zive, Michelle

2017-12-01

Evidence indicates that restaurant-based interventions have the potential to promote healthier purchasing and improve the nutrients consumed. This study adds to this body of research by reporting the results of a trial focused on promoting the sale of healthy child menu items in independently owned restaurants. Eight pair-matched restaurants that met the eligibility criteria were randomized to a menu-only versus a menu-plus intervention condition. Both of the conditions implemented new healthy child menu items and received support for implementation for eight weeks. The menu-plus condition also conducted a marketing campaign involving employee trainings and promotional materials. Process evaluation data captured intervention implementation. Sales of new and existing child menu items were tracked for 16 weeks. Results indicated that the interventions were implemented with moderate to high fidelity depending on the component. Sales of new healthy child menu items occurred immediately, but decreased during the post-intervention period in both conditions. Sales of existing child menu items demonstrated a time by condition effect with restaurants in the menu-plus condition observing significant decreases and menu-only restaurants observing significant increases in sales of existing child menu items. Additional efforts are needed to inform sustainable methods for improving access to healthy foods and beverages in restaurants.

Increasing children's physical activity through a teaching-assistant led extracurricular intervention: process evaluation of the action 3:30 randomised feasibility trial.

PubMed

Jago, Russell; Sebire, Simon J; Davies, Ben; Wood, Lesley; Banfield, Kathryn; Edwards, Mark J; Powell, Jane E; Montgomery, Alan A; Thompson, Janice L; Fox, Kenneth R

2015-02-18

Many children do not engage in recommended levels of physical activity (PA), highlighting the need to find ways to increase children's PA. Process evaluations play an important role in improving the science of randomised controlled trials. We recently reported the results of the Action 3:30 cluster randomised feasibility trial illustrating higher levels of moderate to vigorous intensity PA among boys but not girls. The aim of this paper is to report the process evaluation results including intervention fidelity, implementation, context and how intervention components and trial design could be improved before proceeding to a definitive RCT. Children's session enjoyment was assessed every two weeks. Reasons for non-attendance were provided by questionnaire at the end of the intervention. Post intervention interviews were held with participating teaching assistants (TAs) and school key contacts (KCs), and focus groups were conducted with children in all 10 intervention schools. Interviews and focus groups examined how recruitment and session attendance might be improved and established which elements of the programme that were and were not well received. Data indicated good intervention fidelity with TA's adopting enjoyment-focussed teaching styles and the sessions improving children's skills and self-esteem. Several positive aspects of implementation were identified, including high session variety, the opportunity to work in teams, the child-led sessions and the engaging leader style. In terms of context there was evidence that TA's faced difficulties managing challenging behaviour and that further training in this area was needed. TAs and KCs felt that recruitment could be improved by providing taster sessions during PE lessons and clarifying the days that the clubs would run at the point of recruitment. The programme could be improved to enhance interest for girls, by including training for managing disruptive behaviour and making some activities more age-group appropriate. Action 3.30 showed promise but could be improved by ensuring age appropriate activities, providing more appeal to girls and improving recruitment through taster sessions and early establishment of days of the week it is to be offered on. ISRCTN58502739.

Individualization and quality improvement: two new scales to complement measurement of program fidelity.

PubMed

Bond, Gary R; Drake, Robert E; Rapp, Charles A; McHugo, Gregory J; Xie, Haiyi

2009-09-01

Fidelity scales have been widely used to assess program adherence to the principles of an evidence-based practice, but they do not measure important aspects of quality of care. Pragmatic scales measuring clinical quality of services are needed to complement fidelity scales measuring structural aspects of program implementation. As part of the instrumentation developed for the National Implementing Evidence-Based Practices Project, we piloted a new instrument with two 5-item quality scales, Individualization (a client-level quality scale) and Quality Improvement (an organizational-level quality scale). Pairs of independent fidelity assessors conducted fidelity reviews in 49 sites in 8 states at baseline and at four subsequent 6-month intervals over a 2-year follow-up period. The assessors followed a standardized protocol to administer these quality scales during daylong site visits; during these same visits they assessed programs on fidelity to the evidence-based practice that the site was seeking to implement. Assessors achieved acceptable interrater reliability for both Individualization and Quality Improvement. Principal components factor analysis confirmed the 2-scale structure. The two scales were modestly correlated with each other and with the evidence-based practice fidelity scales. Over the first year, Individualization and Quality Improvement improved, but showed little or no improvement during the last year of follow-up. The two newly developed scales showed adequate psychometric properties in this preliminary study, but further research is needed to assess their validity and utility in routine clinical practice.

Assessing Implementation Fidelity and Adaptation in a Community-Based Childhood Obesity Prevention Intervention

ERIC Educational Resources Information Center

Richards, Zoe; Kostadinov, Iordan; Jones, Michelle; Richard, Lucie; Cargo, Margaret

2014-01-01

Little research has assessed the fidelity, adaptation or integrity of activities implemented within community-based obesity prevention initiatives. To address this gap, a mixed-method process evaluation was undertaken in the context of the South Australian Obesity Prevention and Lifestyle (OPAL) initiative. An ecological coding procedure assessed…

Fidelity: Snapshots of Implementation of a Curricular Intervention

ERIC Educational Resources Information Center

Foster, Lisa Hall

2011-01-01

Fidelity of implementation (FOI) is the extent to which delivery of an intervention adheres to the original intent of the program designer. FOI in educational studies is hindered by the lack of a universally agreed upon definition or set of criteria for measurement. With the increasing need for justification of reliability and validity of…

The Impact of Principal Training in Diffusion of Innovation Theory on Fidelity of Implementation

ERIC Educational Resources Information Center

Petruzzelli, Anthony

2010-01-01

Districts and schools are constantly trying to find ways to increase student achievement. Research has shown a significant correlation between principal leadership skills and increased student achievement. Research has also shown a correlation between fidelity of implementation of new innovations and positive outcomes. This purpose of this study…

The Relationship between School-Level Characteristics and Implementation Fidelity of a Coordinated School Health Childhood Obesity Prevention Intervention

ERIC Educational Resources Information Center

Lederer, Alyssa M.; King, Mindy H.; Sovinski, Danielle; Seo, Dong-Chul; Kim, Nayoung

2015-01-01

Background: Curtailing childhood obesity is a public health imperative. Although multicomponent school-based programs reduce obesity among children, less is known about the implementation fidelity of these interventions. This study examines process evaluation findings for the Healthy, Energetic Ready, Outstanding, Enthusiastic, Schools (HEROES)…

Linking Implementation Fidelity to Impacts in an RCT

ERIC Educational Resources Information Center

Unlu, Fatih; Bozzi, Laurie; Layzer, Carolyn; Smith, Arthur; Price, Cristofer; Hurtig, Richard

2013-01-01

In experimental studies, researchers are often interested in secondary research questions that explore important aspects of main findings, such as whether or not program effects vary according to the level of fidelity in which the program has been implemented; or according to the dosage received by individuals participated in the program. In this…

Teachers' Personal and Professional Influences Related to School-Wide Positive Behavior Supports (SWPBS)

ERIC Educational Resources Information Center

Broskey, Matthew

2017-01-01

This study focused on understanding teachers' personal and professional experiences that influence the fidelity of implementation of a school-wide positive behavior support (SWPBS) program within their classrooms. Research has focused on the implementation fidelity of school-wide positive support programs, academic impact on students, teacher…

The Value of Fidelity of Implementation Criteria to Evaluate School-Based Science Curriculum Innovations

ERIC Educational Resources Information Center

Lee, Yew-Jin; Chue, Shien

2013-01-01

School-based curriculum innovations, including those in science education, are usually not adequately evaluated, if at all. Furthermore, current procedures and instruments for programme evaluations are often unable to support evidence-based decision-making. We suggest that adopting fidelity of implementation (FOI) criteria from healthcare research…

Fidelity of Implementation and Instructional Alignment in Response to Intervention Research

ERIC Educational Resources Information Center

Hill, David R.; King, Seth A.; Lemons, Christopher J.; Partanen, Jane N.

2012-01-01

In this review, we explore the extent to which researchers evaluating the efficacy of Tier 2 elementary reading interventions within the framework of Response to Intervention reported on fidelity of implementation and alignment of instruction between tiers. A literature search identified 22 empirical studies from which conclusions were drawn.…

Comparability of Fidelity Measures for Assessing Tier 1 School-Wide Positive Behavioral Interventions and Supports

ERIC Educational Resources Information Center

Mercer, Sterett H.; McIntosh, Kent; Hoselton, Robert

2017-01-01

Several reliable and valid fidelity surveys are commonly used to assess Tier 1 implementation in School-Wide Positive Behavioral Interventions and Supports (SWPBIS); however, differences across surveys complicate consequential decisions regarding school implementation status when multiple measures are compared. To address this concern, the current…

Further Validation of the Pathways Housing First Fidelity Scale.

PubMed

Goering, Paula; Veldhuizen, Scott; Nelson, Geoffrey B; Stefancic, Ana; Tsemberis, Sam; Adair, Carol E; Distasio, Jino; Aubry, Tim; Stergiopoulos, Vicky; Streiner, David L

2016-01-01

This study examined whether Housing First fidelity ratings correspond to program operation descriptions from administrative data and predict client outcomes. A multisite, randomized controlled trial (At Home/Chez Soi) in five Canadian cities included two assessments of 12 programs over two years. Outcomes for 1,158 clients were measured every six months. Associations between fidelity ratings and administrative data (Spearman correlations) and participant outcomes (mixed-effects modeling) were examined. Fidelity ratings were generally good (mean ± SD=136.6 ± 10.3 out of a possible range of 38-152; 87% of maximum value). Fidelity was significantly associated with three of four measures of program operation, with correlations between .55 and .60. Greater program fidelity was associated with improvement in housing stability, community functioning, and quality of life. Variation in program fidelity was associated with operations and outcomes, supporting scale validity and intervention effectiveness. These findings reinforced the value of using fidelity monitoring to conduct quality assurance and technical assistance activities.

Strengthening Incarcerated Families: Evaluating a Pilot Program for Children of Incarcerated Parents and Their Caregivers

PubMed Central

Miller, Alison L.; Perryman, Jamie; Markovitz, Lara; Franzen, Susan; Cochran, Shirley; Brown, Shavonnea

2013-01-01

Parental incarceration can be devastating for families. Children may experience difficulties, and the stress on caregivers who take on unexpected childrearing is high. We implemented and evaluated a family-level intervention with caregivers and children experiencing parental (typically maternal) incarceration, in a community setting. We partnered with a community-based organization serving families with an incarcerated parent to conduct a pilot trial of the Strengthening Families Program (SFP). Process evaluation indicated high implementation fidelity, satisfaction, engagement, and attendance. Outcome evaluation results indicated positive changes in family-level functioning, caregivers’ positive parenting, and caregiver depression symptoms from pre- to post-intervention, with some changes retained at follow-up 4 months later. Implications for preventive interventions with children of incarcerated parents, and their caregivers, are discussed. PMID:24353363

Strengthening Incarcerated Families: Evaluating a Pilot Program for Children of Incarcerated Parents and Their Caregivers.

PubMed

Miller, Alison L; Perryman, Jamie; Markovitz, Lara; Franzen, Susan; Cochran, Shirley; Brown, Shavonnea

2013-10-01

Parental incarceration can be devastating for families. Children may experience difficulties, and the stress on caregivers who take on unexpected childrearing is high. We implemented and evaluated a family-level intervention with caregivers and children experiencing parental (typically maternal) incarceration, in a community setting. We partnered with a community-based organization serving families with an incarcerated parent to conduct a pilot trial of the Strengthening Families Program (SFP). Process evaluation indicated high implementation fidelity, satisfaction, engagement, and attendance. Outcome evaluation results indicated positive changes in family-level functioning, caregivers' positive parenting, and caregiver depression symptoms from pre- to post-intervention, with some changes retained at follow-up 4 months later. Implications for preventive interventions with children of incarcerated parents, and their caregivers, are discussed.

Understanding influences on teachers' uptake and use of behaviour management strategies within the STARS trial: process evaluation protocol for a randomised controlled trial.

PubMed

Hansford, Lorraine; Sharkey, Siobhan; Edwards, Vanessa; Ukoumunne, Obioha; Byford, Sarah; Norwich, Brahm; Logan, Stuart; Ford, Tamsin

2015-02-10

The 'Supporting Teachers And childRen in Schools' (STARS) study is a cluster randomised controlled trial evaluating the Incredible Years Teacher Classroom Management (TCM) programme as a public health intervention. TCM is a 6 day training course delivered to groups of 8-12 teachers. The STARS trial will investigate whether TCM can improve children's behaviour, attainment and wellbeing, reduce teachers' stress and improve their self-efficacy. This protocol describes the methodology of the process evaluation embedded within the main trial, which aims to examine the uptake and implementation of TCM strategies within the classroom plus the wider school environment and improve the understanding of outcomes. The STARS trial will work with eighty teachers of children aged 4-9 years from eighty schools. Teachers will be randomised to attend the TCM course (intervention arm) or to "teach as normal" (control arm) and attend the course a year later. The process evaluation will use quantitative and qualitative approaches to assess fidelity to model, as well as explore headteachers' and teachers' experiences of TCM and investigate school factors that influence the translation of skills learnt to practice. Four of the eight groups of teachers (n = 40) will be invited to participate in focus groups within one month of completing the TCM course, and again a year later, while 45 of the 80 headteachers will be invited to take part in telephone interviews. Standardised checklists will be completed by group leaders and each training session will be videotaped to assess fidelity to model. Teachers will also complete standardised session evaluations. This study will provide important information about whether the Teacher Classroom Management course influences child and teacher mental health and well-being in both the short and long term. The process evaluation will provide valuable insights into factors that may facilitate or impede any impact. The trial has been registered with ISCTRN (Controlled Trials Ltd) and assigned an ISRCTN number ISRCTN84130388 . Date assigned: 15 May 2012.

How best to measure implementation of school health curricula: a comparison of three measures.

PubMed

Resnicow, K; Davis, M; Smith, M; Lazarus-Yaroch, A; Baranowski, T; Baranowski, J; Doyle, C; Wang, D T

1998-06-01

The impact of school health education programs is often attenuated by inadequate teacher implementation. Using data from a school-based nutrition education program delivered in a sample of fifth graders, this study examines the discriminant and predictive validity of three measures of curriculum implementation: class-room observation of fidelity, and two measures of completeness, teacher self-report questionnaire and post-implementation interview. A fourth measure, obtained during teacher observations, that assessed student and teacher interaction and student receptivity to the curriculum (labeled Rapport) was also obtained. Predictive validity was determined by examining the association of implementation measures with three study outcomes; health knowledge, asking behaviors related to fruit and vegetables, and fruit and vegetable intake, assessed by 7-day diary. Of the 37 teachers observed, 21 were observed for two sessions and 16 were observed once. Implementation measures were moderately correlated, an indication of discriminant validity. Predictive validity analyses indicated that the observed fidelity, Rapport and interview measures were significantly correlated with post-test student knowledge. The association between health knowledge and observed fidelity (based on dual observation only), Rapport and interview measures remained significant after adjustment for pre-test knowledge values. None of the implementation variables were significantly associated with student fruit and vegetable intake or asking behaviors controlling for pre-test values. These results indicate that the teacher self-report questionnaire was not a valid measure of implementation completeness in this study. Post-implementation completeness interviews and dual observations of fidelity and Rapport appear to be more valid, and largely independent methods of implementation assessment.

Implementation outcomes of Multidimensional Family Therapy-Detention to Community: a reintegration program for drug-using juvenile detainees.

PubMed

Liddle, Howard A; Dakof, Gayle A; Henderson, Craig; Rowe, Cindy

2011-06-01

Responding to urgent calls for effective interventions to address young offenders' multiple and interconnected problems, a new variant of an existing empirically-validated intervention for drug-using adolescents, Multidimensional Family Therapy (MDFT)-Detention to Community (DTC) was tested in a two-site controlled trial. This article (a) outlines the rationale and protocol basics of the MDFT-DTC intervention, a program for substance-using juvenile offenders that links justice and substance abuse treatment systems to facilitate adolescents' post-detention community reintegration; (b) presents implementation outcomes, including fidelity, treatment engagement and retention rates, amount of services received, treatment satisfaction, and substance abuse-juvenile justice system collaboration outcomes; and (c) details the implementation and sustainability challenges in a cross-system (substance abuse treatment and juvenile justice) adolescent intervention. Findings support the effectiveness of the MDFT-DTC intervention, and the need to develop a full implementation model in which transfer and dissemination issues could be explored more fully, and tested experimentally.

Capacity Development and Multi-Tiered Systems of Support: Guiding Principles

ERIC Educational Resources Information Center

Sugai, George; Simonsen, Brandi; Freeman, Jennifer; La Salle, Tamika

2016-01-01

Implementation of multi-tiered systems of support is occurring within and across a number of countries with an increased recent focus on the development of local system capacity to maintain high levels of practice implementation fidelity. The purpose of this paper is to describe the importance of local capacity development in the high fidelity and…

Customizing a Research-Based Reading Practice: Balancing the Importance of Implementation Fidelity with Professional Judgment

ERIC Educational Resources Information Center

McMaster, Kristen L.; Jung, Pyung-Gang; Brandes, Dana; Pinto, Viveca; Fuchs, Douglas; Kearns, Devin; Lemons, Christopher; Sáenz, Laura; Yen, Loulee

2014-01-01

Teachers are often asked to implement research-based instructional programs with fidelity and to ensure that all students reach high academic standards. These requests sometimes conflict when teachers find that not all their students are benefiting from the research-backed programs. In this article, we suggest that researchers and teachers can…

An Exploration of Fidelity of Implementation in Drug Abuse Prevention among Five Professional Groups

ERIC Educational Resources Information Center

Dusenbury, Linda; Brannigan, Rosalind; Falco, Mathea; Lake, Antonia

2004-01-01

To explore the extent of awareness among practitioners of the importance of fidelity of implementation and the factors that inhibit or encourage it, interviews were conducted with nine individuals in each of the following groups: 1) researchers who have developed and evaluated prevention curricula; 3) school administrators; 4) teachers; and 5)…

Implementing trauma-focused CBT with fidelity and flexibility: a family case study.

PubMed

Kerig, Patricia K; Sink, Holli E; Cuellar, Raven E; Vanderzee, Karin L; Elfstrom, Jennifer L

2010-01-01

Effective approaches for the treatment of childhood posttraumatic stress disorder and traumatic grief are needed given the prevalence of trauma and its impact on children's lives. To effectively treat posttraumatic stress disorder in children, evidence-based practices should be implemented with flexibility and responsiveness to culture, developmental level, and the specific needs of the family. This case study illustrates flexibility with fidelity in the use of a manualized treatment, describing the implementation of Trauma Focused-Cognitive Behavior Therapy with three traumatized family members-a caregiver and two children. Particular attention is paid to the use of creative strategies to tailor interventions to the individual clients while maintaining fidelity to the principles and components of this evidence-based treatment.

Treatment Fidelity among Family Health Promoters Delivering a Physical Activity and Nutrition Intervention to Immigrant and Refugee Families

ERIC Educational Resources Information Center

Bronars, Carrie A.; Hanza, Marcelo M.; Meiers, Sonja J.; Patten, Christi A.; Clark, Matthew M.; Nigon, Julie A.; Weis, Jennifer A.; Wieland, Mark L.; Sia, Irene G.

2017-01-01

Lack of treatment fidelity can be an important source of variation affecting the credibility and utility of outcomes from behavioral intervention research. Development and implementation of a well-designed treatment fidelity plan, especially with research involving underserved populations, requires careful conceptualization of study needs in…

Evidence, Fidelity, and Organisational Rationales: Multiple Uses of Motivational Interviewing in a Social Services Agency

ERIC Educational Resources Information Center

Björk, Alexander

2016-01-01

In discussions and empirical investigations of the implementation of evidence-based interventions there is often a narrow focus on treatment fidelity. Studying a social services agency trying to incorporate Motivational Interviewing (MI), commonly regarded as evidence-based, this paper problematises a one-sided attention to treatment fidelity by…

Interventionist training and intervention fidelity monitoring and maintenance for CONNECT, a nurse-led primary palliative care in oncology trial.

PubMed

Robbins-Welty, Gregg A; Mueser, Lisa; Mitchell, Chandler; Pope, Nicole; Arnold, Robert; Park, SeoYoung; White, Doug; Smith, Kenneth J; Reynolds, Charles; Rosenzweig, Margaret; Bakitas, Marie; Schenker, Yael

2018-06-01

Intervention fidelity is a critical component of behavioral research that has received inadequate attention in palliative care studies. With increasing focus on the need for palliative care models that can be widely disseminated and delivered by non-specialists, rigorous yet pragmatic strategies for training interventionists and maintaining intervention fidelity are needed. (1) Describe components of a plan for interventionist training and monitoring and maintaining intervention fidelity as part of a primary palliative care trial (CONNECT) and (2) present data about perceived training effectiveness and delivery of key intervention content. Post-training evaluations, visit checklists, and visit audio-recordings. Data were collected from June, 2016 through April, 2017. We include procedures for (1) identification, training and certification of oncology nurses as CONNECT interventionists; (2) monitoring intervention delivery; and (3) maintaining intervention quality. All nurses (N = 14) felt prepared to deliver key competencies after a 3-day in-person training. As assessed via visit checklists, interventionists delivered an average of 94% (SD 13%) of key content for first intervention visits and 85% (SD 14%) for subsequent visits. As assessed via audio-recordings, interventionists delivered an average of 85% (SD 8%) of key content for initial visits and 85% (SD 12%) for subsequent visits. We present a 3-part strategy for training interventionists and monitoring and maintaining intervention delivery in a primary palliative care trial. Training was effective in having nurses feel prepared to deliver primary palliative care skills. As assessed via nursing checklists and visit audio-recordings, intervention fidelity was high.

Intervention Fidelity: Aspects of Complementary and Alternative Medicine (CAM) Research

PubMed Central

Wyatt, Gwen; Sikorskii, Alla; Rahbar, Mohammad Hossein; Victorson, David; Adams, Lora

2010-01-01

Background The Treatment Fidelity Workgroup (TFW) established by the National Institutes of Health (NIH) provides a 5-point structure for intervention fidelity: dosing, interventionists’ consistency, intervention delivery, receipt and enactment of the intervention. Using our reflexology trial, we apply the first three points. Objectives Study objectives are to: 1) evaluate key dosage dimensions associated with CAM research; 2) evaluate approaches to interventionists’ consistency of delivery of CAM protocols; and 3) evaluate and discuss data that reflect CAM intervention fidelity. Intervention Women with late stage breast cancer (N=318) were randomly assigned to either 4 weeks of reflexology, placebo, or standard care. Results Dosing consists of three dimensions: frequency (4-sessions), duration (30 minutes), and interval between sessions (5–9 days). Interventionist consistency revealed over a 90% accuracy rate in following the protocol; 84% and 89% completion rate of the 4 session in the reflexology and placebo groups respectively; and no differences in attrition after randomization between reflexology and placebo groups (17% and 15%, respectively). Intervention delivery, examined through debriefing data, indicated a significantly higher rate of correct guesses on group assignment in the reflexology group as compared to the placebo (82% versus 46%, p-value=.0002). Conclusions This study points out the relevance of dosing, interventionists’ consistency, and delivery data within a CAM clinical trial, as well as the challenges of blinding. Implications Monitoring intervention fidelity by using the key areas identified by the BCC ensures that findings from a clinical trial are meaningful and have the potential to be translated to clinical practice. PMID:20467309

Implementing the LifeSkills Training drug prevention program: factors related to implementation fidelity.

PubMed

Mihalic, Sharon F; Fagan, Abigail A; Argamaso, Susanne

2008-01-18

Widespread replication of effective prevention programs is unlikely to affect the incidence of adolescent delinquency, violent crime, and substance use until the quality of implementation of these programs by community-based organizations can be assured. This paper presents the results of a process evaluation employing qualitative and quantitative methods to assess the extent to which 432 schools in 105 sites implemented the LifeSkills Training (LST) drug prevention program with fidelity. Regression analysis was used to examine factors influencing four dimensions of fidelity: adherence, dosage, quality of delivery, and student responsiveness. Although most sites faced common barriers, such as finding room in the school schedule for the program, gaining full support from key participants (i.e., site coordinators, principals, and LST teachers), ensuring teacher participation in training workshops, and classroom management difficulties, most schools involved in the project implemented LST with very high levels of fidelity. Across sites, 86% of program objectives and activities required in the three-year curriculum were delivered to students. Moreover, teachers were observed using all four recommended teaching practices, and 71% of instructors taught all the required LST lessons. Multivariate analyses found that highly rated LST program characteristics and better student behavior were significantly related to a greater proportion of material taught by teachers (adherence). Instructors who rated the LST program characteristics as ideal were more likely to teach all lessons (dosage). Student behavior and use of interactive teaching techniques (quality of delivery) were positively related. No variables were related to student participation (student responsiveness). Although difficult, high implementation fidelity by community-based organizations can be achieved. This study suggests some important factors that organizations should consider to ensure fidelity, such as selecting programs with features that minimize complexity while maximizing flexibility. Time constraints in the classroom should be considered when choosing a program. Student behavior also influences program delivery, so schools should train teachers in the use of classroom management skills. This project involved comprehensive program monitoring and technical assistance that likely facilitated the identification and resolution of problems and contributed to the overall high quality of implementation. Schools should recognize the importance of training and technical assistance to ensure quality program delivery.

A Theoretically Based Behavioral Nutrition Intervention for Community Elders at High Risk: The B-NICE Randomized Controlled Clinical Trial

PubMed Central

LOCHER, JULIE L.; BALES, CONNIE W.; ELLIS, AMY C.; LAWRENCE, JEANNINE C.; NEWTON, LAURA; RITCHIE, CHRISTINE S.; ROTH, DAVID L.; BUYS, DAVID L.; VICKERS, KRISTIN S.

2012-01-01

We conducted a study designed to evaluate the efficacy and feasibility of a multilevel self-management intervention to improve nutritional intake in a group of older adults receiving Medicare home health services who were at especially high risk for experiencing undernutrition. The Behavioral Nutrition Intervention for Community Elders (B-NICE) trial used a prospective randomized controlled design to determine whether individually tailored counseling focused on social and behavioral aspects of eating resulted in increased caloric intake and improved nutrition-related health outcomes in a high-risk population of older adults. The study was guided by the theoretical approaches of the Ecological Model and Social Cognitive Theory. The development and implementation of the B-NICE protocol, including the theoretical framework, methodology, specific elements of the behavioral intervention, and assurances of the treatment fidelity, as well as the health policy implications of the trial results, are presented in this article. PMID:22098180

Solving Quantum Ground-State Problems with Nuclear Magnetic Resonance

PubMed Central

Li, Zhaokai; Yung, Man-Hong; Chen, Hongwei; Lu, Dawei; Whitfield, James D.; Peng, Xinhua; Aspuru-Guzik, Alán; Du, Jiangfeng

2011-01-01

Quantum ground-state problems are computationally hard problems for general many-body Hamiltonians; there is no classical or quantum algorithm known to be able to solve them efficiently. Nevertheless, if a trial wavefunction approximating the ground state is available, as often happens for many problems in physics and chemistry, a quantum computer could employ this trial wavefunction to project the ground state by means of the phase estimation algorithm (PEA). We performed an experimental realization of this idea by implementing a variational-wavefunction approach to solve the ground-state problem of the Heisenberg spin model with an NMR quantum simulator. Our iterative phase estimation procedure yields a high accuracy for the eigenenergies (to the 10−5 decimal digit). The ground-state fidelity was distilled to be more than 80%, and the singlet-to-triplet switching near the critical field is reliably captured. This result shows that quantum simulators can better leverage classical trial wave functions than classical computers PMID:22355607

Strengthening organizations to implement evidence-based clinical practices.

PubMed

VanDeusen Lukas, Carol; Engle, Ryann L; Holmes, Sally K; Parker, Victoria A; Petzel, Robert A; Nealon Seibert, Marjorie; Shwartz, Michael; Sullivan, Jennifer L

2010-01-01

Despite recognition that implementation of evidence-based clinical practices (EBPs) usually depends on the structure and processes of the larger health care organizational context, the dynamics of implementation are not well understood. This project's aim was to deepen that understanding by implementing and evaluating an organizational model hypothesized to strengthen the ability of health care organizations to facilitate EBPs. CONCEPTUAL MODEL: The model posits that implementation of EBPs will be enhanced through the presence of three interacting components: active leadership commitment to quality, robust clinical process redesign incorporating EBPs into routine operations, and use of management structures and processes to support and align redesign. In a mixed-methods longitudinal comparative case study design, seven medical centers in one network in the Department of Veterans Affairs participated in an intervention to implement the organizational model over 3 years. The network was selected randomly from three interested in using the model. The target EBP was hand-hygiene compliance. Measures included ratings of implementation fidelity, observed hand-hygiene compliance, and factors affecting model implementation drawn from interviews. Analyses support the hypothesis that greater fidelity to the organizational model was associated with higher compliance with hand-hygiene guidelines. High-fidelity sites showed larger effect sizes for improvement in hand-hygiene compliance than lower-fidelity sites. Adherence to the organizational model was in turn affected by factors in three categories: urgency to improve, organizational environment, and improvement climate. Implementation of EBPs, particularly those that cut across multiple processes of care, is a complex process with many possibilities for failure. The results provide the basis for a refined understanding of relationships among components of the organizational model and factors in the organizational context affecting them. This understanding suggests practical lessons for future implementation efforts and contributes to theoretical understanding of the dynamics of the implementation of EBPs.

Understanding Leisure-related Program Effects by Using Process Data in the HealthWise South Africa Project

PubMed Central

Caldwell, Linda L.; Younker, Anita S.; Wegner, Lisa; Patrick, Megan E.; Vergnani, Tania; Smith, Edward A.; Flisher, Alan J.

2010-01-01

Executive Summary As the push for evidence-based programming gathers momentum, many human services programs and interventions are under increased scrutiny to justify their effectiveness across different conditions and populations. Government agencies and the public want to be assured that their resources are being put to good use on programs that are effective and efficient (Guskey, 2000). Thus, programs are increasingly based on theory and evaluated through randomized control trials using longitudinal data. Despite this progress, hypothesized outcomes are often not detected and/or their effect sizes are small (Gingiss, Roberts-Gray, Boerm, 2006). Moreover, findings may go against intuition or “gut feelings” on the part of project staff. Given the need to understand how program implementation issues relate to outcomes, this study focuses on whether process measures that focus on program implementation and fidelity can shed light on associated outcomes. In particular, we linked the process evaluation of the HealthWise motivation lesson with outcomes across four waves of data collection. We hypothesized that HealthWise would increase learners’ intrinsic and identified forms of motivation, and decrease amotivation and extrinsic motivation. We did not hypothesize a direction of effects on introjected motivation due to its conceptual ambiguity. Data came from youth in four intervention schools (n = 902, 41.1%) and five control schools (n = 1291, 58.9%) who were participating in a multi-cohort, longitudinal study. The schools were in a township near Cape Town, South Africa. For each cohort, baseline data are collected on learners as they begin Grade 8. We currently have four waves of data collected on the first cohort, which is the focus of this paper. The mean age of the sample at Wave 3 was 15.0 years (SD = .86) and 51% of students were female. Results suggested that there was evidence of an overall program effect of the curriculum on amotivation regardless of fidelity of implementation. Compared to the control schools, all treatment school learners reported lower levels of amotivation in Wave 4 compared to Wave 3, as hypothesized. Using process evaluation data to monitor implementation fidelity, however, we also conclude that the school with better trained teachers who also reported higher levels of program fidelity had better outcomes than the other schools. We discuss the implications of linking process data with outcome data and associated methodological challenges in linking these data. PMID:20640186

Exploring the Impact of Engaged Teachers on Implementation Fidelity and Reading Skill Gains in a Blended Learning Reading Program

ERIC Educational Resources Information Center

Schechter, Rachel L.; Kazakoff, Elizabeth R.; Bundschuh, Kristine; Prescott, Jen Elise; Macaruso, Paul

2017-01-01

The number of K-12 classrooms adopting blended learning models is rapidly increasing and represents a cultural shift in teaching and learning; however, fidelity of implementation of these new blended learning programs varies widely. This study aimed to examine the role of teacher engagement in student motivation and achievement in a blended…

The Fidelity of Implementation of the Response to Intervention (RTI) Process in Missouri Public Schools

ERIC Educational Resources Information Center

Drury, Debra A.

2014-01-01

The purpose of this problem-based study was to gather data which analyzed the degree of fidelity of implementation of Response to Intervention as reported by building principals in the State of Missouri. The project began when team members, providing professional development for the Response to Intervention process, came to the conclusion there…

Evaluating the implementation process of a participatory organizational level occupational health intervention in schools.

PubMed

Schelvis, Roosmarijn M C; Wiezer, Noortje M; Blatter, Birgitte M; van Genabeek, Joost A G M; Oude Hengel, Karen M; Bohlmeijer, Ernst T; van der Beek, Allard J

2016-12-01

The importance of process evaluations in examining how and why interventions are (un) successful is increasingly recognized. Process evaluations mainly studied the implementation process and the quality of the implementation (fidelity). However, in adopting this approach for participatory organizational level occupational health interventions, important aspects such as context and participants perceptions are missing. Our objective was to systematically describe the implementation process of a participatory organizational level occupational health intervention aimed at reducing work stress and increasing vitality in two schools by applying a framework that covers aspects of the intervention and its implementation as well as the context and participants perceptions. A program theory was developed, describing the requirements for successful implementation. Each requirement was operationalized by making use of the framework, covering: initiation, communication, participation, fidelity, reach, communication, satisfaction, management support, targeting, delivery, exposure, culture, conditions, readiness for change and perceptions. The requirements were assessed by quantitative and qualitative data, collected at 12 and 24 months after baseline in both schools (questionnaire and interviews) or continuously (logbooks). The intervention consisted of a needs assessment phase and a phase of implementing intervention activities. The needs assessment phase was implemented successfully in school A, but not in school B where participation and readiness for change were insufficient. In the second phase, several intervention activities were implemented at school A, whereas this was only partly the case in school B (delivery). In both schools, however, participants felt not involved in the choice of intervention activities (targeting, participation, support), resulting in a negative perception of and only partial exposure to the intervention activities. Conditions, culture and events hindered the implementation of intervention activities in both schools. The framework helped us to understand why the implementation process was not successful. It is therefore considered of added value for the evaluation of implementation processes in participatory organizational level interventions, foremost because of the context and mental models dimensions. However, less demanding methods for doing detailed process evaluations need to be developed. This can only be done if we know more about the most important process components and this study contributes to that knowledge base. Netherlands Trial Register NTR3284 .

Factors associated with physical therapists' implementation of physical activity interventions in The Netherlands.

PubMed

Huijg, Johanna M; Dusseldorp, Elise; Gebhardt, Winifred A; Verheijden, Marieke W; van der Zouwe, Nicolette; Middelkoop, Barend J C; Duijzer, Geerke; Crone, Mathilde R

2015-04-01

Physical therapists play an important role in the promotion of physical activity (PA) and the effectiveness of PA interventions. However, little is known about the extent to which they implement PA interventions following the intervention protocol and about the factors influencing their implementation behaviors. The study objective was to investigate physical therapists' implementation fidelity regarding PA interventions, including completeness and quality of delivery, and influencing factors with a Theoretical Domains Framework-based questionnaire. The study was based on a cross-sectional design. A total of 268 physical therapists completed the Determinants of Implementation Behavior Questionnaire. Questions about completeness and quality of delivery were based on components and tasks of PA interventions as described by the Royal Dutch Society for Physical Therapy. Multilevel regression analyses were used to identify factors associated with completeness and quality of delivery. High implementation fidelity was found for the physical therapists, with higher scores for completeness of delivery than for quality of delivery. Physical therapists' knowledge, skills, beliefs about capabilities and consequences, positive emotions, behavioral regulation, and the automaticity of PA intervention delivery were the most important predictors of implementation fidelity. Together, the Theoretical Domains Framework accounted for 23% of the variance in both total completeness and total quality scores. The cross-sectional design precluded the determination of causal relationships. Also, the use of a self-report measure to assess implementation fidelity could have led to socially desirable responses, possibly resulting in more favorable ratings for completeness and quality. This study enhances the understanding of how physical therapists implement PA interventions and which factors influence their behaviors. Knowledge about these factors may assist in the development of strategies to improve physical therapists' implementation behaviors. © 2015 American Physical Therapy Association.

Experimental magic state distillation for fault-tolerant quantum computing.

PubMed

Souza, Alexandre M; Zhang, Jingfu; Ryan, Colm A; Laflamme, Raymond

2011-01-25

Any physical quantum device for quantum information processing (QIP) is subject to errors in implementation. In order to be reliable and efficient, quantum computers will need error-correcting or error-avoiding methods. Fault-tolerance achieved through quantum error correction will be an integral part of quantum computers. Of the many methods that have been discovered to implement it, a highly successful approach has been to use transversal gates and specific initial states. A critical element for its implementation is the availability of high-fidelity initial states, such as |0〉 and the 'magic state'. Here, we report an experiment, performed in a nuclear magnetic resonance (NMR) quantum processor, showing sufficient quantum control to improve the fidelity of imperfect initial magic states by distilling five of them into one with higher fidelity.

Toward fidelity between specification and implementation

NASA Technical Reports Server (NTRS)

Callahan, John R.; Montgomery, Todd L.; Morrison, Jeff; Wu, Yunqing

1994-01-01

This paper describes the methods used to specify and implement a complex communications protocol that provides reliable delivery of data in multicast-capable, packet-switching telecommunication networks. The protocol, called the Reliable Multicasting Protocol (RMP), was developed incrementally by two complementary teams using a combination of formal and informal techniques in an attempt to ensure the correctness of the protocol implementation. The first team, called the Design team, initially specified protocol requirements using a variant of SCR requirements tables and implemented a prototype solution. The second team, called the V&V team, developed a state model based on the requirements tables and derived test cases from these tables to exercise the implementation. In a series of iterative steps, the Design team added new functionality to the implementation while the V&V team kept the state model in fidelity with the implementation through testing. Test cases derived from state transition paths in the formal model formed the dialogue between teams during development and served as the vehicles for keeping the model and implementation in fidelity with each other. This paper describes our experiences in developing our process model, details of our approach, and some example problems found during the development of RMP.

Intervention Fidelity and Facilitator Training.

PubMed

Rew, Lynn; Banner, Matthew; Johnson, Karen; Slesnick, Natasha

2018-01-01

Intervention fidelity is an ongoing concern for rigorous research, from the initial stages of planning and study design to the maintenance of internal validity. An added concern is the balance between fidelity and design accommodation to better suit varied populations and individuals. In this article, we describe our process for monitoring intervention fidelity during an individualized, yet standardized, strengths-based intervention with homeless youths, in which we include periodic training of our professional intervention facilitators. In our ongoing study, which is based on a Solomon four-group design with repeated measures, monitoring and training are essential to ensure intervention fidelity. Despite a rich literature about intervention fidelity, little guidance is available to help researchers and practitioners implement fidelity strategies in the real world with vulnerable populations. This article addresses this gap.

Development and Implementation of Tele-Savvy for Dementia Caregivers: A Department of Veterans Affairs Clinical Demonstration Project.

PubMed

Griffiths, Patricia C; Whitney, M Kate; Kovaleva, Mariya; Hepburn, Kenneth

2016-02-01

To test fidelity and preliminary efficacy of Tele-Savvy, an internet-based version of the in-person, evidence-based psychoeducation Savvy Caregiver Program (SCP) for dementia caregivers. Tele-Savvy used synchronous (tele-conferences) and asynchronous components (video modules) to provide program access to caregivers in their homes. SCP experts were surveyed to evaluate Tele-Savvy's fidelity to SCP. A convenience sample of 30 dementia caregivers from the Atlanta VA Medical Center enrolled in the Tele-Savvy clinical demonstration program. Twenty-two caregivers completed both pre- and postprogram evaluations, which included measures of caregiver stress and competence and behavioral and psychological symptoms of dementia (BPSD). Expert review confirmed Tele-Savvy's fidelity with and, in certain domains, improvement on the original. Participants demonstrated moderately high initial levels of burden, anxiety, and depressive symptoms all of which decreased significantly postprogram in an intention to treat analysis. Similar reductions were seen in care recipients' BPSD and caregivers' reactions to them. Marginally significant increases in caregiver competence were observed. Caregiver and expert panel evaluations of program quality were enthusiastic. An effective online caregiver psychoeducation program could provide greatly expanded access for caregivers who cannot attend in person for reasons of distance, transportation limitations, and caregiving responsibilities. Further testing in a controlled trial is needed to establish program efficacy. Published by Oxford University Press on behalf of the Gerontological Society of America 2015.

A Qualitative Analysis of the Concepts of Fidelity and Adaptation in the Implementation of an Evidence-Based HIV Prevention Intervention

ERIC Educational Resources Information Center

Owczarzak, Jill; Broaddus, Michelle; Pinkerton, Steven

2016-01-01

Continued debate about the relative value of fidelity versus adaptation, and lack of clarity about the meaning of fidelity, raise concerns about how frontline service providers resolve similar issues in their daily practice. We use SISTA ("Sisters Informing Sisters on Topics about acquired immune deficiency syndrome"), an evidence-based…

Assessing the Implementation Fidelity of a School-Based Teaching Personal and Social Responsibility Program in Physical Education and Other Subject Areas

ERIC Educational Resources Information Center

Escartí, Amparo; Liops-Goig, Ramon; Wright, Paul M.

2018-01-01

Purpose: The Teaching Personal and Social Responsibility (TPSR) model was developed to foster responsibility and teach life skills that transfer to various settings. The purpose of this study was to assess the implementation fidelity of a school-based TPSR program in physical education and other subject areas. Method: Systematic observation was…

Jail-to-community treatment continuum for adults with co-occurring substance use and mental disorders: study protocol for a pilot randomized controlled trial.

PubMed

Van Dorn, Richard A; Desmarais, Sarah L; Rade, Candalyn B; Burris, Elizabeth N; Cuddeback, Gary S; Johnson, Kiersten L; Tueller, Stephen J; Comfort, Megan L; Mueser, Kim T

2017-08-04

Adults with co-occurring mental and substance use disorders (CODs) are overrepresented in jails. In-custody barriers to treatment, including a lack of evidence-based treatment options and the often short periods of incarceration, and limited communication between jails and community-based treatment agencies that can hinder immediate enrollment into community care once released have contributed to a cycle of limited treatment engagement, unaddressed criminogenic risks, and (re)arrest among this vulnerable and high-risk population. This paper describes a study that will develop research and communication protocols and adapt two evidence-based treatments, dual-diagnosis motivational interviewing (DDMI) and integrated group therapy (IGT), for delivery to adults with CODs across a jail-to-community treatment continuum. Adaptations to DDMI and IGT were guided by the Risk-Need-Responsivity model and the National Institute of Corrections' implementation competencies; the development of the implementation framework and communication protocols were guided by the Evidence-Based Interagency Implementation Model for community corrections and the Inter-organizational Relationship model, respectively. Implementation and evaluation of the protocols and adapted interventions will occur via an open trial and a pilot randomized trial. The clinical intervention consists of two in-jail DDMI sessions and 12 in-community IGT sessions. Twelve adults with CODs and four clinicians will participate in the open trial to evaluate the acceptability and feasibility of, and fidelity to, the interventions and research and communication protocols. The pilot controlled trial will be conducted with 60 inmates who will be randomized to either DDMI-IGT or treatment as usual. A baseline assessment will be conducted in jail, and four community-based assessments will be conducted during a 6-month follow-up period. Implementation, clinical, public health, and treatment preference outcomes will be evaluated. Findings have the potential to improve both jail- and community-based treatment services for adults with CODs as well as inform methods for conducting rigorous pilot implementation and evaluation research in correctional settings and as inmates re-enter the community. Findings will contribute to a growing area of work focused on interrupting the cycle of limited treatment engagement, unaddressed criminogenic risks, and (re)arrest among adults with CODs. ClinicalTrials.gov, NCT02214667 . Registered on 10 August 2014.

Randomized Control Trial of COMPASS for Improving Transition Outcomes of Students with Autism Spectrum Disorder.

PubMed

Ruble, Lisa A; McGrew, John H; Toland, Michael; Dalrymple, Nancy; Adams, Medina; Snell-Rood, Claire

2018-06-01

The postsecondary outcomes of individuals with autism spectrum disorder (ASD) are significantly worse than peers with other disabilities. One problem is the lack of empirically-supported transition planning interventions to guide services and help produce better outcomes. We applied an implementation science approach to adapt and modify an evidence-based consultation intervention originally tested with young children called the Collaborative Model for Promoting Competence and Success (COMPASS; Ruble et al., The collaborative model for promoting competence and success for students with ASD. Springer, New York, 2012a) and evaluate it for efficacy in a randomized controlled trial for transition-age youth. Results replicated findings with younger students with ASD that IEP outcomes were higher for COMPASS compared to the placebo control group (d = 2.1). Consultant fidelity was high and teacher adherence improved over time, replicating the importance of ongoing teacher coaching.

EVIDENCE-BASED PROTOCOLS

PubMed Central

Beissner, Katherine L.; Bach, Eileen; Murtaugh, Christopher M.; Trifilio, MaryGrace; Henderson, Charles R.; Barrón, Yolanda; Trachtenberg, Melissa A.; Reid, M. Carrington

2017-01-01

Activity-limiting pain is common among older home care patients and pain management is complicated by the high prevalence of physical frailty and multimorbidity in the home care population. A comparative effectiveness study was undertaken at a large urban home care agency to examine an evidence-based pain self-management program delivered by physical therapists (PTs). This article focuses on PT training, methods implemented to reinforce content after training and to encourage uptake of the program with appropriate patients, and therapists’ fidelity to the program. Seventeen physical therapy teams were included in the cluster randomized controlled trial, with 8 teams (155 PTs) assigned to a control and 9 teams (165 PTs) assigned to a treatment arm. Treatment therapists received interactive training over two sessions, with a follow-up session 6 months later. Additional support was provided via emails, e-learning materials including videos, and a therapist manual. Program fidelity was assessed by examining PT pain documentation in the agency’s electronic health record. PT feedback on the program was obtained via semistructured surveys. There were no between-group differences in the number of PTs documenting program elements with the exception of instruction in the use of imagery, which was documented by a higher percentage of intervention therapists (p = 0.002). PTs felt comfortable teaching the program elements, but cited time as the biggest barrier to implementing the protocol. Possible explanations for study results suggesting limited adherence to the program protocol by intervention-group PTs include the top-down implementation strategy, competing organizational priorities, program complexity, competing patient priorities, and inadequate patient buy-in. Implications for the implementation of complex new programs in the home healthcare setting are discussed. PMID:28157776

Practical experimental certification of computational quantum gates using a twirling procedure.

PubMed

Moussa, Osama; da Silva, Marcus P; Ryan, Colm A; Laflamme, Raymond

2012-08-17

Because of the technical difficulty of building large quantum computers, it is important to be able to estimate how faithful a given implementation is to an ideal quantum computer. The common approach of completely characterizing the computation process via quantum process tomography requires an exponential amount of resources, and thus is not practical even for relatively small devices. We solve this problem by demonstrating that twirling experiments previously used to characterize the average fidelity of quantum memories efficiently can be easily adapted to estimate the average fidelity of the experimental implementation of important quantum computation processes, such as unitaries in the Clifford group, in a practical and efficient manner with applicability in current quantum devices. Using this procedure, we demonstrate state-of-the-art coherent control of an ensemble of magnetic moments of nuclear spins in a single crystal solid by implementing the encoding operation for a 3-qubit code with only a 1% degradation in average fidelity discounting preparation and measurement errors. We also highlight one of the advances that was instrumental in achieving such high fidelity control.

A Process Evaluation of an Efficacious Family-Based Intervention to Promote Healthy Eating: The Entre Familia: Reflejos de Salud Study.

PubMed

Schmied, Emily; Parada, Humberto; Horton, Lucy; Ibarra, Leticia; Ayala, Guadalupe

2015-10-01

Entre Familia: Reflejos de Salud was a successful family-based randomized controlled trial designed to improve dietary behaviors and intake among U.S. Latino families, specifically fruit and vegetable intake. The novel intervention design merged a community health worker (promotora) model with an entertainment-education component. This process evaluation examined intervention implementation and assessed relationships between implementation factors and dietary change. Participants included 180 mothers randomized to an intervention condition. Process evaluation measures were obtained from participant interviews and promotora notes and included fidelity, dose delivered (i.e., minutes of promotora in-person contact with families, number of promotora home visits), and dose received (i.e., participant use of and satisfaction with intervention materials). Outcome variables included changes in vegetable intake and the use of behavioral strategies to increase dietary fiber and decrease dietary fat intake. Participant satisfaction was high, and fidelity was achieved; 87.5% of families received the planned number of promotora home visits. In the multivariable model, satisfaction with intervention materials predicted more frequent use of strategies to increase dietary fiber (p ≤ .01). Trends suggested that keeping families in the prescribed intervention timeline and obtaining support from other social network members through sharing of program materials may improve changes. Study findings elucidate the relationship between specific intervention processes and dietary changes. © 2015 Society for Public Health Education.

Fidelity and outcomes in six integrated dual disorders treatment programs.

PubMed

Chandler, Daniel W

2011-02-01

Fidelity scores and outcomes were measured in six outpatient programs in California which implemented Integrated Dual Disorders Treatment (IDDT). Outcomes were measured for 1 year in four sites and 2 years in two sites; fidelity was assessed at 6 month intervals. Three of the six sites achieved high fidelity (at least a 4 on a 5 point fidelity scale) and three moderate fidelity (at least a 3). Retention in treatment, mental health functioning, stage of substance abuse treatment, abstinence, and psychiatric hospitalization were measured. Outcomes for individual programs were generally positive but not consistent within programs or across programs. Using pooled data in a longitudinal regression model with random effects at person level and adjustment of standard errors for clustering by site, change over time was not statistically significant for the primary outcomes. Fidelity scores had limited association with positive outcomes.

Development and validation of a Housing First fidelity survey.

PubMed

Gilmer, Todd P; Stefancic, Ana; Sklar, Marisa; Tsemberis, Sam

2013-09-01

Programs that use the Housing First model are being implemented throughout the United States and internationally. The authors describe the development and validation of a Housing First fidelity survey. A 46-item survey was developed to measure fidelity across five domains: housing process and structure, separation of housing and services, service philosophy, service array, and team structure. The survey was administered to staff and clients of 93 supported-housing programs in California. Exploratory and confirmatory factor analyses were used to identify the items and model structure that best fit the data. Sixteen items were retained in a two-factor model, one related to approach to housing, separation of housing and services, and service philosophy and one related to service array and team structure. Our survey mapped program practices by using a common metric that captured variation in fidelity to Housing First across a large-scale implementation of supported-housing programs.

Operational fidelity to an evidence-based HIV prevention intervention for people living with HIV/AIDS.

PubMed

Kalichman, Seth C; Hudd, Katie; Diberto, Giorgio

2010-08-01

Evidence-based interventions are often disseminated in public health education with little known about their operational fidelity. This study examined the delivery of intervention components (operational fidelity) of a widely disseminated HIV prevention program designed for people living with HIV/AIDS named Healthy Relationships. Two hundred ninety-nine agencies that had been trained in the intervention by the Centers for Disease Control and Prevention were contacted, and 122 (41%) completed confidential interviews. Among the 93 agencies that implemented the program, 39 (40%) adapted at least one core element activity, and 21 (23%) dropped an activity. Most adaptations were intended to improve the community fit of the intervention. Agencies believed that funders demand that they implement the intervention with fidelity. Models of technology transfer that emphasize behavior change processes rather than specific curriculum content may advance prevention program dissemination.

Individualized Clinical Coaching in the TLE TeachLivE Lab: Enhancing Fidelity of Implementation of System of Least Prompts among Novice Teachers of Students with Autism

ERIC Educational Resources Information Center

Vince Garland, Krista M.; Holden, Kara; Garland, Dennis Patrick

2016-01-01

One of the challenges facing educators who teach students with autism spectrum disorders is implementing evidence-based practices (EBPs) with fidelity. One EBP used to help such learners in acquiring targeted behavioral or academic skills is prompting. In this study, the authors examine the efficacy of individualized clinical coaching (ICC) of…

Factors affecting adoption, implementation fidelity, and sustainability of the Redesigned Community Health Fund in Tanzania: a mixed methods protocol for process evaluation in the Dodoma region

PubMed Central

Kalolo, Albino; Radermacher, Ralf; Stoermer, Manfred; Meshack, Menoris; De Allegri, Manuela

2015-01-01

Background Despite the implementation of various initiatives to address low enrollment in voluntary micro health insurance (MHI) schemes in sub-Saharan Africa, the problem of low enrollment remains unresolved. The lack of process evaluations of such interventions makes it difficult to ascertain whether their poor results are because of design failures or implementation weaknesses. Objective In this paper, we describe a process evaluation protocol aimed at opening the ‘black box’ to evaluate the implementation processes of the Redesigned Community Health Fund (CHF) program in the Dodoma region of Tanzania. Design The study employs a cross-sectional mixed methods design and is being carried out 3 years after the launch of the Redesigned CHF program. The study is grounded in a conceptual framework which rests on the Diffusion of Innovation Theory and the Implementation Fidelity Framework. The study utilizes a mixture of quantitative and qualitative data collection tools (questionnaires, focus group discussions, in-depth interviews, and document review), and aligns the evaluation to the Theory of Intervention developed by our team. Quantitative data will be used to measure program adoption, implementation fidelity, and their moderating factors. Qualitative data will be used to explore the responses of stakeholders to the intervention, contextual factors, and moderators of adoption, implementation fidelity, and sustainability. Discussion This protocol describes a systematic process evaluation in relation to the implementation of a reformed MHI. We trust that the theoretical approaches and methodologies described in our protocol may be useful to inform the design of future process evaluations focused on the assessment of complex interventions, such as MHI schemes. PMID:26679408

Formative Evaluation: A Case Study of the Illinois Birth-to-Three IV-E Waiver.

PubMed

Rolock, Nancy; Snyder, Susan M; Tate, Cynthia

2015-01-01

Few interventions addressing early childhood trauma have adequate evidence. The Illinois Birth-to-Three Waiver is a 5-year federal project to reduce trauma and increase permanency for young children. In this article the authors describe the explorative and formative phases, including: (a) developing a well-built research question; (b) working with purveyors of evidence-supported interventions (ESIs); (c) adapting ESIs to a large public child welfare system; (d) developing fidelity measures, assessment data and outcomes; and (e) managing the challenges of implementing a randomized controlled trial. Public and private agency staff work in conjunction with university partners to merge implementation science, program evaluation, and intervention research. The project's theory of change stipulates that working with the triad of birth parent(s), foster parent(s), and focal child is critical to achieving project outcomes. The authors articulate the importance of working closely with an interdisciplinary team to achieve these outcomes, and the challenges associated with the early stages of implementation.

The Falls In Care Home study: a feasibility randomized controlled trial of the use of a risk assessment and decision support tool to prevent falls in care homes

PubMed Central

Walker, Gemma M; Armstrong, Sarah; Gordon, Adam L; Gladman, John; Robertson, Kate; Ward, Marie; Conroy, Simon; Arnold, Gail; Darby, Janet; Frowd, Nadia; Williams, Wynne; Knowles, Sue; Logan, Pip A

2015-01-01

Objective: To explore the feasibility of implementing and evaluating the Guide to Action Care Home fall prevention intervention. Design: Two-centre, cluster feasibility randomized controlled trial and process evaluation. Setting: Purposive sample of six diverse old age/learning disability, long stay care homes in Nottinghamshire, UK. Subjects: Residents aged over 50 years, who had fallen at least once in the past year, not bed-bound, hoist-dependent or terminally ill. Interventions: Intervention homes (n = 3) received Guide to Action Care Home fall prevention intervention training and support. Control homes (n = 3) received usual care. Outcomes: Recruitment, attrition, baseline and six-month outcome completion, contamination and intervention fidelity, compliance, tolerability, acceptance and impact. Results: A total of 81 of 145 (56%) care homes expressed participatory interest. Six of 22 letter respondent homes (27%) participated. The expected resident recruitment target was achieved by 76% (52/68). Ten (19%) residents did not complete follow-up (seven died, three moved). In intervention homes 36/114 (32%) staff attended training. Two of three (75%) care homes received protocol compliant training. Staff valued the training, but advised greater management involvement to improve intervention implementation. Fall risks were assessed, actioned and recorded in care records. Of 115 recorded falls, 533/570 (93%) of details were complete. Six-month resident fall rates were 1.9 and 4.0 per year for intervention and control homes, respectively. Conclusions: The Guide to Action Care Home is implementable under trial conditions. Recruitment and follow-up rates indicate that a definitive trial can be completed. Falls (primary outcome) can be ascertained reliably from care records. PMID:26385358

Feasibility of a Trial on Improvisational Music Therapy for Children with Autism Spectrum Disorder.

PubMed

Geretsegger, Monika; Holck, Ulla; Bieleninik, Łucja; Gold, Christian

2016-01-01

To conduct generalizable, rigorously designed, adequately powered trials investigating music therapy and other complex interventions, it is essential that study procedures are feasible and acceptable for participants. To date, only limited evidence on feasibility of trial designs and strategies to facilitate study implementation is available in the music therapy literature. Using data from a subsample of a multi-center RCT on improvisational music therapy (IMT) for autism spectrum disorder (ASD), this study aims to evaluate feasibility of study procedures, evaluate safety, document concomitant treatment, and report consistency of individuals' trends over time in chosen outcome measures. Children with ASD aged between 4 years, 0 months, and 6 years, 11 months, were randomly assigned to one of three conditions: one (low intensity) vs. three weekly IMT sessions (high intensity) for five months vs. standard care. Feasibility was evaluated by examining recruitment, implementation of study conditions, assessment procedures, blinding, and retention; we also evaluated safety, concomitant treatment, and consistency of changes in standardized scales completed by blinded assessors and parents before and 5 months after randomization. Within this subsample (n = 15), recruitment rates, session attendance in the high-intensity condition, and consistency between outcome measures were lower than expected. Session attendance in the low-intensity and control conditions, treatment fidelity, measurement completion, blinding, retention, and safety met a priori thresholds for feasibility. By discussing strategies to improve recruitment and to minimize potential burden on study participants, referrers, and researchers, this study helps build knowledge about designing and implementing trials successfully. © the American Music Therapy Association 2016. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Effectiveness of psychological interventions delivered by non-psychologists on low back pain and disability: a qualitative systematic review.

PubMed

Bostick, Geoff P

2017-11-01

Psychological treatments delivered by non-psychologists have been proposed as a way to increase access to care to address important psychological barriers to recovery in people with low back pain (LBP). This review aimed to synthesize randomized controlled trials (RCTs) that assess the effectiveness of psychological interventions delivered by non-psychologists in reducing pain intensity and disability in adults with LBP, compared with usual care. A systematic review without meta-analysis was carried out. Randomized controlled trials including adult patients with all types of musculoskeletal LBP were eligible. Interventions included those based on psychological principles and delivered by non-psychologists. The primary outcomes of interest were self-reported pain intensity and disability. Information sources included Medline, EMBASE, and the Cochrane Central Registrar for Controlled Trials. The Cochrane Collaboration's tool for assessing risk of bias was used for the evaluation of internal validity. There were 1,101 records identified, 159 were assessed for eligibility, 16 were critically appraised, and 11 studies were included. Mild to moderate risk of bias was present in the included studies, with personnel and patient blinding, treatment fidelity, and attrition being the most common sources of bias. Considerable heterogeneity existed for patient population, intervention components, and comparison groups. Although most studies demonstrated statistical and clinical improvements in pain and disability, few were statistically superior to the comparison group. Consistent with the broader psychological literature, psychological interventions delivered by non-psychologists have modest effects on low back pain and disability. Additional high quality research is needed to understand what patients are likely to respond to psychological interventions, the appropriate dose to achieve the desired outcome, the amount of training required to implement psychological interventions, and the optimal procedures to ensure treatment fidelity. Copyright © 2017 Elsevier Inc. All rights reserved.

Exploring Implementation and Fidelity of Evidence-Based Behavioral Interventions for HIV Prevention: Lessons Learned From the Focus on Kids Diffusion Case Study

PubMed Central

Galbraith, Jennifer S.; Stanton, Bonita; Boekeloo, Bradley; King, Winifred; Desmond, Sharon; Howard, Donna; Black, Maureen M.; Carey, James W.

2014-01-01

Evidence-based interventions (EBIs) are used in public health to prevent HIV infection among youth and other groups. EBIs include core elements, features that are thought to be responsible for the efficacy of interventions. The authors evaluate experiences of organizations that adopted an HIV-prevention EBI, Focus on Kids (FOK), and their fidelity to the intervention’s eight core elements. A cross-sectional telephone survey was administered to 34 staff members from organizations that had previously implemented FOK. Questions assessed how the organization adhered to, adapted, dropped, or altered the intervention. None of the organizations implemented all eight core elements. This study underscores the importance for HIV intervention researchers to clearly identify and describe core elements. More effort is needed to reflect the constraints practitioners face in nonresearch settings. To ensure intervention effectiveness, additional research and technical assistance are needed to help organizations implement HIV prevention EBIs with fidelity. PMID:18445739

Strategies to balance fidelity to Housing First principles with local realities: lessons from a large urban centre.

PubMed

O'Campo, Patricia; Zerger, Suzanne; Gozdzik, Agnes; Jeyaratnam, Jeyagobi; Stergiopoulos, Vicky

2015-05-01

The importance of program implementation in achieving desired outcomes is well-documented, but there remains a need for concrete guidance on how to achieve fidelity to evidence-based models within dynamic local contexts. Housing First (HF), an evidence-based model for people experiencing homelessness and mental illness, provides an important test-case for such guidance; it targets a uniquely underserved subpopulation with complex needs, and is delivered by practitioners with varying knowledge and skill levels. Scientific evidence affirms HF's effectiveness, but its rapid dissemination has outpaced the ability to monitor not only whether it is being implemented with fidelity, but also how this can be achieved within variable local contexts and challenges. This qualitative study contributes to this need by capturing insights from practitioners on implementation challenges and specific strategies developed to overcome them. Findings reinforce the importance of developing HF-specific implementation guidelines, and of engaging relevant stakeholders throughout all phases of that development.

Implementation of a Smeared Crack Band Model in a Micromechanics Framework

NASA Technical Reports Server (NTRS)

Pineda, Evan J.; Bednarcyk, Brett A.; Waas, Anthony M.; Arnold, Steven M.

2012-01-01

The smeared crack band theory is implemented within the generalized method of cells and high-fidelity generalized method of cells micromechanics models to capture progressive failure within the constituents of a composite material while retaining objectivity with respect to the size of the discretization elements used in the model. An repeating unit cell containing 13 randomly arranged fibers is modeled and subjected to a combination of transverse tension/compression and transverse shear loading. The implementation is verified against experimental data (where available), and an equivalent finite element model utilizing the same implementation of the crack band theory. To evaluate the performance of the crack band theory within a repeating unit cell that is more amenable to a multiscale implementation, a single fiber is modeled with generalized method of cells and high-fidelity generalized method of cells using a relatively coarse subcell mesh which is subjected to the same loading scenarios as the multiple fiber repeating unit cell. The generalized method of cells and high-fidelity generalized method of cells models are validated against a very refined finite element model.

Improving contraceptive choice: fidelity of implementation and the gap between effectiveness and efficacy.

PubMed

Garbers, Samantha; Flandrick, Kathleen; Bermudez, Dayana; Meserve, Allison; Chiasson, Mary Ann

2014-11-01

Interventions to reduce unintended pregnancy through improved contraceptive use are a public health priority. A comprehensive process evaluation of a contraceptive assessment module intervention with demonstrated efficacy was undertaken. The 12-month process evaluation goal was to describe the extent to which the intervention was implemented as intended over time, and to identify programmatic adjustments to improve implementation fidelity. Quantitative and qualitative methods included staff surveys, electronic health record data, usage monitoring, and observations. Fidelity of implementation was low overall (<10% of eligible patients completed the entire module [dose received]). Although a midcourse correction making the module available in clinical areas led to increased dose delivered (23% vs. 30%, chi-square test p = .006), dose received did not increase significantly after this adjustment. Contextual factors including competing organizational and staff priorities and staff buy-in limited the level of implementation and precluded adoption of some strategies such as adjusting patient flow. Using a process evaluation framework enabled the research team to identify and address complexities inherent in effectiveness studies and facilitated the alignment of program and context. © 2014 Society for Public Health Education.

Development and validation of a high-fidelity phonomicrosurgical trainer.

PubMed

Klein, Adam M; Gross, Jennifer

2017-04-01

To validate the use of a high-fidelity phonomicrosurgical trainer. A high-fidelity phonomicrosurgical trainer, based on a previously validated model by Contag et al., 1 was designed with multilayered vocal folds that more closely mimic the consistency of true vocal folds, containing intracordal lesions to practice phonomicrosurgical removal. A training module was developed to simulate the true phonomicrosurgical experience. A validation study with novice and expert surgeons was conducted. Novices and experts were instructed to remove the lesion from the synthetic vocal folds, and novices were given four training trials. Performances were measured by the amount of time spent and tissue injury (microflap, superficial, deep) to the vocal fold. An independent Student t test and Fisher exact tests were used to compare subjects. A matched-paired t test and Wilcoxon signed rank tests were used to compare novice performance on the first and fourth trials and assess for improvement. Experts completed the excision with less total errors than novices (P = .004) and made less injury to the microflap (P = .05) and superficial tissue (P = .003). Novices improved their performance with training, making less total errors (P = .002) and superficial tissue injuries (P = .02) and spending less time for removal (P = .002) after several practice trials. This high-fidelity phonomicrosurgical trainer has been validated for novice surgeons. It can distinguish between experts and novices; and after training, it helped to improve novice performance. N/A. Laryngoscope, 127:888-893, 2017. © 2016 The American Laryngological, Rhinological and Otological Society, Inc.

Evaluating the Treatment Fidelity of Parents Who Conduct In-Home Functional Communication Training with Coaching via Telehealth

ERIC Educational Resources Information Center

Suess, Alyssa N.; Romani, Patrick W.; Wacker, David P.; Dyson, Shannon M.; Kuhle, Jennifer L.; Lee, John F.; Lindgren, Scott D.; Kopelman, Todd G.; Pelzel, Kelly E.; Waldron, Debra B.

2014-01-01

We conducted a retrospective, descriptive evaluation of the fidelity with which parents of three children with autism spectrum disorders conducted functional communication training (FCT) in their homes. All training was provided to the parents via telehealth by a behavior consultant in a tertiary-level hospital setting. FCT trials coached by the…

Obesity prevention and the Health promoting Schools framework: essential components and barriers to success.

PubMed

Langford, Rebecca; Bonell, Christopher; Jones, Hayley; Campbell, Rona

2015-02-13

Obesity is an important public health issue. Finding ways to increase physical activity and improve nutrition, particularly in children, is a clear priority. Our Cochrane review of the World Health Organization's Health Promoting Schools (HPS) framework found this approach improved students' physical activity and fitness, and increased fruit and vegetable intake. However, there was considerable heterogeneity in reported impacts. This paper synthesises process evaluation data from these studies to identify factors that might explain this variability. We searched 20 health, education and social-science databases, and trials registries and relevant websites in 2011 and 2013. No language or date restrictions were applied. We included cluster randomised controlled trials. Participants were school students aged 4-18 years. Studies were included if they: took an HPS approach (targeting curriculum, environment and family/community); focused on physical activity and/or nutrition; and presented process evaluation data. A framework approach was used to facilitate thematic analysis and synthesis of process data. Twenty-six studies met the inclusion criteria. Most were conducted in America or Europe, with children aged 12 years or younger. Although interventions were acceptable to students and teachers, fidelity varied considerably across trials. Involving families, while an intrinsic element of the HPS approach, was viewed as highly challenging. Several themes emerged regarding which elements of interventions were critical for success: tailoring programmes to individual schools' needs; aligning interventions with schools' core aims; working with teachers to develop programmes; and providing on-going training and support. An emphasis on academic subjects and lack of institutional support were barriers to implementation. Stronger alliances between health and education appear essential to intervention success. Researchers must work with schools to develop and implement interventions, and to evaluate their impact on both health and educational outcomes as this may be a key determinant of scalability. If family engagement is attempted, better ways to achieve this must be developed and evaluated. Further evaluations of interventions to promote physical activity and nutrition during adolescence are needed. Finally, process evaluations must move beyond simple measures of acceptability/fidelity to include detailed contextual information to illuminate exactly what works, for whom, in what contexts and why.

Assessment of the implementation fidelity of the Arctic Char Distribution Project in Nunavik, Quebec

PubMed Central

Gautier, Lara; Pirkle, Catherine M; Furgal, Christopher; Lucas, Michel

2016-01-01

Background In September 2011, the Nunavik Regional Board of Health and Social Services began supporting the Arctic Char Distribution Project (AC/DP) for pregnant women. This initiative promoted consumption of the fish Arctic char—a traditional Inuit food—by pregnant women living in villages of Nunavik, an area in northern Quebec (Canada) inhabited predominantly by people of Inuit ethnicity. This intervention was intended to reduce exposure to contaminants and improve food security in Inuit communities. Methods We assessed the project's implementation based on data collected from background documentation, field notes and qualitative interviews with project recipients and implementers. Themes emerging from the data are critically discussed in the light of the framework for implementation fidelity developed by Carroll et al in 2007. Results Pregnant women fully embraced the initiative because of its cultural appropriateness. However, project implementation was incomplete: first because it did not cover all intended geographic areas, and second because of a recurring inconsistency in the supply and distribution of the fish. In addition, the initiative has been inconsistently funded and relies on multiple funding sources. Discussion This work highlights the extent to which project complexity can impede successful implementation, particularly in terms of communication and coordination. We provide recommendations for improving project implementation and suggest amendments to the implementation fidelity framework. PMID:28588959

Promoting Teachers' Implementation of Classroom-Based Prevention Programming Through Coaching: The Mediating Role of the Coach-Teacher Relationship.

PubMed

Johnson, Stacy R; Pas, Elise T; Bradshaw, Catherine P; Ialongo, Nicholas S

2018-05-01

There is growing awareness of the importance of implementation fidelity and the supports, such as coaching, to optimize it. This study examined how coaching activities (i.e., check-ins, needs assessment, modeling, and technical assistance) related directly and indirectly to implementation dosage and quality of the PAX Good Behavior Game, via a mediating pathway through working relationship. Mediation analyses of 138 teachers revealed direct effects of modeling and working relationship on implementation dosage, whereas needs assessment was associated with greater dosage indirectly, by higher ratings of the working relationship. Understanding how coaching activities promote implementation fidelity elements has implications for improving program effectiveness.

A process evaluation of implementing a vocational enablement protocol for employees with hearing difficulties in clinical practice.

PubMed

Gussenhoven, Arjenne H M; Singh, Amika S; Goverts, S Theo; van Til, Marten; Anema, Johannes R; Kramer, Sophia E

2015-08-01

A multidisciplinary vocational rehabilitation programme, the Vocational Enablement Protocol (VEP) was developed to address the specific needs of employees with hearing difficulties. In the current study we evaluated the process of implementing the VEP in audiologic care among employees with hearing impairment. In conjunction with a randomized controlled trial, we collected and analysed data on seven process parameters: recruitment, reach, fidelity, dose delivered, dose received and implemented, satisfaction, and perceived benefit. Sixty-six employees with hearing impairment participated in the VEP. The multidisciplinary team providing the VEP comprised six professionals. The professionals performed the VEP according to the protocol. Of the recommendations delivered by the professionals, 31% were perceived as implemented by the employees. Compliance rate was highest for hearing-aid uptake (51%). Both employees and professionals were highly satisfied with the VEP. Participants rated good perceived benefit from it. Our results indicate that the VEP could be a useful treatment for employees with hearing difficulties from a process evaluation perspective. Implementation research in the audiological setting should be encouraged in order to further provide insight into parameters facilitating or hindering successful implementation of an intervention and to improve its quality and efficacy.

Determining the impact of a new physiotherapist-led primary care model for back pain: protocol for a pilot cluster randomized controlled trial.

PubMed

Miller, Jordan; Barber, David; Donnelly, Catherine; French, Simon; Green, Michael; Hill, Jonathan; MacDermid, Joy; Marsh, Jacquelyn; Norman, Kathleen; Richardson, Julie; Taljaard, Monica; Wideman, Timothy; Cooper, Lynn; McPhee, Colleen

2017-11-09

Back pain is a leading contributor to disability, healthcare costs, and lost work. Family physicians are the most common first point of contact in the healthcare system for people with back pain, but physiotherapists (PTs) may be able to support the primary care team through evidence-based primary care. A cluster randomized trial is needed to determine the clinical, health system, and societal impact of a primary care model that integrates physiotherapists at the first visit for people with back pain. Prior to conducting a future fully powered cluster randomized trial, we need to demonstrate feasibility of the methods. Therefore, the purpose of this pilot study will be to: 1) Determine feasibility of patient recruitment, assessment procedures, and retention. 2) Determine the feasibility of training and implementation of a new PT-led primary care model for low back pain (LBP) 3) Explore the perspectives of patients and healthcare providers (HCPs) related to their experiences and attitudes towards the new service delivery model, barriers/facilitators to implementation, perceived satisfaction, perceived value, and impact on clinic processes and patient outcomes. This pilot cluster randomized controlled trial will enroll four sites and randomize them to implement a new PT-led primary care model for back pain or a usual physician-led primary care model. All adults booking a primary care visit for back pain will be invited to participate. Feasibility outcomes will include: recruitment and retention rates, completeness of assessment data, PT training participation and confidence after training, and PT treatment fidelity. Secondary outcomes will include the clinical, health system, cost, and process outcomes planned for the future fully powered cluster trial. Results will be analyzed and reported descriptively and qualitatively. To explore perspectives of both HCPs and patients, we will conduct semi-structured qualitative interviews with patients and focus groups with HCPs from participants in the PT-led primary care sites. If this pilot demonstrates feasibility, a fully powered trial will provide evidence that has the potential to transform primary care for back pain. The full trial will inform future service design, whether these models should be more widely implemented, and training agendas. ClinicalTrials.gov, NCT03320148 . Submitted for registration on 17 September 2017.

Evaluation of the Homework, Organization, and Planning Skills (HOPS) Intervention for Middle School Students with ADHD as Implemented by School Mental Health Providers.

PubMed

Langberg, Joshua M; Epstein, Jeffery N; Becker, Stephen P; Girio-Herrera, Erin; Vaughn, Aaron J

2012-09-01

The purpose of the study was to evaluate the Homework, Organization, and Planning Skills (HOPS) intervention for middle school students with Attention-Deficit/Hyperactivity Disorder (ADHD) as implemented by school mental health (SMH) providers using a randomized trial design. Seventeen SMH providers from five school districts implemented the HOPS intervention. Forty-seven middle school students with ADHD (grades 6-8) were randomly assigned to receive the HOPS intervention or to a waitlist comparison group. Parent and teacher ratings of organizational skills and homework problems were collected pre- and post-intervention and at a 3-monoth follow-up, and school grades were also collected. Intervention participants demonstrated significant improvements relative to the waitlist comparison across parent-rated organized action ( d = .88), materials management ( d = .63), planning ( d = 1.05), and homework completion behaviors ( d = .85). Intervention participants did not make significant improvements relative to the comparison group according to teacher ratings. SMH providers were able to implement the HOPS intervention with fidelity despite the fact that no formal ongoing consultation was provided.

Evaluation of the Homework, Organization, and Planning Skills (HOPS) Intervention for Middle School Students with ADHD as Implemented by School Mental Health Providers

PubMed Central

Langberg, Joshua M.; Epstein, Jeffery N.; Becker, Stephen P.; Girio-Herrera, Erin; Vaughn, Aaron J.

2013-01-01

The purpose of the study was to evaluate the Homework, Organization, and Planning Skills (HOPS) intervention for middle school students with Attention-Deficit/Hyperactivity Disorder (ADHD) as implemented by school mental health (SMH) providers using a randomized trial design. Seventeen SMH providers from five school districts implemented the HOPS intervention. Forty-seven middle school students with ADHD (grades 6–8) were randomly assigned to receive the HOPS intervention or to a waitlist comparison group. Parent and teacher ratings of organizational skills and homework problems were collected pre- and post-intervention and at a 3-monoth follow-up, and school grades were also collected. Intervention participants demonstrated significant improvements relative to the waitlist comparison across parent-rated organized action (d = .88), materials management (d = .63), planning (d = 1.05), and homework completion behaviors (d = .85). Intervention participants did not make significant improvements relative to the comparison group according to teacher ratings. SMH providers were able to implement the HOPS intervention with fidelity despite the fact that no formal ongoing consultation was provided. PMID:25355991

Objective coding of content and techniques in workplace-based supervision of an EBT in public mental health.

PubMed

Dorsey, Shannon; Kerns, Suzanne E U; Lucid, Leah; Pullmann, Michael D; Harrison, Julie P; Berliner, Lucy; Thompson, Kelly; Deblinger, Esther

2018-01-24

Workplace-based clinical supervision as an implementation strategy to support evidence-based treatment (EBT) in public mental health has received limited research attention. A commonly provided infrastructure support, it may offer a relatively cost-neutral implementation strategy for organizations. However, research has not objectively examined workplace-based supervision of EBT and specifically how it might differ from EBT supervision provided in efficacy and effectiveness trials. Data come from a descriptive study of supervision in the context of a state-funded EBT implementation effort. Verbal interactions from audio recordings of 438 supervision sessions between 28 supervisors and 70 clinicians from 17 public mental health organizations (in 23 offices) were objectively coded for presence and intensity coverage of 29 supervision strategies (16 content and 13 technique items), duration, and temporal focus. Random effects mixed models estimated proportion of variance in content and techniques attributable to the supervisor and clinician levels. Interrater reliability among coders was excellent. EBT cases averaged 12.4 min of supervision per session. Intensity of coverage for EBT content varied, with some discussed frequently at medium or high intensity (exposure) and others infrequently discussed or discussed only at low intensity (behavior management; assigning/reviewing client homework). Other than fidelity assessment, supervision techniques common in treatment trials (e.g., reviewing actual practice, behavioral rehearsal) were used rarely or primarily at low intensity. In general, EBT content clustered more at the clinician level; different techniques clustered at either the clinician or supervisor level. Workplace-based clinical supervision may be a feasible implementation strategy for supporting EBT implementation, yet it differs from supervision in treatment trials. Time allotted per case is limited, compressing time for EBT coverage. Techniques that involve observation of clinician skills are rarely used. Workplace-based supervision content appears to be tailored to individual clinicians and driven to some degree by the individual supervisor. Our findings point to areas for intervention to enhance the potential of workplace-based supervision for implementation effectiveness. NCT01800266 , Clinical Trials, Retrospectively Registered (for this descriptive study; registration prior to any intervention [part of phase II RCT, this manuscript is only phase I descriptive results]).

Protocol for process evaluation of a randomised controlled trial of family-led rehabilitation post stroke (ATTEND) in India.

PubMed

Liu, Hueiming; Lindley, Richard; Alim, Mohammed; Felix, Cynthia; Gandhi, Dorcas B C; Verma, Shweta J; Tugnawat, Deepak Kumar; Syrigapu, Anuradha; Ramamurthy, Ramaprabhu Krishnappa; Pandian, Jeyaraj D; Walker, Marion; Forster, Anne; Anderson, Craig S; Langhorne, Peter; Murthy, Gudlavalleti Venkata Satyanarayana; Shamanna, Bindiganavale Ramaswamy; Hackett, Maree L; Maulik, Pallab K; Harvey, Lisa A; Jan, Stephen

2016-09-15

We are undertaking a randomised controlled trial (fAmily led rehabiliTaTion aftEr stroke in INDia, ATTEND) evaluating training a family carer to enable maximal rehabilitation of patients with stroke-related disability; as a potentially affordable, culturally acceptable and effective intervention for use in India. A process evaluation is needed to understand how and why this complex intervention may be effective, and to capture important barriers and facilitators to its implementation. We describe the protocol for our process evaluation to encourage the development of in-process evaluation methodology and transparency in reporting. The realist and RE-AIM (Reach, Effectiveness, Adoption, Implementation and Maintenance) frameworks informed the design. Mixed methods include semistructured interviews with health providers, patients and their carers, analysis of quantitative process data describing fidelity and dose of intervention, observations of trial set up and implementation, and the analysis of the cost data from the patients and their families perspective and programme budgets. These qualitative and quantitative data will be analysed iteratively prior to knowing the quantitative outcomes of the trial, and then triangulated with the results from the primary outcome evaluation. The process evaluation has received ethical approval for all sites in India. In low-income and middle-income countries, the available human capital can form an approach to reducing the evidence practice gap, compared with the high cost alternatives available in established market economies. This process evaluation will provide insights into how such a programme can be implemented in practice and brought to scale. Through local stakeholder engagement and dissemination of findings globally we hope to build on patient-centred, cost-effective and sustainable models of stroke rehabilitation. CTRI/2013/04/003557. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

CenteringPregnancy-Africa: a pilot of group antenatal care to address Millennium Development Goals.

PubMed

Patil, Crystal L; Abrams, Elizabeth T; Klima, Carrie; Kaponda, Chrissie P N; Leshabari, Sebalda C; Vonderheid, Susan C; Kamanga, Martha; Norr, Kathleen F

2013-10-01

severe health worker shortages and resource limitations negatively affect quality of antenatal care (ANC) throughout sub-Saharan Africa. Group ANC, specifically CenteringPregnancy (CP), may offer an innovative approach to enable midwives to offer higher quality ANC. our overarching goal was to prepare to conduct a clinical trial of CenteringPregnancy-Africa (CP-Africa) in Malawi and Tanzania. In Phase 1, our goal was to determine the acceptability of CP as a model for ANC in both countries. In Phase 2, our objective was to develop CP-Africa session content consistent with the Essential Elements of CP model and with national standards in both Malawi and Tanzania. In Phase 3, our objective was to pilot CP-Africa in Malawi to determine whether sessions could be conducted with fidelity to the Centering process. Phases 1 and 2 took place in Malawi and Tanzania. Phase 3, the piloting of two sessions of CP-Africa, occurred at two sites in Malawi: a district hospital and a small clinic. we used an Action Research approach to promote partnerships among university researchers, the Centering Healthcare Institute, health care administrators, health professionals and women attending ANC to develop CP-Africa session content and pilot this model of group ANC. for Phases 1 and 2, members of the Ministries of Health, health professionals and pregnant women in Malawi and Tanzania were introduced to and interviewed about CP. In Phase 2, we finalised CP-Africa content and trained 13 health professionals in the Centering Healthcare model. In Phase 3, we conducted a small pilot with 24 pregnant women (12 at each site). participants enthusiastically embraced CP-Africa as an acceptable model of ANC health care delivery. The CP-Africa content met both CP and national standards. The pilot established that the CP model could be implemented with process fidelity to the 13 Essential Elements. Several implementation challenges and strategies to address these challenges were identified. preliminary data suggest that CP-Africa is feasible in resource-constrained, low-literacy, high-HIV settings in sub-Saharan Africa. By improving the quality of ANC delivery, midwives have an opportunity to make a contribution towards Millennium Development Goals (MDG) targeting improvements in child, maternal and HIV-related health outcomes (MDGs 4, 5 and 6). A clinical trial is needed to establish efficacy. CP-Africa also has the potential to reduce job-related stress and enhance job satisfaction for midwives in low income countries. If CP can be transferred with fidelity to process in sub-Saharan Africa and retain similar results to those reported in clinical trials, it has the potential to benefit pregnant women and their infants and could make a positive contribution to MGDs 4, 5 and 6. © 2013 Elsevier Ltd. All rights reserved.

A Structured Process Description of a Pragmatic Implementation Project: Improving Integrated Care for Older Persons in Residential Care Homes

PubMed Central

Poot, Antonius J.; de Waard, Claudia S.; Wind, Annet W.; Caljouw, Monique A. A.; Gussekloo, Jacobijn

2017-01-01

Evaluation of the implementation of integrated care can differ from trial-based research due to complexity. Therefore, we examined whether a theory-based method for process description of implementation can contribute to improvement of evidence-based care. MOVIT, a Dutch project aimed at implementing integrated care for older vulnerable persons in residential care homes, was used as a case study. The project activities were defined according to implementation taxonomy and mapped in a matrix of theoretical levels and domains. Project activities mainly targeted professionals (both individual and group). A few activities targeted the organizational level, whereas none targeted the policy level, or the patient, or the “social, political, and legal” domains. However, the resulting changes in care delivery arrangement had consequences for professionals, patients, organizations, and the social, political, and legal domains. A structured process description of a pragmatic implementation project can help assess the fidelity and quality of the implementation, and identify relevant contextual factors for immediate adaptation and future research. The description showed that, in the MOVIT project, there was a discrepancy between the levels and domains targeted by the implementation activities and those influenced by the resulting changes in delivery arrangement. This could have influenced, in particular, the adoption and sustainability of the project. PMID:29161944

A Structured Process Description of a Pragmatic Implementation Project: Improving Integrated Care for Older Persons in Residential Care Homes.

PubMed

Poot, Antonius J; de Waard, Claudia S; Wind, Annet W; Caljouw, Monique A A; Gussekloo, Jacobijn

2017-01-01

Evaluation of the implementation of integrated care can differ from trial-based research due to complexity. Therefore, we examined whether a theory-based method for process description of implementation can contribute to improvement of evidence-based care. MOVIT, a Dutch project aimed at implementing integrated care for older vulnerable persons in residential care homes, was used as a case study. The project activities were defined according to implementation taxonomy and mapped in a matrix of theoretical levels and domains. Project activities mainly targeted professionals (both individual and group). A few activities targeted the organizational level, whereas none targeted the policy level, or the patient, or the "social, political, and legal" domains. However, the resulting changes in care delivery arrangement had consequences for professionals, patients, organizations, and the social, political, and legal domains. A structured process description of a pragmatic implementation project can help assess the fidelity and quality of the implementation, and identify relevant contextual factors for immediate adaptation and future research. The description showed that, in the MOVIT project, there was a discrepancy between the levels and domains targeted by the implementation activities and those influenced by the resulting changes in delivery arrangement. This could have influenced, in particular, the adoption and sustainability of the project.

High-fidelity gates towards a scalable superconducting quantum processor

NASA Astrophysics Data System (ADS)

Chow, Jerry M.; Corcoles, Antonio D.; Gambetta, Jay M.; Rigetti, Chad; Johnson, Blake R.; Smolin, John A.; Merkel, Seth; Poletto, Stefano; Rozen, Jim; Rothwell, Mary Beth; Keefe, George A.; Ketchen, Mark B.; Steffen, Matthias

2012-02-01

We experimentally explore the implementation of high-fidelity gates on multiple superconducting qubits coupled to multiple resonators. Having demonstrated all-microwave single and two qubit gates with fidelities > 90% on multi-qubit single-resonator systems, we expand the application to qubits across two resonators and investigate qubit coupling in this circuit. The coupled qubit-resonators are building blocks towards two-dimensional lattice networks for the application of surface code quantum error correction algorithms.

Randomized Trial of Family Therapy versus Non-Family Treatment for Adolescent Behavior Problems in Usual Care

PubMed Central

Hogue, Aaron; Dauber, Sarah; Henderson, Craig E.; Bobek, Molly; Johnson, Candace; Lichvar, Emily; Morgenstern, Jon

2014-01-01

Objective A major focus of implementation science is discovering whether evidence-based approaches can be delivered with fidelity and potency in routine practice. This randomized trial compared usual care family therapy (UC-FT), implemented without a treatment manual or extramural support as the standard-of-care approach in a community clinic, to non-family treatment (UC-Other) for adolescent conduct and substance use disorders. Method The study recruited 205 adolescents (mean age 15.7 years; 52% male; 59% Hispanic American, 21% African American) from a community referral network, enrolling 63% for primary mental health problems and 37% for primary substance use problems. Clients were randomly assigned to either the UC-FT site or one of five UC-Other sites. Implementation data confirmed that UC-FT showed adherence to the family therapy approach and differentiation from UC-Other. Follow-ups were completed at 3, 6, and 12 months post-baseline. Results There was no between-group difference in treatment attendance. Both conditions demonstrated improvements in externalizing, internalizing, and delinquency symptoms. However, UC-FT produced greater reductions in youth-reported externalizing and internalizing among the whole sample, in delinquency among substance-using youth, and in alcohol and drug use among substance-using youth. The degree to which UC-FT outperformed UC-Other was consistent with effect sizes from controlled trials of manualized family therapy models. Conclusions Non-manualized family therapy can be effective for adolescent behavior problems within diverse populations in usual care, and it may be superior to non-family alternatives. PMID:25496283

Reciprocal Relations between Coalition Functioning and the Provision of Implementation Support

PubMed Central

Brown, Louis D.; Feinberg, Mark E.; Shapiro, Valerie B.; Greenberg, Mark T.

2014-01-01

Community coalitions have been promoted as a strategy to help overcome challenges to the dissemination and implementation of evidence-based prevention programs. This paper explores the characteristics of coalitions that enable the provision of implementation support for prevention programs in general, and for the implementation of evidence-based prevention programs with fidelity. Longitudinal cross-lagged panel models were used to study 74 Communities That Care (CTC) coalitions in Pennsylvania. These analyses provide evidence of a unidirectional influence of coalition functioning on the provision of implementation support. Coalition member knowledge of the CTC model best predicted the coalition’s provision of support for evidence-based program implementation with fidelity. Implications for developing and testing innovative methods for delivering training and technical assistance to enhance coalition member knowledge are discussed. PMID:24323363

Variation in the Implementation of California's Full Service Partnerships for Persons with Serious Mental Illness

PubMed Central

Gilmer, Todd P; Katz, Marian L; Stefancic, Ana; Palinkas, Lawrence A

2013-01-01

Objective. This study examined variation in the implementation of California's Full Service Partnerships (FSPs), which are supported housing programs that do “whatever it takes” to improve outcomes among persons with serious mental illness who are homeless or at risk of homelessness. Data Sources/Setting. Ninety-three FSPs in California. Study Design. A mixed methods approach was selected to develop a better understanding of the complexity of the FSP programs. The design structure was a combined explanatory and exploratory sequential design (qual→QUAN→qual) where a qualitative focus group was used to develop a quantitative survey that was followed by qualitative site visits. The survey was used to describe the breadth of variation based on fidelity to the Housing First model, while the site visits were used to provide a depth of information on high- versus low-fidelity programs. Principal Findings. We found substantial variation in implementation among FSPs. Fidelity was particularly low along domains related with housing and service philosophy, indicating that many FSPs implemented a rich array of services but applied housing readiness requirements and did not adhere to consumer choice in housing. Conclusions. There remains room for improvement in the recovery-orientation of FSPs. Fortunately, we have identified several processes by which program managers and counties can increase the fidelity of their programs. PMID:24138021

Monitoring intervention fidelity of a lifestyle behavioral intervention delivered through telehealth

PubMed Central

Sineath, Ashley; Lambert, Lauren; Verga, Catherine; Wagstaff, Miranda

2017-01-01

Background Technology-based lifestyle behavioral interventions (i.e., telehealth, mHealth, eHealth, and/or digital health) are becoming an alternative standard of care and possess several advantages over traditional clinical settings such as convenience, cost, and the ability to tailor plans and feedback to a participant’s individual needs. These technology-based interventions also present unique challenges to intervention fidelity due to extra elements involved in executing the intervention. Intervention fidelity monitoring is essential to ensure internal and external validity, yet the development and utilization of fidelity protocols is under-reported in the literature. The purpose of this paper is to describe the intervention fidelity protocol for the 24-START study, a behavior change intervention delivered through telephone and internet. This paper also discusses the results of a pilot audit conducted to determine the feasibility of monitoring adherence to the fidelity protocol. Methods The 24-START fidelity protocol was developed in accordance with the five fidelity areas outlined by the NIH Behavior Change Consortium (NIH BCC) including: design of study, provider training, delivery of treatment, receipt of treatment, and enactment of treatment. The fidelity strategies provided by the NIH BCC in each area were tailored to fit the specific design of the 24-START study. Twenty-six total fidelity strategies were developed in accordance with the five areas and a corresponding fidelity monitoring plan was created. Because these strategies are only beneficial if implemented, the fidelity monitoring plan was developed to ensure the fidelity strategies are consistently implemented over the course of the intervention. Results A pilot audit of nine participant files was conducted to test the feasibility of the fidelity protocol developed. Out of the nine participant files reviewed, 89% of scheduled phone calls between a telehealth coach and participant were successfully completed. Of the completed calls, telehealth coaches delivered the intervention as intended 85.3% of the time, and 74% of planned secondary contacts made through the internet were delivered successfully. Additionally, between treatment group dosing was found to be equal. Several weak areas in the fidelity protocol were identified for improvement. The results were satisfactory and the audit was deemed feasible for ongoing use. Conclusions The NIH BCC provides a valuable framework for telehealth interventions to develop fidelity protocols ultimately contributing to improved internal and external validity, better translation of results, increased transparency, and increased opportunities for replication within the field. The 24-START pilot audit found the fidelity protocol efficacious and feasible while also identifying areas of weakness in need of revision. The refined protocol will continue to be utilized throughout the data collection phase. Future telehealth interventions should develop and disclose fidelity protocols to improve the overall quality and standard of telehealth interventions. PMID:28894745

Monitoring intervention fidelity of a lifestyle behavioral intervention delivered through telehealth.

PubMed

Sineath, Ashley; Lambert, Lauren; Verga, Catherine; Wagstaff, Miranda; Wingo, Brooks C

2017-01-01

Technology-based lifestyle behavioral interventions (i.e., telehealth, mHealth, eHealth, and/or digital health) are becoming an alternative standard of care and possess several advantages over traditional clinical settings such as convenience, cost, and the ability to tailor plans and feedback to a participant's individual needs. These technology-based interventions also present unique challenges to intervention fidelity due to extra elements involved in executing the intervention. Intervention fidelity monitoring is essential to ensure internal and external validity, yet the development and utilization of fidelity protocols is under-reported in the literature. The purpose of this paper is to describe the intervention fidelity protocol for the 24-START study, a behavior change intervention delivered through telephone and internet. This paper also discusses the results of a pilot audit conducted to determine the feasibility of monitoring adherence to the fidelity protocol. The 24-START fidelity protocol was developed in accordance with the five fidelity areas outlined by the NIH Behavior Change Consortium (NIH BCC) including: design of study, provider training, delivery of treatment, receipt of treatment, and enactment of treatment. The fidelity strategies provided by the NIH BCC in each area were tailored to fit the specific design of the 24-START study. Twenty-six total fidelity strategies were developed in accordance with the five areas and a corresponding fidelity monitoring plan was created. Because these strategies are only beneficial if implemented, the fidelity monitoring plan was developed to ensure the fidelity strategies are consistently implemented over the course of the intervention. A pilot audit of nine participant files was conducted to test the feasibility of the fidelity protocol developed. Out of the nine participant files reviewed, 89% of scheduled phone calls between a telehealth coach and participant were successfully completed. Of the completed calls, telehealth coaches delivered the intervention as intended 85.3% of the time, and 74% of planned secondary contacts made through the internet were delivered successfully. Additionally, between treatment group dosing was found to be equal. Several weak areas in the fidelity protocol were identified for improvement. The results were satisfactory and the audit was deemed feasible for ongoing use. The NIH BCC provides a valuable framework for telehealth interventions to develop fidelity protocols ultimately contributing to improved internal and external validity, better translation of results, increased transparency, and increased opportunities for replication within the field. The 24-START pilot audit found the fidelity protocol efficacious and feasible while also identifying areas of weakness in need of revision. The refined protocol will continue to be utilized throughout the data collection phase. Future telehealth interventions should develop and disclose fidelity protocols to improve the overall quality and standard of telehealth interventions.

Nurses' fidelity to theory-based core components when implementing Family Health Conversations - a qualitative inquiry.

PubMed

Östlund, Ulrika; Bäckström, Britt; Lindh, Viveca; Sundin, Karin; Saveman, Britt-Inger

2015-09-01

A family systems nursing intervention, Family Health Conversation, has been developed in Sweden by adapting the Calgary Family Assessment and Intervention Models and the Illness Beliefs Model. The intervention has several theoretical assumptions, and one way translate the theory into practice is to identify core components. This may produce higher levels of fidelity to the intervention. Besides information about how to implement an intervention in accordance to how it was developed, evaluating whether it was actually implemented as intended is important. Accordingly, we describe the nurses' fidelity to the identified core components of Family Health Conversation. Six nurses, working in alternating pairs, conducted Family Health Conversations with seven families in which a family member younger than 65 had suffered a stroke. The intervention contained a series of three-1-hour conversations held at 2-3 week intervals. The nurses followed a conversation structure based on 12 core components identified from theoretical assumptions. The transcripts of the 21 conversations were analysed using manifest qualitative content analysis with a deductive approach. The 'core components' seemed to be useful even if nurses' fidelity varied among the core components. Some components were followed relatively well, but others were not. This indicates that the process for achieving fidelity to the intervention can be improved, and that it is necessary for nurses to continually learn theory and to practise family systems nursing. We suggest this can be accomplished through reflections, role play and training on the core components. Furthermore, as in this study, joint reflections on how the core components have been implemented can lead to deeper understanding and knowledge of how Family Health Conversation can be delivered as intended. © 2014 Nordic College of Caring Science.

Rethinking Curriculum Implementation: Paradigms, Models, and Teachers' Work.

ERIC Educational Resources Information Center

Cho, Jeasik

The paper looks at traditional curriculum implementation strategies, claiming that they have been divided into two polarized perspectives: fidelity implementation and adaptive implementation. These two implementation perspectives have dominated contemporary curriculum textbooks. The paper suggests that curriculum implementation as a field of study…

An intervention to support stroke survivors and their carers in the longer term (LoTS2Care): study protocol for a cluster randomised controlled feasibility trial.

PubMed

Forster, Anne; Hartley, Suzanne; Barnard, Lorna; Ozer, Seline; Hardicre, Natasha; Crocker, Tom; Fletcher, Marie; Moreau, Lauren; Atkinson, Ross; Hulme, Claire; Holloway, Ivana; Schmitt, Laetitia; House, Allan; Hewison, Jenny; Richardson, Gillian; Farrin, Amanda

2018-06-11

Despite the evidence that many stroke survivors report longer term unmet needs, the provision of longer term care is limited. To address this, we are conducting a programme of research to develop an evidence-based and replicable longer term care strategy. The developed complex intervention (named New Start), which includes needs identification, exploration of social networks and components of problem solving and self-management, was designed to improve quality of life by addressing unmet needs and increasing participation. A multicentre, cluster randomised controlled feasibility trial designed to inform the design of a possible future definitive cluster randomised controlled trial (cRCT) and explore the potential clinical and cost-effectiveness of New Start. Ten stroke services across the UK will be randomised on a 1:1 basis either to implement New Start or continue with usual care only. New Start will be delivered by trained facilitators and will be offered to all stroke survivors within the services allocated to the intervention arm. Stroke survivors will be eligible for the trial if they are 4-6 months post-stroke and residing in the community. Carers (if available) will also be invited to take part. Invitation to participate will be initiated by post and outcome measures will be collected via postal questionnaires at 3, 6 and 9 months after recruitment. Outcome data relating to perceived health and disability, wellbeing and quality of life as well as unmet needs will be collected. A 'study within a trial' (SWAT) is planned to determine the most acceptable format in which to provide the postal questionnaires. Details of health and social care service usage will also be collected to inform the economic evaluation. The feasibility of recruiting services and stroke survivors to the trial and of collecting postal outcomes will be assessed and the potential for effectiveness will be investigated. An embedded process evaluation (reported separately) will assess implementation fidelity and explore and clarify causal assumptions regarding implementation. This feasibility trial with embedded process evaluation will allow us to gather important and detailed data regarding methodological and implementation issues to inform the design of a possible future definitive cRCT of this complex intervention. ISRCTN38920246 . Registered 22 June 2016.

Aerodynamic design applying automatic differentiation and using robust variable fidelity optimization

NASA Astrophysics Data System (ADS)

Takemiya, Tetsushi

In modern aerospace engineering, the physics-based computational design method is becoming more important, as it is more efficient than experiments and because it is more suitable in designing new types of aircraft (e.g., unmanned aerial vehicles or supersonic business jets) than the conventional design method, which heavily relies on historical data. To enhance the reliability of the physics-based computational design method, researchers have made tremendous efforts to improve the fidelity of models. However, high-fidelity models require longer computational time, so the advantage of efficiency is partially lost. This problem has been overcome with the development of variable fidelity optimization (VFO). In VFO, different fidelity models are simultaneously employed in order to improve the speed and the accuracy of convergence in an optimization process. Among the various types of VFO methods, one of the most promising methods is the approximation management framework (AMF). In the AMF, objective and constraint functions of a low-fidelity model are scaled at a design point so that the scaled functions, which are referred to as "surrogate functions," match those of a high-fidelity model. Since scaling functions and the low-fidelity model constitutes surrogate functions, evaluating the surrogate functions is faster than evaluating the high-fidelity model. Therefore, in the optimization process, in which gradient-based optimization is implemented and thus many function calls are required, the surrogate functions are used instead of the high-fidelity model to obtain a new design point. The best feature of the AMF is that it may converge to a local optimum of the high-fidelity model in much less computational time than the high-fidelity model. However, through literature surveys and implementations of the AMF, the author xx found that (1) the AMF is very vulnerable when the computational analysis models have numerical noise, which is very common in high-fidelity models, and that (2) the AMF terminates optimization erroneously when the optimization problems have constraints. The first problem is due to inaccuracy in computing derivatives in the AMF, and the second problem is due to erroneous treatment of the trust region ratio, which sets the size of the domain for an optimization in the AMF. In order to solve the first problem of the AMF, automatic differentiation (AD) technique, which reads the codes of analysis models and automatically generates new derivative codes based on some mathematical rules, is applied. If derivatives are computed with the generated derivative code, they are analytical, and the required computational time is independent of the number of design variables, which is very advantageous for realistic aerospace engineering problems. However, if analysis models implement iterative computations such as computational fluid dynamics (CFD), which solves system partial differential equations iteratively, computing derivatives through the AD requires a massive memory size. The author solved this deficiency by modifying the AD approach and developing a more efficient implementation with CFD, and successfully applied the AD to general CFD software. In order to solve the second problem of the AMF, the governing equation of the trust region ratio, which is very strict against the violation of constraints, is modified so that it can accept the violation of constraints within some tolerance. By accepting violations of constraints during the optimization process, the AMF can continue optimization without terminating immaturely and eventually find the true optimum design point. With these modifications, the AMF is referred to as "Robust AMF," and it is applied to airfoil and wing aerodynamic design problems using Euler CFD software. The former problem has 21 design variables, and the latter 64. In both problems, derivatives computed with the proposed AD method are first compared with those computed with the finite differentiation (FD) method, and then, the Robust AMF is implemented along with the sequential quadratic programming (SQP) optimization method with only high-fidelity models. The proposed AD method computes derivatives more accurately and faster than the FD method, and the Robust AMF successfully optimizes shapes of the airfoil and the wing in a much shorter time than SQP with only high-fidelity models. These results clearly show the effectiveness of the Robust AMF. Finally, the feasibility of reducing computational time for calculating derivatives and the necessity of AMF with an optimum design point always in the feasible region are discussed as future work.

Developing an instrument for assessing fidelity of motivational care planning: The Aboriginal and Islander Mental health initiative adherence scale.

PubMed

Prowse, Phuong-Tu; Nagel, Tricia

2014-01-01

The aim of this study was to design and trial an Adherence Scale to measure fidelity of Motivational Care Planning (MCP) within a clinical trial. This culturally adapted therapy MCP uses a client centered holistic approach that emphasises family and culture to motivate healthy life style changes. The Motivational Care Planning-Adherence Scale (MCP-AS) was developed through consultation with Aboriginal and Islander Mental Health Initiative (AIMhi) Indigenous and non-Indigenous trainers, and review of MCP training resources. The resultant ten-item scale incorporates a 9-Point Likert Scale with a supporting protocol manual and uses objective, behaviourally anchored criteria for each scale point. A fidelity assessor piloted the tool through analysis of four audio-recordings of MCP (conducted by Indigenous researchers within a study in remote communities in Northern Australia). File audits of the remote therapy sessions were utilised as an additional source of information. A Gold Standard Motivational Care Planning training video was also assessed using the MCP-AS. The Motivational Care Planning-Adherence Scale contains items measuring both process and content of therapy sessions. This scale was used successfully to assess therapy through observation of audio or video-recorded sessions and review of clinical notes. Treatment fidelity measured by the MCP-AS within the pilot study indicated high fidelity ratings. Ratings were high across the three domains of rapport, motivation, and self-management with especially high ratings for positive feedback and engagement, review of stressors and goal setting. The Motivational Care Planning-Adherence Scale has the potential to provide a measure of quality of delivery of Motivation Care Planning. The pilot findings suggest that despite challenges within the remote Indigenous community setting, Indigenous therapists delivered therapy that was of high fidelity. While developed as a research tool, the scale has the potential to support fidelity of delivery of Motivation Care Planning in clinical, supervision and training settings. Larger studies are needed to establish inter-rater reliability and internal and external validity.

A systematic review of evidence for education and training interventions in microsurgery.

PubMed

Ghanem, Ali M; Hachach-Haram, Nadine; Leung, Clement Chi Ming; Myers, Simon Richard

2013-07-01

Over the past decade, driven by advances in educational theory and pressures for efficiency in the clinical environment, there has been a shift in surgical education and training towards enhanced simulation training. Microsurgery is a technical skill with a steep competency learning curve on which the clinical outcome greatly depends. This paper investigates the evidence for educational and training interventions of traditional microsurgical skills courses in order to establish the best evidence practice in education and training and curriculum design. A systematic review of MEDLINE, EMBASE, and PubMed databases was performed to identify randomized control trials looking at educational and training interventions that objectively improved microsurgical skill acquisition, and these were critically appraised using the BestBETs group methodology. The databases search yielded 1,148, 1,460, and 2,277 citations respectively. These were then further limited to randomized controlled trials from which abstract reviews reduced the number to 5 relevant randomised controlled clinical trials. The best evidence supported a laboratory based low fidelity model microsurgical skills curriculum. There was strong evidence that technical skills acquired on low fidelity models transfers to improved performance on higher fidelity human cadaver models and that self directed practice leads to improved technical performance. Although there is significant paucity in the literature to support current microsurgical education and training practices, simulated training on low fidelity models in microsurgery is an effective intervention that leads to acquisition of transferable skills and improved technical performance. Further research to identify educational interventions associated with accelerated skill acquisition is required.

Beyond surgical care improvement program compliance: antibiotic prophylaxis implementation gaps.

PubMed

Hawkins, Russell B; Levy, Shauna M; Senter, Casey E; Zhao, Jane Y; Doody, Kaitlin; Kao, Lillian S; Lally, Kevin P; Tsao, KuoJen

2013-10-01

Despite increased compliance with Surgical Care Improvement Project infection measures, surgical-site infections are not decreasing. The aim of this study was to test the hypothesis that documented compliance with antibiotic prophylaxis guidelines on a pediatric surgery service does not reflect implementation fidelity or adherence to guidelines as intended. A 7-week observational study of elective pediatric surgical cases was conducted. Adherence was evaluated for appropriate administration, type, timing, weight-based dosing, and redosing of antibiotics. Prophylactic antibiotics were administered appropriately in 141 of 143 cases (99%). Of 100 cases (70%) in which antibiotic prophylaxis was indicated, compliance was documented in 100% cases in the electronic medical record, but only 48% of cases adhered to all 5 guidelines. Lack of adherence was due primarily to dosing or timing errors. Lack of implementation fidelity in antibiotic prophylaxis guidelines may partly explain the lack of expected reduction in surgical-site infections. Future studies of Surgical Care Improvement Project effectiveness should measure adherence and implementation fidelity rather than just documented compliance. Copyright © 2013 Elsevier Inc. All rights reserved.

Examining the association between implementation and outcomes : state-wide scale-up of school-wide positive behavior intervention and supports.

PubMed

Pas, Elise T; Bradshaw, Catherine P

2012-10-01

Although there is an established literature supporting the efficacy of a variety of prevention programs, there has been less empirical work on the translation of such research to everyday practice or when scaled-up state-wide. There is a considerable need for more research on factors that enhance implementation of programs and optimize outcomes, particularly in school settings. The current paper examines how the implementation fidelity of an increasingly popular and widely disseminated prevention model called, School-wide Positive Behavioral Interventions and Supports (SW-PBIS), relates to student outcomes within the context of a state-wide scale-up effort. Data come from a scale-up effort of SW-PBIS in Maryland; the sample included 421 elementary and middle schools trained in SW-PBIS. SW-PBIS fidelity, as measured by one of three fidelity measures, was found to be associated with higher math achievement, higher reading achievement, and lower truancy. School contextual factors were related to implementation levels and outcomes. Implications for scale-up efforts of behavioral and mental health interventions and measurement considerations are discussed.

Examining the Association Between Implementation and Outcomes

PubMed Central

Pas, Elise T.; Bradshaw, Catherine P.

2012-01-01

Although there is an established literature supporting the efficacy of a variety of prevention programs, there has been less empirical work on the tran of such research to everyday practice or when scaled-up state-wide. There is a considerable need for more research on factors that enhance implementation of programs and optimize outcomes, particularly in school settings. The current paper examines how the implementation fidelity of an increasingly popular and widely disseminated prevention model called, School-wide Positive Behavioral Interventions and Supports (SW-PBIS), relates to student outcomes within the context of a state-wide scale-up effort. Data come from a scale-up effort of SW-PBIS in Maryland; the sample included 421 elementary and middle schools trained in SW-PBIS. SW-PBIS fidelity, as measured by one of three fidelity measures, was found to be associated with higher math achievement, higher reading achievement, and lower truancy. School contextual factors were related to implementation levels and outcomes. Implications for scale-up efforts of behavioral and mental health interventions and measurement considerations are discussed. PMID:22836758

Progressive Failure of a Unidirectional Fiber-Reinforced Composite Using the Method of Cells: Discretization Objective Computational Results

NASA Technical Reports Server (NTRS)

Pineda, Evan J.; Bednarcyk, Brett A.; Waas, Anthony M.; Arnold, Steven M.

2012-01-01

The smeared crack band theory is implemented within the generalized method of cells and high-fidelity generalized method of cells micromechanics models to capture progressive failure within the constituents of a composite material while retaining objectivity with respect to the size of the discretization elements used in the model. An repeating unit cell containing 13 randomly arranged fibers is modeled and subjected to a combination of transverse tension/compression and transverse shear loading. The implementation is verified against experimental data (where available), and an equivalent finite element model utilizing the same implementation of the crack band theory. To evaluate the performance of the crack band theory within a repeating unit cell that is more amenable to a multiscale implementation, a single fiber is modeled with generalized method of cells and high-fidelity generalized method of cells using a relatively coarse subcell mesh which is subjected to the same loading scenarios as the multiple fiber repeating unit cell. The generalized method of cells and high-fidelity generalized method of cells models are validated against a very refined finite element model.

Becoming a high-fidelity - super - imitator: what are the contributions of social and individual learning?

PubMed

Subiaul, Francys; Patterson, Eric M; Schilder, Brian; Renner, Elizabeth; Barr, Rachel

2015-11-01

In contrast to other primates, human children's imitation performance goes from low to high fidelity soon after infancy. Are such changes associated with the development of other forms of learning? We addressed this question by testing 215 children (26-59 months) on two social conditions (imitation, emulation) - involving a demonstration - and two asocial conditions (trial-and-error, recall) - involving individual learning - using two touchscreen tasks. The tasks required responding to either three different pictures in a specific picture order (Cognitive: Airplane→Ball→Cow) or three identical pictures in a specific spatial order (Motor-Spatial: Up→Down→Right). There were age-related improvements across all conditions and imitation, emulation and recall performance were significantly better than trial-and-error learning. Generalized linear models demonstrated that motor-spatial imitation fidelity was associated with age and motor-spatial emulation performance, but cognitive imitation fidelity was only associated with age. While this study provides evidence for multiple imitation mechanisms, the development of one of those mechanisms - motor-spatial imitation - may be bootstrapped by the development of another social learning skill - motor-spatial emulation. Together, these findings provide important clues about the development of imitation, which is arguably a distinctive feature of the human species. © 2014 John Wiley & Sons Ltd.

Assessing fidelity of delivery of smoking cessation behavioural support in practice.

PubMed

Lorencatto, Fabiana; West, Robert; Christopherson, Charlotte; Michie, Susan

2013-04-04

Effectiveness of evidence-based behaviour change interventions is likely to be undermined by failure to deliver interventions as planned. Behavioural support for smoking cessation can be a highly cost-effective, life-saving intervention. However, in practice, outcomes are highly variable. Part of this may be due to variability in fidelity of intervention implementation. To date, there have been no published studies on this. The present study aimed to: evaluate a method for assessing fidelity of behavioural support; assess fidelity of delivery in two English Stop-Smoking Services; and compare the extent of fidelity according to session types, duration, individual practitioners, and component behaviour change techniques (BCTs). Treatment manuals and transcripts of 34 audio-recorded behavioural support sessions were obtained from two Stop-Smoking Services and coded into component BCTs using a taxonomy of 43 BCTs. Inter-rater reliability was assessed using percentage agreement. Fidelity was assessed by examining the proportion of BCTs specified in the manuals that were delivered in individual sessions. This was assessed by session type (i.e., pre-quit, quit, post-quit), duration, individual practitioner, and BCT. Inter-coder reliability was high (87.1%). On average, 66% of manual-specified BCTs were delivered per session (SD 15.3, range: 35% to 90%). In Service 1, average fidelity was highest for post-quit sessions (69%) and lowest for pre-quit (58%). In Service 2, fidelity was highest for quit-day (81%) and lowest for post-quit sessions (56%). Session duration was not significantly correlated with fidelity. Individual practitioner fidelity ranged from 55% to 78%. Individual manual-specified BCTs were delivered on average 63% of the time (SD 28.5, range: 0 to 100%). The extent to which smoking cessation behavioural support is delivered as specified in treatment manuals can be reliably assessed using transcripts of audiotaped sessions. This allows the investigation of the implementation of evidence-based practice in relation to smoking cessation, a first step in designing interventions to improve it. There are grounds for believing that fidelity in the English Stop-Smoking Services may be low and that routine monitoring is warranted.

Lapses, infidelities, and creative adaptations: lessons from evaluation of a participatory market development approach in the Andes.

PubMed

Horton, Douglas; Rotondo, Emma; Paz Ybarnegaray, Rodrigo; Hareau, Guy; Devaux, André; Thiele, Graham

2013-08-01

Participatory approaches are frequently recommended for international development programs, but few have been evaluated. From 2007 to 2010 the Andean Change Alliance evaluated an agricultural research and development approach known as the "Participatory Market Chain Approach" (PMCA). Based on a study of four cases, this paper examines the fidelity of implementation, the factors that influenced implementation and results, and the PMCA change model. We identify three types of deviation from the intervention protocol (lapses, creative adaptations, and true infidelities) and five groups of variables that influenced PMCA implementation and results (attributes of the macro context, the market chain, the key actors, rules in use, and the capacity development strategy). There was insufficient information to test the validity of the PMCA change model, but results were greatest where the PMCA was implemented with highest fidelity. Our analysis suggests that the single most critical component of the PMCA is engagement of market agents - not just farmers - throughout the exercise. We present four lessons for planning and evaluating participatory approaches related to the use of action and change models, the importance of monitoring implementation fidelity, the limits of baseline survey data for outcome evaluation, and the importance of capacity development for implementers. Copyright © 2013 Elsevier Ltd. All rights reserved.

Just-in-Time Teaching in undergraduate physics courses: Implementation, learning, and perceptions

NASA Astrophysics Data System (ADS)

Dwyer, Jessica Hewitt

Regardless of discipline, a decades-long battle has ensued within nearly every classroom in higher education: instructors getting students to come to class prepared to learn. In response to this clash between teacher expectations and frequent student neglect, a group of four physics education researchers developed a reformed instructional strategy called Just-in-Time Teaching (JiTT). This dissertation investigates the following three areas: 1) the fidelity with which undergraduate physics instructors implement JiTT, 2) whether student performance predicts student perception of their instructor's fidelity of JiTT implementation, and 3) whether student perception of their instructor's fidelity of JiTT implementation correlates with student views of their physics course. A blend of quantitative data (e.g., students grades, inventory scores, and questionnaire responses) are integrated with qualitative data (e.g., individual faculty interviews, student focus group discussions, and classroom observations). This study revealed no statistically significant relationship between instructors who spent time on a predefined JiTT critical component and their designation as a JiTT user or non-user. While JiTT users implemented the pedagogy in accordance with the creators' intended ideal vision, many also had trouble reconciling personal concerns about their role as a JiTT adopter and the anticipated demand of the innovation. I recommend that this population of faculty members can serve as a JiTT model for other courses, disciplines, and/or institutions. Student performance was not a predictor of student perception instructor fidelity of JiTT implementation. Additionally, the majority of students in this study reported they read their textbook prior to class and that JiTT assignments helped them prepare for in-class learning. I found evidence that exposure to the JiTT strategy may correlate with a more favorable student view of their physics course. Finally, according to students, favorable JiTT implementation occurred when instructors reviewed all questions contained within the JiTT assignment during class and when instructors clearly connected JiTT questions to the textbook reading, lesson discussion, and other assignments. The impact of this work rests in its possibility to set the stage for future education studies on the fidelity of implementation of other research-based instructional strategies in various disciplines and how they affect student performance and perceptions.

Building an Evidence Base for Effective Supervision Practices: An Analogue Experiment of Supervision to Increase EBT Fidelity.

PubMed

Bearman, Sarah Kate; Schneiderman, Robyn L; Zoloth, Emma

2017-03-01

Treatments that are efficacious in research trials perform less well under routine conditions; differences in supervision may be one contributing factor. This study compared the effect of supervision using active learning techniques (e.g. role play, corrective feedback) versus "supervision as usual" on therapist cognitive restructuring fidelity, overall CBT competence, and CBT expertise. Forty therapist trainees attended a training workshop and were randomized to supervision condition. Outcomes were assessed using behavioral rehearsals pre- and immediately post-training, and after three supervision meetings. EBT knowledge, attitudes, and fidelity improved for all participants post-training, but only the SUP+ group demonstrated improvement following supervision.

High Versus Low Theoretical Fidelity Pedometer Intervention Using Social-Cognitive Theory on Steps and Self-Efficacy.

PubMed

Raedeke, Thomas D; Dlugonski, Deirdre

2017-12-01

This study was designed to compare a low versus high theoretical fidelity pedometer intervention applying social-cognitive theory on step counts and self-efficacy. Fifty-six public university employees participated in a 10-week randomized controlled trial with 2 conditions that varied in theoretical fidelity. Participants in the high theoretical fidelity condition wore a pedometer and participated in a weekly group walk followed by a meeting to discuss cognitive-behavioral strategies targeting self-efficacy. Participants in the low theoretical fidelity condition met for a group walk and also used a pedometer as a motivational tool and to monitor steps. Step counts were assessed throughout the 10-week intervention and after a no-treatment follow-up (20 weeks and 30 weeks). Self-efficacy was measured preintervention and postintervention. Participants in the high theoretical fidelity condition increased daily steps by 2,283 from preintervention to postintervention, whereas participants in the low fidelity condition demonstrated minimal change during the same time period (p = .002). Individuals attending at least 80% of the sessions in the high theoretical fidelity condition showed an increase of 3,217 daily steps (d = 1.03), whereas low attenders increased by 925 (d = 0.40). Attendance had minimal impact in the low theoretical fidelity condition. Follow-up data revealed that step counts were at least somewhat maintained. For self-efficacy, participants in the high, compared with those in the low, theoretical fidelity condition showed greater improvements. Findings highlight the importance of basing activity promotion efforts on theory. The high theoretical fidelity intervention that included cognitive-behavioral strategies targeting self-efficacy was more effective than the low theoretical fidelity intervention, especially for those with high attendance.

A multi-level intervention in worksites to increase fruit and vegetable access and intake: Rationale, design and methods of the 'Good to Go' cluster randomized trial.

PubMed

Risica, Patricia M; Gorham, Gemma; Dionne, Laura; Nardi, William; Ng, Doug; Middler, Reese; Mello, Jennifer; Akpolat, Rahmet; Gettens, Katelyn; Gans, Kim M

2018-02-01

Fruit and vegetable (F&V) consumption is an important contributor to chronic disease prevention. However, most Americans do not eat adequate amounts. The worksite is an advantageous setting to reach large, diverse segments of the population with interventions to increase F&V intake, but research gaps exist. No studies have evaluated the implementation of mobile F&V markets at worksites nor compared the effectiveness of such markets with or without nutrition education. This paper describes the protocol for Good to Go (GTG), a cluster randomized trial to evaluate F&V intake change in employees from worksites randomized into three experimental arms: discount, fresh F&V markets (Access Only arm); markets plus educational components including campaigns, cooking demonstrations, videos, newsletters, and a web site (Access Plus arm); and an attention placebo comparison intervention on physical activity and stress reduction (Comparison). Secondary aims include: 1) Process evaluation to determine costs, reach, fidelity, and dose as well as the relationship of these variables with changes in F&V intake; 2) Applying a mediating variable framework to examine relationships of psychosocial factors/determinants with changes in F&V consumption; and 3) Cost effectiveness analysis of the different intervention arms. The GTG study will fill important research gaps in the field by implementing a rigorous cluster randomized trial to evaluate the efficacy of an innovative environmental intervention providing access and availability to F&V at the worksite and whether this access intervention is further enhanced by accompanying educational interventions. GTG will provide an important contribution to public health research and practice. Trial registration number NCT02729675, ClinicalTrials.gov. Copyright © 2017 The Authors. Published by Elsevier Inc. All rights reserved.

A multi-level intervention in worksites to increase fruit and vegetable access and intake: Rationale, design and methods of the ‘Good to Go’ cluster randomized trial

PubMed Central

Risica, Patricia M.; Gorham, Gemma; Dionne, Laura; Nardi, William; Ng, Doug; Middler, Reese; Mello, Jennifer; Akpolat, Rahmet; Gettens, Katelyn; Gans, Kim M.

2018-01-01

Background Fruit and vegetable (F&V) consumption is an important contributor to chronic disease prevention. However, most Americans do not eat adequate amounts. The worksite is an advantageous setting to reach large, diverse segments of the population with interventions to increase F&V intake, but research gaps exist. No studies have evaluated the implementation of mobile F&V markets at worksites nor compared the effectiveness of such markets with or without nutrition education. Methods This paper describes the protocol for Good to Go (GTG), a cluster randomized trial to evaluate F&V intake change in employees from worksites randomized into three experimental arms: discount, fresh F&V markets (Access Only arm); markets plus educational components including campaigns, cooking demonstrations, videos, newsletters, and a web site (Access Plus arm); and an attention placebo comparison intervention on physical activity and stress reduction (Comparison). Secondary aims include: 1) Process evaluation to determine costs, reach, fidelity, and dose as well as the relationship of these variables with changes in F&V intake; 2) Applying a mediating variable framework to examine relationships of psychosocial factors/determinants with changes in F&V consumption; and 3) Cost effectiveness analysis of the different intervention arms. Discussion The GTG study will fill important research gaps in the field by implementing a rigorous cluster randomized trial to evaluate the efficacy of an innovative environmental intervention providing access and availability to F&V at the worksite and whether this access intervention is further enhanced by accompanying educational interventions. GTG will provide an important contribution to public health research and practice. Trial registration number NCT02729675, ClinicalTrials.gov PMID:29242108

Fiscal loss and program fidelity: impact of the economic downturn on HIV/STI prevention program fidelity.

PubMed

Catania, Joseph A; Dolcini, M Margaret; Gandelman, Alice A; Narayanan, Vasudha; McKay, Virginia R

2014-03-01

The economic downturn of 2007 created significant fiscal losses for public and private agencies conducting behavioral prevention. Such macro-economic changes may influence program implementation and sustainability. We examined how public and private agencies conducting RESPECT, a brief HIV/STI (sexually transmitted infection) counseling and testing intervention, adapted to fiscal loss and how these adaptations impacted program fidelity. We collected qualitative and quantitative data in a national sample of 15 agencies experiencing fiscal loss. Using qualitative analyses, we examined how program fidelity varied with different types of adaptations. Agencies reported three levels of adaptation: agency-level, program-level, and direct fiscal remedies. Private agencies tended to use direct fiscal remedies, which were associated with higher fidelity. Some agency-level adaptations contributed to reductions in procedural fit, leading to negative staff morale and decreased confidence in program effectiveness, which in turn, contributed to poor fidelity. Findings describe a "work stress pathway" that links program fiscal losses to poor staff morale and low program fidelity.

Challenges of Applying a Comprehensive Model of Intervention Fidelity

PubMed Central

Bosak, Kelly; Pozehl, Bunny; Yates, Bernice

2014-01-01

Applying a comprehensive model of fidelity to interventions delivered by Information and Communication Technology (ICT) has multiple challenges. Fidelity must be considered in the design, implementation, evaluation, and reporting of the intervention. The fidelity strategies must address the unique aspects of the technology, including training providers to instruct participants to use the technology and to provide standardized feedback, rather than deliver the intervention in-person. Other challenges include the nonspecific effects resulting from participants accessing unintended content in interventions delivered by the Internet. ICT allows participant receipt and enactment of intervention skills to be assessed by electronic evidence, rather than in-person observation. Interventions using ICT, such as the Internet are unique, and there is less control of participant interaction with various electronic components. Monitoring participant use and providing standardized feedback for receipt and enactment of intervention skills is key to ensuring intervention fidelity. The final challenges involve evaluating and reporting fidelity. PMID:21474676

Evaluating Fidelity to a Modified NIATx Process Improvement Strategy for Improving HIV Services in Correctional Facilities.

PubMed

Pankow, Jennifer; Willett, Jennifer; Yang, Yang; Swan, Holly; Dembo, Richard; Burdon, William M; Patterson, Yvonne; Pearson, Frank S; Belenko, Steven; Frisman, Linda K

2018-04-01

In a study aimed at improving the quality of HIV services for inmates, an organizational process improvement strategy using change teams was tested in 14 correctional facilities in 8 US states and Puerto Rico. Data to examine fidelity to the process improvement strategy consisted of quantitative ratings of the structural and process components of the strategy and qualitative notes that explicate challenges in maintaining fidelity to the strategy. Fidelity challenges included (1) lack of communication and leadership within change teams, (2) instability in team membership, and (3) issues with data utilization in decision-making to implement improvements to services delivery.

A New Design for Airway Management Training with Mixed Reality and High Fidelity Modeling.

PubMed

Shen, Yunhe; Hananel, David; Zhao, Zichen; Burke, Daniel; Ballas, Crist; Norfleet, Jack; Reihsen, Troy; Sweet, Robert

2016-01-01

Restoring airway function is a vital task in many medical scenarios. Although various simulation tools have been available for learning such skills, recent research indicated that fidelity in simulating airway management deserves further improvements. In this study, we designed and implemented a new prototype for practicing relevant tasks including laryngoscopy, intubation and cricothyrotomy. A large amount of anatomical details or landmarks were meticulously selected and reconstructed from medical scans, and 3D-printed or molded to the airway intervention model. This training model was augmented by virtually and physically presented interactive modules, which are interoperable with motion tracking and sensor data feedback. Implementation results showed that this design is a feasible approach to develop higher fidelity airway models that can be integrated with mixed reality interfaces.

Permanent Supportive Housing for Transition-Age Youths: Service Costs and Fidelity to the Housing First Model.

PubMed

Gilmer, Todd P

2016-06-01

Permanent supportive housing (PSH) programs are being implemented nationally and on a large scale. However, little is known about PSH for transition-age youths (ages 18 to 24). This study estimated health services costs associated with participation in PSH among youths and examined the relationship between fidelity to the Housing First model and health service outcomes. Administrative data were used in a quasi-experimental, difference-in-differences design with a propensity score-matched contemporaneous control group to compare health service costs among 2,609 youths in PSH and 2,609 youths with serious mental illness receiving public mental health services in California from January 1, 2004, through June 30, 2010. Data from a survey of PSH program practices were merged with the administrative data to examine changes in service use among 1,299 youths in 63 PSH programs by level of fidelity to the Housing First model. Total service costs increased by $13,337 among youths in PSH compared with youths in the matched control group. Youths in higher-fidelity programs had larger declines in use of inpatient services and larger increases in outpatient visits compared with youths in lower-fidelity programs. PSH for youths was associated with substantial increases in costs. Higher-fidelity PSH programs may be more effective than lower-fidelity programs in reducing use of inpatient services and increasing use of outpatient services. As substantial investments are made in PSH for youths, it is imperative that these programs are designed and implemented to maximize their effectiveness and their impact on youth outcomes.

Intervention Fidelity for a Complex Behaviour Change Intervention in Community Pharmacy Addressing Cardiovascular Disease Risk

ERIC Educational Resources Information Center

McNamara, K. P.; O'Reilly, S. L.; George, J.; Peterson, G. M.; Jackson, S. L.; Duncan, G.; Howarth, H.; Dunbar, J. A.

2015-01-01

Background: Delivery of cardiovascular disease (CVD) prevention programs by community pharmacists appears effective and enhances health service access. However, their capacity to implement complex behavioural change processes during patient counselling remains largely unexplored. This study aims to determine intervention fidelity by pharmacists…

A Review of Treatment Adherence Measurement Methods

ERIC Educational Resources Information Center

Schoenwald, Sonja K.; Garland, Ann F.

2013-01-01

Fidelity measurement is critical for testing the effectiveness and implementation in practice of psychosocial interventions. Adherence is a critical component of fidelity. The purposes of this review were to catalogue adherence measurement methods and assess existing evidence for the valid and reliable use of the scores that they generate and the…

Progress towards a microwave-based high-fidelity Toffoli gate with superconducting qubits

NASA Astrophysics Data System (ADS)

Rigetti, Chad; Chow, Jerry; Corcoles, Antonio; Rozen, Jim; Keefe, George; Rothwell, Mary Beth; Rohrs, Jack; Borstelmann, Mark; Divincenzo, David; Ketchen, Mark; Steffen, Matthias

2011-03-01

We describe recent progress at IBM towards a microwave-based implementation of the Toffoli gate using three capacitively shunted flux qubits dispersively coupled to a resonator. We discuss the device architecture and the microwave protocol, along with expected limits to gate fidelity and scaling.

The PD COMM trial: a protocol for the process evaluation of a randomised trial assessing the effectiveness of two types of SLT for people with Parkinson's disease.

PubMed

Masterson-Algar, Patricia; Burton, Christopher R; Brady, Marian C; Nicoll, Avril; Clarke, Carl E; Rick, Caroline; Hughes, Max; Au, Pui; Smith, Christina H; Sackley, Catherine M

2017-08-29

The PD COMM trial is a phase III multi-centre randomised controlled trial whose aim is to evaluate the effectiveness and cost-effectiveness of two approaches to speech and language therapy (SLT) compared with no SLT intervention (control) for people with Parkinson's disease who have self-reported or carer-reported problems with their speech or voice. Our protocol describes the process evaluation embedded within the outcome evaluation whose aim is to evaluate what happened at the time of the PD COMM intervention implementation and to provide findings that will assist in the interpretation of the PD COMM trial results. Furthermore, the aim of the PD COMM process evaluation is to investigate intervention complexity within a theoretical model of how the trialled interventions might work best and why. Drawing from the Normalization Process Theory and frameworks for implementation fidelity, a mixed method design will be used to address process evaluation research questions. Therapists' and participants' perceptions and experiences will be investigated via in-depth interviews. Critical incident reports, baseline survey data from therapists, treatment record forms and home practice diaries also will be collected at relevant time points throughout the running of the PD COMM trial. Process evaluation data will be analysed independently of the outcome evaluation before the two sets of data are then combined. To date, there are a limited number of published process evaluation protocols, and few are linked to trials investigating rehabilitation therapies. Providing a strong theoretical framework underpinning design choices and being tailored to meet the complex characteristics of the trialled interventions, our process evaluation has the potential to provide valuable insight into which components of the interventions being delivered in PD COMM worked best (and what did not), how they worked well and why. ISRCTN Registry, ISRCTN12421382 . Registered on 18 April 2016.

Early ART initiation among HIV-positive pregnant women in central Mozambique: a stepped wedge randomized controlled trial of an optimized Option B+ approach.

PubMed

Cowan, James F; Micek, Mark; Cowan, Jessica F Greenberg; Napúa, Manuel; Hoek, Roxanne; Gimbel, Sarah; Gloyd, Stephen; Sherr, Kenneth; Pfeiffer, James T; Chapman, Rachel R

2015-04-30

Despite effective prevention strategies and increasing investments in global health, maternal to child transmission (MTCT) of HIV remains a significant problem globally, especially in sub-Saharan Africa. In 2012, there were 94,000 HIV-positive pregnant women in Mozambique. Approximately 15% of these women transmitted HIV to their newborn infants, resulting in nearly 14,000 new pediatric HIV infections that year. To address this issue, in 2013, the Mozambican Ministry of Health implemented the World Health Organization-recommended "Option B+" strategy in which all newly diagnosed HIV-positive pregnant women are counseled to initiate combination anti-retroviral therapy (ART) immediately upon diagnosis regardless of CD4 count and to continue treatment for life. Given the limited experience with Option B+ in sub-Saharan Africa, few rigorous pragmatic trials have studied this new treatment strategy. This study utilizes an initial formative research process involving patient and health care provider interviews and focus groups, workforce assessments, value stream mapping, and commodity utilization assessments to understand the strengths and weaknesses in the current Option B+ care cascade. The formative research is intended to guide identification and prioritization of key workflow modifications and the development of an enhanced adherence and retention package. These two components are bundled into a defined intervention implemented and evaluated across six health facilities utilizing a stepped wedge randomized controlled trial study design. The overall objective of this trial is to develop and test a pilot intervention in central Mozambique to implement the new Option B+ guidelines with high fidelity and increase the proportion of HIV-positive pregnant women in target antenatal clinics (ANC) who start ART prior to delivery and are retained in care. This pragmatic study utilizes research strategies that have the potential to meaningfully improve the Option B+ care cascade in central Mozambique and to decrease the MTCT of HIV. This trial is designed to identify critical low-cost improvement strategies that can be bundled into a defined intervention. If this intervention has a measurable impact, it can be rapidly scaled up to other ANC in Mozambique and sub-Saharan Africa. ClinicalTrials.gov: NCT02371265.

Effectiveness and micro-costing of the KiVa school-based bullying prevention programme in Wales: study protocol for a pragmatic definitive parallel group cluster randomised controlled trial.

PubMed

Clarkson, Suzy; Axford, Nick; Berry, Vashti; Edwards, Rhiannon Tudor; Bjornstad, Gretchen; Wrigley, Zoe; Charles, Joanna; Hoare, Zoe; Ukoumunne, Obioha C; Matthews, Justin; Hutchings, Judy

2016-02-01

Bullying refers to verbal, physical or psychological aggression repeated over time that is intended to cause harm or distress to the victims who are unable to defend themselves. It is a key public health priority owing to its widespread prevalence in schools and harmful short- and long-term effects on victims' well-being. There is a need to strengthen the evidence base by testing innovative approaches to preventing bullying. KiVa is a school-based bullying prevention programme with universal and indicated elements and an emphasis on changing bystander behaviour. It achieved promising results in a large trial in Finland, and now requires testing in other countries. This paper describes the protocol for a cluster randomised controlled trial (RCT) of KiVa in Wales. The study uses a two-arm waitlist control pragmatic definitive parallel group cluster RCT design with an embedded process evaluation and calculation of unit cost. Participating schools will be randomised a using a 1:1 ratio to KiVa plus usual provision (intervention group) or usual provision only (control group). The trial has one primary outcome, child self-reported victimisation from bullying, dichotomised as 'victimised' (bullied at least twice a month in the last couple of months) versus 'not victimised'. Secondary outcomes are: bullying perpetration; aspects of child social and emotional well-being (including emotional problems, conduct, peer relations, prosocial behaviour); and school attendance. Follow-up is at 12 months post-baseline. Implementation fidelity is measured through teacher-completed lesson records and independent school-wide observation. A micro-costing analysis will determine the costs of implementing KiVa, including recurrent and non-recurrent unit costs. Factors related to the scalability of the programme will be examined in interviews with head teachers and focus groups with key stakeholders in the implementation of school-based bullying interventions. The results from this trial will provide evidence on whether the KiVa programme is transportable from Finland to Wales in terms of effectiveness and implementation. It will provide information about the costs of delivery and generate insights into factors related to the scalability of the programme. Current Controlled Trials ISRCTN23999021 Date 10-6-13.

The Effects of School Gardens on Children's Science Knowledge: A randomized controlled trial of low-income elementary schools

NASA Astrophysics Data System (ADS)

Wells, Nancy M.; Myers, Beth M.; Todd, Lauren E.; Barale, Karen; Gaolach, Brad; Ferenz, Gretchen; Aitken, Martha; Henderson, Charles R., , Jr.; Tse, Caroline; Pattison, Karen Ostlie; Taylor, Cayla; Connerly, Laura; Carson, Janet B.; Gensemer, Alexandra Z.; Franz, Nancy K.; Falk, Elizabeth

2015-11-01

This randomized controlled trial or 'true experiment' examines the effects of a school garden intervention on the science knowledge of elementary school children. Schools were randomly assigned to a group that received the garden intervention (n = 25) or to a waitlist control group that received the garden intervention at the end of the study (n = 24). The garden intervention consisted of both raised-bed garden kits and a series of 19 lessons. Schools, located in the US states of Arkansas, Iowa, Washington, and New York, were all low-income as defined by having 50% or more children qualifying for the federal school lunch program. Participants were students in second, fourth, and fifth grade (ages 6-12) at baseline (n = 3,061). Science knowledge was measured using a 7-item questionnaire focused on nutritional science and plant science. The survey was administered at baseline (Fall 2011) and at three time points during the intervention (Spring 2012, Fall 2012, and Spring 2013). Garden intervention fidelity (GIF) captured the robustness or fidelity of the intervention delivered in each classroom based on both lessons delivered and garden activities. Analyses were conducted using general linear mixed models. Survey data indicated that among children in the garden intervention, science knowledge increased from baseline to follow-up more than among control group children. However, science knowledge scores were uniformly poor and gains were very modest. GIF, which takes into account the robustness of the intervention, revealed a dose-response relation with science knowledge: more robust or substantial intervention implementations corresponded to stronger treatment effects.

Measurement-based care for refractory depression: a clinical decision support model for clinical research and practice.

PubMed

Trivedi, Madhukar H; Daly, Ella J

2007-05-01

Despite years of antidepressant drug development and patient and provider education, suboptimal medication dosing and duration of exposure resulting in incomplete remission of symptoms remains the norm in the treatment of depression. Additionally, since no one treatment is effective for all patients, optimal implementation focusing on the measurement of symptoms, side effects, and function is essential to determine effective sequential treatment approaches. There is a need for a paradigm shift in how clinical decision making is incorporated into clinical practice and for a move away from the trial-and-error approach that currently determines the "next best" treatment. This paper describes how our experience with the Texas Medication Algorithm Project (TMAP) and the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) trial has confirmed the need for easy-to-use clinical support systems to ensure fidelity to guidelines. To further enhance guideline fidelity, we have developed an electronic decision support system that provides critical feedback and guidance at the point of patient care. We believe that a measurement-based care (MBC) approach is essential to any decision support system, allowing physicians to individualize and adapt decisions about patient care based on symptom progress, tolerability of medication, and dose optimization. We also believe that successful integration of sequential algorithms with MBC into real-world clinics will facilitate change that will endure and improve patient outcomes. Although we use major depression to illustrate our approach, the issues addressed are applicable to other chronic psychiatric conditions including comorbid depression and substance use disorder as well as other medical illnesses.

Measurement-Based Care for Refractory Depression: A Clinical Decision Support Model for Clinical Research and Practice

PubMed Central

Trivedi, Madhukar H.; Daly, Ella J.

2009-01-01

Despite years of antidepressant drug development and patient and provider education, suboptimal medication dosing and duration of exposure resulting in incomplete remission of symptoms remains the norm in the treatment of depression. Additionally, since no one treatment is effective for all patients, optimal implementation focusing on the measurement of symptoms, side effects, and function is essential to determine effective sequential treatment approaches. There is a need for a paradigm shift in how clinical decision making is incorporated into clinical practice and for a move away from the trial-and-error approach that currently determines the “next best” treatment. This paper describes how our experience with the Texas Medication Algorithm Project (TMAP) and the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) trial has confirmed the need for easy-to-use clinical support systems to ensure fidelity to guidelines. To further enhance guideline fidelity, we have developed an electronic decision support system that provides critical feedback and guidance at the point of patient care. We believe that a measurement-based care (MBC) approach is essential to any decision support system, allowing physicians to individualize and adapt decisions about patient care based on symptom progress, tolerability of medication, and dose optimization. We also believe that successful integration of sequential algorithms with MBC into real-world clinics will facilitate change that will endure and improve patient outcomes. Although we use major depression to illustrate our approach, the issues addressed are applicable to other chronic psychiatric conditions including comorbid depression and substance use disorder as well as other medical illnesses. PMID:17320312

Implementation of a community-based secondhand smoke reduction intervention for caregivers of urban children with asthma: process evaluation, successes and challenges

PubMed Central

Blaakman, Susan; Tremblay, Paul J.; Halterman, Jill S.; Fagnano, Maria; Borrelli, Belinda

2013-01-01

Many children, including those with asthma, remain exposed to secondhand smoke. This manuscript evaluates the process of implementing a secondhand smoke reduction counseling intervention using motivational interviewing (MI) for caregivers of urban children with asthma, including reach, dose delivered, dose received and fidelity. Challenges, strategies and successes in applying MI are highlighted. Data for 140 children (3–10 years) enrolled in the School Based Asthma Therapy trial, randomized to the treatment condition and living with one or more smoker, were analyzed. Summary statistics describe the sample, process measures related to intervention implementation, and primary caregiver (PCG) satisfaction with the intervention. The full intervention was completed by 79% of PCGs, but only 17% of other smoking caregivers. Nearly all (98%) PCGs were satisfied with the care study nurses provided and felt the program might be helpful to others. Despite challenges, this intervention was feasible and well received reaching caregivers who were not actively seeking treatment for smoking cessation or secondhand smoke reduction. Anticipating the strategies required to implement such an intervention may help promote participant engagement and retention to enhance the program’s ultimate success. PMID:22717938

Organizational fidelity to a medication management evidence-based practice in the treatment of schizophrenia.

PubMed

El-Mallakh, Peggy; Howard, Patricia B; Rayens, Mary Kay; Roque, Autumn P; Adkins, Sarah

2013-11-01

Organizational support is essential for successful implementation of evidence-based practice (EBP) in clinical settings. This 3-year study used a mixed qualitative and quantitative design to implement a medication management EBP in the treatment of schizophrenia in six community mental health clinics in a south-central state of the United States. Findings from organizational fidelity assessments indicate that support for EBP implementation was moderate. Organizational support was highest for prescriber access to relevant patient information at each medication visit, scheduling flexibility for patients' urgent problems, and availability of medication guidelines. Organizational support was lowest for medication availability and identification of treatment refractory patients. Findings suggest that leadership is essential to support successful implementation. Nurse educators can incorporate implementation research and leadership training into graduate nursing programs to facilitate successful EBP implementation in practice settings. Copyright 2013, SLACK Incorporated.

Beyond the Ask and Advise: Implementation of a Computer Tablet Intervention to Enhance Provider Adherence to the 5As for Smoking Cessation.

PubMed

Kalkhoran, Sara; Appelle, Nicole A; Napoles, Anna M; Munoz, Ricardo F; Lum, Paula J; Alvarado, Nicholas; Gregorich, Steven E; Satterfield, Jason M

2016-01-01

The 5As for smoking cessation is an evidence-based intervention to aid providers in counseling patients to quit smoking. While most providers "ask" patients about their tobacco use patterns and "advise" them to quit, fewer patients report being "assessed" for their interest in quitting, and even fewer report subsequent "assistance" in a quit attempt and having follow-up "arranged". This article describes the design of an implementation study testing a computer tablet intervention to improve provider adherence to the 5As for smoking cessation. Findings will contribute to the existing literature on technology acceptance for addressing addictive behaviors, and how digital tools may facilitate the broader implementation of evidence-based behavioral counseling practices without adversely affecting clinical flow or patient care. This project develops and tests a computer-facilitated 5As (CF-5As) model that administers the 5As intervention to patients with a computer tablet, then prompts providers to reinforce next steps. During the development phase, 5As' content will be programmed onto computer tablets, alpha and beta-testing of the service delivery model will be done, and pre-intervention interview and questionnaire data will be collected from patients, providers, and clinic staff about 5As fidelity and technology adoption. During the program evaluation phase, a randomized controlled trial comparing a group who receives the CF-5As intervention to one that does not will be conducted to assess 5As fidelity. Using the technology acceptance model, a mixed methods study of contextual and human factors influencing both 5As and technology adoption will also be conducted. Technology is increasingly being used in clinical settings. A technological tool that connects patients, providers, and clinic staff to facilitate the promotion of behavioral interventions such as smoking cessation may provide an innovative platform through which to efficiently and effectively implement evidence-based practices. Copyright © 2015 Elsevier Inc. All rights reserved.

A complex culturally targeted intervention to reduce Hispanic disparities in living kidney donor transplantation: an effectiveness-implementation hybrid study protocol.

PubMed

Gordon, Elisa J; Lee, Jungwha; Kang, Raymond H; Caicedo, Juan Carlos; Holl, Jane L; Ladner, Daniela P; Shumate, Michelle D

2018-05-16

The shortage of organs for kidney transplantation for patients with end-stage renal disease (ESRD) is magnified in Hispanics/Latin Americans in the United States. Living donor kidney transplantation (LDKT) is the treatment of choice for ESRD. However, compared to their representation on the transplant waitlist, fewer Hispanics receive a LDKT than non-Hispanic whites. Barriers to LDKT for Hispanics include: lack of knowledge, cultural concerns, and language barriers. Few interventions have been designed to reduce LDKT disparities. This study aims to reduce Hispanic disparities in LDKT through a culturally targeted intervention. Using a prospective effectiveness-implementation hybrid design involving pre-post intervention evaluation with matched controls, we will implement a complex culturally targeted intervention at two transplant centers in Dallas, TX and Phoenix, AZ. The goal of the study is to evaluate the effect of Northwestern Medicine's® Hispanic Kidney Transplant Program's (HKTP) key culturally targeted components (outreach, communication, education) on Hispanic LDKT rates over five years. The main hypothesis is that exposure to the HKTP will reduce disparities by increasing the ratio of Hispanic to non-Hispanic white LDKTs and the number of Hispanic LDKTs. We will also examine other process and outcome measures including: dialysis patient outreach, education session attendance, marketing efforts, Hispanic patients added to the waitlist, Hispanic potential donors per potential recipient, and satisfaction with culturally competent care. We will use mixed methods based on the Promoting Action on Research Implementation in Health Services (revised PARIHS) and the Consolidated Framework for Implementation Research (CFIR) frameworks to formatively evaluate the fidelity and innovative adaptations to HKTP's components at both study sites, to identify moderating factors that most affect implementation fidelity, and to identify adaptations that positively and negatively affect outcomes for patients. Our study will provide new knowledge about implementing culturally targeted interventions and their impact on reducing health disparities. Moreover, the study of a complex organizational-level intervention's implementation over five years is rare in implementation science; as such, this study is poised to contribute new knowledge to the factors influencing how organizational-level interventions are sustained over time. (ClinicalTrials.gov registration # NCT03276390 , date of registration: 9-7-17, retrospectively registered).

Evaluating the Implementation Fidelity of Technology Immersion and Its Relationship with Student Achievement

ERIC Educational Resources Information Center

Shapley, Kelly S.; Sheehan, Daniel; Maloney, Catherine; Caranikas-Walker, Fanny

2010-01-01

In a pilot study of the Technology Immersion model, high-need middle schools were "immersed" in technology by providing a laptop for each student and teacher, wireless Internet access, curricular and assessment resources, professional development, and technical and pedagogical support. This article examines the fidelity of model…

High Fidelity: Investing in Evaluation Training. Ask the Team

ERIC Educational Resources Information Center

Fetters, Jenni

2013-01-01

High-quality training is a crucial investment in establishing and maintaining implementation fidelity as well as building educators' trust in the new process. Training approaches for educator evaluation vary both in format (i.e., how it's delivered) and content (i.e., what is provided). Train-the-trainer sessions, online professional learning…

Did They Get It? The Role of Fidelity in Teaching English Learners

ERIC Educational Resources Information Center

Echevarria, Jana; Richards-Tutor, Catherine; Chinn, Vivan P.; Ratleff, Paige Ann

2011-01-01

Use of research-based practices for improving content area literacy of English learners is important both because of their growing numbers and their overall poor academic performance. However, how do we ensure that teachers are implementing proven methods with fidelity? This study shows the direct relationship between teachers' level of…

Treatment Fidelity: Its Importance and Reported Frequency in Aphasia Treatment Studies

ERIC Educational Resources Information Center

Hinckley, Jacqueline J.; Douglas, Natalie F.

2013-01-01

Purpose: Treatment fidelity is a measure of the reliability of the administration of an intervention in a treatment study. It is an important aspect of the validity of a research study, and it has implications for the ultimate implementation of evidence-supported interventions in typical clinical settings. Method: Aphasia treatment studies…

Implementation: Measuring and Explaining the Fidelity of CSR Implementation

ERIC Educational Resources Information Center

Kurki, Anja; Boyle, Andrea; Aladjem, Daniel K.

2006-01-01

Comprehensive school reform (CSR) is only as effective as its implementation. By using data collected for the National Longitudinal Evaluation of Comprehensive School Reform (NLECSR), this article explores the factors that predict CSR model implementation and the ways that CSR model implementation varies. We found little difference in the fidelity…

How to teach emergency procedural skills in an outdoor environment using low-fidelity simulation.

PubMed

Saxon, Kathleen D; Kapadia, Alison P R; Juneja, Nadia S; Bassin, Benjamin S

2014-03-01

Teaching emergency procedural skills in a wilderness setting can be logistically challenging. To teach these skills as part of a wilderness medicine elective for medical students, we designed an outdoor simulation session with low-fidelity models. The session involved 6 stations in which procedural skills were taught using homemade low-fidelity simulators. At each station, the students encountered a "victim," who required an emergency procedure that was performed using the low-fidelity model. The models are easy and inexpensive to construct, and their design and implementation in the session is described here. Using low-fidelity simulation models in an outdoor setting is an effective teaching tool for emergency wilderness medicine procedures and can easily be reproduced in future wilderness medicine courses. © 2014 Wilderness Medical Society Published by Wilderness Medical Society All rights reserved.

Housing first on a large scale: Fidelity strengths and challenges in the VA's HUD-VASH program.

PubMed

Kertesz, Stefan G; Austin, Erika L; Holmes, Sally K; DeRussy, Aerin J; Van Deusen Lukas, Carol; Pollio, David E

2017-05-01

Housing First (HF) combines permanent supportive housing and supportive services for homeless individuals and removes traditional treatment-related preconditions for housing entry. There has been little research describing strengths and shortfalls of HF implementation outside of research demonstration projects. The U.S. Department of Veterans Affairs (VA) has transitioned to an HF approach in a supportive housing program serving over 85,000 persons. This offers a naturalistic window to study fidelity when HF is adopted on a large scale. We operationalized HF into 20 criteria grouped into 5 domains. We assessed 8 VA medical centers twice (1 year apart), scoring each criterion using a scale ranging from 1 ( low fidelity ) to 4 ( high fidelity ). There were 2 HF domains (no preconditions and rapidly offering permanent housing) for which high fidelity was readily attained. There was uneven progress in prioritizing the most vulnerable clients for housing support. Two HF domains (sufficient supportive services and a modern recovery philosophy) had considerably lower fidelity. Interviews suggested that operational issues such as shortfalls in staffing and training likely hindered performance in these 2 domains. In this ambitious national HF program, the largest to date, we found substantial fidelity in focusing on permanent housing and removal of preconditions to housing entry. Areas of concern included the adequacy of supportive services and adequacy in deployment of a modern recovery philosophy. Under real-world conditions, large-scale implementation of HF is likely to require significant additional investment in client service supports to assure that results are concordant with those found in research studies. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

Community-based implementation and effectiveness in a randomized trial of a risk reduction intervention for HIV-serodiscordant couples: study protocol

PubMed Central

2014-01-01

Background The HIV/AIDS epidemic continues to disproportionately affect African American communities in the US, particularly those located in urban areas. Despite the fact that HIV is often transmitted from one sexual partner to another, most HIV prevention interventions have focused only on individuals, rather than couples. This five-year study investigates community-based implementation, effectiveness, and sustainability of ‘Eban II,’ an evidence-based risk reduction intervention for African-American heterosexual, serodiscordant couples. Methods/design This hybrid implementation/effectiveness implementation study is guided by organizational change theory as conceptualized in the Texas Christian University Program Change Model (PCM), a model of phased organizational change from exposure to adoption, implementation, and sustainability. The primary implementation aims are to assist 10 community-based organizations (CBOs) to implement and sustain Eban II; specifically, to partner with CBOs to expose providers to the intervention; facilitate its adoption, implementation and sustainment; and to evaluate processes and determinants of implementation, effectiveness, fidelity, and sustainment. The primary effectiveness aim is to evaluate the effect of Eban II on participant (n = 200 couples) outcomes, specifically incidents of protected sex and proportion of condom use. We will also determine the cost-effectiveness of implementation, as measured by implementation costs and potential cost savings. A mixed methods evaluation will examine implementation at the agency level; staff members from the CBOs will complete baseline measures of organizational context and climate, while key stakeholders will be interviewed periodically throughout implementation. Effectiveness of Eban II will be assessed using a randomized delayed enrollment (waitlist) control design to evaluate the impact of treatment on outcomes at posttest and three-month follow-up. Multi-level hierarchical modeling with a multi-level nested structure will be used to evaluate the effects of agency- and couples-level characteristics on couples-level outcomes (e.g., condom use). Discussion This study will produce important information regarding the value of the Eban II program and a theory-guided implementation process and tools designed for use in implementing Eban II and other evidence-based programs in demographically diverse, resource-constrained treatment settings. Trial registration NCT00644163 PMID:24950708

National Trainers’ Perspectives on Challenges to Implementation of an Empirically-Supported Mental Health Treatment

PubMed Central

Hanson, Rochelle F.; Gros, Kirstin Stauffacher; Davidson, Tatiana M.; Barr, Simone; Cohen, Judith; Deblinger, Esther; Mannarino, Anthony P.; Ruggiero, Kenneth J.

2013-01-01

This study examined perceived challenges to implementation of an empirically supported mental health treatment for youth (Trauma-Focused Cognitive Behavioral Therapy; TF-CBT) and explored the potential use of technology-based resources in treatment delivery. Thematic interviews were conducted with 19 approved national TF-CBT trainers to assess their perspectives about challenges to implementation of TF-CBT and to explore their perceptions about the potential value of innovative, technology-based solutions to enhance provider fidelity and improve quality of care. These data offer some important insights and implications for training in evidence-based treatments, provider fidelity and competence, and patient engagement, particularly for those interventions targeting trauma-related symptoms among youth. PMID:23605292

Evaluation of interactive computerized training to teach parents to implement photographic activity schedules with children with autism spectrum disorder.

PubMed

Gerencser, Kristina R; Higbee, Thomas S; Akers, Jessica S; Contreras, Bethany P

2017-07-01

Training parents of children with autism spectrum disorder can be a challenge due to limited resources, time, and money. Interactive computerized training (ICT)-a self-paced program that incorporates instructions, videos, and interactive questions-is one method professionals can use to disseminate trainings to broader populations. This study extends previous research on ICT by assessing the effect of ICT to teach three parents how to implement a photographic activity schedule using a systematic prompting procedure with their child. Following ICT, all parents increased their fidelity to implementation of an activity schedule during role-play sessions with an adult. Fidelity remained high during implementation with their child and maintained during a 2-week follow-up. © 2017 Society for the Experimental Analysis of Behavior.

CENTERING PREGANCY- AFRICA: A PILOT OF GROUP ANTENATAL CARE TO ADDRESS MILLENIUM DEVELOPMENT GOALS

PubMed Central

Patil, Crystal L.; Abrams, Elizabeth T.; Klima, Carrie; Kaponda, Chrissie P.N.; Leshabari, Sebalda C.; Vonderheid, Susan C.; Kamaga, Martha; Norr, Kathleen F.

2013-01-01

Background Severe health worker shortages and resource limitations negatively affect quality of antenatal care (ANC) throughout sub-Saharan Africa. Group ANC, specifically CenteringPregnancy (CP), may offer an innovative approach to enable midwives to offer higher quality ANC. Objective Our overarching goal was to prepare to conduct a clinical trial of CenteringPregnancy – Africa (CP-Africa) in Malawi and Tanzania. In Phase 1, our goal was to determine the acceptability of CP as model for ANC in both countries. In Phase 2, our objective was to develop CP-Africa session content consistent with the Essential Elements of CP model and with national standards in both Malawi and Tanzania. In Phase 3, our objective was to pilot CP-Africa in Malawi to determine whether sessions could be conducted with fidelity to the Centering process. Setting Phase 1 and 2 took place in Malawi and Tanzania. Phase 3, the piloting of two sessions of CP-Africa, occurred at two sites in Malawi: a district hospital and a small clinic. Design We used an Action Research approach to promote partnerships among university researchers, the Centering Healthcare Institute, healthcare administrators, health professionals and women attending ANC to develop CP-Africa session content and pilot this model of group ANC. Participants For Phases 1 and 2, members of the Ministries of Health, health professionals and pregnant women in Malawi and Tanzania were introduced to and interviewed about CP. In Phase 2, we finalized CP-Africa content and trained thirteen health professionals in the Centering Healthcare model. In Phase 3, we conducted a small pilot with 24 pregnant women (12 at each site). Measurements and Findings Participants enthusiastically embraced CP-Africa as an acceptable model of ANC healthcare delivery. The CP-Africa content met both CP and national standards. The pilot established that the CP model could be implemented with process fidelity to the 13 Essential Elements. Several implementation challenges and strategies to address these challenges were identified. Key Conclusions Preliminary data suggest that CP-Africa is feasible in resource-constrained, low-literacy, high-HIV settings in sub-Saharan Africa. By improving the quality of ANC delivery, midwives have an opportunity to make a contribution towards Millennium Development Goals (MDG) targeting improvements in child, maternal and HIV-related health outcomes (MDGs 4, 5 and 6). A clinical trial is needed to establish efficacy. Implications for Practice CP-Africa also has the potential to reduce job-related stress and enhance job satisfaction for midwives in low income countries. If CP can be transferred with fidelity to process in sub-Saharan Africa and retain similar results to those reported in clinical trials, it has the potential to benefit pregnant women and their infants and could make a positive contribution to MGDs 4, 5 and .6. PMID:23871278

Formative evaluation on a physical activity health promotion program for the group home setting.

PubMed

Dixon-Ibarra, Alicia; Driver, Simon; VanVolkenburg, Haley; Humphries, Kathleen

2017-02-01

Physical inactivity and high rates of chronic conditions is a public health concern for adults with intellectual disability. Few health promotion programs target the group home setting which is the pre-dominant form of residential accommodation for persons with intellectual disability. A process evaluation of a physical activity health promotion program, Menu-Choice, was conducted with five group home sites for adults with intellectual and developmental disabilities. Menu-Choice assists group home staff in including physical activity goals within resident schedules. The physical activity program was designed based on theoretical frameworks, community-based participatory approaches, and established health promotion guidelines for adults with disabilities. Fourteen program coordinators (age M 39; 77% females), 22 staff (age M 39; 82% females), and 18 residents (age M 59; 72% females; 56% ambulatory) participated. Results from the fidelity survey and program completion highlight potential challenges with implementation. Findings will assist with the refinement of the program for continued implementation trials in the group home community. Published by Elsevier Ltd.

A qualitative study of staff's perspectives on implementing an after school program promoting youth physical activity.

PubMed

Zarrett, Nicole; Skiles, Brittany; Wilson, Dawn K; McClintock, Lauren

2012-08-01

Minimal effects found across youth physical activity (PA) interventions, and increased attention to circumstances that impede adequate delivery of program components, has highlighted the importance of learning from staff what is needed to foster staff comprehension and engagement for developing, adopting, and successfully implementing PA-based youth interventions. The purpose of this study is to address this knowledge gap by conducting a qualitative assessment of school staff perspectives on the positive aspects and challenges of implementing the 17-week ACT program, an after-school intervention that integrated motivational and behavioral components to promote PA in underserved adolescents. Interviews were conducted with one school staff member from each participating school for all four trial cohorts (N=12). Transcripts were coded by independent coders (r=.84) and content analyses of themes was performed using QSR NVivo. Themes were organized into five meta-themes: (1) Logistics; (2) Essential Elements; (3) Staff and Child Challenges; (4) Staff Comprehension, Value, and Enjoyment; (5) Spill-Over Effects. Findings indicate that staff can be successful at understanding, valuing, and reaching fidelity in implementing climate-based mediation components. The insight gained from this study lays the foundation for understanding the components needed for establishing well-implemented, effective, and generalizable interventions for increasing youth PA. Copyright © 2012 Elsevier Ltd. All rights reserved.

Engagement matters: lessons from assessing classroom implementation of steps to respect: a bullying prevention program over a one-year period.

PubMed

Low, Sabina; Van Ryzin, Mark J; Brown, Eric C; Smith, Brian H; Haggerty, Kevin P

2014-04-01

Steps to Respect: A Bullying Prevention Program (STR) relies on a social-ecological model of prevention to increase school staff awareness and responsiveness, foster socially responsible beliefs among students, and teach social-emotional skills to students to reduce bullying behavior. As part of a school-randomized controlled trial of STR, we examined predictors and outcomes associated with classroom curriculum implementation in intervention schools. Data on classroom implementation (adherence and engagement) were collected from a sample of teachers using a weekly on-line Teacher Implementation Checklist system. Pre-post data related to school bullying-related outcomes were collected from 1,424 students and archival school demographic data were obtained from the National Center for Education Statistics. Results of multilevel analyses indicated that higher levels of program engagement were influenced by school-level percentage of students receiving free/reduced lunch, as well as classroom-level climate indicators. Results also suggest that higher levels of program engagement were related to lower levels of school bullying problems, enhanced school climate and attitudes less supportive of bullying. Predictors and outcomes related to program fidelity (i.e., adherence) were largely nonsignificant. Results suggest that student engagement is a key element of program impact, though implementation is influenced by both school-level demographics and classroom contexts.

Testing the 4Rs and 2Ss Multiple Family Group intervention: study protocol for a randomized controlled trial.

PubMed

Acri, Mary; Hamovitch, Emily; Mini, Maria; Garay, Elene; Connolly, Claire; McKay, Mary

2017-12-04

Oppositional defiant disorder (ODD) is a major mental health concern and highly prevalent among children living in poverty-impacted communities. Despite that treatments for ODD are among the most effective, few children living in poverty receive these services due to substantial barriers to access, as well as difficulties in the uptake and sustained adoption of evidence-based practices (EBPs) in community settings. The purpose of this study is to examine implementation processes that impact uptake of an evidence-based practice for childhood ODD, and the impact of a Clinic Implementation Team (CIT)-driven structured adaptation to enhance its fit within the public mental health clinic setting. This study, a Hybrid Type II effectiveness-implementation research trial, blends clinical effectiveness and implementation research methods to examine the impact of the 4Rs and 2Ss Multiple Family Group (MFG) intervention, family level mediators of child outcomes, clinic/provider-level mediators of implementation, and the impact of CITs on uptake and long-term utilization of this model. All New York City public outpatient mental health clinics have been invited to participate. A sampling procedure that included randomization at the agency level and a sub-study to examine the impact of clinic choice upon outcomes yielded a distribution of clinics across three study conditions. Quantitative data measuring child outcomes, organizational factors and implementation fidelity will be collected from caregivers and providers at baseline, 8, and 16 weeks from baseline, and 6 months from treatment completion. The expected participation is 134 clinics, 268 providers, and 2688 caregiver/child dyads. We will use mediation analysis with a multi-level Structural Equation Modeling (SEM) (MSEM including family level variables, provider variables, and clinic variables), as well as mediation tests to examine study hypotheses. The aim of the study is to generate knowledge about effectiveness and mediating factors in the treatment of ODDs in children in the context of family functioning, and to propose an innovative approach to the adaptation and implementation of new treatment interventions within clinic settings. The proposed CIT adaptation and implementation model has the potential to enhance implementation and sustainability, and ultimately increase the extent to which effective interventions are available and can impact children and families in need of services for serious behavior problems. ClinicalTrials.gov, ID: NCT02715414 . Registered on 3 March 2016.

Fidelity to the housing first model and effectiveness of permanent supported housing programs in California.

PubMed

Gilmer, Todd P; Stefancic, Ana; Katz, Marian L; Sklar, Marisa; Tsemberis, Sam; Palinkas, Lawrence A

2014-11-01

Permanent supported housing programs are being implemented throughout the United States. This study examined the relationship between fidelity to the Housing First model and residential outcomes among clients of full service partnerships (FSPs) in California. This study had a mixed-methods design. Quantitative administrative and survey data were used to describe FSP practices and to examine the association between fidelity to Housing First and residential outcomes in the year before and after enrollment of 6,584 FSP clients in 86 programs. Focus groups at 20 FSPs provided qualitative data to enhance the understanding of these findings with actual accounts of housing-related experiences in high- and low-fidelity programs. Prior to enrollment, the mean days of homelessness were greater at high- versus low-fidelity (101 versus 46 days) FSPs. After adjustment for individual characteristics, the analysis found that days spent homeless after enrollment declined by 87 at high-fidelity programs and by 34 at low-fidelity programs. After adjustment for days spent homeless before enrollment, days spent homeless after enrollment declined by 63 at high-fidelity programs and by 53 at low-fidelity programs. After enrollment, clients at high-fidelity programs spent more than 60 additional days in apartments than clients at low-facility programs. Differences were found between high- and low-fidelity FSPs in client choice in housing and how much clients' goals were considered in housing placement. Programs with greater fidelity to the Housing First model enrolled clients with longer histories of homelessness and placed most of them in apartments.

Evaluating the sub-national fidelity of national Initiatives in decentralized health systems: Integrated Primary Health Care Governance in Nigeria.

PubMed

Eboreime, Ejemai Amaize; Abimbola, Seye; Obi, Felix Abrahams; Ebirim, Obinna; Olubajo, Olalekan; Eyles, John; Nxumalo, Nonhlanhla Lynette; Mambulu, Faith Nankasa

2017-03-21

Policy making, translation and implementation in politically and administratively decentralized systems can be challenging. Beyond the mere sub-national acceptance of national initiatives, adherence to policy implementation processes is often poor, particularly in low and middle-income countries. In this study, we explore the implementation fidelity of integrated PHC governance policy in Nigeria's decentralized governance system and its implications on closing implementation gaps with respect to other top-down health policies and initiatives. Having engaged policy makers, we identified 9 core components of the policy (Governance, Legislation, Minimum Service Package, Repositioning, Systems Development, Operational Guidelines, Human Resources, Funding Structure, and Office Establishment). We evaluated the level and pattern of implementation at state level as compared to the national guidelines using a scorecard approach. Contrary to national government's assessment of level of compliance, we found that sub-national governments exercised significant discretion with respect to the implementation of core components of the policy. Whereas 35 and 32% of states fully met national criteria for the structural domains of "Office Establishment" and Legislation" respectively, no state was fully compliant to "Human Resource Management" and "Funding" requirements, which are more indicative of functionality. The pattern of implementation suggests that, rather than implementing to improve outcomes, state governments may be more interested in executing low hanging fruits in order to access national incentives. Our study highlights the importance of evaluating implementation fidelity in providing evidence of implementation gaps towards improving policy execution, particularly in decentralized health systems. This approach will help national policy makers identify more effective ways of supporting lower tiers of governance towards improvement of health systems and outcomes.

Treatment fidelity instrument to measure a brief opportunistic intervention for prenatal substance use.

PubMed

Torrey, Antonia Rae

2012-01-01

To develop and psychometrically evaluate an instrument designed to measure the treatment fidelity associated with implementation of the I Am Concerned (IAC) brief opportunistic intervention by frontline, prenatal, primary care staff. A methodologic approach framed development of the IAC Treatment Fidelity Instrument in a six-phase protocol. A simulated prenatal clinic with standardized patients portraying substance-using pregnant women. Prenatal, primary care, frontline staff (N = 6), experienced in IAC implementation. Following development of the IAC treatment fidelity instrument, independent raters used the instrument to evaluate audio recordings (N = 49) of frontline staff implementing the IAC brief opportunistic intervention with standardized patients representing substance-using pregnant women. Psychometric analysis provided evidence of content validity. Intraclass correlation coefficients calculated for inter-rater reliability were satisfactory for subscales (0.64) and (0.62) and ranged from -0.07 to 0.81 for individual items. Internal consistency alpha coefficients were satisfactory for the total scale (0.72) and lower than acceptable for adherence (0.54) and competence (0.56) subscales. Overall high rater percentage agreement and negatively skewed ratings distribution indicated reliability results were paradoxically low due to the base rate problem. Results support revision and ongoing testing of the IAC treatment fidelity instrument. The impact on reliability statistics exerted by this study's skewed data distribution has implications for nursing research as low variance can be anticipated when measuring care provided to homogenous patient populations. It is important to recognize the resulting influence on inter-rater agreement to avoid making inaccurate interpretations about the reliability of an instrument's measurements. © 2012 AWHONN, the Association of Women's Health, Obstetric and Neonatal Nurses.

Increasing In-Service Teacher Implementation of Classroom Management Practices through Consultation, Implementation Planning, and Participant Modeling

ERIC Educational Resources Information Center

Hagermoser Sanetti, Lisa M.; Williamson, Kathleen M.; Long, Anna C. J.; Kratochwill, Thomas R.

2018-01-01

Numerous evidence-based classroom management strategies to prevent and respond to problem behavior have been identified, but research consistently indicates teachers rarely implement them with sufficient implementation fidelity. The purpose of this study was to evaluate the effectiveness of implementation planning, a strategy involving logistical…

The OPERA trial: a protocol for the process evaluation of a randomised trial of an exercise intervention for older people in residential and nursing accommodation

PubMed Central

2011-01-01

Background The OPERA trial is large cluster randomised trial testing a physical activity intervention to address depression amongst people living in nursing and residential homes for older people. A process evaluation was commissioned alongside the trial and we report the protocol for this process evaluation. Challenges included the cognitive and physical ability of the participants, the need to respect the privacy of all home residents, including study non-participants, and the physical structure of the homes. Evaluation activity had to be organised around the structured timetable of homes, leaving limited opportunities for data collection. The aims of this process evaluation are to provide findings that will assist in the interpretation of the clinical trial results, and to inform potential implementation of the physical activity intervention on a wider scale. Methods/design Quantitative data on recruitment of homes and individuals is being collected. For homes in the intervention arm, data on dose and fidelity of the intervention delivered; including individual rates of participation in exercise classes are collected. In the control homes, uptake and delivery of depression awareness training is monitored. These data will be combined with qualitative data from an in-depth study of a purposive sample of eight homes (six intervention and two control). Discussion Although process evaluations are increasingly funded alongside trials, it is still rare to see the findings published, and even rarer to see the protocol for such an evaluation published. Process evaluations have the potential to assist in interpreting and understanding trial results as well as informing future roll-outs of interventions. If such evaluations are funded they should also be reported and reviewed in a similar way to the trial outcome evaluation. Trial Registration ISRCTN No: ISRCTN43769277 PMID:21288341

A participatory and capacity-building approach to healthy eating and physical activity – SCIP-school: a 2-year controlled trial

PubMed Central

2012-01-01

Background Schools can be effective settings for improving eating habits and physical activity, whereas it is more difficult to prevent obesity. A key challenge is the “implementation gap”. Trade-off must be made between expert-driven programmes on the one hand and contextual relevance, flexibility, participation and capacity building on the other. The aim of the Stockholm County Implementation Programme was to improve eating habits, physical activity, self-esteem, and promote a healthy body weight in children aged 6–16 years. We describe the programme, intervention fidelity, impacts and outcomes after two years of intervention. Methods Nine out of 18 schools in a middle-class municipality in Sweden agreed to participate whereas the other nine schools served as the comparison group (quasi-experimental study). Tailored action plans were developed by school health teams on the basis of a self-assessment questionnaire called KEY assessing strengths and weaknesses of each school’s health practices and environments. Process evaluation was carried out by the research staff. Impacts at school level were assessed yearly by the KEY. Outcome measures at student level were anthropometry (measured), and health behaviours assessed by a questionnaire, at baseline and after 2 years. All children in grade 2, 4 and 7 were invited to participate (n=1359) of which 59.8% consented. The effect of the intervention on health behaviours, self-esteem, weight status and BMIsds was evaluated by unilevel and multilevel regression analysis adjusted for gender and baseline values. Results Programme fidelity was high demonstrating feasibility, but fidelity to school action plans was only 48% after two years. Positive and significant (p<.05) impacts were noted in school health practices and environments after 2 years. At student level no significant intervention effects were seen for the main outcomes. Conclusions School staff has the capacity to create their own solutions and make changes at school level on the basis of self-assessment and facilitation by external agents. However these changes were challenging to sustain over time and had little impact on student behaviours or weight status. Better student outcomes could probably be attained by a more focused and evidence-based approach with stepwise implementation of action plans. PMID:23245473

Workflow standardization of a novel team care model to improve chronic care: a quasi-experimental study.

PubMed

Panattoni, Laura; Hurlimann, Lily; Wilson, Caroline; Durbin, Meg; Tai-Seale, Ming

2017-04-19

Team-based chronic care models have not been widely adopted in community settings, partly due to their varying effectiveness in randomized control trials, implementation challenges, and concerns about physician acceptance. The Palo Alto Medical Foundation designed and implemented "Champion," a novel team-based model that includes new standard work (e.g. proactive patient outreach, pre-visit schedule grooming, depression screening, care planning, health coaching) to support patients' self-management of hypertension and diabetes. We investigated whether Champion improved clinical outcomes. We conducted a quasi-experimental study comparing the Champion clinic-level intervention (n = 38 physicians) with a usual care clinic (n = 37 physicians) in Northern California. The primary outcomes, blood pressure and glycohemoglobin (A1c), were analyzed using a piecewise linear growth curve model for patients exposed to a Champion physician visit (n = 3156) or usual care visit (n = 8034) in the two years prior and one year post implementation. Secondary outcomes were provider experience, compared at baseline and 12 months in both the intervention and usual care clinics using multi-level ordered logistic modeling, and electronic health record based fidelity measures. Compared to usual care, in the first 6 months after a Champion physician visit, diabetes patients aged 18-75 experienced an additional -1.13 mm Hg (95% CI: -2.23 to -0.04) decline in diastolic blood pressure and -0.47 (95% CI: -0.61 to -0.33) decline in A1c. There were no additional improvements in blood pressure or A1c 6 to 12 months post physician visit. At 12 months, Champion physicians reported improved experience with managing chronic care patients in 6 of 7 survey items (p < 0.05), but compared to usual, this difference was only statistically significant for one item (p < 0.05). Fidelity to standard work was uneven; depression screening was the most commonly documented element (85% of patients), while care plans were the least (30.8% of patients). Champion standard work improved glycemic control over the first 6 months and physicians' experience with managing chronic care; changes in blood pressure were not clinically meaningful. Our results suggest the need to understand the relationship between the intervention, the contextual features of implementation, and fidelity to further improve chronic disease outcomes. This study was retrospectively registered with the ISRCTN Registry on March 15, 2017 (ISRCTN11341906).

Using mixed methods to assess fidelity of delivery and its influencing factors in a complex self-management intervention for people with osteoarthritis and low back pain.

PubMed

Toomey, Elaine; Matthews, James; Hurley, Deirdre A

2017-08-04

Despite an increasing awareness of the importance of fidelity of delivery within complex behaviour change interventions, it is often poorly assessed. This mixed methods study aimed to establish the fidelity of delivery of a complex self-management intervention and explore the reasons for these findings using a convergent/triangulation design. Feasibility trial of the Self-management of Osteoarthritis and Low back pain through Activity and Skills (SOLAS) intervention (ISRCTN49875385), delivered in primary care physiotherapy. 60 SOLAS sessions were delivered across seven sites by nine physiotherapists. Fidelity of delivery of prespecified intervention components was evaluated using (1) audio-recordings (n=60), direct observations (n=24) and self-report checklists (n=60) and (2) individual interviews with physiotherapists (n=9). Quantitatively, fidelity scores were calculated using percentage means and SD of components delivered. Associations between fidelity scores and physiotherapist variables were analysed using Spearman's correlations. Interviews were analysed using thematic analysis to explore potential reasons for fidelity scores. Integration of quantitative and qualitative data occurred at an interpretation level using triangulation. Quantitatively, fidelity scores were high for all assessment methods; with self-report (92.7%) consistently higher than direct observations (82.7%) or audio-recordings (81.7%). There was significant variation between physiotherapists' individual scores (69.8% - 100%). Both qualitative and quantitative data (from physiotherapist variables) found that physiotherapists' knowledge (Spearman's association at p=0.003) and previous experience (p=0.008) were factors that influenced their fidelity. The qualitative data also postulated participant-level (eg, individual needs) and programme-level factors (eg, resources) as additional elements that influenced fidelity. The intervention was delivered with high fidelity. This study contributes to the limited evidence regarding fidelity assessment methods within complex behaviour change interventions. The findings suggest a combination of quantitative methods is suitable for the assessment of fidelity of delivery. A mixed methods approach provided a more insightful understanding of fidelity and its influencing factors. ISRCTN49875385; Pre-results. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Behavioral health coaching for rural veterans with diabetes and depression: a patient randomized effectiveness implementation trial.

PubMed

Cully, Jeffrey A; Breland, Jessica Y; Robertson, Suzanne; Utech, Anne E; Hundt, Natalie; Kunik, Mark E; Petersen, Nancy J; Masozera, Nicholas; Rao, Radha; Naik, Aanand D

2014-04-28

Depression and diabetes cause significant burden for patients and the healthcare system and, when co-occurring, result in poorer self-care behaviors and worse glycemic control than for either condition alone. However, the clinical management of these comorbid conditions is complicated by a host of patient, provider, and system-level barriers that are especially problematic for patients in rural locations. Patient-centered medical homes provide an opportunity to integrate mental and physical health care to address the multifaceted needs of complex comorbid conditions. Presently, there is a need to not only develop robust clinical interventions for complex medically ill patients but also to find feasible ways to embed these interventions into the frontlines of existing primary care practices. This randomized controlled trial uses a hybrid effectiveness-implementation design to evaluate the Healthy Outcomes through Patient Empowerment (HOPE) intervention, which seeks to simultaneously address diabetes and depression for rural veterans in Southeast Texas. A total of 242 Veterans with uncontrolled diabetes and comorbid symptoms of depression will be recruited and randomized to either the HOPE intervention or to a usual-care arm. Participants will be evaluated on a host of diabetes and depression-related measures at baseline and 6- and 12-month follow-up. The trial has two primary goals: 1) to examine the effectiveness of the intervention on both physical (diabetes) and emotional health (depression) outcomes and 2) to simultaneously pilot test a multifaceted implementation strategy designed to increase fidelity and utilization of the intervention by coaches interfacing within the primary care setting. This ongoing blended effectiveness-implementation design holds the potential to advance the science and practice of caring for complex medically ill patients within the constraints of a busy patient-centered medical home. Behavioral Activation Therapy for Rural Veterans with Diabetes and Depression: NCT01572389.

A Transfer of Training Study of Control Loader Dynamics

NASA Technical Reports Server (NTRS)

Cardullo, Frank M.; Stanco, Anthony A.; Kelly, Lon C.; Houck, Jacob A.; Grube, Richard C.

2011-01-01

The control inceptor used in a simulated vehicle is an important part in maintaining the fidelity of a simulation. The force feedback provided by the control inceptor gives the operator important cues to maintain adequate performance. The dynamics of a control inceptor are typically based on a second order spring mass damper system with damping, force gradient, breakout force, and natural frequency parameters. Changing these parameters can have a great effect on pilot or driver control of the vehicle. The neuromuscular system has a very important role in manipulating the control inceptor within a vehicle. Many studies by McRuer, Aponso, and Hess have dealt with modeling the neuromuscular system and quantifying the effects of a high fidelity control loader as compared to a low fidelity control loader. Humans are adaptive in nature and their control behavior changes based on different control loader dynamics. Humans will change their control behavior to maintain tracking bandwidth and minimize tracking error. This paper reports on a quasi-transfer of training experiment which was performed at the NASA Langley Research Center. The quasi transfer of training study used a high fidelity control loader and a low fidelity control loader. Subjects trained in both simulations and then were transferred to the high fidelity control loader simulation. The parameters for the high fidelity control loader were determined from the literature. The low fidelity control loader parameters were found through testing of a simple computer joystick. A disturbance compensatory task is employed. The compensatory task involves implementing a simple horizon out the window display. A disturbance consisting of a sum of sines is used. The task consists of the subject compensating for the disturbance on the roll angle of the aircraft. The vehicle dynamics are represented as 1/s and 1/s2. The subject will try to maintain level flight throughout the experiment. The subjects consist of non-pilots to remove any effects of pilot experience. First, this paper discusses the implementation of the disturbance compensation task. Second, the high and low fidelity parameters used within the experiment are presented. Finally, an explanation of results from the experiments is presented.

The influence of fidelity of implementation on teacher-student interaction quality in the context of a randomized controlled trial of the Responsive Classroom approach.

PubMed

Abry, Tashia; Rimm-Kaufman, Sara E; Larsen, Ross A; Brewer, Alexis J

2013-08-01

This study examined the direct and indirect effects between training in the Responsive Classroom® (RC) approach, teachers' uptake of RC practices, and teacher-student interaction quality, using a structural equation modeling framework. A total of 24 schools were randomly assigned to experimental or control conditions. Third- and fourth-grade teachers in treatment schools (n=132) received training in the RC approach, whereas teachers in control schools (n=107) continued "business as usual." Observers rated teachers' fidelity of implementation (FOI) of RC practices 5 times throughout the year using the Classroom Practices Observation Measure. In addition, teachers completed self-report measures of FOI, the Classroom Practices Teacher Survey and Classroom Practices Frequency Survey, at the end of the school year. Teacher-student interactions were rated during classroom observations using the Classroom Assessment Scoring System. Controlling for teachers' grade level and teacher-student interaction quality at pretest, RC training was expected to predict posttest teacher-student interaction quality directly and indirectly through FOI. Results supported only a significant indirect effect, β=0.85, p=.002. Specifically, RC teachers had higher levels of FOI of RC practices, β=1.62, p<.001, R2=.69. In turn, FOI related to greater improvement in teacher-student interaction quality, β=0.52, p=.001, R2=.32. Discussion highlights factors contributing to variability in FOI and school administrators roles in supporting FOI. Copyright © 2013 Society for the Study of School Psychology. Published by Elsevier Ltd. All rights reserved.

Electro-Optic Frequency Beam Splitters and Tritters for High-Fidelity Photonic Quantum Information Processing

DOE PAGES

Lu, Hsuan-Hao; Lukens, Joseph M.; Peters, Nicholas A.; ...

2018-01-18

In this paper, we report the experimental realization of high-fidelity photonic quantum gates for frequency-encoded qubits and qutrits based on electro-optic modulation and Fourier-transform pulse shaping. Our frequency version of the Hadamard gate offers near-unity fidelity (0.99998±0.00003), requires only a single microwave drive tone for near-ideal performance, functions across the entire C band (1530–1570 nm), and can operate concurrently on multiple qubits spaced as tightly as four frequency modes apart, with no observable degradation in the fidelity. For qutrits, we implement a 3×3 extension of the Hadamard gate: the balanced tritter. This tritter—the first ever demonstrated for frequency modes—attains fidelitymore » 0.9989±0.0004. Finally, these gates represent important building blocks toward scalable, high-fidelity quantum information processing based on frequency encoding.« less

Electro-Optic Frequency Beam Splitters and Tritters for High-Fidelity Photonic Quantum Information Processing

DOE Office of Scientific and Technical Information (OSTI.GOV)

Lu, Hsuan-Hao; Lukens, Joseph M.; Peters, Nicholas A.

In this paper, we report the experimental realization of high-fidelity photonic quantum gates for frequency-encoded qubits and qutrits based on electro-optic modulation and Fourier-transform pulse shaping. Our frequency version of the Hadamard gate offers near-unity fidelity (0.99998±0.00003), requires only a single microwave drive tone for near-ideal performance, functions across the entire C band (1530–1570 nm), and can operate concurrently on multiple qubits spaced as tightly as four frequency modes apart, with no observable degradation in the fidelity. For qutrits, we implement a 3×3 extension of the Hadamard gate: the balanced tritter. This tritter—the first ever demonstrated for frequency modes—attains fidelitymore » 0.9989±0.0004. Finally, these gates represent important building blocks toward scalable, high-fidelity quantum information processing based on frequency encoding.« less

Comment on ``Teleportation of two-mode squeezed states''

NASA Astrophysics Data System (ADS)

He, Guangqiang; Zhang, Jingtao

2011-10-01

We investigate the teleportation scheme of two-mode squeezed states proposed by Adhikari [S. Adhikari , Phys. Rev. APLRAAN1050-294710.1103/PhysRevA.77.012337 77, 012337 (2008)]. It uses four-mode entangled states to teleport two-mode squeezed states. The fidelity between the original two-mode squeezed states and teleported ones is calculated. The maximal fidelity value of Adhikari's protocol is 0.38, which is incompatible with the fidelity definition with the maximal value 1. In our opinion, one reason is that they calculate the fidelity for multimodes Gaussian states using the fidelity formula for single-mode ones. Another reason is that the covariance matrix of output states should be what is obtained after applying the linear unitary Bogoliubov operations (two cascaded Fourier transformations) on the covariance matrix given in Eq. (12) in their paper. These two reasons result in the incomparable results. In addition, Adhikari's protocol can be simplified to be easily implemented.

Comment on ''Teleportation of two-mode squeezed states''

DOE Office of Scientific and Technical Information (OSTI.GOV)

He Guangqiang; Zhang Jingtao

2011-10-15

We investigate the teleportation scheme of two-mode squeezed states proposed by Adhikari et al.[S. Adhikari et al., Phys. Rev. A 77, 012337 (2008)]. It uses four-mode entangled states to teleport two-mode squeezed states. The fidelity between the original two-mode squeezed states and teleported ones is calculated. The maximal fidelity value of Adhikari's protocol is 0.38, which is incompatible with the fidelity definition with the maximal value 1. In our opinion, one reason is that they calculate the fidelity for multimodes Gaussian states using the fidelity formula for single-mode ones. Another reason is that the covariance matrix of output states shouldmore » be what is obtained after applying the linear unitary Bogoliubov operations (two cascaded Fourier transformations) on the covariance matrix given in Eq. (12) in their paper. These two reasons result in the incomparable results. In addition, Adhikari's protocol can be simplified to be easily implemented.« less

High-Fidelity Single-Shot Toffoli Gate via Quantum Control.

PubMed

Zahedinejad, Ehsan; Ghosh, Joydip; Sanders, Barry C

2015-05-22

A single-shot Toffoli, or controlled-controlled-not, gate is desirable for classical and quantum information processing. The Toffoli gate alone is universal for reversible computing and, accompanied by the Hadamard gate, forms a universal gate set for quantum computing. The Toffoli gate is also a key ingredient for (nontopological) quantum error correction. Currently Toffoli gates are achieved by decomposing into sequentially implemented single- and two-qubit gates, which require much longer times and yields lower overall fidelities compared to a single-shot implementation. We develop a quantum-control procedure to construct a single-shot Toffoli gate for three nearest-neighbor-coupled superconducting transmon systems such that the fidelity is 99.9% and is as fast as an entangling two-qubit gate under the same realistic conditions. The gate is achieved by a nongreedy quantum control procedure using our enhanced version of the differential evolution algorithm.

A measure to evaluate classroom teaching practices in nursing.

PubMed

Herinckx, Heidi; Munkvold, Julia Paschall; Winter, Elisabeth; Tanner, Christine A

2014-01-01

The Oregon Consortium for Nursing Education (OCNE) Classroom Teaching Fidelity Scale was created to measure the implementation of the OCNE curriculum and its related pedagogy. OCNE is a partnership of eight community colleges and the five-campus state-supported university. OCNE developed a shared competency-based curriculum and pedagogical practices. An essential part of the OCNE evaluation was to measure the extent the curriculum and pedagogical model were implemented on each partner campus. The scale was developed using a multistep methodology, including review of the literature and OCNE guidelines and materials, frequent consultation with local and national advisory boards, and multiple observations of OCNE classrooms over a two-year period. Fidelity scores are reported for 10 OCNE colleges observed in 2009. CONCLUSlON: The creation and use of this fidelity scale and similar measures may contribute to the emerging science of nursing education by more clearly documenting educational reform efforts..

A methodology for the assessment of manned flight simulator fidelity

NASA Technical Reports Server (NTRS)

Hess, Ronald A.; Malsbury, Terry N.

1989-01-01

A relatively simple analytical methodology for assessing the fidelity of manned flight simulators for specific vehicles and tasks is offered. The methodology is based upon an application of a structural model of the human pilot, including motion cue effects. In particular, predicted pilot/vehicle dynamic characteristics are obtained with and without simulator limitations. A procedure for selecting model parameters can be implemented, given a probable pilot control strategy. In analyzing a pair of piloting tasks for which flight and simulation data are available, the methodology correctly predicted the existence of simulator fidelity problems. The methodology permitted the analytical evaluation of a change in simulator characteristics and indicated that a major source of the fidelity problems was a visual time delay in the simulation.

Manualization of Occupational Therapy Interventions: Illustrations from the Pressure Ulcer Prevention Research Program

PubMed Central

Blanche, Erna Imperatore; Fogelberg, Donald; Diaz, Jesus; Carlson, Mike; Clark, Florence

2011-01-01

The manualization of a complex occupational therapy intervention is a crucial step in ensuring treatment fidelity for both clinical application and research purposes. Towards this latter end, intervention manuals are essential for assuring trustworthiness and replicability of randomized controlled trials (RCT’s) that aim to provide evidence of the effectiveness of occupational therapy. In this paper, literature on the process of intervention manualization is reviewed. The prescribed steps are then illustrated through our experience in implementing the University of Southern California/Rancho Los Amigos National Rehabilitation Center’s collaborative Pressure Ulcer Prevention Project (PUPP). In this research program, qualitative research provided the initial foundation for manualization of a multifaceted occupational therapy intervention designed to reduce incidence of medically serious pressure ulcers in people with SCI. PMID:22214116

Implementing Curriculum-Embedded Formative Assessment in Primary School Science Classrooms

ERIC Educational Resources Information Center

Hondrich, Annika Lena; Hertel, Silke; Adl-Amini, Katja; Klieme, Eckhard

2016-01-01

The implementation of formative assessment strategies is challenging for teachers. We evaluated teachers' implementation fidelity of a curriculum-embedded formative assessment programme for primary school science education, investigating both material-supported, direct application and subsequent transfer. Furthermore, the relationship between…

Universal non-adiabatic holonomic quantum computation in decoherence-free subspaces with quantum dots inside a cavity

NASA Astrophysics Data System (ADS)

Liu, Jun; Dong, Ping; Zhou, Jian; Cao, Zhuo-Liang

2017-05-01

A scheme for implementing the non-adiabatic holonomic quantum computation in decoherence-free subspaces is proposed with the interactions between a microcavity and quantum dots. A universal set of quantum gates can be constructed on the encoded logical qubits with high fidelities. The current scheme can suppress both local and collective noises, which is very important for achieving universal quantum computation. Discussions about the gate fidelities with the experimental parameters show that our schemes can be implemented in current experimental technology. Therefore, our scenario offers a method for universal and robust solid-state quantum computation.

Cluster-state quantum computing enhanced by high-fidelity generalized measurements.

PubMed

Biggerstaff, D N; Kaltenbaek, R; Hamel, D R; Weihs, G; Rudolph, T; Resch, K J

2009-12-11

We introduce and implement a technique to extend the quantum computational power of cluster states by replacing some projective measurements with generalized quantum measurements (POVMs). As an experimental demonstration we fully realize an arbitrary three-qubit cluster computation by implementing a tunable linear-optical POVM, as well as fast active feedforward, on a two-qubit photonic cluster state. Over 206 different computations, the average output fidelity is 0.9832+/-0.0002; furthermore the error contribution from our POVM device and feedforward is only of O(10(-3)), less than some recent thresholds for fault-tolerant cluster computing.

Study protocol: a pragmatic randomised controlled trial of a 12-week physical activity and nutritional education program for overweight Aboriginal and Torres Strait Islander women.

PubMed

Canuto, Karla J; McDermott, Robyn A; Cargo, Margaret; Esterman, Adrian J

2011-08-19

Aboriginal and Torres Strait Islander women have a higher prevalence and incidence of obesity and type 2 diabetes than non-Indigenous Australian women. Physical inactivity is a key modifiable risk factor for obesity and evidence shows that even modest reductions in waist circumference (WC) have significant health benefits. Trialing physical activity programs in difficult-to-reach high risk groups, especially urban Indigenous Australians poses distinct implementation challenges. The trial objective is to evaluate the effectiveness of a structured 12-week physical activity group program with nutritional advice. The design is a pragmatic randomised controlled trial. This study protocol describes the implementation and evaluation of the program. Participants are randomised into either an intervention or waitlisted group. The waitlisted group have a 12 month waiting period before commencing the 12-week program. Participant data is collected at baseline, 12, 24 and 52 weeks. Participants are Aboriginal and Torres Strait Islander women, aged 18-64 years with a waist circumference greater than 80 centimetres residing in Adelaide. The primary outcome measure is WC change immediately post program from baseline. Secondary outcomes include short term and long term changes in WC, weight, blood pressure, fasting blood glucose, insulin, insulin resistance (calculated HOMA), haemoglobin A1C (HbA1C), triglycerides and C-reactive protein (CRP). Behavioural and psychosocial surveys are administered to assess physical activity, dietary intake and the participant's motivation, self-efficacy and perceived social support for physical activity. Qualitative interviews focusing on participants' motivation, enablers and barriers to healthy eating and physical activity will be undertaken. Implementation fidelity and participation are also assessed. The Aboriginal and Torres Strait Islander Women's Fitness Program (WFP) is designed to provide a rigorous physiological and client-based evaluation of a structured 12-week program aimed to increase metabolic fitness and reduce WC in this high risk population. Evaluation results aim to provide the support necessary to design programs that are accessible, affordable and effective at reducing WC, while also improving the metabolic profile of overweight Aboriginal and Torres Strait Islander women. Australian New Zealand Clinical Trials Registry ACTRN12610000224022.

High fidelity quantum gates with vibrational qubits.

PubMed

Berrios, Eduardo; Gruebele, Martin; Shyshlov, Dmytro; Wang, Lei; Babikov, Dmitri

2012-11-26

Physical implementation of quantum gates acting on qubits does not achieve a perfect fidelity of 1. The actual output qubit may not match the targeted output of the desired gate. According to theoretical estimates, intrinsic gate fidelities >99.99% are necessary so that error correction codes can be used to achieve perfect fidelity. Here we test what fidelity can be accomplished for a CNOT gate executed by a shaped ultrafast laser pulse interacting with vibrational states of the molecule SCCl(2). This molecule has been used as a test system for low-fidelity calculations before. To make our test more stringent, we include vibrational levels that do not encode the desired qubits but are close enough in energy to interfere with population transfer by the laser pulse. We use two complementary approaches: optimal control theory determines what the best possible pulse can do; a more constrained physical model calculates what an experiment likely can do. Optimal control theory finds pulses with fidelity >0.9999, in excess of the quantum error correction threshold with 8 × 10(4) iterations. On the other hand, the physical model achieves only 0.9992 after 8 × 10(4) iterations. Both calculations converge as an inverse power law toward unit fidelity after >10(2) iterations/generations. In principle, the fidelities necessary for quantum error correction are reachable with qubits encoded by molecular vibrations. In practice, it will be challenging with current laboratory instrumentation because of slow convergence past fidelities of 0.99.

Are inner context factors related to implementation outcomes in cognitive-behavioral therapy for youth anxiety?

PubMed Central

Edmunds, Julie; Ditty, Matthew; Watkins, Jessica; Walsh, Lucia; Marcus, Steven; Kendall, Philip

2013-01-01

Among the challenges facing the mental health field are the dissemination and implementation of evidence-based practices. The present study investigated the relationships between inner context variables (i.e., adopter characteristics and individual perceptions of intra-organizational factors) and two implementation outcomes – independently rated therapist fidelity on a performance-based role-play (i.e., adherence and skill) and self-reported penetration of cognitive behavioral therapy for youth anxiety following training. A significant relationship was found between inner context variables and fidelity. Specifically, adopter characteristics were associated with adherence and skill; individual perceptions of intra-organizational factors were associated with adherence. Inner context variables were not associated with penetration. Future directions are discussed. PMID:24202067

Cross-cultural adaptation of the Window to Hope: a psychological intervention to reduce hopelessness among U.S. veterans with traumatic brain injury.

PubMed

Matarazzo, Bridget B; Hoffberg, Adam S; Clemans, Tracy A; Signoracci, Gina M; Simpson, Grahame K; Brenner, Lisa A

2014-01-01

To conduct a cross-cultural adaptation of Window to Hope (WtoH), a treatment to reduce hopelessness after traumatic brain injury (TBI), from the Australian civilian context to that of U.S. Veterans. Three-stage mixed-methods approach. Stage 1: Consensus conference with stakeholders to revise the manual. Stage 2: Pilot study of the revised manual with US Veterans to examine acceptability, feasibility and fidelity. Stage 3: Review of results with consensus conference attendees and further revisions. Stage 1: Conference attendees reached 100% consensus regarding changes made to the manual. Stage 2: Qualitative results yielded themes that suggest that participants benefitted from the intervention and that multiple factors contributed to successful implementation (Narrative Evaluation of Intervention Interview, User Feedback Survey-Modified, Post-Treatment Interviews). Therapists achieved 100% treatment fidelity. Quantitative results from the Client Satisfaction Questionnaire-8 suggest that the intervention was acceptable. Stage 3: The culturally adapted manual was finalized. RESULTS of this study suggest that the revised WtoH manual is acceptable and feasible. US therapists exhibited adherence to the protocol. The three-stage methodology was successfully employed to cross-culturally adapt an intervention that is well-suited for a Phase II randomized controlled trial among US military Veterans.

Examining the Variation of Fidelity across an Intervention: Implications for Measuring and Evaluating Student Learning

ERIC Educational Resources Information Center

Harn, Beth A.; Parisi Damico, Danielle; Stoolmiller, Michael

2017-01-01

Fidelity of implementation is a commonly used term that is not well understood yet frequently assessed in schools today. This paper will discuss the different approaches commonly used in schools (e.g., structural, procedural, etc.) and then present them in relation to student outcomes within a seven-month intervention study with at-risk…

Fidelity Moderates the Association Between Negative Condom Attitudes and Outcome Behavior in an Evidence-Based Sexual Risk Reduction Intervention for Female Sex Workers.

PubMed

Pitpitan, Eileen V; Chavarin, Claudia V; Semple, Shirley J; Mendoza, Doroteo; Rodriguez, Carlos Magis; Staines, Hugo; Aarons, Gregory A; Patterson, Thomas L

2017-06-01

Intervention fidelity and participant-level variables, such as negative attitudes towards condoms, are important variables to consider in the successful implementation of evidence-based HIV prevention interventions. Mujer Segura is an intervention that has been shown to be efficacious at reducing condomless sex for female sex workers (FSWs) in Mexico [1]. We examined main effects of fidelity, negative condom attitudes, and their interaction on the effectiveness of the Mujer Segura intervention at reducing condomless sex at intervention follow-up. Of the FSWs recruited from 13 cities across Mexico, 528 participated in the Mujer Segura intervention. We measured negative condom attitudes at baseline (comprising of beliefs and outcome evaluations) and condomless sex with clients at baseline and 6-month follow-up. Fidelity was measured by a fidelity checklist completed by independent raters; the sum of potentially 43 total elements completed by the counselor constituted fidelity. Complete fidelity was found in only 15.1% (n = 73) of sessions. There was no significant main effect of intervention fidelity on condomless sex with clients at follow-up. There was a significant and positive main effect of negative condom attitudes and a significant two-way interaction. At lower levels of fidelity, negative condom attitudes predicted greater condomless sex acts, whereas at higher levels of fidelity, the effect of condom attitudes became weaker. The results also indicated that the interaction between negative condom attitudes and fidelity were driven primarily by negative condom beliefs, as opposed to negative condom outcome evaluations. Ensuring treatment fidelity in an HIV prevention intervention is particularly important when participants have negative attitudes towards condoms.

Implementation of an efficacious intervention for high risk women in Mexico: protocol for a multi-site randomized trial with a parallel study of organizational factors.

PubMed

Patterson, Thomas L; Semple, Shirley J; Chavarin, Claudia V; Mendoza, Doroteo V; Santos, Lorena E; Chaffin, Mark; Palinkas, Lawrence A; Strathdee, Steffanie A; Aarons, Gregory A

2012-10-29

Studies of implementation of efficacious human immunodeficiency virus (HIV) prevention interventions are rare, especially in resource-poor settings, but important, because they have the potential to increase the impact of interventions by improving uptake and sustainability. Few studies have focused on provider and organizational factors that may influence uptake and fidelity to core intervention components. Using a hybrid design, we will study the implementation of an efficacious intervention to reduce sexually transmitted infections (STIs) among female sex workers (FSWs) in 12 cities across Mexico. Our protocol will test a 'train-the-trainer' implementation model for transporting the Mujer Segura (Healthy Woman) intervention into community-based organizations (CBOs). We have partnered with Mexican Foundation for Family Planning (Mexfam), a non-governmental organization that has CBOs throughout Mexico. At each CBO, trained ethnographers will survey CBO staff on characteristics of their organization and on their attitudes toward their CBO and toward the implementation of evidence-based interventions (EBIs). Then, after CBO staff recruit a sample of 80 eligible FSWs and deliver a standard-care, didactic intervention to 40 women randomly selected from that pool, a Mexfam staff person will be trained in the Mujer Segura intervention and will then train other counselors to deliver Mujer Segura to the 40 remaining participating FSWs. FSW participants will receive a baseline behavioral assessment and be tested for HIV and STIs (syphilis, gonorrhea, and chlamydia); they will be reassessed at six months post-intervention to measure for possible intervention effects. At the same time, both qualitative and quantitative data will be collected on the implementation process, including measures of counselors' fidelity to the intervention model. After data collection at each CBO is complete, the relative efficacy of the Mujer Segura intervention will be analyzed, and across CBOs, correlations will be examined between individual and organizational provider characteristics and intervention efficacy. This cooperative, bi-national research study will provide critical insights into barriers and facilitating factors associated with implementing interventions in CBOs using the 'train the trainer' model. Our work builds on similar scale-up strategies that have been effective in the United States. This study has the potential to increase our knowledge of the generalizability of such strategies across health issues, national contexts, and organizational contexts. NCT01465607.

Critical Features Predicting Sustained Implementation of School-Wide Positive Behavioral Interventions and Supports

ERIC Educational Resources Information Center

Mathews, Susanna; McIntosh, Kent; Frank, Jennifer L.; May, Seth L.

2014-01-01

The current study explored the extent to which a common measure of perceived implementation of critical features of Positive Behavioral Interventions and Supports (PBIS) predicted fidelity of implementation 3 years later. Respondents included school personnel from 261 schools across the United States implementing PBIS. School teams completed the…

Development and Delivery of a Physical Activity Intervention for People With Huntington Disease: Facilitating Translation to Clinical Practice

PubMed Central

Quinn, Lori; Trubey, Rob; Gobat, Nina; Dawes, Helen; Edwards, Rhiannon Tudor; Jones, Carys; Townson, Julia; Drew, Cheney; Kelson, Mark; Poile, Vincent; Rosser, Anne; Hood, Kerenza

2016-01-01

Background and Purpose: We studied the development and delivery of a 14-week complex physical activity intervention for people with Huntington disease, where detailed information about the intervention was fully embedded in the trial design process. Methods: Intervention Development: The intervention was developed through a series of focus groups. The findings from the focus groups informed the development of a logic model for the physical activity intervention that was broadly consistent with the framework of self-determination theory. Intervention Delivery: Key components underpinning the delivery of the intervention were implemented including a defined coach training program and intervention fidelity assessment methods. Training of coaches (physical therapists, occupational therapists, research nurses, and exercise trainers) was delivered via group and 1:1 training sessions using a detailed coach's manual, and with ongoing support via video calls, and e-mail communication as needed. Detailed documentation was provided to determine costs of intervention development and coach training. Results: Intervention delivery coaches at 8 sites across the United Kingdom participated in the face-to-face training. Self-report checklists completed by each of the coaches indicated that all components of the intervention were delivered in accordance with the protocol. Mean (standard deviation) intervention fidelity scores (n = 15), as measured using a purpose-developed rating scale, was 11 (2.4) (out of 16 possible points). Coaches' perceptions of intervention fidelity were similarly high. The total cost of developing the intervention and providing training was £30,773 ($47,042 USD). Discussion and Conclusions: An important consideration in promoting translation of clinical research into practice is the ability to convey the detailed components of how the intervention was delivered to facilitate replication if the results are favorable. This report presents an illustrative example of a physical activity intervention, including the development and the training required to deliver it. This approach has the potential to facilitate reproducibility, evidence synthesis, and implementation in clinical practice. Video Abstract available for more insights from the authors (see Supplemental Digital Content 1, http://links.lww.com/JNPT/A122). PMID:26863152

Intervention fidelity in a school-based diet and physical activity intervention in the UK: Active for Life Year 5.

PubMed

Campbell, Rona; Rawlins, Emma; Wells, Sian; Kipping, Ruth R; Chittleborough, Catherine R; Peters, Tim J; Lawlor, Debbie A; Jago, Russell

2015-11-11

Active for Life Year 5 (AFLY5) is an educational programme for Year 5 children (aged 9-10) designed to increase children's physical activity, decrease sedentary behaviour and increase fruit and vegetable intake. This paper reports findings from a process evaluation embedded within a randomised controlled trial evaluating the programme's effectiveness. It considers the fidelity of implementation of AFLY5 with a focus on three research questions: 1. To what extent was the intervention delivered as planned? 2. In what ways, if any, did the teachers amend the programme? and 3. What were the reasons for any amendments? Mixed methods were used including data collection via observation of the intervention delivery, questionnaire, teacher's intervention delivery log and semi-structured interviews with teachers and parents. Qualitative data were analysed thematically and quantitative data were summarised using descriptive statistics. Following training, 42 of the 43 intervention school teachers/teaching staff (98%) were confident they could deliver the nutrition and physical activity lessons according to plan. The mean number of lessons taught was 12.3 (s.d. 3.7), equating to 77% of the intervention. Reach was high with 95% of children in intervention schools receiving lessons. A mean of 6.2 (s.d. 2.6) out of 10 homeworks were delivered. Median lesson preparation time was 10 min (IQR 10-20) and 28% of lessons were reported as having been amended. Qualitative findings revealed that those who amended the lessons did so to differentiate for student ability, update them for use with new technologies and to enhance teacher and student engagement. Teachers endorsed the aims of the intervention, but some were frustrated with having to adapt the lesson materials. Teachers also a reported tendency to delegate the physical activity lessons to other staff not trained in the intervention. Fidelity of intervention implementation was good but teachers' enthusiasm for the AFLY5 programme was mixed despite them believing that the messages behind the lessons were important. This may have meant that the intervention messages were not delivered as anticipated and explain why the intervention was found not to be effective. ISRCTN50133740.

Brief cognitive behavioral therapy in primary care: a hybrid type 2 patient-randomized effectiveness-implementation design.

PubMed

Cully, Jeffrey A; Armento, Maria E A; Mott, Juliette; Nadorff, Michael R; Naik, Aanand D; Stanley, Melinda A; Sorocco, Kristen H; Kunik, Mark E; Petersen, Nancy J; Kauth, Michael R

2012-07-11

Despite the availability of evidence-based psychotherapies for depression and anxiety, they are underused in non-mental health specialty settings such as primary care. Hybrid effectiveness-implementation designs have the potential to evaluate clinical and implementation outcomes of evidence-based psychotherapies to improve their translation into routine clinical care practices. This protocol article discusses the study methodology and implementation strategies employed in an ongoing, hybrid, type 2 randomized controlled trial with two primary aims: (1) to determine whether a brief, manualized cognitive behavioral therapy administered by Veterans Affairs Primary Care Mental Health Integration program clinicians is effective in treating depression and anxiety in a sample of medically ill (chronic cardiopulmonary diseases) primary care patients and (2) to examine the acceptability, feasibility, and preliminary outcomes of a focused implementation strategy on improving adoption and fidelity of brief cognitive behavioral therapy at two Primary Care-Mental Health Integration clinics. The study uses a hybrid type 2 effectiveness/implementation design to simultaneously test clinical effectiveness and to collect pilot data on a multifaceted implementation strategy that includes an online training program, audit and feedback of session content, and internal and external facilitation. Additionally, the study engages the participation of an advisory council consisting of stakeholders from Primary Care-Mental Health Integration, as well as regional and national mental health leaders within the Veterans Administration. It targets recruitment of 320 participants randomized to brief cognitive behavioral therapy (n = 200) or usual care (n = 120). Both effectiveness and implementation outcomes are being assessed using mixed methods, including quantitative evaluation (e.g., intent-to-treat analyses across multiple time points) and qualitative methods (e.g., focus interviews and surveys from patients and providers). Patient-effectiveness outcomes include measures of depression, anxiety, and physical health functioning using blinded independent evaluators. Implementation outcomes include patient engagement and adherence and clinician brief cognitive behavioral therapy adoption and fidelity. Hybrid designs are needed to advance clinical effectiveness and implementation knowledge to improve healthcare practices. The current article describes the rationale and challenges associated with the use of a hybrid design for the study of brief cognitive behavioral therapy in primary care. Although trade-offs exist between scientific control and external validity, hybrid designs are part of an emerging approach that has the potential to rapidly advance both science and practice. NCT01149772 at http://www.clinicaltrials.gov/ct2/show/NCT01149772.

Technology Implementation in Education--Identifying Barriers to Fidelity

ERIC Educational Resources Information Center

Dennis, William J.; Johnson, Daniel L.; Monroe, Arla K.

2012-01-01

This report describes a problem-based learning project focused on determining the barriers to the implementation of technological innovations. Research findings offered evidence that properly executed technology implementation is an instructional variable related to student achievement; yet, school district leaders are faced with the problem of…

Technology Implementation in Education--Identifying Barriers to Fidelity

ERIC Educational Resources Information Center

Monroe, Arla K.; Dennis, William J.; Johnson, Daniel L.

2012-01-01

This report describes a problem-based learning project focused on determining the barriers to the implementation of technological innovations. that properly executed technology implementation is an instructional variable related to student achievement; yet, school district leaders are faced with the problem of recognizing and identifying the…

Implementation in Practice: Adaptations to Sexuality Education Curricula in California

ERIC Educational Resources Information Center

Arons, Abigail; Decker, Mara; Yarger, Jennifer; Malvin, Jan; Brindis, Claire D.

2016-01-01

Background: Local implementation of evidence-based curricula, including sex education, has received increasing attention. Although there are expectations that practitioners will implement evidence-based programs with fidelity, little is known regarding the experiences of instructors in meeting such standards. During 2005 to 2009, the California…

Lessons learned from Action Schools! BC--an 'active school' model to promote physical activity in elementary schools.

PubMed

Naylor, Patti-Jean; Macdonald, Heather M; Zebedee, Janelle A; Reed, Katherine E; McKay, Heather A

2006-10-01

The 'active school' model offers promise for promoting school-based physical activity (PA); however, few intervention trials have evaluated its effectiveness. Thus, our purpose was to: (1) describe Action Schools! BC (AS! BC) and its implementation (fidelity and feasibility) and (2) evaluate the impact of AS! BC on school provision of PA. Ten elementary schools were randomly assigned to one of the three conditions: Usual Practice (UP, three schools), Liaison (LS, four schools) or Champion (CS, three schools). Teachers in LS and CS schools received AS! BC training and resources but differed on the level of facilitation provided. UP schools continued with regular PA. Delivery of PA during the 11-month intervention was assessed with weekly Activity Logs and intervention fidelity and feasibility were assessed using Action Plans, workshop evaluations, teacher surveys and focus groups with administrators, teachers, parents and students. Physical activity delivered was significantly greater in LS (+67.4 min/week; 95% CI: 18.7-116.1) and CS (+55.2 min/week; 95% CI: 26.4-83.9) schools than UP schools. Analysis of Action Plans and Activity Logs showed fidelity to the model and moderate levels of compliance (75%). Teachers were highly satisfied with training and support. Benefits of AS! BC included positive changes in the children and school climate, including provision of resources, improved communication and program flexibility. These results support the use of the 'active school' model to positively alter the school environment. The AS! BC model was effective, providing more opportunities for "more children to be more active more often" and as such has the potential to provide health benefits to elementary school children.

The Interface of Clinical Decision-Making With Study Protocols for Knowledge Translation From a Walking Recovery Trial.

PubMed

Hershberg, Julie A; Rose, Dorian K; Tilson, Julie K; Brutsch, Bettina; Correa, Anita; Gallichio, Joann; McLeod, Molly; Moore, Craig; Wu, Sam; Duncan, Pamela W; Behrman, Andrea L

2017-01-01

Despite efforts to translate knowledge into clinical practice, barriers often arise in adapting the strict protocols of a randomized, controlled trial (RCT) to the individual patient. The Locomotor Experience Applied Post-Stroke (LEAPS) RCT demonstrated equal effectiveness of 2 intervention protocols for walking recovery poststroke; both protocols were more effective than usual care physical therapy. The purpose of this article was to provide knowledge-translation tools to facilitate implementation of the LEAPS RCT protocols into clinical practice. Participants from 2 of the trial's intervention arms: (1) early Locomotor Training Program (LTP) and (2) Home Exercise Program (HEP) were chosen for case presentation. The two cases illustrate how the protocols are used in synergy with individual patient presentations and clinical expertise. Decision algorithms and guidelines for progression represent the interface between implementation of an RCT standardized intervention protocol and clinical decision-making. In each case, the participant presents with a distinct clinical challenge that the therapist addresses by integrating the participant's unique presentation with the therapist's expertise while maintaining fidelity to the LEAPS protocol. Both participants progressed through an increasingly challenging intervention despite their own unique presentation. Decision algorithms and exercise progression for the LTP and HEP protocols facilitate translation of the RCT protocol to the real world of clinical practice. The two case examples to facilitate translation of the LEAPS RCT into clinical practice by enhancing understanding of the protocols, their progression, and their application to individual participants.Video Abstract available for more insights from the authors (see Supplemental Digital Content 1, available at: http://links.lww.com/JNPT/A147).

Contextual community prevention theory: building interventions with community agency collaboration.

PubMed

Morales, Eduardo S

2009-11-01

Translation from research to practice faces numerous problems that include replicating effectiveness, fidelity to the protocol and processes, and adaptations to different types of target populations. Working collaboratively with existing service providers can speed up the time for development and can ease the implementation of empirical randomized trials. Contextual community prevention theory is an innovative approach that focuses on changing behaviors of community members by creating a visible institutional presence that draws and pulls the targeted population into the organization's activities and interventions. The result is an institution or organization within the community that provides a new active and dynamic context, engaging its community members into its activities, interventions, and functions. An HIV prevention program developed collaboratively from the ground up for Latino gay/bisexual men is presented. Results from the program evaluation efforts across the years suggest promise for testing its efficacy through a randomized trial. HIV prevention efforts need to develop dynamic support systems within communities where these men have ownership, have control, and feel safe; otherwise HIV infection rates in this population will increase. Copyright 2009 by the American Psychological Association

REACH VA: Moving from Translation to System Implementation.

PubMed

Nichols, Linda O; Martindale-Adams, Jennifer; Burns, Robert; Zuber, Jeffrey; Graney, Marshall J

2016-02-01

Resources for Enhancing All Caregivers Health in the Department of Veterans Affairs (REACH VA) has been implemented in the VA system as a national program for caregivers. We describe the trajectory of REACH VA from national randomized clinical trial through translation to national implementation. The implementation is examined through the six stages of the Fixsen and Blasé implementation process model: exploration and adoption, program installation, initial implementation, full operation, innovation, and sustainability. Different drivers that move the implementation process forward are important at each stage, including staff selection, staff training, consultation and coaching, staff evaluation, administrative support, program evaluation/fidelity, and systems interventions. Caregivers in the REACH VA 4 session intervention currently implemented in the VA had similar outcomes to longer REACH interventions, including Resources for Enhancing Alzheimer's Caregivers Health (REACH II). Caregivers experienced significant decreases in burden, depression, anxiety, number of troubling patient behaviors reported, caregiving frustrations, stress symptoms (feeling overwhelmed, feeling like crying, being frustrated as a result of caregiving, being lonely), and general stress. Effect sizes (Cohen's d) for these significant variables were between small and medium ranging from .24 to .46. The implementation of REACH VA provides a road map for implementation of other behavioral interventions in health care delivery settings. Lessons learned include the importance of implementing a proven, needed intervention, support from both leadership and clinical staff, willingness to respond to staff and organization needs and modify the intervention while preserving its integrity, and fitting the intervention into ongoing routines and practices. Published by Oxford University Press on behalf of the Gerontological Society of America 2014.

Development and Implementation of CFD-Informed Models for the Advanced Subchannel Code CTF

DOE Office of Scientific and Technical Information (OSTI.GOV)

Blyth, Taylor S.; Avramova, Maria

The research described in this PhD thesis contributes to the development of efficient methods for utilization of high-fidelity models and codes to inform low-fidelity models and codes in the area of nuclear reactor core thermal-hydraulics. The objective is to increase the accuracy of predictions of quantities of interests using high-fidelity CFD models while preserving the efficiency of low-fidelity subchannel core calculations. An original methodology named Physics- based Approach for High-to-Low Model Information has been further developed and tested. The overall physical phenomena and corresponding localized effects, which are introduced by the presence of spacer grids in light water reactor (LWR)more » cores, are dissected in corresponding four building basic processes, and corresponding models are informed using high-fidelity CFD codes. These models are a spacer grid-directed cross-flow model, a grid-enhanced turbulent mixing model, a heat transfer enhancement model, and a spacer grid pressure loss model. The localized CFD-models are developed and tested using the CFD code STAR-CCM+, and the corresponding global model development and testing in sub-channel formulation is performed in the thermal- hydraulic subchannel code CTF. The improved CTF simulations utilize data-files derived from CFD STAR-CCM+ simulation results covering the spacer grid design desired for inclusion in the CTF calculation. The current implementation of these models is examined and possibilities for improvement and further development are suggested. The validation experimental database is extended by including the OECD/NRC PSBT benchmark data. The outcome is an enhanced accuracy of CTF predictions while preserving the computational efficiency of a low-fidelity subchannel code.« less

Development and Implementation of CFD-Informed Models for the Advanced Subchannel Code CTF

NASA Astrophysics Data System (ADS)

Blyth, Taylor S.

The research described in this PhD thesis contributes to the development of efficient methods for utilization of high-fidelity models and codes to inform low-fidelity models and codes in the area of nuclear reactor core thermal-hydraulics. The objective is to increase the accuracy of predictions of quantities of interests using high-fidelity CFD models while preserving the efficiency of low-fidelity subchannel core calculations. An original methodology named Physics-based Approach for High-to-Low Model Information has been further developed and tested. The overall physical phenomena and corresponding localized effects, which are introduced by the presence of spacer grids in light water reactor (LWR) cores, are dissected in corresponding four building basic processes, and corresponding models are informed using high-fidelity CFD codes. These models are a spacer grid-directed cross-flow model, a grid-enhanced turbulent mixing model, a heat transfer enhancement model, and a spacer grid pressure loss model. The localized CFD-models are developed and tested using the CFD code STAR-CCM+, and the corresponding global model development and testing in sub-channel formulation is performed in the thermal-hydraulic subchannel code CTF. The improved CTF simulations utilize data-files derived from CFD STAR-CCM+ simulation results covering the spacer grid design desired for inclusion in the CTF calculation. The current implementation of these models is examined and possibilities for improvement and further development are suggested. The validation experimental database is extended by including the OECD/NRC PSBT benchmark data. The outcome is an enhanced accuracy of CTF predictions while preserving the computational efficiency of a low-fidelity subchannel code.

Brief interventions to reduce Ecstasy use: a multi-site randomized controlled trial.

PubMed

Norberg, Melissa M; Hides, Leanne; Olivier, Jake; Khawar, Laila; McKetin, Rebecca; Copeland, Jan

2014-11-01

Studies examining the ability of motivational enhancement therapy (MET) to augment education provision among ecstasy users have produced mixed results and none have examined whether treatment fidelity was related to ecstasy use outcomes. The primary objectives of this multi-site, parallel, two-group randomized controlled trial were to determine if a single-session of MET could instill greater commitment to change and reduce ecstasy use and related problems more so than an education-only intervention and whether MET sessions delivered with higher treatment fidelity are associated with better outcomes. The secondary objective was to assess participants' satisfaction with their assigned interventions. Participants (N=174; Mage=23.62) at two Australian universities were allocated randomly to receive a 15-minute educational session on ecstasy use (n=85) or a 50-minute session of MET that included an educational component (n=89). Primary outcomes were assessed at baseline, and then at 4-, 16-, and 24-weeks postbaseline, while the secondary outcome measure was assessed 4-weeks postbaseline by researchers blind to treatment allocation. Overall, the treatment fidelity was acceptable to good in the MET condition. There were no statistical differences at follow-up between the groups on the primary outcomes of ecstasy use, ecstasy-related problems, and commitment to change. Both intervention groups reported a 50% reduction in their ecstasy use and a 20% reduction in the severity of their ecstasy-related problems at the 24-week follow up. Commitment to change slightly improved for both groups (9%-17%). Despite the lack of between-group statistical differences on primary outcomes, participants who received a single session of MET were slightly more satisfied with their intervention than those who received education only. MI fidelity was not associated with ecstasy use outcomes. Given these findings, future research should focus on examining mechanisms of change. Such work may suggest new methods for enhancing outcomes. Australia and New Zealand Clinical Trial Registry: ACTRN12611000136909. Copyright © 2014. Published by Elsevier Ltd.

A process evaluation exploring the lay counsellor experience of delivering a task shared psycho-social intervention for perinatal depression in Khayelitsha, South Africa.

PubMed

Munodawafa, Memory; Lund, Crick; Schneider, Marguerite

2017-07-01

Task sharing of psycho-social interventions for perinatal depression has been shown to be feasible, acceptable and effective in low and middle-income countries. This study conducted a process evaluation exploring the perceptions of counsellors who delivered a task shared psycho-social counselling intervention for perinatal depression in Khayelitsha, Cape Town together with independent fidelity ratings. Post intervention qualitative semi-structured interviews were conducted with six counsellors from the AFrica Focus on Intervention Research for Mental health (AFFIRM-SA) randomised controlled trial on their perceptions of delivering a task shared psycho-social intervention for perinatal depression. Themes were identified using the framework approach and were coded and analysed using Nvivo v11. These interviews were supplemented with fidelity ratings for each counsellor and supervision notes. Facilitating factors in the delivery of the intervention included intervention related factors such as: the content of the intervention, ongoing training and supervision, using a counselling manual, conducting counselling sessions in the local language (isiXhosa) and fidelity to the manual; counsellor factors included counsellors' confidence and motivation to conduct the sessions; participant factors included older age, commitment and a desire to be helped. Barriers included contextual factors such as poverty, crime and lack of space to conduct counselling sessions and participant factors such as the nature of the participant's problem, young age, and avoidance of contact with counsellors. Fidelity ratings and dropout rates varied substantially between counsellors. These findings show that a variety of intervention, counsellor, participant and contextual factors need to be considered in the delivery of task sharing counselling interventions. Careful attention needs to be paid to ongoing supervision and quality of care if lay counsellors are to deliver good quality task shared counselling interventions in under-resourced communities. Clinical Trials: NCT01977326 , registered on 24/10/2013; Pan African Clinical Trials Registry: PACTR201403000676264 , registered on 11/10/2013.

An Evaluation of Immediate Outcomes and Fidelity of a Drug Abuse Prevention Program in Continuation High Schools: Project towards No Drug Abuse (TND)

ERIC Educational Resources Information Center

Lisha, Nadra E.; Sun, Ping; Rohrbach, Louise A.; Spruijt-Metz, Donna; Unger, Jennifer B.; Sussman, Steve

2012-01-01

The present study provides an implementation fidelity, process, and immediate outcomes evaluation of Project Towards No Drug Abuse (TND), a drug prevention program targeting continuation high school youth (n = 1426) at risk for drug abuse. A total of 24 schools participated in three randomized conditions: TND Only, TND and motivational…

Motivation and Technological Readiness in the Use of High-Fidelity Simulation: A Descriptive Comparative Study of Nurse Educators

ERIC Educational Resources Information Center

Duvall, Judy Jo

2012-01-01

There are many driving forces to increase the use of high-fidelity simulation (HFS) in nursing education, as well as many factors that may influence the implementation of this teaching strategy. These include the motivation of nurse educators to use HFS, the technological readiness of nurse educators to use HFS and the changing demographics of the…

Evidence-Based Programming within Cooperative Extension: How Can We Maintain Program Fidelity While Adapting to Meet Local Needs?

ERIC Educational Resources Information Center

Olson, Jonathan R.; Welsh, Janet A.; Perkins, Daniel F.

2015-01-01

In this article, we describe how the recent movement towards evidence-based programming has impacted Extension. We review how the emphasis on implementing such programs with strict fidelity to an underlying program model may be at odds with Extension's strong history of adapting programming to meet the unique needs of children, youth, families,…

Process evaluation of the RaDIANT community study: a dialysis facility-level intervention to increase referral for kidney transplantation.

PubMed

Hamoda, Reem E; Gander, Jennifer C; McPherson, Laura J; Arriola, Kimberly J; Cobb, Loren; Pastan, Stephen O; Plantinga, Laura; Browne, Teri; Hartmann, Erica; Mulloy, Laura; Zayas, Carlos; Krisher, Jenna; Patzer, Rachel E

2018-01-15

The Reducing Disparities in Access to kidNey Transplantation Community Study (RaDIANT) was an End-Stage Renal Disease (ESRD) Network 6-developed, dialysis facility-level randomized trial testing the effectiveness of a 1-year multicomponent education and quality improvement intervention in increasing referral for kidney transplant evaluation among selected Georgia dialysis facilities. To assess implementation of the RaDIANT intervention, we conducted a process evaluation at the conclusion of the intervention period (January-December 2014). We administered a 20-item survey to the staff involved with transplant education in 67 dialysis facilities randomized to participate in intervention activities. Survey items assessed facility participation in the intervention (fidelity and reach), helpfulness and willingness to continue intervention activities (sustainability), suggestions for improving intervention components (sustainability), and factors that may have influenced participation and study outcomes (context). We defined high fidelity to the intervention as completing 11 or more activities, and high participation in an activity as having at least 75% participation across intervention facilities. Staff from 65 of the 67 dialysis facilities completed the questionnaire, and more than half (50.8%) reported high adherence (fidelity) to RaDIANT intervention requirements. Nearly two-thirds (63.1%) of facilities reported that RaDIANT intervention activities were helpful or very helpful, with 90.8% of facilities willing to continue at least one intervention component beyond the study period. Intervention components with high participation emphasized staff and patient-level education, including in-service staff orientations, patient and family education programs, and patient educational materials. Suggested improvements for intervention activities emphasized addressing financial barriers to transplantation, with financial education materials perceived as most helpful among RaDIANT educational materials. Variation in facility-level fidelity of the RADIANT intervention did not significantly influence the mean difference in proportion of patients referred pre- (2013) and post-intervention (2014). We found high fidelity to the RaDIANT multicomponent intervention at the majority of intervention facilities, with sustainability of select intervention components at intervention facilities and feasibility for dissemination across ESRD Networks. Future modification of the intervention should emphasize financial education regarding kidney transplantation and amend intervention components that facilities perceive as time-intensive or non-sustainable. Clinicaltrials.gov number NCT02092727 . Registered 13 Mar 2014 (retrospectively registered).

Efficient experimental design of high-fidelity three-qubit quantum gates via genetic programming

NASA Astrophysics Data System (ADS)

Devra, Amit; Prabhu, Prithviraj; Singh, Harpreet; Arvind; Dorai, Kavita

2018-03-01

We have designed efficient quantum circuits for the three-qubit Toffoli (controlled-controlled-NOT) and the Fredkin (controlled-SWAP) gate, optimized via genetic programming methods. The gates thus obtained were experimentally implemented on a three-qubit NMR quantum information processor, with a high fidelity. Toffoli and Fredkin gates in conjunction with the single-qubit Hadamard gates form a universal gate set for quantum computing and are an essential component of several quantum algorithms. Genetic algorithms are stochastic search algorithms based on the logic of natural selection and biological genetics and have been widely used for quantum information processing applications. We devised a new selection mechanism within the genetic algorithm framework to select individuals from a population. We call this mechanism the "Luck-Choose" mechanism and were able to achieve faster convergence to a solution using this mechanism, as compared to existing selection mechanisms. The optimization was performed under the constraint that the experimentally implemented pulses are of short duration and can be implemented with high fidelity. We demonstrate the advantage of our pulse sequences by comparing our results with existing experimental schemes and other numerical optimization methods.

Understanding the Social Context of School Health Promotion Program Implementation

ERIC Educational Resources Information Center

Cargo, Margaret; Salsberg, Jon; Delormier, Treena; Desrosiers, Serge; Macaulay, Ann C.

2006-01-01

Purpose: Although implementation fidelity is an important component in the evaluation of school health promotion programs, it assumes that teaching is the most relevant teacher role. To understand the social context of program implementation, a qualitative study was undertaken with the aim of identifying the schoolteacher's role in implementing…

Coaching Early Childhood Special Educators to Implement a Comprehensive Model for Promoting Young Children's Social Competence

ERIC Educational Resources Information Center

Fox, Lise; Hemmeter, Mary; Snyder, Patricia; Binder, Denise Perez; Clarke, Shelley

2011-01-01

Growing evidence suggests the importance of practitioners implementing promotion, prevention, and intervention practices to foster children's social-emotional competence and address challenging behavior within schools. Limited research exists, however, on how to support teachers of school-age children to implement with fidelity comprehensive…

Prevalence and Implementation Fidelity of Research-Based Prevention Programs in Public Schools. Final Report

ERIC Educational Resources Information Center

Crosse, Scott; Williams, Barbara; Hagen, Carol A.; Harmon, Michele; Ristow, Liam; DiGaetano, Ralph; Broene, Pamela; Alexander, Debbie; Tseng, Margaret; Derzon, James H.

2011-01-01

This report presents descriptive information about the prevalence and quality of implementation of research-based programs from the Study of the Implementation of Research-Based Programs to Prevent Youth Substance Abuse and School Crime. The study found that, while schools reported implementing a large number of prevention programs during the…

Implementation of the World Starts with Me, a Comprehensive Rights-Based Sex Education Programme in Uganda

ERIC Educational Resources Information Center

Rijsdijk, Liesbeth E.; Bos, Arjan E. R.; Lie, Rico; Leerlooijer, Joanne N.; Eiling, Ellen; Atema, Vera; Gebhardt, Winifred A.; Ruiter, Robert A. C.

2014-01-01

This article presents a process evaluation of the implementation of the sex education programme the World Starts With Me (WSWM) for secondary school students in Uganda. The purpose of this mixed-methods study was to examine factors associated with dose delivered (number of lessons implemented) and fidelity of implementation (implementation…

Implementation of Supplemental Education Services: 2009-10. Implementation Insights. E&R Report No. 11.05

ERIC Educational Resources Information Center

Paeplow, Colleen

2011-01-01

This report, the second of three reports examining Supplemental Education Services (SES) within WCPSS, focused on the implementation of SES, and the degree to which short-term and intermediate goals were met. In 2009-10, the SES program was largely implemented with fidelity, with some areas needing refinement related to communication and…

Predictors of Sustained Implementation of School-Wide Positive Behavioral Interventions and Supports

ERIC Educational Resources Information Center

McIntosh, Kent; Mercer, Sterett H.; Nese, Rhonda N. T.; Strickland-Cohen, M. Kathleen; Hoselton, Robert

2016-01-01

In this analysis of extant data from 3,011 schools implementing school-wide positive behavioral interventions and supports (SWPBIS) across multiple years, we assessed the predictive power of various school characteristics and speed of initial implementation on sustained fidelity of implementation of SWPBIS at 1, 3, and 5 years. In addition, we…

A multisite study of implementing supported employment in the Netherlands.

PubMed

van Erp, Nicole H J; Giesen, Femke B M; van Weeghel, Jaap; Kroon, Hans; Michon, Harry W C; Becker, Deborah; McHugo, Gregory J; Drake, Robert E

2007-11-01

This article reports on the implementation of the individual placement and support model of supported employment in four Dutch regions. The authors used structured site visits, employment data, and semistructured interviews to assess fidelity, employment outcomes, and facilitators of and barriers to successful implementation. At 24 months, the four sites reached a mean+/-SD fidelity score of 4.1+/-.3 (possible scores range from 1 to 5, with higher scores indicating closer adherence to the model). Of the 316 persons with mental illnesses, 57 (18%) obtained competitive jobs. Barriers to implementation included lack of organizational standards, loss of vocational staff, funding problems, insufficient time for program leaders, and inadequate cooperation between the involved organizations. Important facilitators were the skills and commitment of the vocational team members and the integration of vocational and mental health staff. To implement evidence-based supported employment in the Netherlands will require changes in financing, organizational structures, attitudes, cultural beliefs, and labor and disability regulations.

Virtual Reality Compared with Bench-Top Simulation in the Acquisition of Arthroscopic Skill: A Randomized Controlled Trial.

PubMed

Banaszek, Daniel; You, Daniel; Chang, Justues; Pickell, Michael; Hesse, Daniel; Hopman, Wilma M; Borschneck, Daniel; Bardana, Davide

2017-04-05

Work-hour restrictions as set forth by the Accreditation Council for Graduate Medical Education (ACGME) and other governing bodies have forced training programs to seek out new learning tools to accelerate acquisition of both medical skills and knowledge. As a result, competency-based training has become an important part of residency training. The purpose of this study was to directly compare arthroscopic skill acquisition in both high-fidelity and low-fidelity simulator models and to assess skill transfer from either modality to a cadaveric specimen, simulating intraoperative conditions. Forty surgical novices (pre-clerkship-level medical students) voluntarily participated in this trial. Baseline demographic data, as well as data on arthroscopic knowledge and skill, were collected prior to training. Subjects were randomized to 5-week independent training sessions on a high-fidelity virtual reality arthroscopic simulator or on a bench-top arthroscopic setup, or to an untrained control group. Post-training, subjects were asked to perform a diagnostic arthroscopy on both simulators and in a simulated intraoperative environment on a cadaveric knee. A more difficult surprise task was also incorporated to evaluate skill transfer. Subjects were evaluated using the Global Rating Scale (GRS), the 14-point arthroscopic checklist, and a timer to determine procedural efficiency (time per task). Secondary outcomes focused on objective measures of virtual reality simulator motion analysis. Trainees on both simulators demonstrated a significant improvement (p < 0.05) in arthroscopic skills compared with baseline scores and untrained controls, both in and ex vivo. The virtual reality simulation group consistently outperformed the bench-top model group in the diagnostic arthroscopy crossover tests and in the simulated cadaveric setup. Furthermore, the virtual reality group demonstrated superior skill transfer in the surprise skill transfer task. Both high-fidelity and low-fidelity simulation trainings were effective in arthroscopic skill acquisition. High-fidelity virtual reality simulation was superior to bench-top simulation in the acquisition of arthroscopic skills, both in the laboratory and in vivo. Further clinical investigation is needed to interpret the importance of these results.

High Fidelity Preparation of a Single Atom in Its 2D Center of Mass Ground State

NASA Astrophysics Data System (ADS)

Sompet, Pimonpan; Fung, Yin Hsien; Schwartz, Eyal; Hunter, Matthew D. J.; Phrompao, Jindaratsamee; Andersen, Mikkel F.

2017-04-01

Complete control over quantum states of individual atoms is important for the study of the microscopic world. Here, we present a push button method for high fidelity preparation of a single 85Rb atom in the vibrational ground state of tightly focused optical tweezers. The method combines near-deterministic preparation of a single atom with magnetically-insensitive Raman sideband cooling. We achieve 2D cooling in the radial plane with a ground state population of 0.85, which provides a fidelity of 0.7 for the entire procedure (loading and cooling). The Raman beams couple two sublevels (| F = 3 , m = 0 〉 and | F = 2 , m = 0 〉) that are indifferent to magnetic noise to first order. This leads to long atomic coherence times, and allows us to implement the cooling in an environment where magnetic field fluctuations prohibit previously demonstrated variations. Additionally, we implement the trapping and manipulation of two atoms confined in separate dynamically reconfigurable optical tweezers, to study few-body dynamics.

Preserving photon qubits in an unknown quantum state with Knill Dynamical Decoupling - Towards an all optical quantum memory

NASA Astrophysics Data System (ADS)

Gupta, Manish K.; Navarro, Erik J.; Moulder, Todd A.; Mueller, Jason D.; Balouchi, Ashkan; Brown, Katherine L.; Lee, Hwang; Dowling, Jonathan P.

2015-05-01

The storage of quantum states and its distribution over long distances is essential for emerging quantum technologies such as quantum networks and long distance quantum cryptography. The implementation of polarization-based quantum communication is limited by signal loss and decoherence caused by the birefringence of a single-mode fiber. We investigate the Knill dynamical decoupling scheme, implemented using half-wave plates in a single mode fiber, to minimize decoherence of polarization qubit and show that a fidelity greater than 99 % can be achieved in absence of rotation error and fidelity greater than 96 % can be achieved in presence of rotation error. Such a scheme can be used to preserve any quantum state with high fidelity and has potential application for constructing all optical quantum memory, quantum delay line, and quantum repeater. The authors would like to acknowledge the support from the Air Force office of Scientific Research, the Army Research office, and the National Science Foundation.

Lessons learned: Evaluating the program fidelity of UNWomen Partnership for Peace domestic violence diversion program in the Eastern Caribbean.

PubMed

Jeremiah, Rohan D; Quinn, Camille R; Alexis, Jicinta M

2018-08-01

To date, there have been a plethora of punitive and diversion programs to address domestic violence around the world. However, the evaluative scholarship of such programs overwhelmingly reflects studies in developed countries while barely showcasing the realities of addressing domestic violence in developing countries. This paper features a multi-year (2008-2011) evaluation study that measured the fidelity of the United Nations Partnership for Peace (PfP) domestic violence diversion program in the Eastern Caribbean country of Grenada. Our findings illuminate organic engagement strategies that were built within existing multi-sectoral partnerships that included magistrate court judges, law enforcement officials, and social service agencies. Furthermore, we documented how the locally-devised implementation strategies ensured the program's fidelity within a resource-limited context. This paper contributes to the global evaluative scholarship, highlighting the lessons learned about implementing culturally-adapted and theoretically-driven domestic violence diversion within a developing country. Copyright © 2018 Elsevier Ltd. All rights reserved.

A training program for nurse scientists to promote intervention translation.

PubMed

Santacroce, Sheila Judge; Leeman, Jennifer; Song, Mi-Kyung

To reduce the burden of chronic illness, prevention and management interventions must be efficacious, adopted and implemented with fidelity, and reach those at greatest risk. Yet, many research-tested interventions are slow to translate into practice. This paper describes how The University of North Carolina at Chapel Hill School of Nursing's NINR-funded institutional pre- and postdoctoral research-training program is addressing the imperative to speed knowledge translation across the research cycle. The training emphasizes six research methods ("catalysts") to speed translation: stakeholder engagement, patient-centered outcomes, intervention optimization and sequential multiple randomized trials (SMART), pragmatic trials, mixed methods approaches, and dissemination and implementation science strategies. Catalysts are integrated into required coursework, biweekly scientific and integrative seminars, and experiential research training. Trainee and program success is evaluated based on benchmarks applicable to all PhD program students, supplemented by indicators specific to the catalysts. Trainees must also demonstrate proficiency in at least two of the six catalysts in their scholarly products. Proficiency is assessed through their works in progress presentations and peer reviews at T32 integrative seminars. While maintaining the emphasis on theory-based interventions, we have integrated six catalysts into our ongoing research training to expedite the dynamic process of intervention development, testing, dissemination and implementation. Through a variety of training activities, our research training focused on theory-based interventions and the six catalysts will generate future nurse scientists who speed translation of theory-based interventions into practice to maximize health outcomes for patients, families, communities and populations affected by chronic illness. Copyright © 2017 Elsevier Inc. All rights reserved.

What is actually measured in process evaluations for worksite health promotion programs: a systematic review

PubMed Central

2013-01-01

Background Numerous worksite health promotion program (WHPPs) have been implemented the past years to improve employees’ health and lifestyle (i.e., physical activity, nutrition, smoking, alcohol use and relaxation). Research primarily focused on the effectiveness of these WHPPs. Whereas process evaluations provide essential information necessary to improve large scale implementation across other settings. Therefore, this review aims to: (1) further our understanding of the quality of process evaluations alongside effect evaluations for WHPPs, (2) identify barriers/facilitators affecting implementation, and (3) explore the relationship between effectiveness and the implementation process. Methods Pubmed, EMBASE, PsycINFO, and Cochrane (controlled trials) were searched from 2000 to July 2012 for peer-reviewed (randomized) controlled trials published in English reporting on both the effectiveness and the implementation process of a WHPP focusing on physical activity, smoking cessation, alcohol use, healthy diet and/or relaxation at work, targeting employees aged 18-65 years. Results Of the 307 effect evaluations identified, twenty-two (7.2%) published an additional process evaluation and were included in this review. The results showed that eight of those studies based their process evaluation on a theoretical framework. The methodological quality of nine process evaluations was good. The most frequently reported process components were dose delivered and dose received. Over 50 different implementation barriers/facilitators were identified. The most frequently reported facilitator was strong management support. Lack of resources was the most frequently reported barrier. Seven studies examined the link between implementation and effectiveness. In general a positive association was found between fidelity, dose and the primary outcome of the program. Conclusions Process evaluations are not systematically performed alongside effectiveness studies for WHPPs. The quality of the process evaluations is mostly poor to average, resulting in a lack of systematically measured barriers/facilitators. The narrow focus on implementation makes it difficult to explore the relationship between effectiveness and implementation. Furthermore, the operationalisation of process components varied between studies, indicating a need for consensus about defining and operationalising process components. PMID:24341605

A Multi-Level Parallelization Concept for High-Fidelity Multi-Block Solvers

NASA Technical Reports Server (NTRS)

Hatay, Ferhat F.; Jespersen, Dennis C.; Guruswamy, Guru P.; Rizk, Yehia M.; Byun, Chansup; Gee, Ken; VanDalsem, William R. (Technical Monitor)

1997-01-01

The integration of high-fidelity Computational Fluid Dynamics (CFD) analysis tools with the industrial design process benefits greatly from the robust implementations that are transportable across a wide range of computer architectures. In the present work, a hybrid domain-decomposition and parallelization concept was developed and implemented into the widely-used NASA multi-block Computational Fluid Dynamics (CFD) packages implemented in ENSAERO and OVERFLOW. The new parallel solver concept, PENS (Parallel Euler Navier-Stokes Solver), employs both fine and coarse granularity in data partitioning as well as data coalescing to obtain the desired load-balance characteristics on the available computer platforms. This multi-level parallelism implementation itself introduces no changes to the numerical results, hence the original fidelity of the packages are identically preserved. The present implementation uses the Message Passing Interface (MPI) library for interprocessor message passing and memory accessing. By choosing an appropriate combination of the available partitioning and coalescing capabilities only during the execution stage, the PENS solver becomes adaptable to different computer architectures from shared-memory to distributed-memory platforms with varying degrees of parallelism. The PENS implementation on the IBM SP2 distributed memory environment at the NASA Ames Research Center obtains 85 percent scalable parallel performance using fine-grain partitioning of single-block CFD domains using up to 128 wide computational nodes. Multi-block CFD simulations of complete aircraft simulations achieve 75 percent perfect load-balanced executions using data coalescing and the two levels of parallelism. SGI PowerChallenge, SGI Origin 2000, and a cluster of workstations are the other platforms where the robustness of the implementation is tested. The performance behavior on the other computer platforms with a variety of realistic problems will be included as this on-going study progresses.

Multidisciplinary management of chronic heart failure: principles and future trends.

PubMed

Davidson, Patricia M; Newton, Phillip J; Tankumpuan, Thitipong; Paull, G; Dennison-Himmelfarb, Cheryl

2015-10-01

Globally, the management of chronic heart failure (CHF) challenges health systems. The high burden of disease and the costs associated with hospitalization adversely affect individuals, families, and society. Improved quality, access, efficiency, and equity of CHF care can be achieved by using multidisciplinary care approaches if there is adherence and fidelity to the program's elements. The goal of this article was to summarize evidence and make recommendations for advancing practice, education, research, and policy in the multidisciplinary management of patients with CHF. Essential elements of multidisciplinary management of CHF were identified from meta-analyses and clinical practice guidelines. The study factors were discussed from the perspective of the health care system, providers, patients, and their caregivers. Identified gaps in evidence were used to identify areas for future focus in CHF multidisciplinary management. Although there is high-level evidence (including several meta-analyses) for the efficacy of management programs for CHF, less evidence exists to determine the benefit attributable to individual program components or to identify the specific content of effective components and the manner of their delivery. Health care system, provider, and patient factors influence health care models and the effective management of CHF and require focus and attention. Extrapolating trial findings to clinical practice settings is limited by the heterogeneity of study populations and the implementation of models of intervention beyond academic health centers, where practice environments differ considerably. Ensuring that individual programs are both developed and assessed that consider these factors is integral to ensuring adherence and fidelity with the core dimensions of disease management necessary to optimize patient and organizational outcomes. Recognizing the complexity of the multidisciplinary CHF interventions will be important in advancing the design, implementation, and evaluation of the interventions. Copyright © 2015 Elsevier HS Journals, Inc. All rights reserved.

SOWFA Super-Controller: A High-Fidelity Tool for Evaluating Wind Plant Control Approaches

DOE Office of Scientific and Technical Information (OSTI.GOV)

Fleming, P.; Gebraad, P.; van Wingerden, J. W.

2013-01-01

This paper presents a new tool for testing wind plant controllers in the Simulator for Offshore Wind Farm Applications (SOWFA). SOWFA is a high-fidelity simulator for the interaction between wind turbine dynamics and the fluid flow in a wind plant. The new super-controller testing environment in SOWFA allows for the implementation of the majority of the wind plant control strategies proposed in the literature.

Comparing synthetic imagery with real imagery for visible signature analysis: human observer results

NASA Astrophysics Data System (ADS)

Culpepper, Joanne B.; Richards, Noel; Madden, Christopher S.; Winter, Neal; Wheaton, Vivienne C.

2017-10-01

Synthetic imagery could potentially enhance visible signature analysis by providing a wider range of target images in differing environmental conditions than would be feasible to collect in field trials. Achieving this requires a method for generating synthetic imagery that is both verified to be realistic and produces the same visible signature analysis results as real images. Is target detectability as measured by image metrics the same for real images and synthetic images of the same scene? Is target detectability as measured by human observer trials the same for real images and synthetic images of the same scene, and how realistic do the synthetic images need to be? In this paper we present the results of a small scale exploratory study on the second question: a photosimulation experiment conducted using digital photographs and synthetic images generated of the same scene. Two sets of synthetic images were created: a high fidelity set created using an image generation tool, E-on Vue, and a low fidelity set created using a gaming engine, Unity 3D. The target detection results obtained using digital photographs were compared with those obtained using the two sets of synthetic images. There was a moderate correlation between the high fidelity synthetic image set and the real images in both the probability of correct detection (Pd: PCC = 0.58, SCC = 0.57) and mean search time (MST: PCC = 0.63, SCC = 0.61). There was no correlation between the low fidelity synthetic image set and the real images for the Pd, but a moderate correlation for MST (PCC = 0.67, SCC = 0.55).

Exploring Barriers to Implementing a School-Wide Positive Behavioral Intervention and Support Program

ERIC Educational Resources Information Center

Gay, Ronald Lynn

2016-01-01

This study examined factors related to the implementation of a School Wide Positive Behavioral Intervention and Support (SWPBIS) program at a large middle school in the United States. Parent Teacher Student Association volunteers at the school reported that teacher fidelity to implementation of SWPBIS activities was inconsistent, threatening the…

From Theory to Practice: An Illustrative Case for Selecting Evidence-Based Practices and Building Implementation Capacity in Three Canadian Health Jurisdictions

ERIC Educational Resources Information Center

Duda, Michelle A.; Riopelle, Richard J.; Brown, Jacquie

2013-01-01

Using principles of Applied Implementation Science, this paper examines strategies for systematically selecting and operationalising National clinical practice guidelines and intentionally creating implementation supports to ensure high fidelity use and sustainable application and outcomes. In the spirit of participatory action research, key…

Implementing a Primary Prevention Social Skills Intervention in Urban Preschools: Factors Associated with Quality and Fidelity

ERIC Educational Resources Information Center

Wenz-Gross, Melodie; Upshur, Carole

2012-01-01

Research Findings: Preschool behavior problems are of increasing concern to early childhood educators. Preventive interventions are being developed, but implementation in underresourced child care programs is challenging. This study describes the implementation of an adapted Second Step curriculum to increase children's social skills and decrease…

Voices from the Field: Stakeholder Perspectives on PBIS Implementation in Alternative Educational Settings

ERIC Educational Resources Information Center

Swain-Bradway, Jessica; Swoszowski, Nicole Cain; Boden, Lauren J.; Sprague, Jeffrey R.

2013-01-01

The positive behavior interventions and supports (PBIS) framework is currently implemented in over 18,000 schools in the United States. Schools implementing PBIS with fidelity report school level benefits including decreases in problem behavior, increases in academic engaged time, and improved perceptions of school safety. While potential benefits…

Using Policy Attributes Theory to Examine Comprehensive School Reform Implementation in Two Title I Middle Schools

ERIC Educational Resources Information Center

Patterson, Jean A.; Campbell, J. K.; Johnson, Dawn M.; Marx, Gina; Whitener, Mark

2013-01-01

Findings from a qualitative study of two Title I middle schools that were in their second year of implementing an externally developed Comprehensive School Reform (CSR) model are presented. Policy attributes theory was used as a framework for examining implementation. The theory argues fidelity of implementation of a CSR is strongest when it is…

Explaining outcomes in major system change: a qualitative study of implementing centralised acute stroke services in two large metropolitan regions in England.

PubMed

Fulop, Naomi J; Ramsay, Angus I G; Perry, Catherine; Boaden, Ruth J; McKevitt, Christopher; Rudd, Anthony G; Turner, Simon J; Tyrrell, Pippa J; Wolfe, Charles D A; Morris, Stephen

2016-06-03

Implementing major system change in healthcare is not well understood. This gap may be addressed by analysing change in terms of interrelated components identified in the implementation literature, including decision to change, intervention selection, implementation approaches, implementation outcomes, and intervention outcomes. We conducted a qualitative study of two cases of major system change: the centralisation of acute stroke services in Manchester and London, which were associated with significantly different implementation outcomes (fidelity to referral pathway) and intervention outcomes (provision of evidence-based care, patient mortality). We interviewed stakeholders at national, pan-regional, and service-levels (n = 125) and analysed 653 documents. Using a framework developed for this study from the implementation science literature, we examined factors influencing implementation approaches; how these approaches interacted with the models selected to influence implementation outcomes; and their relationship to intervention outcomes. London and Manchester's differing implementation outcomes were influenced by the different service models selected and implementation approaches used. Fidelity to the referral pathway was higher in London, where a 'simpler', more inclusive model was used, implemented with a 'big bang' launch and 'hands-on' facilitation by stroke clinical networks. In contrast, a phased approach of a more complex pathway was used in Manchester, and the network acted more as a platform to share learning. Service development occurred more uniformly in London, where service specifications were linked to financial incentives, and achieving standards was a condition of service launch, in contrast to Manchester. 'Hands-on' network facilitation, in the form of dedicated project management support, contributed to achievement of these standards in London; such facilitation processes were less evident in Manchester. Using acute stroke service centralisation in London and Manchester as an example, interaction between model selected and implementation approaches significantly influenced fidelity to the model. The contrasting implementation outcomes may have affected differences in provision of evidence-based care and patient mortality. The framework used in this analysis may support planning and evaluating major system changes, but would benefit from application in different healthcare contexts.

The role of staff turnover in the implementation of evidence-based practices in mental health care.

PubMed

Woltmann, Emily M; Whitley, Rob; McHugo, Gregory J; Brunette, Mary; Torrey, William C; Coots, Laura; Lynde, David; Drake, Robert E

2008-07-01

This study examined turnover rates of teams implementing psychosocial evidence-based practices in public-sector mental health settings. It also explored the relationship between turnover and implementation outcomes in an effort to understand whether practitioner perspectives on turnover are related to implementation outcomes. Team turnover was measured for 42 implementing teams participating in a national demonstration project examining implementation of five evidence-based practices between 2002 and 2005. Regression techniques were used to analyze the effects of team turnover on penetration and fidelity. Qualitative data collected throughout the project were blended with the quantitative data to examine the significance of team turnover to those attempting to implement the practices. High team turnover was common (M+/-SD=81%+/-46%) and did not vary by practice. The 24-month turnover rate was inversely related to fidelity scores at 24 months (N=40, beta=-.005, p=.01). A negative trend was observed for penetration. Further analysis indicated that 71% of teams noted that turnover was a relevant factor in implementation. The behavioral health workforce remains in flux. High turnover most often had a negative impact on implementation, although some teams were able to use strategies to improve implementation through turnover. Implementation models must consider turbulent behavioral health workforce conditions.

Emotion-based preventive intervention: Effectively promoting emotion knowledge and adaptive behavior among at-risk preschoolers.

PubMed

Finlon, Kristy J; Izard, Carroll E; Seidenfeld, Adina; Johnson, Stacy R; Cavadel, Elizabeth Woodburn; Ewing, E Stephanie Krauthamer; Morgan, Judith K

2015-11-01

Effectiveness studies of preschool social-emotional programs are needed in low-income, diverse populations to help promote the well-being of at-risk children. Following an initial program efficacy study 2 years prior, 248 culturally diverse Head Start preschool children participated in the current effectiveness trial and received either the Emotion-Based Prevention Program (EBP) or the I Can Problem Solve (ICPS) intervention. Pre- and postdata collection included direct child assessment, teacher report, parent interview, and independent observations. Teachers implementing the EBP intervention demonstrated good and consistent fidelity to the program. Overall, children in EBP classrooms gained more emotion knowledge and displayed greater decreases in negative emotion expressions and internalizing behaviors across the implementation period as compared to children in ICPS classrooms. In addition, cumulative risk, parental depressive symptoms, and classroom climate significantly moderated treatment effects. For children experiencing more stress or less support, EBP produced more successful outcomes than did ICPS. These results provide evidence of EBP sustainability and program effectiveness, as did previous findings that demonstrated EBP improvements in emotion knowledge, regulation skills, and behavior problems replicated under unsupervised program conditions.

Fidelity in complex behaviour change interventions: a standardised approach to evaluate intervention integrity

PubMed Central

Mars, Tom; Ellard, David; Carnes, Dawn; Homer, Kate; Underwood, Martin; Taylor, Stephanie J C

2013-01-01

Objectives The aim of this study was to (1) demonstrate the development and testing of tools and procedures designed to monitor and assess the integrity of a complex intervention for chronic pain (COping with persistent Pain, Effectiveness Research into Self-management (COPERS) course); and (2) make recommendations based on our experiences. Design Fidelity assessment of a two-arm randomised controlled trial intervention, assessing the adherence and competence of the facilitators delivering the intervention. Setting The intervention was delivered in the community in two centres in the UK: one inner city and one a mix of rural and urban locations. Participants 403 people with chronic musculoskeletal pain were enrolled in the intervention arm and 300 attended the self-management course. Thirty lay and healthcare professionals were trained and 24 delivered the courses (2 per course). We ran 31 courses for up to 16 people per course and all were audio recorded. Interventions The course was run over three and a half days; facilitators delivered a semistructured manualised course. Outcomes We designed three measures to evaluate fidelity assessing adherence to the manual, competence and overall impression. Results We evaluated a random sample of four components from each course (n=122). The evaluation forms were reliable and had good face validity. There were high levels of adherence in the delivery: overall adherence was two (maximum 2, IQR 1.67–2.00), facilitator competence exhibited more variability, and overall competence was 1.5 (maximum 2, IQR 1.25–2.00). Overall impression was three (maximum 4, IQR 2.00–3.00). Conclusions Monitoring and assessing adherence and competence at the point of intervention delivery can be realised most efficiently by embedding the principles of fidelity measurement within the design stage of complex interventions and the training and assessment of those delivering the intervention. More work is necessary to ensure that more robust systems of fidelity evaluation accompany the growth of complex interventions. Trial Registration ISRCTN No ISRCTN24426731. PMID:24240140

A protocol for evaluating progressive levels of simulation fidelity in the development of technical skills, integrated performance and woman centred clinical assessment skills in undergraduate midwifery students

PubMed Central

2013-01-01

Background Simulation as a pedagogical approach has been used in health professional education to address the need to safely develop effective clinical skills prior to undertaking clinical practice. However, evidence for the use of simulation in midwifery is largely anecdotal, and research evaluating the effectiveness of different levels of simulation fidelity are lacking. Woman centred care is a core premise of the midwifery profession and describes the behaviours of an individual midwife who demonstrates safe and effective care of the individual woman. Woman centred care occurs when the midwife modifies the care to ensure the needs of each individual woman are respected and addressed. However, a review of the literature demonstrates an absence of a valid and reliable tool to measure the development of woman centred care behaviours. This study aims to determine which level of fidelity in simulated learning experiences provides the most effective learning outcomes in the development of woman centred clinical assessment behaviors and skills in student midwives. Methods/Design Three-arm, randomised, intervention trial. In this research we plan to: a) trial three levels of simulation fidelity - low, medium and progressive, on student midwives performing the procedure of vaginal examination; b) measure clinical assessment skills using the Global Rating Scale (GRS) and Integrated Procedural Performance Instrument (IPPI); and c) pilot the newly developed Woman Centred Care Scale (WCCS) to measure clinical behaviors related to Woman-Centredness. Discussion This project aims to enhance knowledge in relation to the appropriate levels of fidelity in simulation that yield the best educational outcomes for the development of woman centred clinical assessment in student midwives. The outcomes of this project may contribute to improved woman centred clinical assessment for student midwives, and more broadly influence decision making regarding education resource allocation for maternity simulation. PMID:23706037

Assessment of Fidelity in Interventions to Improve Hand Hygiene of Healthcare Workers: A Systematic Review

PubMed Central

Musuuza, Jackson S.; Barker, Anna; Ngam, Caitlyn; Vellardita, Lia; Safdar, Nasia

2016-01-01

OBJECTIVE Compliance with hand hygiene in healthcare workers is fundamental to infection prevention yet remains a challenge to sustain. We examined fidelity reporting in interventions to improve hand hygiene compliance, and we assessed 5 measures of intervention fidelity: (1) adherence, (2) exposure or dose, (3) quality of intervention delivery, (4) participant responsiveness, and (5) program differentiation. DESIGN Systematic review METHODS A librarian performed searches of the literature in PubMed, Cumulative Index to Nursing and Allied Health (CINAHL), Cochrane Library, and Web of Science of material published prior to June 19, 2015. The review protocol was registered in PROSPERO International Prospective Register of Systematic Reviews, and assessment of study quality was conducted for each study reviewed. RESULTS A total of 100 studies met the inclusion criteria. Only 8 of these 100 studies reported all 5 measures of intervention fidelity. In addition, 39 of 100 (39%) failed to include at least 3 fidelity measures; 20 of 100 (20%) failed to include 4 measures; 17 of 100 (17%) failed to include 2 measures, while 16 of 100 (16%) of the studies failed to include at least 1 measure of fidelity. Participant responsiveness and adherence to the intervention were the most frequently unreported fidelity measures, while quality of the delivery was the most frequently reported measure. CONCLUSIONS Almost all hand hygiene intervention studies failed to report at least 1 fidelity measurement. To facilitate replication and effective implementation, reporting fidelity should be standard practice when describing results of complex behavioral interventions such as hand hygiene. PMID:26861117

A community-based randomized controlled trial of Mom Power parenting intervention for mothers with interpersonal trauma histories and their young children.

PubMed

Rosenblum, Katherine L; Muzik, Maria; Morelen, Diana M; Alfafara, Emily A; Miller, Nicole M; Waddell, Rachel M; Schuster, Melisa M; Ribaudo, Julie

2017-10-01

We conducted a study to evaluate the effectiveness of Mom Power, a multifamily parenting intervention to improve mental health and parenting among high-risk mothers with young children in a community-based randomized controlled trial (CB-RCT) design. Participants (N = 122) were high-risk mothers (e.g., interpersonal trauma histories, mental health problems, poverty) and their young children (age <6 years), randomized either to Mom Power, a parenting intervention (treatment condition), or weekly mailings of parenting information (control condition). In this study, the 13-session intervention was delivered by community clinicians trained to fidelity. Pre- and post-trial assessments included mothers' mental health symptoms, parenting stress and helplessness, and connection to care. Mom Power was delivered in the community with fidelity and had good uptake (>65%) despite the risk nature of the sample. Overall, we found improvements in mental health and parenting stress for Mom Power participants but not for controls; in contrast, control mothers increased in parent-child role reversal across the trial period. The benefits of Mom Power treatment (vs. control) were accentuated for mothers with interpersonal trauma histories. Results of this CB-RCT confirm the effectiveness of Mom Power for improving mental health and parenting outcomes for high-risk, trauma-exposed women with young children. ClinicalTrials.gov Identifier: NCT01554215.

The use of high-fidelity human patient simulation as an evaluative tool in the development of clinical research protocols and procedures.

PubMed

Wright, Melanie C; Taekman, Jeffrey M; Barber, Linda; Hobbs, Gene; Newman, Mark F; Stafford-Smith, Mark

2005-12-01

Errors in clinical research can be costly, in terms of patient safety, data integrity, and data collection. Data inaccuracy in early subjects of a clinical study may be associated with problems in the design of the protocol, procedures, and data collection tools. High-fidelity patient simulation centers provide an ideal environment to apply human-centered design to clinical trial development. A draft of a complex clinical protocol was designed, evaluated and modified using a high-fidelity human patient simulator in the Duke University Human Simulation and Patient Safety Center. The process included walk-throughs, detailed modifications of the protocol and development of procedural aids. Training of monitors and coordinators provided an opportunity for observation of performance that was used to identify further improvements to the protocol. Evaluative steps were used to design the research protocol and procedures. Iterative modifications were made to the protocol and data collection tools. The success in use of human simulation in the preparation of a complex clinical drug trial suggests the benefits of human patient simulation extend beyond training and medical equipment evaluation. Human patient simulation can provide a context for informal expert evaluation of clinical protocol design and for formal "rehearsal" to evaluate the efficacy of procedures and support tools.

Methodological quality of behavioural weight loss studies: a systematic review

PubMed Central

Lemon, S. C.; Wang, M. L.; Haughton, C. F.; Estabrook, D. P.; Frisard, C. F.; Pagoto, S. L.

2018-01-01

Summary This systematic review assessed the methodological quality of behavioural weight loss intervention studies conducted among adults and associations between quality and statistically significant weight loss outcome, strength of intervention effectiveness and sample size. Searches for trials published between January, 2009 and December, 2014 were conducted using PUBMED, MEDLINE and PSYCINFO and identified ninety studies. Methodological quality indicators included study design, anthropometric measurement approach, sample size calculations, intent-to-treat (ITT) analysis, loss to follow-up rate, missing data strategy, sampling strategy, report of treatment receipt and report of intervention fidelity (mean = 6.3). Indicators most commonly utilized included randomized design (100%), objectively measured anthropometrics (96.7%), ITT analysis (86.7%) and reporting treatment adherence (76.7%). Most studies (62.2%) had a follow-up rate >75% and reported a loss to follow-up analytic strategy or minimal missing data (69.9%). Describing intervention fidelity (34.4%) and sampling from a known population (41.1%) were least common. Methodological quality was not associated with reporting a statistically significant result, effect size or sample size. This review found the published literature of behavioural weight loss trials to be of high quality for specific indicators, including study design and measurement. Identified for improvement include utilization of more rigorous statistical approaches to loss to follow up and better fidelity reporting. PMID:27071775

Study protocol: implementation of a computer-assisted intervention for autism in schools: a hybrid type II cluster randomized effectiveness-implementation trial.

PubMed

Pellecchia, Melanie; Beidas, Rinad S; Marcus, Steven C; Fishman, Jessica; Kimberly, John R; Cannuscio, Carolyn C; Reisinger, Erica M; Rump, Keiran; Mandell, David S

2016-11-25

The number of children diagnosed with autism has rapidly outpaced the capacities of many public school systems to serve them, especially under-resourced, urban school districts. The intensive nature of evidence-based autism interventions, which rely heavily on one-to-one delivery, has caused schools to turn to computer-assisted interventions (CAI). There is little evidence regarding the feasibility, effectiveness, and implementation of CAI in public schools. While CAI has the potential to increase instructional time for students with autism, it may also result in unintended consequences such as reduction in the amount of interpersonal (as opposed to computerized) instruction students receive. The purpose of this study is to test the effectiveness of one such CAI-TeachTown-its implementation, and its effects on teachers' use of other evidence-based practices. This study protocol describes a type II hybrid cluster randomized effectiveness-implementation trial. We will train and coach 70 teachers in autism support classrooms in one large school district in the use of evidence-based practices for students with autism. Half of the teachers then will be randomly selected to receive training and access to TeachTown: Basics, a CAI for students with autism, for the students in their classrooms. The study examines: (1) the effectiveness of TeachTown for students with autism; (2) the extent to which teachers implement TeachTown the way it was designed (i.e., fidelity); and (3) whether its uptake increases or reduces the use of other evidence-based practices. This study will examine the implementation of new technology for children with ASD in public schools and will be the first to measure the effectiveness of CAI. As importantly, the study will investigate whether adding a new technology on top of existing practices increases or decreases their use. This study presents a unique method to studying both the implementation and exnovation of evidence-based practices for children with autism in school settings. NCT02695693 . Retrospectively registered on July 8, 2016.

Intermediate Fidelity Closed Brayton Cycle Power Conversion Model

NASA Technical Reports Server (NTRS)

Lavelle, Thomas M.; Khandelwal, Suresh; Owen, Albert K.

2006-01-01

This paper describes the implementation of an intermediate fidelity model of a closed Brayton Cycle power conversion system (Closed Cycle System Simulation). The simulation is developed within the Numerical Propulsion Simulation System architecture using component elements from earlier models. Of particular interest, and power, is the ability of this new simulation system to initiate a more detailed analysis of compressor and turbine components automatically and to incorporate the overall results into the general system simulation.

Demonstration of universal parametric entangling gates on a multi-qubit lattice

PubMed Central

Reagor, Matthew; Osborn, Christopher B.; Tezak, Nikolas; Staley, Alexa; Prawiroatmodjo, Guenevere; Scheer, Michael; Alidoust, Nasser; Sete, Eyob A.; Didier, Nicolas; da Silva, Marcus P.; Acala, Ezer; Angeles, Joel; Bestwick, Andrew; Block, Maxwell; Bloom, Benjamin; Bradley, Adam; Bui, Catvu; Caldwell, Shane; Capelluto, Lauren; Chilcott, Rick; Cordova, Jeff; Crossman, Genya; Curtis, Michael; Deshpande, Saniya; El Bouayadi, Tristan; Girshovich, Daniel; Hong, Sabrina; Hudson, Alex; Karalekas, Peter; Kuang, Kat; Lenihan, Michael; Manenti, Riccardo; Manning, Thomas; Marshall, Jayss; Mohan, Yuvraj; O’Brien, William; Otterbach, Johannes; Papageorge, Alexander; Paquette, Jean-Philip; Pelstring, Michael; Polloreno, Anthony; Rawat, Vijay; Ryan, Colm A.; Renzas, Russ; Rubin, Nick; Russel, Damon; Rust, Michael; Scarabelli, Diego; Selvanayagam, Michael; Sinclair, Rodney; Smith, Robert; Suska, Mark; To, Ting-Wai; Vahidpour, Mehrnoosh; Vodrahalli, Nagesh; Whyland, Tyler; Yadav, Kamal; Zeng, William; Rigetti, Chad T.

2018-01-01

We show that parametric coupling techniques can be used to generate selective entangling interactions for multi-qubit processors. By inducing coherent population exchange between adjacent qubits under frequency modulation, we implement a universal gate set for a linear array of four superconducting qubits. An average process fidelity of ℱ = 93% is estimated for three two-qubit gates via quantum process tomography. We establish the suitability of these techniques for computation by preparing a four-qubit maximally entangled state and comparing the estimated state fidelity with the expected performance of the individual entangling gates. In addition, we prepare an eight-qubit register in all possible bitstring permutations and monitor the fidelity of a two-qubit gate across one pair of these qubits. Across all these permutations, an average fidelity of ℱ = 91.6 ± 2.6% is observed. These results thus offer a path to a scalable architecture with high selectivity and low cross-talk. PMID:29423443

Engineering integrated photonics for heralded quantum gates

NASA Astrophysics Data System (ADS)

Meany, Thomas; Biggerstaff, Devon N.; Broome, Matthew A.; Fedrizzi, Alessandro; Delanty, Michael; Steel, M. J.; Gilchrist, Alexei; Marshall, Graham D.; White, Andrew G.; Withford, Michael J.

2016-06-01

Scaling up linear-optics quantum computing will require multi-photon gates which are compact, phase-stable, exhibit excellent quantum interference, and have success heralded by the detection of ancillary photons. We investigate the design, fabrication and characterisation of the optimal known gate scheme which meets these requirements: the Knill controlled-Z gate, implemented in integrated laser-written waveguide arrays. We show device performance to be less sensitive to phase variations in the circuit than to small deviations in the coupler reflectivity, which are expected given the tolerance values of the fabrication method. The mode fidelity is also shown to be less sensitive to reflectivity and phase errors than the process fidelity. Our best device achieves a fidelity of 0.931 ± 0.001 with the ideal 4 × 4 unitary circuit and a process fidelity of 0.680 ± 0.005 with the ideal computational-basis process.

Engineering integrated photonics for heralded quantum gates

PubMed Central

Meany, Thomas; Biggerstaff, Devon N.; Broome, Matthew A.; Fedrizzi, Alessandro; Delanty, Michael; Steel, M. J.; Gilchrist, Alexei; Marshall, Graham D.; White, Andrew G.; Withford, Michael J.

2016-01-01

Scaling up linear-optics quantum computing will require multi-photon gates which are compact, phase-stable, exhibit excellent quantum interference, and have success heralded by the detection of ancillary photons. We investigate the design, fabrication and characterisation of the optimal known gate scheme which meets these requirements: the Knill controlled-Z gate, implemented in integrated laser-written waveguide arrays. We show device performance to be less sensitive to phase variations in the circuit than to small deviations in the coupler reflectivity, which are expected given the tolerance values of the fabrication method. The mode fidelity is also shown to be less sensitive to reflectivity and phase errors than the process fidelity. Our best device achieves a fidelity of 0.931 ± 0.001 with the ideal 4 × 4 unitary circuit and a process fidelity of 0.680 ± 0.005 with the ideal computational-basis process. PMID:27282928

Engineering integrated photonics for heralded quantum gates.

PubMed

Meany, Thomas; Biggerstaff, Devon N; Broome, Matthew A; Fedrizzi, Alessandro; Delanty, Michael; Steel, M J; Gilchrist, Alexei; Marshall, Graham D; White, Andrew G; Withford, Michael J

2016-06-10

Scaling up linear-optics quantum computing will require multi-photon gates which are compact, phase-stable, exhibit excellent quantum interference, and have success heralded by the detection of ancillary photons. We investigate the design, fabrication and characterisation of the optimal known gate scheme which meets these requirements: the Knill controlled-Z gate, implemented in integrated laser-written waveguide arrays. We show device performance to be less sensitive to phase variations in the circuit than to small deviations in the coupler reflectivity, which are expected given the tolerance values of the fabrication method. The mode fidelity is also shown to be less sensitive to reflectivity and phase errors than the process fidelity. Our best device achieves a fidelity of 0.931 ± 0.001 with the ideal 4 × 4 unitary circuit and a process fidelity of 0.680 ± 0.005 with the ideal computational-basis process.

High fidelity quantum teleportation assistance with quantum neural network

NASA Astrophysics Data System (ADS)

Huang, Chunhui; Wu, Bichun

2014-09-01

In this paper, a high fidelity scheme of quantum teleportation based on quantum neural network (QNN) is proposed. The QNN is composed of multi-bit control-not gates. The quantum teleportation of a qubit state via two-qubit entangled channels is investigated by solving the master equation in Lindblad operators with a noisy environment. To ensure the security of quantum teleportation, the indirect training of QNN is employed. Only 10% of teleported information is extracted for the training of QNN parameters. Then the outputs are corrected by the other QNN at Bob's side. We build a random series of numbers ranged in [0, π] as inputs and simulate the properties of our teleportation scheme. The results show that the fidelity of quantum teleportation system is significantly improved to approach 1 by the error-correction of QNN. It illustrates that the distortion can be eliminated perfectly and the high fidelity of quantum teleportation could be implemented.

Informed shared decision-making programme on the prevention of myocardial infarction in type 2 diabetes: a randomised controlled trial.

PubMed

Buhse, Susanne; Mühlhauser, Ingrid; Heller, Tabitha; Kuniss, Nadine; Müller, Ulrich Alfons; Kasper, Jürgen; Lehmann, Thomas; Lenz, Matthias

2015-11-13

To evaluate an informed shared decision-making programme (ISDM-P) for people with type 2 diabetes under high fidelity conditions. Randomised, single-blinded trial with sham control intervention and follow-up of 6 months. Single-centre diabetes clinic providing care according to the national disease management programme in Germany. 154 people with type 2 diabetes without diagnosis of ischaemic heart disease or stroke. The ISDM-P is executed by diabetes educators. Core component is a patient decision aid on the prevention of myocardial infarction supplemented by a 90 min group teaching session. The structurally equivalent control intervention addresses stress issues. Primary outcome was risk comprehension, including realistic expectations about benefits and harms of interventions. It was assessed by a 12-item questionnaire after the teaching session when patients set and prioritise their treatment goals. Key secondary outcome was adherence to treatment goals, operationalised as achievement of individual goals and medication uptake. ISDM-P teaching sessions were video-taped to monitor intervention fidelity. 72 of 77 ISDM-P and 71 of 77 control patients completed the questionnaire (score 0-12). ISDM-P patients achieved higher levels of risk comprehension, mean score 8.25 vs 2.62, difference 5.63 (95% CI 4.82 to 6.44), and realistic expectations (score 0-6), 4.51 vs 0.85, 3.67 (3.23 to 4.11). More ISDM-P patients wished to take statins, 59.2% vs 30.4%, 28.7% (12.9% to 44.5%); more prioritised blood pressure control, 51.4% vs 25.7%, and fewer intensive glucose control, 33.3% vs 60%, p=0.002. More ISDM-P patients achieved their glycated haemoglobin goals, 95.8% vs 85.7%, 10.1% (0.6% to 19.5%). Achievement of prioritised goals and medication uptake were comparable between groups. The ISDM-P on preventive measures in type 2 diabetes was effective under high fidelity conditions. Involvement of diabetes educators may facilitate implementation of the informed shared decision-making. ISRCTN84636255. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

DOE Office of Scientific and Technical Information (OSTI.GOV)

Wang Yinan; Shi Handuo; Xiong Zhaoxi

We present a unified universal quantum cloning machine, which combines several different existing universal cloning machines together, including the asymmetric case. In this unified framework, the identical pure states are projected equally into each copy initially constituted by input and one half of the maximally entangled states. We show explicitly that the output states of those universal cloning machines are the same. One importance of this unified cloning machine is that the cloning procession is always the symmetric projection, which reduces dramatically the difficulties for implementation. Also, it is found that this unified cloning machine can be directly modified tomore » the general asymmetric case. Besides the global fidelity and the single-copy fidelity, we also present all possible arbitrary-copy fidelities.« less

Verification and validation of a reliable multicast protocol

NASA Technical Reports Server (NTRS)

Callahan, John R.; Montgomery, Todd L.

1995-01-01

This paper describes the methods used to specify and implement a complex communications protocol that provides reliable delivery of data in multicast-capable, packet-switching telecommunication networks. The protocol, called the Reliable Multicasting Protocol (RMP), was developed incrementally by two complementary teams using a combination of formal and informal techniques in an attempt to ensure the correctness of the protocol implementation. The first team, called the Design team, initially specified protocol requirements using a variant of SCR requirements tables and implemented a prototype solution. The second team, called the V&V team, developed a state model based on the requirements tables and derived test cases from these tables to exercise the implementation. In a series of iterative steps, the Design team added new functionality to the implementation while the V&V team kept the state model in fidelity with the implementation through testing. Test cases derived from state transition paths in the formal model formed the dialogue between teams during development and served as the vehicles for keeping the model and implementation in fidelity with each other. This paper describes our experiences in developing our process model, details of our approach, and some example problems found during the development of RMP.

Improving oxygen therapy for children and neonates in secondary hospitals in Nigeria: study protocol for a stepped-wedge cluster randomised trial.

PubMed

Graham, Hamish R; Ayede, Adejumoke I; Bakare, Ayobami A; Oyewole, Oladapo B; Peel, David; Gray, Amy; McPake, Barbara; Neal, Eleanor; Qazi, Shamim; Izadnegahdar, Rasa; Falade, Adegoke G; Duke, Trevor

2017-10-27

Oxygen is a life-saving, essential medicine that is important for the treatment of many common childhood conditions. Improved oxygen systems can reduce childhood pneumonia mortality substantially. However, providing oxygen to children is challenging, especially in small hospitals with weak infrastructure and low human resource capacity. This trial will evaluate the implementation of improved oxygen systems at secondary-level hospitals in southwest Nigeria. The improved oxygen system includes: a standardised equipment package; training of clinical and technical staff; infrastructure support (including improved power supply); and quality improvement activities such as supportive supervision. Phase 1 will involve the introduction of pulse oximetry alone; phase 2 will involve the introduction of the full, improved oxygen system package. We have based the intervention design on a theory-based analysis of previous oxygen projects, and used quality improvement principles, evidence-based teaching methods, and behaviour-change strategies. We are using a stepped-wedge cluster randomised design with participating hospitals randomised to receive an improved oxygen system at 4-month steps (three hospitals per step). Our mixed-methods evaluation will evaluate effectiveness, impact, sustainability, process and fidelity. Our primary outcome measures are childhood pneumonia case fatality rate and inpatient neonatal mortality rate. Secondary outcome measures include a range of clinical, quality of care, technical, and health systems outcomes. The planned study duration is from 2015 to 2018. Our study will provide quality evidence on the effectiveness of improved oxygen systems, and how to better implement and scale-up oxygen systems in resource-limited settings. Our results should have important implications for policy-makers, hospital administrators, and child health organisations in Africa and globally. Australian New Zealand Clinical Trials Registry: ACTRN12617000341325 . Retrospectively registered on 6 March 2017.

How Educators Perceive the Process and Implementation of a Multi-Tiered System of Supports (MTSS): A Case Study

ERIC Educational Resources Information Center

Maniglia, Linda M.

2017-01-01

In this qualitative case study the researcher examined educators' perceptions of an RTI/MTSS model of instruction and how staff perceptions relate to fidelity of implementation. The purpose of this research was to investigate educators' understanding of the critical components necessary to support implementation of RTI/MTSS framework. The…

Data Decision-Making and Program-Wide Implementation of the Pyramid Model. Roadmap to Effective Intervention Practices #7

ERIC Educational Resources Information Center

Fox, Lise; Veguilla, Myrna; Perez Binder, Denise

2014-01-01

The Technical Assistance Center on Social Emotional Intervention for Young Children (TACSEI) Roadmap on "Data Decision-Making and Program-Wide Implementation of the Pyramid Model" provides programs with guidance on how to collect and use data to ensure the implementation of the Pyramid Model with fidelity and decision-making that…

The Implementation of Content Enhancement Routines for Improved Content Literacy for Middle and Secondary Social Studies Students

ERIC Educational Resources Information Center

Little, Mary E.; Hahs-Vaughn, Debbie L.

2007-01-01

The increased demands for access to and accountability for mastery of the general education curriculum by all students, including students with disabilities, is a current reality within schools. A number of factors have been suggested to promote school reform, classroom implementation, and increase the rates of high fidelity implementation of…

Effects of Coaching on Teachers' Implementation of Tier 1 School-Wide Positive Behavioral Interventions and Support Strategies

ERIC Educational Resources Information Center

Bethune, Keri S.

2017-01-01

Fidelity of implementation of School-Wide Positive Behavioral Interventions and Supports (SWPBIS) procedures within schools is critical to the success of the program. Coaching has been suggested as one approach to helping ensure accuracy of implementation of SWPBIS plans. This study used a multiple baseline across participants design to examine…

Predicting Abandonment of School-Wide Behavior Support Interventions

ERIC Educational Resources Information Center

Nese, Rhonda N. T.; McIntosh, Kent; Nese, Joseph F. T.; Ghemraoui, Adam; Bloom, Jerry; Johnson, Nanci W.; Phillips, Danielle; Richter, Mary F.; Hoselton, Robert

2016-01-01

This study examines predictors of abandonment of evidence-based practices through descriptive analyses of extant state-level training data, fidelity of implementation data, and nationally reported school demographic data across 915 schools in 3 states implementing school-wide positive behavioral interventions and supports (SWPBIS). Schools…

Parallel processes: using motivational interviewing as an implementation coaching strategy.

PubMed

Hettema, Jennifer E; Ernst, Denise; Williams, Jessica Roberts; Miller, Kristin J

2014-07-01

In addition to its clinical efficacy as a communication style for strengthening motivation and commitment to change, motivational interviewing (MI) has been hypothesized to be a potential tool for facilitating evidence-based practice adoption decisions. This paper reports on the rationale and content of MI-based implementation coaching Webinars that, as part of a larger active dissemination strategy, were found to be more effective than passive dissemination strategies at promoting adoption decisions among behavioral health and health providers and administrators. The Motivational Interviewing Treatment Integrity scale (MITI 3.1.1) was used to rate coaching Webinars from 17 community behavioral health organizations and 17 community health centers. The MITI coding system was found to be applicable to the coaching Webinars, and raters achieved high levels of agreement on global and behavior count measurements of fidelity to MI. Results revealed that implementation coaches maintained fidelity to the MI model, exceeding competency benchmarks for almost all measures. Findings suggest that it is feasible to implement MI as a coaching tool.

An empirical assessment of implementation, adaptation, and tailoring: the evaluation of CDC's National Diffusion of VOICES/VOCES.

PubMed

Harshbarger, Camilla; Simmons, Gretchen; Coelho, Helen; Sloop, Kira; Collins, Charles

2006-08-01

The Centers for Disease Control and Prevention (CDC), through its Diffusion of Effective Behavioral Interventions (DEBI) program, trained over 260 agencies on VOICES/VOCES between August 2003 and April 2005. ORC Macro conducted interviews with agency staff 3 months after receiving VOICES/VOCES training. This article discusses the diffusion of VOICES/VOCES; agencies' adoption, adaptation, and implementation of this intervention; and needs for ongoing proactive technical assistance (TA) for agencies to successfully integrate behavioral interventions into their programs. The vastmajority of agencies implemented VOICES/VOCES with fidelity to the core elements, and agencies successfully adapted the intervention to make it more appealing to target populations. TA is needed for interventions to be successfully adapted and implemented with fidelity to the core elements, and to ensure program sustainability. More effective interventions of short duration and minimum complexity to easily match with existing resources and conditions of agency capacity among HIV prevention providers in the community are needed.

Relationship Between Implementation of Collaborative Strategic Reading and Student Outcomes for Adolescents With Disabilities.

PubMed

Boardman, Alison G; Buckley, Pamela; Vaughn, Sharon; Roberts, Gregory; Scornavacco, Karla; Klingner, Janette K

2016-11-01

This study examines the interaction between the fidelity of implementation of a set of research-based strategies-Collaborative Strategic Reading (CSR)-and outcomes for students with mild to moderate disabilities using data from two nonoverlapping studies in middle school language arts and reading classrooms (Study 1) and middle school social studies and science classrooms (Study 2). The authors use a definition of fidelity that includes both the amount of CSR instruction delivered by teachers and the quality of implementation. Although there were no main effects for quality or amount of CSR instruction, in both studies there was an interaction effect between quality of implementation and special education status. The study used a within-groups design and multilevel analyses, and the results demonstrate that higher quality CSR instruction was associated with higher reading outcomes for students with disabilities. This finding was consistent across Study 1 and Study 2. Implications for practice and future research are discussed. © Hammill Institute on Disabilities 2016.

The current implementation status of the integration of sports and physical activity into Dutch rehabilitation care.

PubMed

Hoekstra, Femke; Hettinga, Florentina J; Alingh, Rolinde A; Duijf, Marjo; Dekker, Rienk; van der Woude, Lucas H V; van der Schans, Cees P

2017-01-01

To describe the current status of the nationwide implementation process of a sports and physical activity stimulation programme to gain insight into how sports and physical activity were integrated into Dutch rehabilitation care. The current implementation status of a sports and physical activity stimulation programme in 12 rehabilitation centres and 5 hospitals with a rehabilitation department was described by scoring fidelity and satisfaction. Seventy-one rehabilitation professionals filled out a questionnaire on how sports and physical activity, including stimulation activities, were implemented into rehabilitation care. Total fidelity scores (in %) were calculated for each organization. Professionals' satisfaction was rated on a scale from 1 to 10. In most organizations sports and physical activity were to some extent integrated during and after rehabilitation (fidelity scores: median = 54%, IQR = 23%). Physical activity stimulation was not always embedded as standard component of a rehabilitation treatment. Professionals' satisfaction rated a median value of 8.0 (IQR = 0.0) indicating high satisfaction rates. The fidelity outcome showed that activities to stimulate sports and physical activity during and after rehabilitation were integrated into rehabilitation care, but not always delivered as standardized component. These findings have emphasized the importance to focus on integrating these activities into routines of organizations. Implications for Rehabilitation Components of an evidence-based programme to stimulate sports and physical activity during and after rehabilitation can be used to measure the current status of the integration of sports and physical activity in rehabilitation care in a structural and effective way. The method described in the current study can be used to compare the content of the rehabilitation care regarding the integration of sports and physical activity among organizations both on a national and international level. Sports and physical activity are seen as important ingredients for successful rehabilitation care in The Netherlands.

Impact of a Web-Based Electronic Health Record on Behavioral Health Service Delivery for Children and Adolescents: Randomized Controlled Trial.

PubMed

Bruns, Eric J; Hook, Alyssa N; Parker, Elizabeth M; Esposito, Isabella; Sather, April; Parigoris, Ryan M; Lyon, Aaron R; Hyde, Kelly L

2018-06-14

Electronic health records (EHRs) have been widely proposed as a mechanism for improving health care quality. However, rigorous research on the impact of EHR systems on behavioral health service delivery is scant, especially for children and adolescents. The current study evaluated the usability of an EHR developed to support the implementation of the Wraparound care coordination model for children and youth with complex behavioral health needs, and impact of the EHR on service processes, fidelity, and proximal outcomes. Thirty-four Wraparound facilitators working in two programs in two states were randomized to either use the new EHR (19/34, 56%) or to continue to implement Wraparound services as usual (SAU) using paper-based documentation (15/34, 44%). Key functions of the EHR included standard fields such as youth and family information, diagnoses, assessment data, and progress notes. In addition, there was the maintenance of a coordinated plan of care, progress measurement on strategies and services, communication among team members, and reporting on services, expenditures, and outcomes. All children and youth referred to services for eight months (N=211) were eligible for the study. After excluding those who were ineligible (69/211, 33%) and who declined to participate (59/211, 28%), a total of 83/211 (39%) children and youth were enrolled in the study with 49/211 (23%) in the EHR condition and 34/211 (16%) in the SAU condition. Facilitators serving these youth and families and their supervisors completed measures of EHR usability and appropriateness, supervision processes and activities, work satisfaction, and use of and attitudes toward standardized assessments. Data from facilitators were collected by web survey and, where necessary, by phone interviews. Parents and caregivers completed measures via phone interviews. Related to fidelity and quality of behavioral health care, including Wraparound team climate, working alliance with providers, fidelity to the Wraparound model, and satisfaction with services. EHR-assigned facilitators from both sites demonstrated the robust use of the system. Facilitators in the EHR group reported spending significantly more time reviewing client progress (P=.03) in supervision, and less time overall sending reminders to youth/families (P=.04). A trend toward less time on administrative tasks (P=.098) in supervision was also found. Facilitators in both groups reported significantly increased use of measurement-based care strategies overall, which may reflect cross-group contamination (given that randomization of staff to the EHR occurred within agencies and supervisors supervised both types of staff). Although not significant at P<.05, there was a trend (P=.10) toward caregivers in the EHR group reporting poorer shared agreement on tasks on the measure of working alliance with providers. No other significant between-group differences were found. Results support the proposal that use of EHR systems can promote the use of client progress data and promote efficiency; however, there was little evidence of any impact (positive or negative) on overall service quality, fidelity, or client satisfaction. The field of children's behavioral health services would benefit from additional research on EHR systems using designs that include larger sample sizes and longer follow-up periods. ClinicalTrials.gov NCT02421874; https://clinicaltrials.gov/ct2/show/NCT02421874 (Archived by WebCite at http://www.webcitation.org/6yyGPJ3NA). ©Eric J Bruns, Alyssa N Hook, Elizabeth M Parker, Isabella Esposito, April Sather, Ryan M Parigoris, Aaron R Lyon, Kelly L Hyde. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 14.06.2018.

High-Fidelity Simulation for Advanced Cardiac Life Support Training

PubMed Central

Davis, Lindsay E.; Storjohann, Tara D.; Spiegel, Jacqueline J.; Beiber, Kellie M.

2013-01-01

Objective. To determine whether a high-fidelity simulation technique compared with lecture would produce greater improvement in advanced cardiac life support (ACLS) knowledge, confidence, and overall satisfaction with the training method. Design. This sequential, parallel-group, crossover trial randomized students into 2 groups distinguished by the sequence of teaching technique delivered for ACLS instruction (ie, classroom lecture vs high-fidelity simulation exercise). Assessment. Test scores on a written examination administered at baseline and after each teaching technique improved significantly from baseline in all groups but were highest when lecture was followed by simulation. Simulation was associated with a greater degree of overall student satisfaction compared with lecture. Participation in a simulation exercise did not improve pharmacy students’ knowledge of ACLS more than attending a lecture, but it was associated with improved student confidence in skills and satisfaction with learning and application. Conclusions. College curricula should incorporate simulation to complement but not replace lecture for ACLS education. PMID:23610477

Adolescent and parent motivation for change affects psychotherapy outcomes among youth with poorly controlled diabetes.

PubMed

Ellis, Deborah A; Berio, Heidi; Carcone, April Idalski; Naar-King, Sylvie

2012-01-01

Investigate effect of baseline motivation for change on treatment fidelity, therapeutic alliance, treatment dose, and treatment outcome in a randomized controlled trial of family therapy for youth with poorly controlled diabetes. Seventy-four adolescents and caregivers completed measures of motivation for change. Measures of fidelity, alliance, dose, and youth health status were collected. Structural equation modeling was used to test the direct and indirect effects of motivation on treatment outcomes. Parent motivation was significantly related to alliance and fidelity. Only alliance was significantly related to posttreatment metabolic control. In adolescent models, only motivation was significantly related to alliance. In both models, motivation had a significant indirect effect on metabolic control through alliance. Findings demonstrate the importance of parent and youth initial motivational status and treatment alliance to treatment outcome among youth with poorly controlled diabetes. Additional research on treatment techniques that promote motivation for change is needed.

High-fidelity simulation for advanced cardiac life support training.

PubMed

Davis, Lindsay E; Storjohann, Tara D; Spiegel, Jacqueline J; Beiber, Kellie M; Barletta, Jeffrey F

2013-04-12

OBJECTIVE. To determine whether a high-fidelity simulation technique compared with lecture would produce greater improvement in advanced cardiac life support (ACLS) knowledge, confidence, and overall satisfaction with the training method. DESIGN. This sequential, parallel-group, crossover trial randomized students into 2 groups distinguished by the sequence of teaching technique delivered for ACLS instruction (ie, classroom lecture vs high-fidelity simulation exercise). ASSESSMENT. Test scores on a written examination administered at baseline and after each teaching technique improved significantly from baseline in all groups but were highest when lecture was followed by simulation. Simulation was associated with a greater degree of overall student satisfaction compared with lecture. Participation in a simulation exercise did not improve pharmacy students' knowledge of ACLS more than attending a lecture, but it was associated with improved student confidence in skills and satisfaction with learning and application. CONCLUSIONS. College curricula should incorporate simulation to complement but not replace lecture for ACLS education.

Advancing methods for reliably assessing motivational interviewing fidelity using the Motivational Interviewing Skills Code

PubMed Central

Lord, Sarah Peregrine; Can, Doğan; Yi, Michael; Marin, Rebeca; Dunn, Christopher W.; Imel, Zac E.; Georgiou, Panayiotis; Narayanan, Shrikanth; Steyvers, Mark; Atkins, David C.

2014-01-01

The current paper presents novel methods for collecting MISC data and accurately assessing reliability of behavior codes at the level of the utterance. The MISC 2.1 was used to rate MI interviews from five randomized trials targeting alcohol and drug use. Sessions were coded at the utterance-level. Utterance-based coding reliability was estimated using three methods and compared to traditional reliability estimates of session tallies. Session-level reliability was generally higher compared to reliability using utterance-based codes, suggesting that typical methods for MISC reliability may be biased. These novel methods in MI fidelity data collection and reliability assessment provided rich data for therapist feedback and further analyses. Beyond implications for fidelity coding, utterance-level coding schemes may elucidate important elements in the counselor-client interaction that could inform theories of change and the practice of MI. PMID:25242192

Advancing methods for reliably assessing motivational interviewing fidelity using the motivational interviewing skills code.

PubMed

Lord, Sarah Peregrine; Can, Doğan; Yi, Michael; Marin, Rebeca; Dunn, Christopher W; Imel, Zac E; Georgiou, Panayiotis; Narayanan, Shrikanth; Steyvers, Mark; Atkins, David C

2015-02-01

The current paper presents novel methods for collecting MISC data and accurately assessing reliability of behavior codes at the level of the utterance. The MISC 2.1 was used to rate MI interviews from five randomized trials targeting alcohol and drug use. Sessions were coded at the utterance-level. Utterance-based coding reliability was estimated using three methods and compared to traditional reliability estimates of session tallies. Session-level reliability was generally higher compared to reliability using utterance-based codes, suggesting that typical methods for MISC reliability may be biased. These novel methods in MI fidelity data collection and reliability assessment provided rich data for therapist feedback and further analyses. Beyond implications for fidelity coding, utterance-level coding schemes may elucidate important elements in the counselor-client interaction that could inform theories of change and the practice of MI. Copyright © 2015 Elsevier Inc. All rights reserved.

Describing teacher-student interactions: a qualitative assessment of teacher implementation of the 7th grade keepin' it REAL substance use intervention.

PubMed

Pettigrew, Jonathan; Miller-Day, Michelle; Shin, Youngju; Hecht, Michael L; Krieger, Janice L; Graham, John W

2013-03-01

Variations in the delivery of school-based substance use prevention curricula affect students' acquisition of the lesson content and program outcomes. Although adaptation is sometimes viewed as a lack of fidelity, it is unclear what types of variations actually occur in the classroom. This observational study investigated teacher and student behaviors during implementation of a middle school-based drug prevention curriculum in 25 schools across two Midwestern states. Trained observers coded videos of 276 lessons, reflecting a total of 31 predominantly Caucasian teachers (10 males and 21 females) in 73 different classes. Employing qualitative coding procedures, the study provides a working typology of implementation patterns based on varying levels of teacher control and student participation. These patterns are fairly consistent across lessons and across classes of students, suggesting a teacher-driven delivery model where teachers create a set of constraints within which students vary their engagement. Findings provide a descriptive basis grounded in observation of classroom implementation that can be used to test models of implementation fidelity and quality as well as impact training and other dissemination research.

Findings From a Prospective Randomized Controlled Trial of an Individualized Music Listening Program for Persons With Dementia.

PubMed

Kwak, Jung; Anderson, Keith; O'Connell Valuch, Katharine

2018-06-01

Music & Memory (M&M) is a passive music intervention that uses personalized music playlists delivered on digital music players. This program has been increasingly adopted in nursing homes across the United States to facilitate communication, engagement, and socialization among persons with dementia (PWDs); however, few studies have evaluated the program's effect on PWDs' outcomes. In the present study, a randomized controlled crossover design was used to examine the impact of the M&M program on 59 PWDs in 10 nursing homes over a 14-week period. Residents' evaluated outcomes included agitation, behavioral symptoms, and use of psychotropic medications. Although trends supported the positive effects of M&M, no statistically significant differences were found in any of the outcomes measured over time. Methodological limitations withstanding, these findings call into question the effectiveness of the M&M program and the ability of facility staff to implement this intervention with fidelity.

High-Fidelity Simulation: Preparing Dental Hygiene Students for Managing Medical Emergencies.

PubMed

Bilich, Lisa A; Jackson, Sarah C; Bray, Brenda S; Willson, Megan N

2015-09-01

Medical emergencies can occur at any time in the dental office, so being prepared to properly manage the situation can be the difference between life and death. The entire dental team must be properly trained regarding all aspects of emergency management in the dental clinic. The aim of this study was to evaluate a new educational approach using a high-fidelity simulator to prepare dental hygiene students for medical emergencies. This study utilized high-fidelity simulation (HFS) to evaluate the abilities of junior dental hygiene students at Eastern Washington University to handle a medical emergency in the dental hygiene clinic. Students were given a medical emergency scenario requiring them to assess the emergency and implement life-saving protocols in a simulated "real-life" situation using a high-fidelity manikin. Retrospective data were collected for four years from the classes of 2010 through 2013 (N=114). The results indicated that learning with simulation was effective in helping the students identify the medical emergency in a timely manner, implement emergency procedures correctly, locate and correctly utilize contents of the emergency kit, administer appropriate intervention/treatment for a specific patient, and provide the patient with appropriate follow-up instructions. For dental hygiene programs seeking to enhance their curricula in the area of medical emergencies, this study suggests that HFS is an effective tool to prepare students to appropriately handle medical emergencies. Faculty calibration is essential to standardize simulation.

Informing the design of a randomised controlled trial of an exercise-based programme for long term stroke survivors: lessons from a before-and-after case series study

PubMed Central

2013-01-01

Background To inform the design of a randomised controlled trial (RCT) of an exercise-based programme for long term stroke survivors, we conducted a mixed methods before-and-after case series with assessment at three time points. We evaluated Action for Rehabilitation from Neurological Injury (ARNI), a personalised, functionally-focussed programme. It was delivered through 24 hours of one-to-one training by an Exercise Professional (EP), plus at least 2 hours weekly unsupervised exercise, over 12- 14 weeks. Assessment was by patient-rated questionnaires addressing function, physical activity, confidence, fatigue and health-related quality of life; objective assessment of gait quality and speed; qualitative individual interviews conducted with participants. Data were collected at baseline, 3 months and 6 months. Fidelity and acceptability was assessed by participant interviews, audit of participant and EP records, and observation of training. Findings Four of six enrolled participants completed the exercise programme. Quantitative data demonstrated little change across the sample, but marked changes on some measures for some individuals. Qualitative interviews suggested that small benefits in physical outcomes could be of great psychological significance to participants. Participant-reported fatigue levels commonly increased, and non-completers said they found the programme too demanding. Most key components of the intervention were delivered, but there were several potentially important departures from intervention fidelity. Discussion The study provided data and experience that are helping to inform the design of an RCT of this intervention. It suggested the need for a broader recruitment strategy; indicated areas that could be explored in more depth in the qualitative component of the trial; and highlighted issues that should be addressed to enhance and evaluate fidelity, particularly in the preparation and monitoring of intervention providers. The experience illustrates the value of even small sample before-and-after studies in the development of trials of complex interventions. PMID:23941470

Evaluation of the Coaches Educators training implementation of the PAPA project: A comparison between Norway and France.

PubMed

Van Hoye, A; Larsen, T; Sovik, M; Wold, B; Heuzé, J-P; Samdal, O; Ommundsen, Y; Sarrazin, P

2015-10-01

This study cross-culturally compares the implementation process of the Empowering Coaching™ training program to grassroots coaches (GCs) by trained Coach Educators (CEs) from the Promoting Adolescent Physical Activity project in Norway and France. This program targets children's health through coach training to create a more positive environment. Using the RE-AIM framework, indicators of implementation processes at staff level were defined for adoption, implementation, and maintenance. CEs (n = 18) were interviewed about their professional use of the training, and video-filmed delivering the intervention trial workshops. GCs (n = 185) responded to a questionnaire after the workshops. Results showed that CEs were keen to participate in this project (16/18; 89%), to stay involved (12/16; 75%), and to diffuse the principles after the research project (12/12; 100%). CE training seemed effective as results indicated that CEs applied the principles of the program during the workshops and covered a high degree of the content (78%). Cross-cultural differences were found as the French Football Federation CE dropped out of the program, as Norwegian CEs delivered the content with higher fidelity and had higher enjoyment and pace scores than French CEs. More indicators are needed to use the RE-AIM framework for cross-cultural comparison. © 2014 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

The Effects of Fluency Training on Implementation Fidelity of a Reading Intervention Conducted by Paraprofessionals

ERIC Educational Resources Information Center

O'Keeffe, Breda Victoria

2009-01-01

Improving educational outcomes involves many variables, including identifying effective interventions and ensuring that they are effectively implemented in schools. Within a "response to intervention" model, treatment integrity of academic interventions has become increasingly important. However, recent research has suggested that…

Predicting Abandonment of School-Wide Positive Behavioral Interventions and Supports

ERIC Educational Resources Information Center

Nese, Rhonda; McIntosh, Kent; Nese, Joseph; Hoselton, Robert; Bloom, Jerry; Johnson, Nanci; Richter, Mary; Phillips, Danielle; Ghemraoui, Adam

2016-01-01

This study examines predictors of abandonment of evidence-based practices through descriptive analyses of extant state-level training data, fidelity of implementation data, and nationally reported school demographic data across 915 schools in three states implementing school-wide positive behavioral interventions and supports (SWPBIS). Schools…

Process evaluation of Internet-based cognitive behavioural therapy for adults with tinnitus in the context of a randomised control trial.

PubMed

Beukes, Eldré W; Manchaiah, Vinaya; Baguley, David M; Allen, Peter M; Andersson, Gerhard

2018-02-01

The research objective was to identify processes that could either facilitate or hinder clinical implementation of an Internet-based cognitive behavioural therapy intervention for tinnitus in the UK. This was done by exploring the research context, the intervention components and the factors that contributed to the outcomes obtained. This study investigated eight processes including the recruitment strategies, reach, research context, treatment dose delivered and received, implementation fidelity, barriers to implementation and effectiveness of the intervention. Of the 169 registered participants, 146 were randomly assigned to the experimental or control groups (23 were excluded). The mean age was 55.57 years with an average tinnitus duration of 11.63 years. The intended sample of people with distressing tinnitus who were underserved with evidence-based tinnitus interventions was reached. The full guided intervention was delivered. The recommended modules were read more than the optional modules. Intervention components such as the easily readable format and the benefits of the applied relaxation programme facilitated significant positive post-intervention outcomes. Barriers hampering the intervention application included time pressures and low self-motivation. Results of this process evaluation together with the outcome data can be used to facilitate translating this research into clinical practice.

From Biological to Program Efficacy: Promoting Dialogue among the Research, Policy, and Program Communities12

PubMed Central

Habicht, Jean-Pierre; Pelto, Gretel H.

2014-01-01

The biological efficacy of nutritional supplements to complement usual diets in poor populations is well established. This knowledge rests on decades of methodologic research development and, more recently, on codification of methods to compile and interpret results across studies. The challenge now is to develop implementation (delivery) science knowledge and achieve a similar consensus on efficacy criteria for the delivery of these nutrients by public health and other organizations. This requires analysis of the major policy instruments for delivery and well-designed program delivery studies that examine the flow of a nutrient through a program impact pathway. This article discusses the differences between biological and program efficacy, and why elucidating the fidelity of delivery along the program impact pathways is essential for implementing a program efficacy trial and for assessing its internal and external validity. Research on program efficacy is expanding, but there is a lack of adequate frameworks to facilitate the process of harmonizing concepts and vocabulary, which is essential for communication among scientists, policy planners, and program implementers. There is an urgent need to elaborate these frameworks at national and program levels not only for program efficacy studies but also for the broader research agenda to support and improve the science of delivering adequate nutrition to those who need it most. PMID:24425719

Barriers to implementing the DSM-5 cultural formulation interview: a qualitative study.

PubMed

Aggarwal, Neil Krishan; Nicasio, Andel Veronica; DeSilva, Ravi; Boiler, Marit; Lewis-Fernández, Roberto

2013-09-01

The Outline for Cultural Formulation (OCF) in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) marked an attempt to apply anthropological concepts within psychiatry. The OCF has been criticized for not providing guidelines to clinicians. The DSM-5 Cultural Issues Subgroup has since converted the OCF into the Cultural Formulation Interview (CFI) for use by any clinician with any patient in any clinical setting. This paper presents perceived barriers to CFI implementation in clinical practice reported by patients (n = 32) and clinicians (n = 7) at the New York site within the DSM-5 international field trial. We used an implementation fidelity paradigm to code debriefing interviews after each CFI session through deductive content analysis. The most frequent patient threats were lack of differentiation from other treatments, lack of buy-in, ambiguity of design, over-standardization of the CFI, and severity of illness. The most frequent clinician threats were lack of conceptual relevance between intervention and problem, drift from the format, repetition, severity of patient illness, and lack of clinician buy-in. The Subgroup has revised the CFI based on these barriers for final publication in DSM-5. Our findings expand knowledge on the cultural formulation by reporting the CFI's reception among patients and clinicians.

Barriers to Implementing the DSM-5 Cultural Formulation Interview: A Qualitative Study

PubMed Central

Aggarwal, Neil Krishan; Nicasio, Andel Veronica; DeSilva, Ravi; Boiler, Marit; Lewis-Fernández, Roberto

2015-01-01

The Outline for Cultural Formulation (OCF) in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) marked an attempt to apply anthropological concepts within psychiatry. The OCF has been criticized for not providing guidelines to clinicians. The DSM-5 Cultural Issues Subgroup has since converted the OCF into the Cultural Formulation Interview (CFI) for use by any clinician with any patient in any clinical setting. This paper presents perceived barriers to CFI implementation in clinical practice reported by patients (n=32) and clinicians (n=7) at the New York site within the DSM-5 international field trial. We used an implementation fidelity paradigm to code debriefing interviews after each CFI session through deductive content analysis. The most frequent patient threats were lack of differentiation from other treatments, lack of buy-in, ambiguity of design, over-standardization of the CFI, and severity of illness. The most frequent clinician threats were lack of conceptual relevance between intervention and problem, drift from the format, repetition, severity of patient illness, and lack of clinician buy-in. The Subgroup has revised the CFI based on these barriers for final publication in DSM-5. Our findings expand knowledge on the cultural formulation by reporting the CFI’s reception among patients and clinicians. PMID:23836098

Effectiveness of cognitive-behavioral trauma treatment for incarcerated women with mental illnesses and substance abuse disorders.

PubMed

Wolff, Nancy; Frueh, B Christopher; Shi, Jing; Schumann, Brooke E

2012-10-01

An open trial design was used to examine the implementation and effectiveness of a cognitive-behavioral intervention (Seeking Safety) for comorbid post-traumatic stress disorder (PTSD) and substance use disorder (SUD) for incarcerated women with Axis I mental disorders who self-referred for specialty trauma treatment. The study sample was female inmates aged 18 and old who were primarily minority, had experienced childhood-based trauma, committed violent crimes, had a serious mental illness, and resided in maximum, medium, and minimum compounds of a women's prison. A total of 74 women completed the group intervention, with the average attending 23 of the 28 sessions (82%). Implementation feasibility was demonstrated by the ability to recruit, screen, assign, and retain participation. Effectiveness was supported by changes pre-post intervention on the PTSD Checklist (ES=0.56) and Global Severity Index (ES=0.47). Of the 19 completers with PCL scores of 50 or higher pre-intervention, 16 (84%) had scores below 50, the "cut score" consistent with or supportive of a PTSD diagnosis. Three-quarters or more of participants reported that Seeking Safety was helpful in each of the following areas: overall, for traumatic stress symptoms, for substance use, to focus on safety, and to learn safe coping skills. Future directions include the need for larger scale randomized controlled trials in medium or maximum security prisons and fidelity evaluations of non-research dissemination efforts. Published by Elsevier Ltd.

Providing activity for people with dementia in care homes: a cluster randomised controlled trial.

PubMed

Wenborn, Jennifer; Challis, David; Head, Jenny; Miranda-Castillo, Claudia; Popham, Carolyn; Thakur, Ruchi; Illes, Jacqueline; Orrell, Martin

2013-12-01

Activity levels remain low in care homes, but activity engagement can enhance residents' quality of life. This study aimed to assess an occupational therapy programme designed to enable care home staff to increase activity provision. A cluster randomised controlled trial with blinded assessment of outcome was conducted. A total of 210 residents with dementia in 16 care homes were recruited. Intervention homes received the programme, and control homes were provided usual care. Primary outcome is quality of life; secondary measures are dependency, challenging behaviour, depression, anxiety, severity of dementia and number and type of medication. Quality of life decreased overall with statistically significant change in staff ratings (p < 0.001). At follow-up, staff-rated quality of life was slightly lower in the intervention group (mean difference in staff ratings = -1.91, 95% CI -3.39 to -0.43, p = 0.01). There were no significant differences between the two groups for self-rated quality of life or secondary outcomes. The results may be related to the following: wide variability in how the intervention was implemented, such as low staff attendance at the education and coaching sessions, and patchy provision of additional activities to residents; or the residents' severity of dementia or the choice of outcome measures. Future studies need to pay more attention to process measures such as implementation and fidelity strategies, and outcome measures that better capture the focus of the intervention such as level of engagement and activity. Copyright © 2013 John Wiley & Sons, Ltd.

Implementation of quantum logic gates via Stark-tuned Förster resonance in Rydberg atoms

NASA Astrophysics Data System (ADS)

Huang, Xi-Rong; Hu, Chang-Sheng; Shen, Li-Tuo; Yang, Zhen-Biao; Wu, Huai-Zhi

2018-02-01

We present a scheme for implementation of controlled-Z and controlled-NOT gates via rapid adiabatic passage and Stark-tuned Förster resonance. By sweeping the Förster resonance once without passing through it and adiabatically tuning the angle-dependent Rydberg-Rydberg interaction of the dipolar nature, the system can be effectively described by a two-level system with the adiabatic theorem. The single adiabatic passage leads to a gate fidelity as high as 0.999 and a greatly reduced gate operation time. We investigate the scheme by considering an actual atomic level configuration with rubidium atoms, where the fidelity of the controlled-Z gate is still higher than 0.99 under the influence of the Zeeman effect.

Implementing Response to Intervention: Use of Innovation Configuration Maps within a Multiple-Case Study Analysis

ERIC Educational Resources Information Center

Fernando, Sheara

2010-01-01

The success of an implementation effort depends on the ability for a system to utilize the innovation effectively; the effective usage of an innovation can be determined by monitoring for program integrity and fidelity, and assessing the degree to which the program implementation matches the intended plan (Fixsen, Blase, Horner, & Sugai 2007). The…

Implemented or Not Implemented? Process Evaluation of the School-Based Obesity Prevention Program DOiT and Associations with Program Effectiveness

ERIC Educational Resources Information Center

van Nassau, Femke; Singh, Amika S.; Hoekstra, Trynke; van Mechelen, Willem; Brug, Johannes; Chinapaw, Mai J. M.

2016-01-01

This study investigates if and to what extent the Dutch Obesity Intervention in Teenagers (DOiT) program was implemented as intended and how this affected program effectiveness. We collected data at 20 prevocational education schools in the Netherlands. We assessed seven process indicators: recruitment, reach, dosage, fidelity, satisfaction,…

Exploring the value of mixed methods within the At Home/Chez Soi housing first project: a strategy to evaluate the implementation of a complex population health intervention for people with mental illness who have been homeless.

PubMed

Macnaughton, Eric L; Goering, Paula N; Nelson, Geoffrey B

2012-05-02

This paper is a methodological case study that describes the At Home/Chez Soi (Housing First) Initiative's mixed-methods strategy for implementation evaluation and discusses the value of these methods in evaluating the implementation of such complex population health interventions. The Housing First (HF) model is being implemented in five cities: Vancouver, Winnipeg, Toronto, Montréal and Moncton. At Home/Chez Soi is an intervention trial that aims to address the issue of homelessness in people with mental health issues. The HF model emphasizes choices, hopefulness and connecting people with resources that make a difference to their quality of life. A component of HF is supported housing, which provides a rent subsidy and rapid access to housing of choice in private apartments; a second component is support. Quantitative and qualitative methods were used to evaluate HF implementation. The findings of this case study illustrate how the critical ingredients of complex interventions, such as HF, can be adapted to different contexts while implementation fidelity is maintained at a theoretical level. The findings also illustrate how the project's mixed methods approach helped to facilitate the adaptation process. Another value of this approach is that it identifies systemic and organizational factors (e.g., housing supply, discrimination, housing procurement strategy) that affect implementation of key elements of HF. In general, the approach provides information about both whether and how key aspects of the intervention are implemented effectively across different settings. It thus provides implementation data that are rigorous, contextually relevant and practical.

School climate and teachers' beliefs and attitudes associated with implementation of the positive action program: a diffusion of innovations model.

PubMed

Beets, Michael W; Flay, Brian R; Vuchinich, Samuel; Acock, Alan C; Li, Kin-Kit; Allred, Carol

2008-12-01

Teacher- and school-level factors influence the fidelity of implementation of school-based prevention and social character and development (SACD) programs. Using a diffusion of innovations framework, the relationships among teacher beliefs and attitudes towards a prevention/SACD program and the influence of a school's administrative support and perceptions of school connectedness, characteristics of a school's climate, were specified in two cross-sectional mediation models of program implementation. Implementation was defined as the amount of the programs' curriculum delivered (e.g., lessons taught), and use of program-specific materials in the classroom (e.g., ICU boxes and notes) and in relation to school-wide activities (e.g., participation in assemblies). Teachers from 10 elementary schools completed year-end process evaluation reports for year 2 (N = 171) and 3 (N = 191) of a multi-year trial. Classroom and school-wide material usage were each favorably associated with the amount of the curriculum delivered, which were associated with teachers' attitudes toward the program which, in turn, were related to teachers' beliefs about SACD. These, in turn, were associated with teachers' perceptions of school climate. Perceptions of school climate were indirectly related to classroom material usage and both indirectly and directly related to the use of school-wide activities. Program developers need to consider the importance of a supportive environment on program implementation and attempt to incorporate models of successful school leadership and collaboration among teachers that foster a climate promoting cohesiveness, shared visions, and support.

DiAlert: a prevention program for overweight first degree relatives of type 2 diabetes patients: results of a pilot study to test feasibility and acceptability.

PubMed

Heideman, Wieke H; de Wit, Maartje; Middelkoop, Barend J C; Nierkens, Vera; Stronks, Karien; Verhoeff, Arnoud P; Snoek, Frank J

2012-09-27

Prevalence of type 2 diabetes mellitus is increasing due to lifestyle changes, particularly affecting those genetically at risk. We developed DiAlert as a targeted group-based intervention aimed to promote intrinsic motivation and action planning for lifestyle changes and weight loss in first degree relatives of patients with type 2 diabetes mellitus.The main objective of the pilot of the DiAlert intervention was to assess fidelity, feasibility and acceptability prior to starting the randomized controlled trial. Individuals with a family history of type 2 diabetes mellitus were self-identified and screened for eligibility. DiAlert consists of two group sessions. Feasibility, fidelity, acceptability and self-reported perceptions and behavioral determinants were evaluated in a pre-post study using questionnaires and observations. Determinants of behavior change were analyzed using paired-samples t tests and Wilcoxon signed rank tests. DiAlert was delivered to two groups of first degree relatives of patients with type 2 diabetes mellitus (N = 9 and N = 12). Feasibility and fidelity were confirmed. Overall, the DiAlert group sessions were positively evaluated (8.0 on a scale of 1 to 10) by participants. The intervention did not impact perceived susceptibility or worry about personal diabetes risk. Action planning with regard to changing diet and physical activity increased. DiAlert proved feasible and was well-accepted by participants. Positive trends in action planning indicate increased likelihood of actual behavior change following DiAlert. Testing the effectiveness in a randomized controlled trial is imperative. Netherlands National Trial Register (NTR): NTR2036.

Community Intervention and Public Policy in the Prevention of Antisocial Behavior

ERIC Educational Resources Information Center

Dodge, Kenneth A.

2009-01-01

As academic clinical science moves to community intervention to achieve impact on population prevalence of antisocial behavior disorders, exciting potential is tempered by realistic caution. Three kinds of efforts are noted. First, individual evidence-based therapies are being implemented at scale. Difficulties in high-fidelity implementation are…

Collaborative Strategic Reading: Findings from Experienced Implementers

ERIC Educational Resources Information Center

Vaughn, Sharon; Roberts, Greg; Klingner, Janette K.; Swanson, Elizabeth A.; Boardman, Alison; Stillman-Spisak, Stephanie J.; Mohammed, Sarojani S.; Leroux, Audrey J.

2013-01-01

This study examined the effects and fidelity of collaborative strategic reading (CSR) implemented by experienced CSR teachers (participated in previous study; Vaughn et al., 2011) on the reading comprehension outcomes of students in English/Language Arts (ELA) or Reading classes. Eligible teachers (12 of 17; others reassigned to teach…

Rigorous Measures of Implementation: A Methodological Framework for Evaluating Innovative STEM Programs

ERIC Educational Resources Information Center

Cassata-Widera, Amy; Century, Jeanne; Kim, Dae Y.

2011-01-01

The practical need for multidimensional measures of fidelity of implementation (FOI) of reform-based science, technology, engineering, and mathematics (STEM) instructional materials, combined with a theoretical need in the field for a shared conceptual framework that could support accumulating knowledge on specific enacted program elements across…

Implementing Trauma-Focused CBT with Fidelity and Flexibility: A Family Case Study

ERIC Educational Resources Information Center

Kerig, Patricia K.; Sink, Holli E.; Cuellar, Raven E.; Vanderzee, Karin L.; Elfstrom, Jennifer L.

2010-01-01

Effective approaches for the treatment of childhood posttraumatic stress disorder and traumatic grief are needed given the prevalence of trauma and its impact on children's lives. To effectively treat posttraumatic stress disorder in children, evidence-based practices should be implemented with flexibility and responsiveness to culture,…

To Adapt or Not to Adapt: Navigating an Implementation Conundrum

ERIC Educational Resources Information Center

Leko, Melinda M.

2015-01-01

Maximizing the effectiveness of evidence-based practices (EBPs) requires an optimal balance of implementation fidelity and adaptation so EBPs fit local contexts and meet the individual learning needs of students with disabilities. The framework for classifying adaptations presented in this article can help educators make decisions about whether…

Moving Triadic Gaze Intervention into Practice: Measuring Clinician Attitude and Implementation Fidelity

ERIC Educational Resources Information Center

Feuerstein, Julie; Olswang, Lesley B.; Greenslade, Kathryn; Pinder, Gay Lloyd; Dowden, Patricia; Madden, Jodi

2017-01-01

Purpose: This research investigated a first step in implementing the dynamic assessment (DA) component of Triadic Gaze Intervention (Olswang, Feuerstein, Pinder, & Dowden, 2013; Olswang et al., 2014), an evidence-based protocol for teaching early signals of communication to young children with physical disabilities. Clinician attitudes about…

From innovation to implementation: the long and winding road.

PubMed

Galavotti, Christine; Kuhlmann, Anne K Sebert; Kraft, Joan Marie; Harford, Nicola; Petraglia, Joseph

2008-06-01

Building on theory and past research, in early 2000 scientists in the Division of Reproductive Health developed a prevention innovation for CDC's Global AIDS Program for use in countries severely affected by the HIV/AIDS epidemic. This innovative program model is called MARCH: Modeling and Reinforcement to Combat HIV/AIDS (Galavotti et al. Am J Public Health 91:1602-1607, 2001). MARCH promotes behavioral changes that reduce the risk of HIV infection and creates normative environments that sustain these changes through two key program components: entertainment-education using mass media, particularly long-running radio serial dramas, and reinforcement activities at the community level. Using the framework developed by Wandersman et al. (Am J Commun Psychol, 41(3-4), 2008), we describe the key elements of the MARCH prevention innovation and outline how we support its adaptation and implementation. We focus on the following questions: How do we get from an innovative model to effective program implementation in the field? How do we support implementation with fidelity when adaptation is required? And, once implemented, can we demonstrate fidelity of the adaptation to the original program model? Because our program model requires local adaptation for every instance of implementation, we suggest a potential enhancement to the Interactive Systems Framework-support for adaptation of the innovation-as part of the Prevention Support System. In this paper we describe how we supported adaptation of the radio serial drama component for unique contexts in several African countries. We focus attention on the tools and trainings we developed to build innovation specific capacity for implementation, including capacities for adaptation. We then present results of a qualitative analysis of scripts from the MARCH radio serial drama in Zimbabwe to assess the adapted program's fidelity to the original design of the innovation. Finally, we discuss lessons learned and explore implications for the field.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Dong Yuli; Zou Xubo; Guo Guangcan

We investigate the economical Gaussian cloning of coherent states with the known phase, which produces M copies from N input replica and can be implemented with degenerate parametric amplifiers and beam splitters.The achievable fidelity of single copy is given by 2M{radical}(N)/[{radical}(N)(M-1)+{radical}((1+N)(M{sup 2}+N))], which is bigger than the optimal fidelity of the universal Gaussian cloning. The cloning machine presented here works without ancillary optical modes and can be regarded as the continuous variable generalization of the economical cloning machine for qudits.

High-Fidelity e-Learning: SEI’s Virtual Training Environment (VTE)

DTIC Science & Technology

2009-01-01

Assessment 2.4 Collaboration 2.4.1 Peer-Student Collaboration 2.4.2 Instructor Support 2.5 Accessibility 2.6 Modularity 2.6.1 Design for Re-Use 2.6.2 Design ...ing Environment as an implementation of a high-fidelity e-Ieaming system. This report does not cover concepts of pedagogy or instructional design in e...pedagogical agents. This is the basis for Clark and Mayer’s Personalization principle for designing media for e-learning [Clark & Mayer 2003]. E-learning

Degrees of reality: airway anatomy of high-fidelity human patient simulators and airway trainers.

PubMed

Schebesta, Karl; Hüpfl, Michael; Rössler, Bernhard; Ringl, Helmut; Müller, Michael P; Kimberger, Oliver

2012-06-01

Human patient simulators and airway training manikins are widely used to train airway management skills to medical professionals. Furthermore, these patient simulators are employed as standardized "patients" to evaluate airway devices. However, little is known about how realistic these patient simulators and airway-training manikins really are. This trial aimed to evaluate the upper airway anatomy of four high-fidelity patient simulators and two airway trainers in comparison with actual patients by means of radiographic measurements. The volume of the pharyngeal airspace was the primary outcome parameter. Computed tomography scans of 20 adult trauma patients without head or neck injuries were compared with computed tomography scans of four high-fidelity patient simulators and two airway trainers. By using 14 predefined distances, two cross-sectional areas and three volume parameters of the upper airway, the manikins' similarity to a human patient was assessed. The pharyngeal airspace of all manikins differed significantly from the patients' pharyngeal airspace. The HPS Human Patient Simulator (METI®, Sarasota, FL) was the most realistic high-fidelity patient simulator (6/19 [32%] of all parameters were within the 95% CI of human airway measurements). The airway anatomy of four high-fidelity patient simulators and two airway trainers does not reflect the upper airway anatomy of actual patients. This finding may impact airway training and confound comparative airway device studies.

Multi-tiered system of support incorporating the R.E.N.E.W. process and its relationship to perception of school safety and office discipline referrals

NASA Astrophysics Data System (ADS)

Flood, Molly M.

This study examined the relationship between the fidelity of multi-tier school-wide positive behavior interventions and supports (SWPBIS) and staff perception of school safety and office discipline referrals. This research provided a case study on multi-tier supports and interventions, and the RENEW person-centered planning process in an alternative special education center following the implementation of a multi-tier SWPBIS model. Pennsylvania is one of several states looking to adopt an effective Tier III behavioral tool. The research described the results of an analysis of implementation fidelity on a multi-tiered school-wide positive behavior support model developed at a special education center operated by a public school system entity. This research explored the fidelity of SWPBIS implementation; analyzed the relationship of SWPBIS to school climate as measured by staff perceptions and reduction of office discipline referrals (ODR); explored tier III supports incorporating a process Rehabilitation and Empowerment, Natural Supports, Education and Work (RENEW); and investigated the potential sustainability of the RENEW process as a multi-tier system of support. This study investigated staff perceptions on integrated supports between schools and communities and identified the degree of relationship to school risk factors, school protective factors, and office discipline referrals following the building of cooperative partnerships between Systems of Care and Local Education Agencies.