Development and Construct Validation of the Interprofessional Attitudes Scale

PubMed Central

Norris, Jeffrey; Carpenter, Joan G.; Eaton, Jacqueline; Guo, Jia-Wen; Lassche, Madeline; Pett, Marjorie A.; Blumenthal, Donald K.

2015-01-01

Purpose Training of health professionals requires development of interprofessional competencies and assessment of these competencies. No validated tools exist to assess all four competency domains described in the 2011 Core Competencies for Interprofessional Collaborative Practice (the IPEC Report). The purpose of this study was to develop and validate a scale based on the IPEC competency domains that assesses interprofessional attitudes of students in the health professions. Method In 2012, a survey tool was developed and administered to 1,549 students from the University of Utah Health Science Center, an academic health center composed of four schools and colleges (Health, Medicine, Nursing, and Pharmacy). Exploratory and confirmatory factor analyses (EFA and CFA) were performed to validate the assessment tool, eliminate redundant questions, and to identify subscales. Results The EFA and CFA focused on aligning subscales with IPEC core competencies, and demonstrating good construct validity and internal consistency reliability. A response rate of 45% (n = 701) was obtained. Responses with complete data (n=678) were randomly split into two datasets which were independently analyzed using EFA and CFA. The EFA produced a 27-item scale, with five subscales (Cronbach’s alpha coefficients: 0.62 to 0.92). CFA indicated the content of the five subscales was consistent with the EFA model. Conclusions The Interprofessional Attitudes Scale (IPAS) is a novel tool that, compared to previous tools, better reflects current trends in interprofessional competencies. The IPAS should be useful to health sciences educational institutions and others training people to work collaboratively in interprofessional teams. PMID:25993280

Comparing assessments of the decision-making competencies of psychiatric inpatients as provided by physicians, nurses, relatives and an assessment tool.

PubMed

Aydin Er, Rahime; Sehiralti, Mine

2014-07-01

To compare assessments of the decision-making competencies of psychiatric inpatients as provided by physicians, nurses, relatives and an assessment tool. This study was carried out at the psychiatry clinic of Kocaeli University Hospital from June 2007 to February 2008. The decision-making competence of the 83 patients who participated in the study was assessed by physicians, nurses, relatives and MacCAT-T. Of the 83 patients, the relatives of 73.8% of them, including the parents of 47.7%, were interviewed during the study. A moderately good consistency between the competency assessments of the nurses versus those of the physicians, but a poor consistency between the assessments of the physicians and nurses versus those of the patients' relatives, was determined. The differences in the competency assessment obtained with the MacCAT-T versus the evaluations of the physicians, nurses and patients' relatives were statistically significant. Our findings demonstrate those physicians, nurses and the patients' relatives have difficulty in identifying patients lacking decision-making competence. Therefore, an objective competence assessment tool should be used along with the assessments of physicians and nurses, both of whom can provide clinical data, as well as those of relatives, who can offer insights into the patient's moral values and expectations. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

A Suggested Model for Developing and Assessing Competence of Prospective Teachers in Faculties of Education

ERIC Educational Resources Information Center

Alqiawi, Dalal A.; Ezzeldin, Sawsan M.

2015-01-01

Background: Competence assessment of teachers has long been a concern for teachers in the Faculties of Education. The decision makers have been looking for a model to assess the competence of prospective students. Tools and assessment models are used to gauge the competency of teachers in all disciplines. Objective: The present study aimed at…

Competency-Based Behavioral Anchors as Authentication Tools To Document Distance Education Competencies.

ERIC Educational Resources Information Center

Dooley, Kim E.; Lindner, James R.

2002-01-01

A study of 20 graduate students learning distance education methods found that great variance in idnviudal competence at course begining moved to similar levels at course end. Open-ended verification of competence using behavioral anchors worked well as a self-assessment and benchmarking tool to document growth in learning. (Contains 19…

Coercion, competence, and consent in offenders with personality disorder

PubMed Central

Zlodre, J; Yiend, J; Burns, T

2016-01-01

Competence to consent to treatment has not previously been examined in a personality disorder cohort without comorbid mental disorder. We examined competence and coercion in 174 individuals diagnosed with severe personality disorder using two validated tools (the MacArthur Competence Assessment Tool for Treatment and the MacArthur Coercion Assessment Scale – Short Form). Competence was not categorically impaired, but there were variations within the sample on dimensional competence measures. Further, there were significant negative correlations between experienced coercion and competence. Higher coercion scores were associated with two components of competence: lower understanding and reasoning. Patients who consented to treatment had higher scores on competence measures and experienced less coercion. These findings suggest that therapeutic approaches that decrease experienced coercion and increase competence may increase the engagement of individuals diagnosed with severe personality disorders in treatment. PMID:27284235

[Perspectives on patient competence in psychiatry: cognitive functions, emotions and values].

PubMed

Ruissen, A; Meynen, G; Widdershoven, G A M

2011-01-01

Informed consent, a central concept in the doctor-patient relationship, is only valid if it is given by a competent patient. To review the literature on competence or decision-making capacity in psychiatry. We studied the international literature and relevant Dutch material such as health acts and medical guidelines. We found a consensus in the literature about the assessment criteria and the basic principles, but we did not find any consensus about the exact definition of competence. We review a number of perspectives on competence. The conceptualisations of competence, particularly in the field of psychiatry, are still being debated. The best known clinical tool to assess patients’ capacities to make treatment decisions is the MacArthur Competence Assessment Tool (MacCAT). There are three perspectives on competence: a cognitive perspective, a perspective concerning emotions and a perspective relating to values. Further research is needed in order to make the conceptual debate on competence relevant to psychiatric practice.

An Examination of Cultural Competence Training in US Medical Education Guided by the Tool for Assessing Cultural Competence Training

PubMed Central

Jernigan, Valarie Blue Bird; Hearod, Jordan B.; Tran, Kim; Norris, Keith C.; Buchwald, Dedra

2015-01-01

In the United States, medical students must demonstrate a standard level of “cultural competence,” upon graduation. Cultural competence is most often defined as a set of congruent behaviors, attitudes, and policies that come together in a system, organization, or among professionals that enables effective work in cross-cultural situations. The Association of American Medical Colleges developed the Tool for Assessing Cultural Competence Training (TACCT) to assist schools in developing and evaluating cultural competence curricula to meet these requirements. This review uses the TACCT as a guideline to describe and assess pedagogical approaches to cultural competence training in US medical education and identify content gaps and opportunities for curriculum improvement. A total of 18 programs are assessed. Findings support previous research that cultural competence training can improve the knowledge, attitudes, and skills of medical trainees. However, wide variation in the conceptualization, implementation, and evaluation of cultural competence training programs exists, leading to differences in training quality and outcomes. More research is needed to establish optimal approaches to implementing and evaluating cultural competence training that incorporate cultural humility, the social determinants of health, and broader structural competency within the medical system. PMID:27818848

An Examination of Cultural Competence Training in US Medical Education Guided by the Tool for Assessing Cultural Competence Training.

PubMed

Jernigan, Valarie Blue Bird; Hearod, Jordan B; Tran, Kim; Norris, Keith C; Buchwald, Dedra

2016-01-01

In the United States, medical students must demonstrate a standard level of "cultural competence," upon graduation. Cultural competence is most often defined as a set of congruent behaviors, attitudes, and policies that come together in a system, organization, or among professionals that enables effective work in cross-cultural situations. The Association of American Medical Colleges developed the Tool for Assessing Cultural Competence Training (TACCT) to assist schools in developing and evaluating cultural competence curricula to meet these requirements. This review uses the TACCT as a guideline to describe and assess pedagogical approaches to cultural competence training in US medical education and identify content gaps and opportunities for curriculum improvement. A total of 18 programs are assessed. Findings support previous research that cultural competence training can improve the knowledge, attitudes, and skills of medical trainees. However, wide variation in the conceptualization, implementation, and evaluation of cultural competence training programs exists, leading to differences in training quality and outcomes. More research is needed to establish optimal approaches to implementing and evaluating cultural competence training that incorporate cultural humility, the social determinants of health, and broader structural competency within the medical system.

Assessing children’s competence to consent in research by a standardized tool: a validity study

PubMed Central

2012-01-01

Background Currently over 50% of drugs prescribed to children have not been evaluated properly for use in their age group. One key reason why children have been excluded from clinical trials is that they are not considered able to exercise meaningful autonomy over the decision to participate. Dutch law states that competence to consent can be presumed present at the age of 12 and above; however, in pediatric practice children’s competence is not that clearly presented and the transition from assent to active consent is gradual. A gold standard for competence assessment in children does not exist. In this article we describe a study protocol on the development of a standardized tool for assessing competence to consent in research in children and adolescents. Methods/design In this study we modified the MacCAT-CR, the best evaluated competence assessment tool for adults, for use in children and adolescents. We will administer the tool prospectively to a cohort of pediatric patients from 6 to18 years during the selection stages of ongoing clinical trials. The outcomes of the MacCAT-CR interviews will be compared to a reference standard, established by the judgments of clinical investigators, and an expert panel consisting of child psychiatrists, child psychologists and medical ethicists. The reliability, criterion-related validity and reproducibility of the tool will be determined. As MacCAT-CR is a multi-item scale consisting of 13 items, power was justified at 130–190 subjects, providing a minimum of 10–15 observations per item. MacCAT-CR outcomes will be correlated with age, life experience, IQ, ethnicity, socio-economic status and competence judgment of the parent(s). It is anticipated that 160 participants will be recruited over 2 years to complete enrollment. Discussion A validity study on an assessment tool of competence to consent is strongly needed in research practice, particularly in the child and adolescent population. In this study we will establish a reference standard of children’s competence to consent, combined with validation of an assessment instrument. Results can facilitate responsible involvement of children in clinical trials by further development of guidelines, health-care policies and legal policies. PMID:23009102

Evaluating the integration of cultural competence skills into health and physical assessment tools: a survey of Canadian schools of nursing.

PubMed

Chircop, Andrea; Edgecombe, Nancy; Hayward, Kathryn; Ducey-Gilbert, Cherie; Sheppard-Lemoine, Debbie

2013-04-01

Currently used audiovisual (AV) teaching tools to teach health and physical assessment reflect a Eurocentric bias using the biomedical model. The purpose of our study was to (a) identify commonly used AV teaching tools of Canadian schools of nursing and (b) evaluate the identified tools. A two-part descriptive quantitative method design was used. First, we surveyed schools of nursing across Canada. Second, the identified AV teaching tools were evaluated for content and modeling of cultural competence. The majority of the schools (67%) used publisher-produced videos associated with a physical assessment textbook. Major findings included minimal demonstration of negotiation with a client around cultural aspects of the interview including the need for an interpreter, modesty, and inclusion of support persons. Identification of culturally specific examples given during the videos was superficial and did not provide students with a comprehensive understanding of necessary culturally competent skills.

An overview of clinical tools used to assess neonatal abstinence syndrome.

PubMed

Orlando, Susan

2014-01-01

Several clinical tools have been developed to quantify the severity of withdrawal signs and symptoms exhibited by infants born to substance-using mothers. Scores from the systematic assessments are used to guide treatment of infants with moderate to severe clinical signs. This article provides an overview of published assessment tools developed for infants with neonatal abstinence syndrome. Nurses caring for infants at risk for neonatal abstinence syndrome should be knowledgeable about the tools used to evaluate these infants and guide their treatment. The ideal assessment tool should be published and include item definitions and a protocol for administering the tool. Nurses need education and training to achieve competency and interobserver reliability in the use of a selected tool. Tool-specific materials should be used to standardize training and improve accuracy in assessments. Competent and knowledgeable nurses play a critical role in improving outcomes for infants with neonatal abstinence syndrome.

Competence assessment for vocational school students based on business and industry chamber to improve graduate entrepreneurship

NASA Astrophysics Data System (ADS)

Samsudi, Widodo, Joko; Margunani

2017-03-01

Vocational school's skill competence assessment is an important phase to complete learning process at vocational school. For vocational school this phase should be designed and implemented not only to measure learning objective target, but also to provide entrepreneurship experience for the graduates. Therefore competence assessment implementation should be done comprehensively in cooperation with Business and Industry Chamber. The implementation of skill competence aspect covering materials, methods, strategies, tools and assessors, need to be designed and optimized with respect to vocational school together with Business and Industry Chamber. This aims to measure the learning objective target and produce improved entrepreneurship graduates. 4M-S strategy in students' skill competence assessment could be done to ensure that the material, method, tool and assessor have been well designed and implemented in both institutions: vocational school and Business and Industry Chamber to improve entrepreneurship graduates.

Competency-Based Training and Simulation: Making a "Valid" Argument.

PubMed

Noureldin, Yasser A; Lee, Jason Y; McDougall, Elspeth M; Sweet, Robert M

2018-02-01

The use of simulation as an assessment tool is much more controversial than is its utility as an educational tool. However, without valid simulation-based assessment tools, the ability to objectively assess technical skill competencies in a competency-based medical education framework will remain challenging. The current literature in urologic simulation-based training and assessment uses a definition and framework of validity that is now outdated. This is probably due to the absence of awareness rather than an absence of comprehension. The following review article provides the urologic community an updated taxonomy on validity theory as it relates to simulation-based training and assessments and translates our simulation literature to date into this framework. While the old taxonomy considered validity as distinct subcategories and focused on the simulator itself, the modern taxonomy, for which we translate the literature evidence, considers validity as a unitary construct with a focus on interpretation of simulator data/scores.

Relationships between high-stakes clinical skills exam scores and program director global competency ratings of first-year pediatric residents

PubMed Central

Langenau, Erik E.; Pugliano, Gina; Roberts, William L.

2011-01-01

Background Responding to mandates from the Accreditation Council for Graduate Medical Education (ACGME) and American Osteopathic Association (AOA), residency programs have developed competency-based assessment tools. One such tool is the American College of Osteopathic Pediatricians (ACOP) program directors’ annual report. High-stakes clinical skills licensing examinations, such as the Comprehensive Osteopathic Medical Licensing Examination Level 2-Performance Evaluation (COMLEX-USA Level 2-PE), also assess competency in several clinical domains. Objective The purpose of this study is to investigate the relationships between program director competency ratings of first-year osteopathic residents in pediatrics and COMLEX-USA Level 2-PE scores from 2005 to 2009. Methods The sample included all 94 pediatric first-year residents who took COMLEX-USA Level 2-PE and whose training was reviewed by the ACOP for approval of training between 2005 and 2009. Program director competency ratings and COMLEX-USA Level 2-PE scores (domain and component) were merged and analyzed for relationships. Results Biomedical/biomechanical domain scores were positively correlated with overall program director competency ratings. Humanistic domain scores were not significantly correlated with overall program director competency ratings, but did show moderate correlation with ratings for interpersonal and communication skills. The six ACGME or seven AOA competencies assessed empirically by the ACOP program directors’ annual report could not be recovered by principal component analysis; instead, three factors were identified, accounting for 86% of the variance between competency ratings. Discussion A few significant correlations were noted between COMLEX-USA Level 2-PE scores and program director competency ratings. Exploring relationships between different clinical skills assessments is inherently difficult because of the heterogeneity of tools used and overlap of constructs within the AOA and ACGME core competencies. PMID:21927550

Issues in developing valid assessments of speech pathology students' performance in the workplace.

PubMed

McAllister, Sue; Lincoln, Michelle; Ferguson, Alison; McAllister, Lindy

2010-01-01

Workplace-based learning is a critical component of professional preparation in speech pathology. A validated assessment of this learning is seen to be 'the gold standard', but it is difficult to develop because of design and validation issues. These issues include the role and nature of judgement in assessment, challenges in measuring quality, and the relationship between assessment and learning. Valid assessment of workplace-based performance needs to capture the development of competence over time and account for both occupation specific and generic competencies. This paper reviews important conceptual issues in the design of valid and reliable workplace-based assessments of competence including assessment content, process, impact on learning, measurement issues, and validation strategies. It then goes on to share what has been learned about quality assessment and validation of a workplace-based performance assessment using competency-based ratings. The outcomes of a four-year national development and validation of an assessment tool are described. A literature review of issues in conceptualizing, designing, and validating workplace-based assessments was conducted. Key factors to consider in the design of a new tool were identified and built into the cycle of design, trialling, and data analysis in the validation stages of the development process. This paper provides an accessible overview of factors to consider in the design and validation of workplace-based assessment tools. It presents strategies used in the development and national validation of a tool COMPASS, used in an every speech pathology programme in Australia, New Zealand, and Singapore. The paper also describes Rasch analysis, a model-based statistical approach which is useful for establishing validity and reliability of assessment tools. Through careful attention to conceptual and design issues in the development and trialling of workplace-based assessments, it has been possible to develop the world's first valid and reliable national assessment tool for the assessment of performance in speech pathology.

The COA360: a tool for assessing the cultural competency of healthcare organizations.

PubMed

LaVeist, Thomas A; Relosa, Rachel; Sawaya, Nadia

2008-01-01

The U.S. Census Bureau projects that by 2050, non-Hispanic whites will be in the numerical minority. This rapid diversification requires healthcare organizations to pay closer attention to cross-cultural issues if they are to meet the healthcare needs of the nation and continue to maintain a high standard of care. Although scorecards and benchmarking are widely used to gauge healthcare organizations' performance in various areas, these tools have been underused in relation to cultural preparedness or initiatives. The likely reason for this is the lack of a validated tool specifically designed to examine cultural competency. Existing validated cultural competency instruments evaluate individuals, not organizations. In this article, we discuss a study to validate the Cultural Competency Organizational Assessment--360 or the COA360, an instrument designed to appraise a healthcare organization's cultural competence. The Office of Minority Health and the Joint Commission have each developed standards for measuring the cultural competency of organizations. The COA360 is designed to assess adherence to both of these sets of standards. For this validation study, we enlisted a panel of national experts. The panel rated each dimension of the COA360, and the combination of items for each of the scale's 14 dimensions was rated above 4.13 (on 5-point scale). Our conclusion points to the validity of the COA360. As such, it is a valuable tool not only for assessing a healthcare organization's cultural readiness but also for benchmarking its progress in addressing cultural and diversity issues.

Face and Content Validity of the MacArthur Competence Assessment Tool for the Treatment of Iranian Patients.

PubMed

Saber, Ali; Tabatabaei, Seyed Mahmoud; Akasheh, Godarz; Sehat, Mojtaba; Zanjani, Zahra; Larijani, Bagher

2017-01-01

There is not a valid Persian tool for measuring the decision-making competency of patients. The aim of this study is to evaluate the face and content validity of the MacArthur Competence Assessment Tool for the treatment of Iranian Persian-speaking patients. To assess the validity of the Persian version of the tool, a self-administrated questionnaire was designed. The Lawshe method was also used for assessing each item. Content validity ratio (CVR) and content validity index (CVI) were used to assess the content validity quantitatively. According to the experts' judgment, questions with a CVR ≥0.62 and CVR <0.62 were maintainable and unmaintainable, respectively. The questions were designed in a manner to achieve the desirable result (CVR ≥0.62). The CVI scale (S-CVI) and CVI (S-CVI/Ave) were 0.94 (higher than 0.79). Thus, the content validity was confirmed. Since capacity assessments are usually based on physician's subjective judgment, they are likely to bias and therefore, with this suitably validated tool, we can improve judgment of physicians and health-care providers in out- and in-patient cases.

The Development and Validation of the Transition Competence Battery for Adolescents and Young Adults with Deafness.

ERIC Educational Resources Information Center

Reiman, John; Bullis, Michael

1988-01-01

The lack of appropriate assessment tools designed for deaf adolescents and young adults making the transition from educational programs to adult life is cited as one of the most glaring deficits in the field of deafness. The Transition Competence Battery (TCB) is being developed as an assessment tool that will guide individual training decisions…

Meeting the challenge of assessing clinical competence of occupational therapists within a program management environment.

PubMed

Salvatori, Penny; Simonavicius, Nijole; Moore, Joan; Rimmer, Georgina; Patterson, Michele

2008-02-01

Program management models have raised concerns among occupational therapists about professional standards related to clinical competence, performance review procedures, and quality improvement initiatives. This paper describes how a chart-stimulated recall (CSR) peer-review process and interview tool was revised, implemented, and evaluated as a pilot project to assess the clinical competence of occupational therapy staff at a large urban health centre in southern Ontario. Fourteen pairs (n=28) of occupational therapists representing various practice areas participated in this project. Half served as peer assessors and half as interviewees. Peer assessors conducted an independent chart review followed by a one-hour personal interview with a peer partner to discuss clinical management issues related to the client cases. Each interviewer rated his or her partner's clinical competence in eight areas of performance using a 7-point Likert scale. Results indicated that the CSR tool could discriminate among occupational therapists in terms of overall levels of clinical competence and also identify specific areas of concern that could be targeted for professional development. Feedback from participants was positive. The CSR tool was found to be useful for assessing clinical competence of occupational therapists in this large health centre as a quality improvement initiative within that discipline group. Further research is needed to establish the reliability and validity of the CSR tool.

Cultural competency assessment tool for hospitals: evaluating hospitals' adherence to the culturally and linguistically appropriate services standards.

PubMed

Weech-Maldonado, Robert; Dreachslin, Janice L; Brown, Julie; Pradhan, Rohit; Rubin, Kelly L; Schiller, Cameron; Hays, Ron D

2012-01-01

The U.S. national standards for culturally and linguistically appropriate services (CLAS) in health care provide guidelines on policies and practices aimed at developing culturally competent systems of care. The Cultural Competency Assessment Tool for Hospitals (CCATH) was developed as an organizational tool to assess adherence to the CLAS standards. First, we describe the development of the CCATH and estimate the reliability and validity of the CCATH measures. Second, we discuss the managerial implications of the CCATH as an organizational tool to assess cultural competency. We pilot tested an initial draft of the CCATH, revised it based on a focus group and cognitive interviews, and then administered it in a field test with a sample of California hospitals. The reliability and validity of the CCATH were evaluated using factor analysis, analysis of variance, and Cronbach's alphas. Exploratory and confirmatory factor analyses identified 12 CCATH composites: leadership and strategic planning, data collection on inpatient population, data collection on service area, performance management systems and quality improvement, human resources practices, diversity training, community representation, availability of interpreter services, interpreter services policies, quality of interpreter services, translation of written materials, and clinical cultural competency practices. All the CCATH scales had internal consistency reliability of .65 or above, and the reliability was .70 or above for 9 of the 12 scales. Analysis of variance results showed that not-for-profit hospitals have higher CCATH scores than for-profit hospitals in five CCATH scales and higher CCATH scores than government hospitals in two CCATH scales. The CCATH showed adequate psychometric properties. Managers and policy makers can use the CCATH as a tool to evaluate hospital performance in cultural competency and identify and target improvements in hospital policies and practices that undergird the provision of CLAS.

Emotional Intelligence and the ACGME Competencies.

PubMed

Webb, Anita R; Young, Richard A; Baumer, Joane G

2010-12-01

Residency programs desire assessment tools for teaching and measuring resident attainment of the Accreditation Council for Graduate Medical Education competencies, including interpersonal and communication skills. We sought to evaluate the use of emotional intelligence (EI) assessment and training tools in assessing and enhancing interpersonal and communication skills. We used a quasi-experimental design, with an intervention and control group composed of 1 class each of family medicine residents. The intervention was EI coaching. The assessment used the Emotional and Social Competence Inventory, a 360-degree EI survey consisting of self and other (colleague) ratings for 12 EI competencies. There were 21 participants in each of the 3 assessments (test, posttest, and control). Our EI coaching intervention had very limited participation due to a lack of protected time for EI coaching and residents' competing obligations. Return rates for self surveys were 86% to 91% and 66% to 68% for others. On all 3 trials, ratings by others were significantly higher than self ratings for every competence (range, P < .001-.045). None of the self ratings by the intervention group increased significantly for any of the competencies. None of the intervention group self ratings increased significantly on posttesting, whereas ratings by others increased significantly for coach/mentor (P < .001). The teamwork rating decreased significantly on both self and other ratings (P < .001). Achievement orientation was the highest intervention group posttest rating, and teamwork was the lowest. EI is a necessary skill in today's health care environment, and our study found that a tool from another sector was useful in assessing resident EI skills. Because our EI coaching intervention was unsuccessful, the effects of coaching on interpersonal and communication skills could not be assessed.

A Cultural Competence Organizational Review for Community Health Services: Insights From a Participatory Approach.

PubMed

Truong, Mandy; Gibbs, Lisa; Pradel, Veronika; Morris, Michal; Gwatirisa, Pauline; Tadic, Maryanne; de Silva, Andrea; Hall, Martin; Young, Dana; Riggs, Elisha; Calache, Hanny; Gussy, Mark; Watt, Richard; Gondal, Iqbal; Waters, Elizabeth

2017-05-01

Cultural competence is an important aspect of health service access and delivery in health promotion and community health. Although a number of frameworks and tools are available to assist health service organizations improve their services to diverse communities, there are few published studies describing organizational cultural competence assessments and the extent to which these tools facilitate cultural competence. This article addresses this gap by describing the development of a cultural competence assessment, intervention, and evaluation tool called the Cultural Competence Organizational Review (CORe) and its implementation in three community sector organizations. Baseline and follow-up staff surveys and document audits were conducted at each participating organization. Process data and organizational documentation were used to evaluate and monitor the experience of CORe within the organizations. Results at follow-up indicated an overall positive trend in organizational cultural competence at each organization in terms of both policy and practice. Organizations that are able to embed actions to improve organizational cultural competence within broader organizational plans increase the likelihood of sustainable changes to policies, procedures, and practice within the organization. The benefits and lessons learned from the implementation of CORe are discussed.

The development and assessment of an evaluation tool for pediatric resident competence in leading simulated pediatric resuscitations.

PubMed

Grant, Estée C; Grant, Vincent J; Bhanji, Farhan; Duff, Jonathan P; Cheng, Adam; Lockyer, Jocelyn M

2012-07-01

It is critical that competency in pediatric resuscitation is achieved and assessed during residency or post graduate medical training. The purpose of this study was to create and evaluate a tool to measure all elements of pediatric resuscitation team leadership competence. An initial set of items, derived from a literature review and a brainstorming session, were refined to a 26 item assessment tool through the use of Delphi methodology. The tool was tested using videos of standardized resuscitations. A psychometric assessment of the evidence for instrument validity and reliability was undertaken. The performance of 30 residents on two videotaped scenarios was assessed by 4 pediatricians using the tool, with 12 items assessing 'leadership and communication skills' (LCS) and 14 items assessing 'knowledge and clinical skills' (KCS). The instrument showed evidence of reliability; the Cronbach's alpha and generalizability co-efficients for the overall instrument were α=0.818 and Ep(2)=0.76, for LCS were α=0.827 and Ep(2)=0.844, and for KCS were α=0.673 and Ep(2)=0.482. While validity was initially established through literature review and brainstorming by the panel of experts, it was further built through the high strength of correlation between global scores and scores for overall performance (r=0.733), LCS (r=0.718) and KCS (r=0.662) as well as the factor analysis which accounted for 40.2% of the variance. The results of the study demonstrate that the instrument is a valid and reliable tool to evaluate pediatric resuscitation team leader competence. Copyright © 2012 Elsevier Ireland Ltd. All rights reserved.

Teaching core competencies of reconstructive microsurgery with the use of standardized patients.

PubMed

Son, Ji; Zeidler, Kamakshi R; Echo, Anthony; Otake, Leo; Ahdoot, Michael; Lee, Gordon K

2013-04-01

The Accreditation Council of Graduate Medical Education has defined 6 core competencies that residents must master before completing their training. Objective structured clinical examinations (OSCEs) using standardized patients are effective educational tools to assess and teach core competencies. We developed an OSCE specific for microsurgical head and neck reconstruction. Fifteen plastic surgery residents participated in the OSCE simulating a typical new patient consultation, which involved a patient with oral cancer. Residents were scored in all 6 core competencies by the standardized patients and faculty experts. Analysis of participant performance showed that although residents performed well overall, many lacked proficiency in systems-based practice. Junior residents were also more likely to omit critical elements of the physical examination compared to senior residents. We have modified our educational curriculum to specifically address these deficiencies. Our study demonstrates that the OSCE is an effective assessment tool for teaching and assessing all core competencies in microsurgery.

Competency Analytics Tool: Analyzing Curriculum Using Course Competencies

ERIC Educational Resources Information Center

Gottipati, Swapna; Shankararaman, Venky

2018-01-01

The applications of learning outcomes and competency frameworks have brought better clarity to engineering programs in many universities. Several frameworks have been proposed to integrate outcomes and competencies into course design, delivery and assessment. However, in many cases, competencies are course-specific and their overall impact on the…

An Analytical Method for Measuring Competence in Project Management

ERIC Educational Resources Information Center

González-Marcos, Ana; Alba-Elías, Fernando; Ordieres-Meré, Joaquín

2016-01-01

The goal of this paper is to present a competence assessment method in project management that is based on participants' performance and value creation. It seeks to close an existing gap in competence assessment in higher education. The proposed method relies on information and communication technology (ICT) tools and combines Project Management…

Tools to Assess Behavioral and Social Science Competencies in Medical Education: A Systematic Review.

PubMed

Carney, Patricia A; Palmer, Ryan T; Fuqua Miller, Marissa; Thayer, Erin K; Estroff, Sue E; Litzelman, Debra K; Biagioli, Frances E; Teal, Cayla R; Lambros, Ann; Hatt, William J; Satterfield, Jason M

2016-05-01

Behavioral and social science (BSS) competencies are needed to provide quality health care, but psychometrically validated measures to assess these competencies are difficult to find. Moreover, they have not been mapped to existing frameworks, like those from the Liaison Committee on Medical Education (LCME) and Accreditation Council for Graduate Medical Education (ACGME). This systematic review aimed to identify and evaluate the quality of assessment tools used to measure BSS competencies. The authors searched the literature published between January 2002 and March 2014 for articles reporting psychometric or other validity/reliability testing, using OVID, CINAHL, PubMed, ERIC, Research and Development Resource Base, SOCIOFILE, and PsycINFO. They reviewed 5,104 potentially relevant titles and abstracts. To guide their review, they mapped BSS competencies to existing LCME and ACGME frameworks. The final included articles fell into three categories: instrument development, which were of the highest quality; educational research, which were of the second highest quality; and curriculum evaluation, which were of lower quality. Of the 114 included articles, 33 (29%) yielded strong evidence supporting tools to assess communication skills, cultural competence, empathy/compassion, behavioral health counseling, professionalism, and teamwork. Sixty-two (54%) articles yielded moderate evidence and 19 (17%) weak evidence. Articles mapped to all LCME standards and ACGME core competencies; the most common was communication skills. These findings serve as a valuable resource for medical educators and researchers. More rigorous measurement validation and testing and more robust study designs are needed to understand how educational strategies contribute to BSS competency development.

Tools to Assess Behavioral and Social Science Competencies in Medical Education: A Systematic Review

PubMed Central

Carney, Patricia A.; Palmer, Ryan T.; Miller, Marissa Fuqua; Thayer, Erin K.; Estroff, Sue E.; Litzelman, Debra K.; Biagioli, Frances E.; Teal, Cayla R.; Lambros, Ann; Hatt, William J.; Satterfield, Jason M.

2015-01-01

Purpose Behavioral and social science (BSS) competencies are needed to provide quality health care, but psychometrically validated measures to assess these competencies are difficult to find. Moreover, they have not been mapped to existing frameworks, like those from the Liaison Committee on Medical Education (LCME) and Accreditation Council for Graduate Medical Education (ACGME). This systematic review aimed to identify and evaluate the quality of assessment tools used to measure BSS competencies. Method The authors searched the literature published between January 2002 and March 2014 for articles reporting psychometric or other validity/reliability testing, using OVID, CINAHL, PubMed, ERIC, Research and Development Resource Base, SOCIOFILE, and PsycINFO. They reviewed 5,104 potentially relevant titles and abstracts. To guide their review, they mapped BSS competencies to existing LCME and ACGME frameworks. The final, included articles fell into three categories: instrument development, which were of the highest quality; educational research, which were of the second highest quality; and curriculum evaluation, which were of lower quality. Results Of the 114 included articles, 33 (29%) yielded strong evidence supporting tools to assess communication skills, cultural competence, empathy/compassion, behavioral health counseling, professionalism, and teamwork. Sixty-two (54%) articles yielded moderate evidence and 19 (17%) weak evidence. Articles mapped to all LCME standards and ACGME core competencies; the most common was communication skills. Conclusions These findings serve as a valuable resource for medical educators and researchers. More rigorous measurement validation and testing and more robust study designs are needed to understand how educational strategies contribute to BSS competency development. PMID:26796091

Assessment of competence in video-assisted thoracoscopic surgery lobectomy: A Danish nationwide study.

PubMed

Petersen, René Horsleben; Gjeraa, Kirsten; Jensen, Katrine; Møller, Lars Borgbjerg; Hansen, Henrik Jessen; Konge, Lars

2018-04-18

Competence in video-assisted thoracoscopic surgery lobectomy has previously been established on the basis of numbers of procedures performed, but this approach does not ensure competence. Specific assessment tools, such as the newly developed video-assisted thoracoscopic surgery lobectomy assessment tool, allow for structured and objective assessment of competence. Our aim was to provide validity evidence for the video-assisted thoracoscopic surgery lobectomy assessment tool. Video recordings of 60 video-assisted thoracoscopic surgery lobectomies performed by 18 thoracic surgeons were rated using the video-assisted thoracoscopic surgery lobectomy assessment tool. All 4 centers of thoracic surgery in Denmark participated in the study. Two video-assisted thoracoscopic surgery experts rated the videos. They were blinded to surgeon and center. The total internal consistency reliability Cronbach's alpha was 0.93. Inter-rater reliability between the 2 raters was Pearson's r = 0.71 (P < .001). The mean video-assisted thoracoscopic surgery lobectomy assessment tool scores for the 10 procedures performed by beginners were 22.1 (standard deviation [SD], 8.6) for the 28 procedures performed by the intermediate surgeons, 31.2 (SD, 4.4), and for the 20 procedures performed by experts 35.9 (SD, 2.9) (P < .001). Bonferroni post hoc tests showed that experts were significantly better than intermediates (P < .008) and beginners (P < .001). Intermediates' mean scores were significantly better than beginners (P < .001). The pass/fail standard calculated using the contrasting group's method was 31 points. One of the beginners passed, and 2 experts failed the test. Validity evidence was provided for a newly developed assessment tool for video-assisted thoracoscopic surgery lobectomy (video-assisted thoracoscopic surgery lobectomy assessment tool) in a clinical setting. The discriminatory ability among expert surgeons, intermediate surgeons, and beginners proved highly significant. The video-assisted thoracoscopic surgery lobectomy assessment tool could be an important aid in the future training and certification of thoracic surgeons. Copyright © 2018 The American Association for Thoracic Surgery. Published by Elsevier Inc. All rights reserved.

Implementation of a competency assessment tool for agency nurses working in an acute paediatric setting.

PubMed

Hennerby, Cathy; Joyce, Pauline

2011-03-01

This paper reports on the implementation of a competency assessment tool for registered general agency nurses working in an acute paediatric setting, using a change management framework. The increased number of registered general agency nurses working in an acute children's hospital alerted concerns around their competency in working with children. These concerns were initially raised via informal complaints about 'near misses', parental dissatisfaction, perceived competency weaknesses and rising cost associated with their use. [Young's (2009) Journal of Organisational Change, 22, 524-548] nine-stage change framework was used to guide the implementation of the competency assessment tool within a paediatric acute care setting. The ongoing success of the initiative, from a nurse manager's perspective, relies on structured communication with the agency provider before employing competent agency nurses. Sustainability of the change will depend on nurse managers' persistence in attending the concerns of those resisting the change while simultaneously supporting those championing the change. These key communication and supporting roles highlight the pivotal role held by nurse managers, as gate keepers, in safe-guarding children while in hospital. Leadership qualities of nurse managers will also be challenged in continuing to manage and drive the change where resistance might prevail. © 2011 The Authors. Journal compilation © 2011 Blackwell Publishing Ltd.

A Student Assessment Tool for Standardized Patient Simulations (SAT-SPS): Psychometric analysis.

PubMed

Castro-Yuste, Cristina; García-Cabanillas, María José; Rodríguez-Cornejo, María Jesús; Carnicer-Fuentes, Concepción; Paloma-Castro, Olga; Moreno-Corral, Luis Javier

2018-05-01

The evaluation of the level of clinical competence acquired by the student is a complex process that must meet various requirements to ensure its quality. The psychometric analysis of the data collected by the assessment tools used is a fundamental aspect to guarantee the student's competence level. To conduct a psychometric analysis of an instrument which assesses clinical competence in nursing students at simulation stations with standardized patients in OSCE-format tests. The construct of clinical competence was operationalized as a set of observable and measurable behaviors, measured by the newly-created Student Assessment Tool for Standardized Patient Simulations (SAT-SPS), which was comprised of 27 items. The categories assigned to the items were 'incorrect or not performed' (0), 'acceptable' (1), and 'correct' (2). 499 nursing students. Data were collected by two independent observers during the assessment of the students' performance at a four-station OSCE with standardized patients. Descriptive statistics were used to summarize the variables. The difficulty levels and floor and ceiling effects were determined for each item. Reliability was analyzed using internal consistency and inter-observer reliability. The validity analysis was performed considering face validity, content and construct validity (through exploratory factor analysis), and criterion validity. Internal reliability and inter-observer reliability were higher than 0.80. The construct validity analysis suggested a three-factor model accounting for 37.1% of the variance. These three factors were named 'Nursing process', 'Communication skills', and 'Safe practice'. A significant correlation was found between the scores obtained and the students' grades in general, as well as with the grades obtained in subjects with clinical content. The assessment tool has proven to be sufficiently reliable and valid for the assessment of the clinical competence of nursing students using standardized patients. This tool has three main components: the nursing process, communication skills, and safety management. Copyright © 2018 Elsevier Ltd. All rights reserved.

Competency-based tool for evaluation of community-based training in undergraduate medical education in India - a Delphi approach.

PubMed

Shewade, Hemant Deepak; Jeyashree, Kathiresan; Kalaiselvi, Selvaraj; Palanivel, Chinnakali; Panigrahi, Krishna Chandra

2017-01-01

A community-based training (CBT) program, where teaching and training are carried out in the community outside of the teaching hospital, is a vital part of undergraduate medical education. Worldwide, there is a shift to competency-based training, and CBT is no exception. We attempted to develop a tool that uses a competency-based approach for assessment of CBT. Based on a review on competencies, we prepared a preliminary list of major domains with items under each domain. We used the Delphi technique to arrive at a consensus on this assessment tool. The Delphi panel consisted of eight purposively selected experts from the field of community medicine. The panel rated each item for its relevance, sensitivity, specificity, and understandability on a scale of 0-4. Median ratings were calculated at the end of each round and shared with the panel. Consensus was predefined as when 70% of the experts gave a rating of 3 or above for an item under relevance, sensitivity, and specificity. If an item failed to achieve consensus after being rated in 2 consecutive rounds, it was excluded. Anonymity of responses was maintained. The panel arrived at a consensus at the end of 3 rounds. The final version of the self-assessment tool consisted of 7 domains and 74 items. The domains (number of items) were Public health - epidemiology and research methodology (13), Public health - biostatistics (6), Public health administration at primary health center level (17), Family medicine (24), Cultural competencies (3), Community development and advocacy (2), and Generic competence (9). Each item was given a maximum score of 5 and minimum score of 1. This is the first study worldwide to develop a tool for competency-based evaluation of CBT in undergraduate medical education. The competencies identified in the 74-item questionnaire may provide the base for development of authentic curricula for CBT.

Technical skills assessment toolbox: a review using the unitary framework of validity.

PubMed

Ghaderi, Iman; Manji, Farouq; Park, Yoon Soo; Juul, Dorthea; Ott, Michael; Harris, Ilene; Farrell, Timothy M

2015-02-01

The purpose of this study was to create a technical skills assessment toolbox for 35 basic and advanced skills/procedures that comprise the American College of Surgeons (ACS)/Association of Program Directors in Surgery (APDS) surgical skills curriculum and to provide a critical appraisal of the included tools, using contemporary framework of validity. Competency-based training has become the predominant model in surgical education and assessment of performance is an essential component. Assessment methods must produce valid results to accurately determine the level of competency. A search was performed, using PubMed and Google Scholar, to identify tools that have been developed for assessment of the targeted technical skills. A total of 23 assessment tools for the 35 ACS/APDS skills modules were identified. Some tools, such as Operative Performance Rating System (OSATS) and Objective Structured Assessment of Technical Skill (OPRS), have been tested for more than 1 procedure. Therefore, 30 modules had at least 1 assessment tool, with some common surgical procedures being addressed by several tools. Five modules had none. Only 3 studies used Messick's framework to design their validity studies. The remaining studies used an outdated framework on the basis of "types of validity." When analyzed using the contemporary framework, few of these studies demonstrated validity for content, internal structure, and relationship to other variables. This study provides an assessment toolbox for common surgical skills/procedures. Our review shows that few authors have used the contemporary unitary concept of validity for development of their assessment tools. As we progress toward competency-based training, future studies should provide evidence for various sources of validity using the contemporary framework.

Self-Assessing of the Emotional Intelligence and Organizational Intelligence in Schools

ERIC Educational Resources Information Center

Dagiene, Valentina; Juškeviciene, Anita; Carneiro, Roberto; Child, Camilla; Cullen, Joe

2015-01-01

The paper presents the results of an evaluation of the Emotional Intelligence (EI) and Organisational Intelligence (OI) competences self-assessment tools developed and applied by the IGUANA project. In the paper Emotional Intelligence and Organisational Intelligence competences are discussed, their use in action research experiments to assess and…

Development of a Multi-Domain Assessment Tool for Quality Improvement Projects.

PubMed

Rosenbluth, Glenn; Burman, Natalie J; Ranji, Sumant R; Boscardin, Christy K

2017-08-01

Improving the quality of health care and education has become a mandate at all levels within the medical profession. While several published quality improvement (QI) assessment tools exist, all have limitations in addressing the range of QI projects undertaken by learners in undergraduate medical education, graduate medical education, and continuing medical education. We developed and validated a tool to assess QI projects with learner engagement across the educational continuum. After reviewing existing tools, we interviewed local faculty who taught QI to understand how learners were engaged and what these faculty wanted in an ideal assessment tool. We then developed a list of competencies associated with QI, established items linked to these competencies, revised the items using an iterative process, and collected validity evidence for the tool. The resulting Multi-Domain Assessment of Quality Improvement Projects (MAQIP) rating tool contains 9 items, with criteria that may be completely fulfilled, partially fulfilled, or not fulfilled. Interrater reliability was 0.77. Untrained local faculty were able to use the tool with minimal guidance. The MAQIP is a 9-item, user-friendly tool that can be used to assess QI projects at various stages and to provide formative and summative feedback to learners at all levels.

Development of RAD-Score: A Tool to Assess the Procedural Competence of Diagnostic Radiology Residents.

PubMed

Isupov, Inga; McInnes, Matthew D F; Hamstra, Stan J; Doherty, Geoffrey; Gupta, Ashish; Peddle, Susan; Jibri, Zaid; Rakhra, Kawan; Hibbert, Rebecca M

2017-04-01

The purpose of this study is to develop a tool to assess the procedural competence of radiology trainees, with sources of evidence gathered from five categories to support the construct validity of tool: content, response process, internal structure, relations to other variables, and consequences. A pilot form for assessing procedural competence among radiology residents, known as the RAD-Score tool, was developed by evaluating published literature and using a modified Delphi procedure involving a group of local content experts. The pilot version of the tool was tested by seven radiology department faculty members who evaluated procedures performed by 25 residents at one institution between October 2014 and June 2015. Residents were evaluated while performing multiple procedures in both clinical and simulation settings. The main outcome measure was the percentage of residents who were considered ready to perform procedures independently, with testing conducted to determine differences between levels of training. A total of 105 forms (for 52 procedures performed in a clinical setting and 53 procedures performed in a simulation setting) were collected for a variety of procedures (eight vascular or interventional, 42 body, 12 musculoskeletal, 23 chest, and 20 breast procedures). A statistically significant difference was noted in the percentage of trainees who were rated as being ready to perform a procedure independently (in postgraduate year [PGY] 2, 12% of residents; in PGY3, 61%; in PGY4, 85%; and in PGY5, 88%; p < 0.05); this difference persisted in the clinical and simulation settings. User feedback and psychometric analysis were used to create a final version of the form. This prospective study describes the successful development of a tool for assessing the procedural competence of radiology trainees with high levels of construct validity in multiple domains. Implementation of the tool in the radiology residency curriculum is planned and can play an instrumental role in the transition to competency-based radiology training.

Assessment of Proficiency and Competency in Laboratory Animal Biomethodologies

PubMed Central

Clifford, Paula; Melfi, Natasha; Bogdanske, John; Johnson, Elizabeth J; Kehler, James; Baran, Szczepan W

2013-01-01

Personnel working with laboratory animals are required by laws and guidelines to be trained and qualified to perform biomethodologic procedures. The assessment of competency and proficiency is a vital component of a laboratory animal training program, because this process confirms that the trainees have met the learning objectives for a particular procedure. The approach toward qualification assessment differs between organizations because laws and guidelines do not outline how the assessment should be performed or which methods and tools should be used. Assessment of clinical and surgical medicine has received considerable attention over the last few decades and has progressed from simple subjective methods to well-defined and objective methods of assessing competency. Although biomethodology competency and proficiency assessment is discussed in the literature, a standard and objective assessment method has not yet been developed. The development and implementation of an objective and standardized biomethodologic assessment program can serve as a tool to improve standards, ensure consistent training, and decrease research variables yet ensure animal welfare. Here we review the definition and goals of training and assessment, review assessment methods, and propose a method to develop a standard and objective assessment program for the laboratory animal science field, particularly training departments and IACUC. PMID:24351758

Feasibility of an Assessment Tool for Children's Competence to Consent to Predictive Genetic Testing: a Pilot Study.

PubMed

Hein, Irma M; Troost, Pieter W; Lindeboom, Robert; Christiaans, Imke; Grisso, Thomas; van Goudoever, Johannes B; Lindauer, Ramón J L

2015-12-01

Knowledge on children's capacities to consent to medical treatment is limited. Also, age limits for asking children's consent vary considerably between countries. Decision-making on predictive genetic testing (PGT) is especially complicated, considering the ongoing ethical debate. In order to examine just age limits for alleged competence to consent in children, we evaluated feasibility of a standardized assessment tool, and investigated cutoff ages for children's competence to consent to PGT. We performed a pilot study, including 17 pediatric outpatients between 6 and 18 years at risk for an autosomal dominantly inherited cardiac disease, eligible for predictive genetic testing. The reference standard for competence was established by experts trained in the relevant criteria for competent decision-making. The MacArthur Competence Assessment Tool for Treatment (MacCAT-T) served as index test. Data analysis included raw agreement between competence classifications, difference in mean ages between children judged competent and judged incompetent, and estimation of cutoff ages for judgments of competence. Twelve (71 %) children were considered competent by the reference standard, and 16 (94 %) by the MacCAT-T, with an overall agreement of 76 %. The expert judgments disagreed in most cases, while the MacCAT-T judgments agreed in 65 %. Mean age of children judged incompetent was 9.3 years and of children judged competent 12.1 years (p = .035). With 90 % sensitivity, children younger than 10.0 years were judged incompetent, with 90 % specificity children older than 11.8 years were judged competent. Feasibility of the MacCAT-T in children is confirmed. Initial findings on age cutoffs are indicative for children between the age of 12 and 18 to be judged competent for involvement in the informed consent process. Future research on appropriate age-limits for children's alleged competence to consent is needed.

Measuring competence in endoscopic sinus surgery.

PubMed

Syme-Grant, J; White, P S; McAleer, J P G

2008-02-01

Competence based education is currently being introduced into higher surgical training in the UK. Valid and reliable performance assessment tools are essential to ensure competencies are achieved. No such tools have yet been reported in the UK literature. We sought to develop and pilot test an Endoscopic Sinus Surgery Competence Assessment Tool (ESSCAT). The ESSCAT was designed for in-theatre assessment of higher surgical trainees in the UK. The ESSCAT rating matrix was developed through task analysis of ESS procedures. All otolaryngology consultants and specialist registrars in Scotland were given the opportunity to contribute to its refinement. Two cycles of in-theatre testing were used to ensure utility and gather quantitative data on validity and reliability. Videos of trainees performing surgery were used in establishing inter-rater reliability. National consultation, the consensus derived minimum standard of performance, Cronbach's alpha = 0.89 and demonstration of trainee learning (p = 0.027) during the in vivo application of the ESSCAT suggest a high level of validity. Inter-rater reliability was moderate for competence decisions (Cohen's Kappa = 0.5) and good for total scores (Intra-Class Correlation Co-efficient = 0.63). Intra-rater reliability was good for both competence decisions (Kappa = 0.67) and total scores (Kendall's Tau-b = 0.73). The ESSCAT generates a valid and reliable assessment of trainees' in-theatre performance of endoscopic sinus surgery. In conjunction with ongoing evaluation of the instrument we recommend the use of the ESSCAT in higher specialist training in otolaryngology in the UK.

Nursing Informatics Competencies: Psychometric Validation, Dissemination, and Maintenance of Self-Assessment Tool for Nurse Leaders.

PubMed

Collins, Sarah

2016-01-01

Due to rapid advances in technology, HIT competencies for nursing leaders require frequent attention and updating from experts in the field to ensure relevance to nursing leaders' work. This workshop will target nursing informatics researchers and leaders to: 1) learn methods and findings from a study validating a Self-Assessment Scale for Nursing Informatics Competencies for Nurse Leaders, 2) generate awareness of the Self-Assessment scale, 3) discuss strategies for maintenance of competencies overtime and 4) identify strategies to engage nursing leaders in this pursuit.

A Novel Scenario-Based Interview Tool to Evaluate Nontechnical Skills and Competencies in Global Health Delivery.

PubMed

Wroe, Emily B; McBain, Ryan K; Michaelis, Annie; Dunbar, Elizabeth L; Hirschhorn, Lisa R; Cancedda, Corrado

2017-08-01

Despite rapid growth in the number of physicians and academic institutions entering the field of global health, there are few tools that inform global health curricula and assess physician readiness for this field. To address this gap, we describe the development and pilot testing of a new tool to assess nontechnical competencies and values in global health. Competencies assessed include systems-based practice, interpersonal and cross-cultural communication, professionalism and self-care, patient care, mentoring, teaching, management, and personal motivation and experience. The Global Health Delivery Competency Assessment Tool presents 15 case vignettes and open-ended questions related to situations a global health practitioner might encounter, and grades the quality of responses on a 6-point ordinal scale. We interviewed 17 of 18 possible global health residents (94%), matched with 17 residents not training in global health, for a total of 34 interviews. A second reviewer independently scored recordings of 13 interviews for reliability. Pilot testing indicated a high degree of discriminant validity, as measured by the instrument's ability to distinguish between residents who were and were not enrolled in a global health program ( P  < .001). It also demonstrated acceptable consistency, as assessed by interrater reliability (κ = 0.53), with a range of item-level agreement from 84%-96%. The tool has potential applicability to a variety of academic and programmatic activities, including evaluation of candidates for global health positions and evaluating the success of training programs in equipping practitioners for entry into this field.

Cultural competency assessment tool for hospitals: Evaluating hospitals’ adherence to the culturally and linguistically appropriate services standards

PubMed Central

Weech-Maldonado, Robert; Dreachslin, Janice L.; Brown, Julie; Pradhan, Rohit; Rubin, Kelly L.; Schiller, Cameron; Hays, Ron D.

2016-01-01

Background The U.S. national standards for culturally and linguistically appropriate services (CLAS) in health care provide guidelines on policies and practices aimed at developing culturally competent systems of care. The Cultural Competency Assessment Tool for Hospitals (CCATH) was developed as an organizational tool to assess adherence to the CLAS standards. Purposes First, we describe the development of the CCATH and estimate the reliability and validity of the CCATH measures. Second, we discuss the managerial implications of the CCATH as an organizational tool to assess cultural competency. Methodology/Approach We pilot tested an initial draft of the CCATH, revised it based on a focus group and cognitive interviews, and then administered it in a field test with a sample of California hospitals. The reliability and validity of the CCATH were evaluated using factor analysis, analysis of variance, and Cronbach’s alphas. Findings Exploratory and confirmatory factor analyses identified 12 CCATH composites: leadership and strategic planning, data collection on inpatient population, data collection on service area, performance management systems and quality improvement, human resources practices, diversity training, community representation, availability of interpreter services, interpreter services policies, quality of interpreter services, translation of written materials, and clinical cultural competency practices. All the CCATH scales had internal consistency reliability of .65 or above, and the reliability was .70 or above for 9 of the 12 scales. Analysis of variance results showed that not-for-profit hospitals have higher CCATH scores than for-profit hospitals in five CCATH scales and higher CCATH scores than government hospitals in two CCATH scales. Practice Implications The CCATH showed adequate psychometric properties. Managers and policy makers can use the CCATH as a tool to evaluate hospital performance in cultural competency and identify and target improvements in hospital policies and practices that undergird the provision of CLAS. PMID:21934511

Psychometric properties of the Peer Proficiency Assessment (PEPA): a tool for evaluation of undergraduate peer counselors' motivational interviewing fidelity.

PubMed

Mastroleo, Nadine R; Mallett, Kimberly A; Turrisi, Rob; Ray, Anne E

2009-09-01

Despite the expanding use of undergraduate student peer counseling interventions aimed at reducing college student drinking, few programs evaluate peer counselors' competency to conduct these interventions. The present research describes the development and psychometric assessments of the Peer Proficiency Assessment (PEPA), a new tool for examining Motivational Interviewing adherence in undergraduate student peer delivered interventions. Twenty peer delivered sessions were evaluated by master and undergraduate student coders using a cross-validation design to examine peer based alcohol intervention sessions. Assessments revealed high inter-rater reliability between student and master coders and good correlations between previously established fidelity tools. Findings lend support for the use of the PEPA to examine peer counselor competency. The PEPA, training for use, inter-rater reliability information, construct and predictive validity, and tool usefulness are described.

Competence evaluation processes for nursing students abroad: Findings from an international case study.

PubMed

Tommasini, Cristina; Dobrowolska, Beata; Zarzycka, Danuta; Bacatum, Claudia; Bruun, Anne Marie Gran; Korsath, Dag; Roel, Siv; Jansen, Mette Bro; Milling, Tine; Deschamps, Anne; Mantzoukas, Stefanos; Mantzouka, Christine; Palese, Alvisa

2017-04-01

Assessing clinical competence in nursing students abroad is a challenge, and requires both methods and instruments capable of capturing the multidimensional nature of the clinical competences acquired. The aim of the study was to compare the clinical competence assessment processes and instruments adopted for nursing students during their clinical placement abroad. A case study design was adopted in 2015. A purposeful sample of eight nursing programmes located in seven countries (Belgium, Denmark, Greece, Norway, Poland, Portugal and Italy) were approached. Tools as instruments for evaluating competences developed in clinical training by international nursing students, and written procedures aimed at guiding the evaluation process, were scrutinised through a content analysis method. All clinical competence evaluation procedures and instruments used in the nursing programmes involved were provided in English. A final evaluation of the competences was expected by all nursing programmes at the end of the clinical placement, while only four provided an intermediate evaluation. Great variability emerged in the tools, with between five and 88 items included. Through content analysis, 196 items emerged, classified into 12 different core competence categories, the majority were categorised as 'Technical skills' (=60), 'Self-learning and critical thinking' (=27) and 'Nursing care process' (=25) competences. Little emphasis was given in the tools to competences involving 'Self-adaptation', 'Inter-professional skills', 'Clinical documentation', 'Managing nursing care', 'Patient communication', and 'Theory and practice integration'. Institutions signing Bilateral Agreements should agree upon the competences expected from students during their clinical education abroad. The tools used in the process, as well as the role expected by the student, should also be agreed upon. Intercultural competences should be further addressed in the process of evaluation, in addition to adaptation to different settings. There is also a need to establish those competences achievable or not in the host country, aiming at increasing transparency in learning expectations and evaluation. Copyright © 2017 Elsevier Ltd. All rights reserved.

Development and psychometric testing of Holistic Clinical Assessment Tool (HCAT) for undergraduate nursing students.

PubMed

Wu, Xi Vivien; Enskär, Karin; Pua, Lay Hoon; Heng, Doreen Gek Noi; Wang, Wenru

2016-09-22

A major focus in nursing education is on the judgement of clinical performance, and it is a complex process due to the diverse nature of nursing practice. A holistic approach in assessment of competency is advocated. Difficulties in the development of valid and reliable assessment measures in nursing competency have resulted in the development of assessment instruments with an increase in face and content validity, but few studies have tested these instruments psychometrically. It is essential to develop a holistic assessment tool to meet the needs of the clinical education. The study aims to develop a Holistic Clinical Assessment Tool (HCAT) and test its psychometric properties. The HCAT was developed based on the systematic literature review and the findings of qualitative studies. An expert panel was invited to evaluate the content validity of the tool. A total of 130 final-year nursing undergraduate students were recruited to evaluate the psychometric properties (i.e. factor structure, internal consistency and test-retest reliability) of the tool. The HCAT has good content validity with content validity index of .979. The exploratory factor analysis reveals a four-factor structure of the tool. The internal consistency and test-retest reliability of the HCAT are satisfactory with Cronbach alpha ranging from .789 to .965 and Intraclass Correlation Coefficient ranging from .881 to .979 for the four subscales and total scale. HCAT has the potential to be used as a valid measure to evaluate clinical competence in nursing students, and provide specific and ongoing feedback to enhance the holistic clinical learning experience. In addition, HCAT functions as a tool for self-reflection, peer-assessment and guides preceptors in clinical teaching and assessment.

Development of the Competency Assessment Tool-Mental Health, an instrument to assess core competencies for mental health care workers.

PubMed

Clasen, Carla; Meyer, Cheryl; Brun, Carl; Mase, William; Cauley, Kate

2003-01-01

As the focus on accountability in health care increases, there has been a corresponding emphasis on establishing core competencies for health care workers. This article discusses the development of an instrument to establish core competencies for workers in inpatient mental health settings. Twenty-six competencies were identified and rated by mental health care personnel on two subscales: the importance of the competency and how much behavioral health care workers could benefit from training on the competency. The reliability of the scale and its contributions to the training, retention and recruitment of direct care workers for behavioral health are discussed.

Structured journal club as a tool to teach and assess resident competence in practice-based learning and improvement.

PubMed

Lee, Andrew G; Boldt, H Culver; Golnik, Karl C; Arnold, Anthony C; Oetting, Thomas A; Beaver, Hilary A; Olson, Richard J; Zimmerman, M Bridget; Carter, Keith

2006-03-01

To describe the use of the journal club as a tool to teach and assess competency in practice-based learning (PBL) and improvement among residents in ophthalmology. Interventional case series. Ophthalmology residents. Three academic ophthalmology residency programs in the United States. A survey was performed of self-assessed skills in PBL among residents in ophthalmology training before and after the implementation of a structured review checklist during a traditional resident journal club. The survey had 5 domains, including (A) appraise and assimilate evidence, (B) read a journal article critically, (C) use a systematic and standardized checklist, (D) apply knowledge of study designs and statistical methods, and (E) maintain a self-documented written record of compliance. The respondents scored their ability (range, 1-5). The use of a structured journal club tool was associated with a statistically significant improvement in self-assessed ability in all 5 domains. Although validity, reliability, and long-term efficacy studies are necessary, the structured journal club is one method of teaching and assessing resident competency in PBL and improvement.

Investigation of Standardized Patient Ratings of Humanistic Competence on a Medical Licensure Examination Using Many-Facet Rasch Measurement and Generalizability Theory

ERIC Educational Resources Information Center

Zhang, Xiuyuan; Roberts, William L.

2013-01-01

Humanistic doctor-patient interaction has been measured for eight years using the Global Patient Assessment (GPA) tool in the national osteopathic clinical skills medical licensure examination. Standardized patients (SPs) apply the GPA tool to rate examinees' competence on doctor-patient communication, interpersonal skills, and professionalism.…

Development of a Tool to Assess Resident Physicians' Perceived Competence for Patient-Centered Obesity Counseling

ERIC Educational Resources Information Center

Burton, Amy M.; Brezausek, Carl M.; Hendricks, Peter S.; Agne, April A.; Hankins, Shirley L.; Cherrington, Andrea L.

2015-01-01

Physicians report a number of barriers to obesity counseling, among them low perceived competence in the ability to counsel. While there is increasing recognition of the need for resident training on counseling, implementation requires residency programs to have the necessary curricula and tools to evaluate training effectiveness. This study's…

The impact of culturally competent diabetes care interventions for improving diabetes-related outcomes in ethnic minority groups: a systematic review.

PubMed

Zeh, P; Sandhu, H K; Cannaby, A M; Sturt, J A

2012-10-01

To examine the evidence on culturally competent interventions tailored to the needs of people with diabetes from ethnic minority groups. MEDLINE (NHS Evidence), CINAHL and reference lists of retrieved papers were searched from inception to September 2011; two National Health Service specialist libraries were also searched. Google, Cochrane and DARE databases were interrogated and experts consulted. Studies were included if they reported primary research on the impact of culturally competent interventions on outcome measures of any ethnic minority group with diabetes. Paper selection and appraisal were conducted independently by two reviewers. The heterogeneity of the studies required narrative analysis. A novel culturally competent assessment tool was used to systematically assess the cultural competency of each intervention. Three hundred and twenty papers were retrieved and 11 included. Study designs varied with a diverse range of service providers. Of the interventions, 64% were found to be highly culturally competent (scoring 90-100%) and 36% moderately culturally competent (70-89%). Data were collected from 2616 participants on 22 patient-reported outcome measures. A consistent finding from 10 of the studies was that any structured intervention, tailored to ethnic minority groups by integrating elements of culture, language, religion and health literacy skills, produced a positive impact on a range of patient-important outcomes. Benefits in using culturally competent interventions with ethnic minority groups with diabetes were identified. The majority of interventions described as culturally competent were confirmed as so, when assessed using the culturally competent assessment tool. Further good quality research is required to determine effectiveness and cost-effectiveness of culturally competent interventions to influence diabetes service commissioners. © 2012 The Authors. Diabetic Medicine © 2012 Diabetes UK.

MacArthur Competence Assessment Tool for Treatment in Alzheimer disease: cross-cultural adaptation.

PubMed

Santos, Raquel Luiza; Sousa, Maria Fernanda Barroso de; Simões, José Pedro; Bertrand, Elodie; Mograbi, Daniel C; Landeira-Fernandez, Jesus; Laks, Jerson; Dourado, Marcia Cristina Nascimento

2017-01-01

We adapted the MacArthur Competence Assessment Tool for Treatment (MacCAT-T) to Brazilian Portuguese, pilot testing it on mild and moderate patients with Alzheimer's disease (AD). The cross-cultural process required six steps. Sixty-six patients with AD were assessed for competence to consent to treatment, global cognition, working memory, awareness of disease, functionality, depressive symptoms and dementia severity. The items had semantic, idiomatic, conceptual and experiential equivalence. We found no difference between mild and moderate patients with AD on the MacCAT-T domains. The linear regressions showed that reasoning (p = 0.000) and functional status (p = 0.003) were related to understanding. Understanding (p = 0.000) was related to appreciation and reasoning. Awareness of disease (p = 0.001) was related to expressing a choice. The MacCAT-T adaptation was well-understood and the constructs of the original version were maintained. The results of the pilot study demonstrated an available Brazilian tool focused on decision-making capacity in AD.

The development of multi-level critical care competency statements for self-assessment by ICU nurses.

PubMed

Bourgault, Annette M; Smith, Sherry

2004-01-01

Multi-levelled critical care competency statements were developed based on the levels of novice to expert (Benner, 1984). These competency statements provide a framework for the development of knowledge and skills specific to critical care. The purpose of this tool is to guide personal development in critical care, facilitating the assessment of individual learning needs. Competency levels are attained through the completion of performance criteria. Multi-levelled competency statements define clear expectations for the new orientee, in addition to providing a framework for the advancement of the intermediate and experienced nurse.

A Novel Tool for Assessment of Emergency Medicine Resident Skill in Determining Diagnosis and Management for Emergent Electrocardiograms: A Multicenter Study.

PubMed

Hartman, Nicholas D; Wheaton, Natasha B; Williamson, Kelly; Quattromani, Erin N; Branzetti, Jeremy B; Aldeen, Amer Z

2016-12-01

Reading emergent electrocardiograms (ECGs) is one of the emergency physician's most crucial tasks, yet no well-validated tool exists to measure resident competence in this skill. To assess validity of a novel tool measuring emergency medicine resident competency for interpreting, and responding to, critical ECGs. In addition, we aim to observe trends in this skill for resident physicians at different levels of training. This is a multi-center, prospective study of postgraduate year (PGY) 1-4 residents at five emergency medicine (EM) residency programs in the United States. An assessment tool was created that asks the physician to identify either the ECG diagnosis or the best immediate management. One hundred thirteen EM residents from five EM residency programs submitted completed assessment surveys, including 43 PGY-1s, 33 PGY-2s, and 37 PGY-3/4s. PGY-3/4s averaged 74.6% correct (95% confidence interval [CI] 70.9-78.4) and performed significantly better than PGY-1s, who averaged 63.2% correct (95% CI 58.0-68.3). PGY-2s averaged 69.0% (95% CI 62.2-73.7). Year-to-year differences were more pronounced in management than in diagnosis. Residency training in EM seems to be associated with improved ability to interpret "critical" ECGs as measured by our assessment tool. This lends validity evidence for the tool by correlating with a previously observed association between residency training and improved ECG interpretation. Resident skill in ECG interpretation remains less than ideal. Creation of this sort of tool may allow programs to assess resident performance as well as evaluate interventions designed to improve competency. Copyright © 2016 Elsevier Inc. All rights reserved.

Suicide Risk Assessment Training for Psychology Doctoral Programs: Core Competencies and a Framework for Training

PubMed Central

Cramer, Robert J.; Johnson, Shara M.; McLaughlin, Jennifer; Rausch, Emilie M.; Conroy, Mary Alice

2014-01-01

Clinical and counseling psychology programs currently lack adequate evidence-based competency goals and training in suicide risk assessment. To begin to address this problem, this article proposes core competencies and an integrated training framework that can form the basis for training and research in this area. First, we evaluate the extent to which current training is effective in preparing trainees for suicide risk assessment. Within this discussion, sample and methodological issues are reviewed. Second, as an extension of these methodological training issues, we integrate empirically- and expert-derived suicide risk assessment competencies from several sources with the goal of streamlining core competencies for training purposes. Finally, a framework for suicide risk assessment training is outlined. The approach employs Objective Structured Clinical Examination (OSCE) methodology, an approach commonly utilized in medical competency training. The training modality also proposes the Suicide Competency Assessment Form (SCAF), a training tool evaluating self- and observer-ratings of trainee core competencies. The training framework and SCAF are ripe for empirical evaluation and potential training implementation. PMID:24672588

Suicide Risk Assessment Training for Psychology Doctoral Programs: Core Competencies and a Framework for Training.

PubMed

Cramer, Robert J; Johnson, Shara M; McLaughlin, Jennifer; Rausch, Emilie M; Conroy, Mary Alice

2013-02-01

Clinical and counseling psychology programs currently lack adequate evidence-based competency goals and training in suicide risk assessment. To begin to address this problem, this article proposes core competencies and an integrated training framework that can form the basis for training and research in this area. First, we evaluate the extent to which current training is effective in preparing trainees for suicide risk assessment. Within this discussion, sample and methodological issues are reviewed. Second, as an extension of these methodological training issues, we integrate empirically- and expert-derived suicide risk assessment competencies from several sources with the goal of streamlining core competencies for training purposes. Finally, a framework for suicide risk assessment training is outlined. The approach employs Objective Structured Clinical Examination (OSCE) methodology, an approach commonly utilized in medical competency training. The training modality also proposes the Suicide Competency Assessment Form (SCAF), a training tool evaluating self- and observer-ratings of trainee core competencies. The training framework and SCAF are ripe for empirical evaluation and potential training implementation.

Computerized Assessment of Competence-Related Abilities in Living Liver Donors: The Adult-to-Adult Living Donor Liver Transplantation Cohort Study (A2ALL)

PubMed Central

Freeman, Jason; Emond, Jean; Gillespie, Brenda W.; Appelbaum, Paul S.; Weinrieb, Robert; Hill-Callahan, Peg; Gordon, Elisa J.; Terrault, Norah; Trotter, James; Ashworth, April; Dew, Mary Amanda; Pruett, Timothy

2014-01-01

Background Despite its importance, determination of competence to consent to organ donation varies widely based on local standards. We piloted a new tool to aid transplant centers in donor assessment. Methods We assessed competence-related abilities among potential living liver donors (LDs) in the 9-center A2ALL study. Prospective LDs viewed an educational video, and were queried to assess Understanding, Appreciation, Reasoning, and ability to express a Final Choice using the MacArthur Competence Assessment Tool for Clinical Research, adapted for computerized administration in LDs (“MacLiver”). Videotaped responses were scored by a clinical neuropsychologist (JF). Results Ninety-three LDs were assessed. Mean (standard deviation; domain maximum) scores were: Understanding: 18.1 (2.6; max=22), Appreciation: 5.1 (1.0; max=6), Reasoning: 3.1 (0.8; max=4), and Final Choice: 3.8 (0.5; max=4). Scores did not differ by demographics, relationship to the recipient, eligibility to donate, or eventual donation (p>0.4). Higher education was associated with greater Understanding (p=0.004) and Reasoning (p=0.03). Conclusion Standardized, computerized education with independent ratings of responses may (1) alert the clinical staff to potential donors who may not be competent to donate, and (2) highlight areas needing further assessment and education, leading to better informed decision-making. PMID:23859354

Procedural training and assessment of competency utilizing simulation.

PubMed

Sawyer, Taylor; Gray, Megan M

2016-11-01

This review examines the current environment of neonatal procedural learning, describes an updated model of skills training, defines the role of simulation in assessing competency, and discusses potential future directions for simulation-based competency assessment. In order to maximize impact, simulation-based procedural training programs should follow a standardized and evidence-based approach to designing and evaluating educational activities. Simulation can be used to facilitate the evaluation of competency, but must incorporate validated assessment tools to ensure quality and consistency. True competency evaluation cannot be accomplished with simulation alone: competency assessment must also include evaluations of procedural skill during actual clinical care. Future work in this area is needed to measure and track clinically meaningful patient outcomes resulting from simulation-based training, examine the use of simulation to assist physicians undergoing re-entry to practice, and to examine the use of procedural skills simulation as part of a maintenance of competency and life-long learning. Copyright © 2016 Elsevier Inc. All rights reserved.

Creating Trails: Tool for Real-Time Assessment of Information Literacy Skills

ERIC Educational Resources Information Center

Schloman, Barbara F.; Gedeon, Julie A.

2007-01-01

This article describes Tool for Real-time Assessment of Information Literacy Skills (TRAILS), a freely available, online tool designed to measure the information literacy skills of high school students. It is based on information literacy competencies for ninth-graders found in the "Ohio Academic Content Standards" (Ohio Department of Education…

A competency framework for librarians involved in systematic reviews.

PubMed

Townsend, Whitney A; Anderson, Patricia F; Ginier, Emily C; MacEachern, Mark P; Saylor, Kate M; Shipman, Barbara L; Smith, Judith E

2017-07-01

The project identified a set of core competencies for librarians who are involved in systematic reviews. A team of seven informationists with broad systematic review experience examined existing systematic review standards, conducted a literature search, and used their own expertise to identify core competencies and skills that are necessary to undertake various roles in systematic review projects. The team identified a total of six competencies for librarian involvement in systematic reviews: "Systematic review foundations," "Process management and communication," "Research methodology," "Comprehensive searching," "Data management," and "Reporting." Within each competency are the associated skills and knowledge pieces (indicators). Competence can be measured using an adaptation of Miller's Pyramid for Clinical Assessment, either through self-assessment or identification of formal assessment instruments. The Systematic Review Competencies Framework provides a standards-based, flexible way for librarians and organizations to identify areas of competence and areas in need of development to build capacity for systematic review integration. The framework can be used to identify or develop appropriate assessment tools and to target skill development opportunities.

Innovative Approaches to Assessment of Results of Higher School Students Training

ERIC Educational Resources Information Center

Vaganova, Olga I.; Medvedeva, Tatiana Yu.; Kirdyanova, Elena R.; Kazantseva, Galina A.; Karpukova, Albina A.

2016-01-01

The basis of assessment tools selection for performance of control and evaluation of training results subject to requirements of modular-competence approach has been disclosed. The experience in implementation of assessment tools during "General and professional pedagogy" course has been observed. The objective of the study is rationale…

Competency-based medical education for plastic surgery: where do we begin?

PubMed

Knox, Aaron D C; Gilardino, Mirko S; Kasten, Steve J; Warren, Richard J; Anastakis, Dimitri J

2014-05-01

North American surgical education is beginning to shift toward competency-based medical education, in which trainees complete their training only when competence has been demonstrated through objective milestones. Pressure is mounting to embrace competency-based medical education because of the perception that it provides more transparent standards and increased public accountability. In response to calls for reform from leading bodies in medical education, competency-based medical education is rapidly becoming the standard in training of physicians. The authors summarize the rationale behind the recent shift toward competency-based medical education and creation of the milestones framework. With respect to procedural skills, initial efforts will require the field of plastic surgery to overcome three challenges: identifying competencies (principles and procedures), modeling teaching strategies, and developing assessment tools. The authors provide proposals for how these challenges may be addressed and the educational rationale behind each proposal. A framework for identification of competencies and a stepwise approach toward creation of a principles oriented competency-based medical education curriculum for plastic surgery are presented. An assessment matrix designed to sample resident exposure to core principles and key procedures is proposed, along with suggestions for generating validity evidence for assessment tools. The ideal curriculum should provide exposure to core principles of plastic surgery while demonstrating competence through performance of index procedures that are most likely to benefit graduating residents when entering independent practice and span all domains of plastic surgery. The authors advocate that exploring the role and potential benefits of competency-based medical education in plastic surgery residency training is timely.

Validation of Assessment Vignettes and Scoring Rubric of Multicultural and International Competency in Faculty Teaching

ERIC Educational Resources Information Center

Henderson, Sheila J.; Horton, Ruth A.; Saito, Paul K.; Shorter-Gooden, Kumea

2016-01-01

The purpose of this research was to develop a new tool for assessing multicultural and international competency in faculty teaching through vignette scenarios of university classroom critical incidents--across disciplines of clinical and forensics psychology, business, and education. Construct and content validity of the initial draft vignettes…

Using the Journal Club to teach and assess competence in practice-based learning and improvement: a literature review and recommendation for implementation.

PubMed

Lee, Andrew G; Boldt, H Culver; Golnik, Karl C; Arnold, Anthony C; Oetting, Thomas A; Beaver, Hilary A; Olson, Richard J; Carter, Keith

2005-01-01

The traditional journal club has historically been used to teach residents about critically reading and reviewing the literature in order to improve patient care. The Accreditation Council for Graduate Medical Education competencies mandate requires that ophthalmology residency programs both teach and assess practice-based learning and improvement. A systematically conducted review of the literature regarding the use of the journal club in resident medical education was performed to define specific recommendations for implementation of a journal club tool. Selected best practices for a successful journal club were gleaned from the existing medical literature. These include the following: 1) the use of a structured review checklist, 2) explicit written learning objectives, and 3) a formalized meeting structure and process. The journal club might prove to be an excellent tool for the assessment of competencies like practice-based learning which may be difficult to assess by other means. Future study is necessary to determine if journal club can improve educational outcomes and promote lifelong competence in practice-based learning.

Evaluating competency to stand trial with evidence-based practice.

PubMed

Rogers, Richard; Johansson-Love, Jill

2009-01-01

Evaluations for competency to stand trial are distinguished from other areas of forensic consultation by their long history of standardized assessment beginning in the 1970s. As part of a special issue of the Journal on evidence-based forensic practice, this article examines three published competency measures: the MacArthur Competence Assessment Tool-Criminal Adjudication (MacCAT-CA), the Evaluation of Competency to Stand Trial-Revised (ECST-R), and the Competence Assessment for Standing Trial for Defendants with Mental Retardation (CAST-MR). Using the Daubert guidelines as a framework, we examined each competency measure regarding its relevance to the Dusky standard and its error and classification rates. The article acknowledges the past polarization of forensic practitioners on acceptance versus rejection of competency measures. It argues that no valuable information, be it clinical acumen or standardized data, should be systematically ignored. Consistent with the American Academy of Psychiatry and the Law Practice Guideline, it recommends the integration of competency interview findings with other sources of data in rendering evidence-based competency determinations.

A mixed methods study to develop and pilot a competency assessment tool to support midwifery care of women with intellectual disabilities.

PubMed

Beake, Sarah; Clark, Louise L; Turner, Toni; Bick, Debra

2013-08-01

Recent reports have highlighted the poor quality of health care received by people with intellectual disabilities (otherwise known as 'learning disabilities') in the United Kingdom (UK). UK Confidential Enquiries into maternal deaths have highlighted adverse pregnancy outcomes for women with intellectual disabilities and need for timely and appropriate clinical care. To develop and test a competency assessment tool to support midwifery care of women with intellectual disabilities. A mixed methods study. Large inner city maternity unit. Midwives and key experts in intellectual disabilities, maternity policy and midwifery education. Phase one comprised a systematic narrative review of the literature. Evidence identified informed phase two which included focus groups and interviews. Emergent themes informed the development of a competency assessment tool which was piloted in phase three. Phase one: Four primary research papers and two systematic reviews met the review inclusion criteria. Support to develop parenting skills of women with intellectual disabilities was highlighted as was the need to optimise organisation of maternity services. No studies specifically considered midwifery competencies to support women with intellectual disabilities. Phase two: 23 midwives attended three focus groups and individual interviews were conducted with national leaders in intellectual disability (n=6) and midwifery policy and education (n=7). Themes identified included need for individualised care provided by a known midwife, the importance of effective communication skills and need for clear knowledge and understanding of the legislative framework relevant to intellectual disability. Phase three: A convenience sample of 60 midwives was asked to participate in a pilot study to test the tool, 46 (77%) of whom responded. Thirty midwives (65%) felt competent in their ability to recognise intellectual disability and 37 (80%) competent or expert in understanding women have the right to be offered choice and make informed decisions. A high proportion (n=40, 87%) reported little or no knowledge regarding consent issues. Work to inform timely and appropriate care of women with intellectual disability has been neglected in the UK maternity services. Use of a tool could aid assessment of midwifery competencies to support women with intellectual disability and highlight where further midwifery education and development are needed. Evidence of optimal care for women with intellectual disabilities and their families across the continuum of pregnancy and birth is required. Copyright © 2012 Elsevier Ltd. All rights reserved.

The Case for the Perceived Social Competence Scale II

ERIC Educational Resources Information Center

Anderson-Butcher, Dawn; Amorose, Anthony J.; Lower, Leeann M.; Riley, Allison; Gibson, Allison; Ruch, Donna

2016-01-01

Objective: This study examines the psychometric properties of the revised Perceived Social Competence Scale (PSCS), a brief, user-friendly tool used to assess social competence among youth. Method: Confirmatory factor analyses (CFAs) examined the factor structure and invariance of an enhanced scale (PSCS-II), among a sample of 420 youth.…

Computer Simulation in Mass Emergency and Disaster Response: An Evaluation of Its Effectiveness as a Tool for Demonstrating Strategic Competency in Emergency Department Medical Responders

ERIC Educational Resources Information Center

O'Reilly, Daniel J.

2011-01-01

This study examined the capability of computer simulation as a tool for assessing the strategic competency of emergency department nurses as they responded to authentically computer simulated biohazard-exposed patient case studies. Thirty registered nurses from a large, urban hospital completed a series of computer-simulated case studies of…

Evaluation of Curriculum and Student Learning Needs Using 360 Degree Assessment

ERIC Educational Resources Information Center

Ladyshewsky, Richard; Taplin, Ross

2015-01-01

This research used a 360 degree assessment tool modelled from the competing values framework to assess the curriculum. A total of 100 Master's of Business Administration students and 746 of their work colleagues completed the 360 degree assessment tool. The students were enrolled in a course on leadership and management. The results of the…

Evaluating a standardised clinical assessment tool for pre-registration midwifery students: A cross-sectional survey of midwifery students and midwives in Australia.

PubMed

Morrow, Jane; Biggs, Laura; Stelfox, Sara; Phillips, Diane; McKellar, Lois; McLachlan, Helen

2016-02-01

Assessment of clinical competence is a core component of midwifery education. Clinical assessment tools have been developed to help increase consistency and overcome subjectivity of assessment. The study had two main aims. The first was to explore midwifery students and educators/clinical midwives' views and experiences of a common clinical assessment tool used for all preregistration midwifery programmes in Victoria and the University of South Australia. The second was to assess the need for changes to the tool to align with developments in clinical practice and evidence-based care. A cross-sectional, web-based survey including Likert-type scales and open-ended questions was utilised. Students enrolled in all four entry pathways to midwifery at seven Victorian and one South Australian university and educators/clinical midwives across both states. One hundred and ninety-one midwifery students' and 86 educators/clinical midwives responded. Overall, students and educators/clinical midwives were positive about the Clinical Assessment Tool with over 90% reporting that it covered the necessary midwifery skills. Students and educators/clinical midwives reported high levels of satisfaction with the content of the learning tools. Only 4% of educators/clinical midwives and 6% of students rated the Clinical Assessment Tool as poor overall. Changes to some learning tools were necessary in order to reflect recent practice and evidence. A common clinical assessment tool for evaluating midwifery students' clinical practice may facilitate the provision of consistent, reliable and objective assessment of student skills and competency. Copyright © 2015 Australian College of Midwives. Published by Elsevier Ltd. All rights reserved.

The Culture Audit: A Leadership Tool for Assessment and Strategic Planning in Diverse Schools and Colleges

ERIC Educational Resources Information Center

Bustamante, Rebecca M.

2006-01-01

This module is designed to introduce educational leaders to an organizational assessment tool called a "culture audit." Literature on organizational cultural competence suggests that culture audits are a valuable tool for determining how well school policies, programs, and practices respond to the needs of diverse groups and prepare…

Identifying Factors Associated with Risk Assessment Competencies of Public Health Emergency Responders.

PubMed

Hao, Jiejing; Ren, Jiaojiao; Wu, Qunhong; Hao, Yanhua; Sun, Hong; Ning, Ning; Ding, Ding

2017-06-04

This study aimed to better understand the current situation of risk assessment and identify the factors associated with competence of emergency responders in public health risk assessment. The participants were selected by a multi-stage, stratified cluster sampling method in Heilongjiang Centers for Disease Control and Prevention (CDC). The questionnaires that measured their perceptions on risk assessment competences were administered through the face-to-face survey. A final sample of 1889 staff was obtained. Of this sample, 78.6% of respondents rated their own risk assessment competences as "relatively low", contrasting with 21.4% rated as "relatively high". Most of the respondents (62.7%) did not participate in any risk assessment work. Only 13.7% and 42.7% of respondents reported participating in risk assessment training and were familiar with risk assessment tools. There existed statistical significance between risk assessment-related characteristics of respondents and their self-rated competences scores. Financial support from the government and administrative attention were regarded as the important factors contributing to risk assessment competences of CDC responders. Higher attention should be given to risk assessment training and enhancing the availability of surveillance data. Continuous efforts should be made to remove the financial and technical obstacles to improve the competences of risk assessment for public health emergency responders.

Identifying Factors Associated with Risk Assessment Competencies of Public Health Emergency Responders

PubMed Central

Hao, Jiejing; Ren, Jiaojiao; Wu, Qunhong; Hao, Yanhua; Sun, Hong; Ning, Ning; Ding, Ding

2017-01-01

This study aimed to better understand the current situation of risk assessment and identify the factors associated with competence of emergency responders in public health risk assessment. The participants were selected by a multi-stage, stratified cluster sampling method in Heilongjiang Centers for Disease Control and Prevention (CDC). The questionnaires that measured their perceptions on risk assessment competences were administered through the face-to-face survey. A final sample of 1889 staff was obtained. Of this sample, 78.6% of respondents rated their own risk assessment competences as “relatively low”, contrasting with 21.4% rated as “relatively high”. Most of the respondents (62.7%) did not participate in any risk assessment work. Only 13.7% and 42.7% of respondents reported participating in risk assessment training and were familiar with risk assessment tools. There existed statistical significance between risk assessment-related characteristics of respondents and their self-rated competences scores. Financial support from the government and administrative attention were regarded as the important factors contributing to risk assessment competences of CDC responders. Higher attention should be given to risk assessment training and enhancing the availability of surveillance data. Continuous efforts should be made to remove the financial and technical obstacles to improve the competences of risk assessment for public health emergency responders. PMID:28587226

Standards of Competence: A Multi-Site Case Study of School Reform. Project: Effects of Testing Reforms and Standards.

ERIC Educational Resources Information Center

Ellwein, Mary Catherine; Glass, Gene V.

A qualitative case study involving five educational institutions assessed the use of competency testing as a prerequisite for high school graduation, criterion for admission into college, criterion for teacher certification, and statewide assessment tool. Focus was on persons and processes involved in setting educational standards associated with…

Experiences in Evaluating Outcomes in Tool-Based, Competence Building Education in Dynamical Systems Using Symbolic Computer Algebra

ERIC Educational Resources Information Center

Perram, John W.; Andersen, Morten; Ellekilde, Lars-Peter; Hjorth, Poul G.

2004-01-01

This paper discusses experience with alternative assessment strategies for an introductory course in dynamical systems, where the use of computer algebra and calculus is fully integrated into the learning process, so that the standard written examination would not be appropriate. Instead, students' competence was assessed by grading three large…

Reliable and valid assessment of competence in endoscopic ultrasonography and fine-needle aspiration for mediastinal staging of non-small cell lung cancer.

PubMed

Konge, L; Vilmann, P; Clementsen, P; Annema, J T; Ringsted, C

2012-10-01

Fine-needle aspiration (FNA) guided by endoscopic ultrasonography (EUS) is important in mediastinal staging of non-small cell lung cancer (NSCLC). Training standards and implementation strategies of this technique are currently under discussion. The aim of this study was to explore the reliability and validity of a newly developed EUS Assessment Tool (EUSAT) designed to measure competence in EUS - FNA for mediastinal staging of NSCLC. A total of 30 patients with proven or suspected NSCLC underwent EUS - FNA for mediastinal staging by three trainees and three experienced physicians. Their performances were assessed prospectively by three experts in EUS under direct observation and again 2 months later in a blinded fashion using digital video-recordings. Based on the assessments, intra-rater reliability, inter-rater reliability, and construct validity were explored. The intra-rater reliability was good (Cronbach's α = 0.80), but comparison of results based on direct observations and blinded video-recordings indicated a significant bias favoring consultants (P = 0.022). Inter-rater reliability was very good (Cronbach's α = 0.93). However, one rater assessing five procedures or two raters each assessing four procedures were necessary to secure a generalizability coefficient of 0.80. The assessment tool demonstrated construct validity by discriminating between trainees and experienced physicians (P = 0.034). Competency in mediastinal staging of NSCLC using EUS and EUS - FNA can be assessed in a reliable and valid way using the EUSAT assessment tool. Measuring and defining competency and training requirements could improve EUS quality and benefit patient care. © Georg Thieme Verlag KG Stuttgart · New York.

Investigating the validity and usability of an interactive computer programme for assessing competence in telephone-based mental health triage.

PubMed

Sands, Natisha; Elsom, Stephen; Keppich-Arnold, Sandra; Henderson, Kathryn; King, Peter; Bourke-Finn, Karen; Brunning, Debra

2016-02-01

Telephone-based mental health triage services are frontline health-care providers that operate 24/7 to facilitate access to psychiatric assessment and intervention for people requiring assistance with a mental health problem. The mental health triage clinical role is complex, and the populations triage serves are typically high risk; yet to date, no evidence-based methods have been available to assess clinician competence to practice telephone-based mental health triage. The present study reports the findings of a study that investigated the validity and usability of the Mental Health Triage Competency Assessment Tool, an evidence-based, interactive computer programme designed to assist clinicians in developing and assessing competence to practice telephone-based mental health triage. © 2015 Australian College of Mental Health Nurses Inc.

The State of Evaluation in Internal Medicine Residency

PubMed Central

Holmboe, Eric; Beasley, Brent W.

2008-01-01

Background There are no nationwide data on the methods residency programs are using to assess trainee competence. The Accreditation Council for Graduate Medical Education (ACGME) has recommended tools that programs can use to evaluate their trainees. It is unknown if programs are adhering to these recommendations. Objective To describe evaluation methods used by our nation’s internal medicine residency programs and assess adherence to ACGME methodological recommendations for evaluation. Design Nationwide survey. Participants All internal medicine programs registered with the Association of Program Directors of Internal Medicine (APDIM). Measurements Descriptive statistics of programs and tools used to evaluate competence; compliance with ACGME recommended evaluative methods. Results The response rate was 70%. Programs were using an average of 4.2–6.0 tools to evaluate their trainees with heavy reliance on rating forms. Direct observation and practice and data-based tools were used much less frequently. Most programs were using at least 1 of the Accreditation Council for Graduate Medical Education (ACGME)’s “most desirable” methods of evaluation for all 6 measures of trainee competence. These programs had higher support staff to resident ratios than programs using less desirable evaluative methods. Conclusions Residency programs are using a large number and variety of tools for evaluating the competence of their trainees. Most are complying with ACGME recommended methods of evaluation especially if the support staff to resident ratio is high. PMID:18612734

Components of cognitive competence predictive of occupational competence in persons with dementia: a Delphi study.

PubMed

Zur, Briana M; Rudman, Debbie Laliberte; Johnson, Andrew M; Roy, Eric A; Wells, Jennie L

2013-04-01

Within the area of dementia care, occupational therapists are asked to predict occupational competence in everyday living and often do so by assessing cognitive competence. Considering the cognitive changes that occur with dementia over time, the construct of cognitive competence is a key consideration. Still, a gap exists in the literature examining the relationship between cognitive competence and occupational competence. This study developed a consensus among participating Canadian occupational therapists regarding the components of cognitive competence they considered essential to predict occupational competence in people with dementia. A three-round Delphi study was completed with English- and French-speaking occupational therapists (n = 127; 116; 125) experienced in dementia care. Ten cognitive components were identified as essential to predict occupational competence in individuals with dementia. The 10 identified components provide direction for assessment practices and education in dementia care and for development of measurement tools.

The East Anglian specialist registrar assessment tool

PubMed Central

Robinson, Susan; Boursicot, Katharine; Hayhurst, Catherine

2007-01-01

Background In our region, it was acknowledged that the process of assessment needed to be improved, but before developing a system for this, there was a need to define the “competent or satisfactory trainee”. Objective To outline the process by which a consensus was achieved on this standard, and how a system for formally assessing competency across a wide range of knowledge skills and attitudes was subsequently agreed on, thus enabling increased opportunities for training and feedback and improving the accuracy of assessment in the region. Methods The opinions of trainees and trainers from across the region were collated, and a consensus was achieved with regard to the minimum acceptable standard for a trainee in emergency medicine, thus defining a competent trainee. The group that set the standard then focused on identifying the assessment methods most appropriate for the evaluation of the knowledge, skills and attitudes required of an emergency medicine trainee. The tool was subsequently trialled for a period of 6 months, and opinion evaluated by use of a questionnaire. Results The use of the tool was reviewed from both the trainers' and trainees' perspectives. 42% (n = 11) of trainers and 31% (n = 8) trainees responded to the questionnaire. In the region, there were 26 trainers and 26 trainees. Five trainees and nine trainers had used the tool. 93% (14/15) of respondents thought that the descriptors used to describe the satisfactory trainee were acceptable; 89% (8/9) of trainers thought that it helped them assess trainees more accurately. 60% (3/5) of trainees thought that, as a result, they had a better understanding of their weak areas. Conclusion We believe that we achieved a consensus across our region as to what defined a satisfactory trainee and set the standard against which all our trainees would subsequently be evaluated. The use of this tool to assess trainees during the pilot period was disappointing; however, we were encouraged that most of those using the tool thought that it allowed an objective assessment of trainees and feedback on areas requiring further work. Those who used the tool identified important reasons that may have hindered widespread use of the assessment tool. PMID:17351222

A Comparative Study of Competency-Based Courses Demonstrating a Potential Measure of Course Quality and Student Success

ERIC Educational Resources Information Center

Krause, Jackie; Dias, Laura Portolese; Schedler, Chris

2015-01-01

While competency-based education is growing, standardized tools for evaluating the unique characteristics of course design in this domain are still under development. This preliminary research study evaluated the effectiveness of a rubric developed for assessing course design of competency-based courses in an undergraduate Information Technology…

The portfolio approach to competency-based assessment at the Cleveland Clinic Lerner College of Medicine.

PubMed

Dannefer, Elaine F; Henson, Lindsey C

2007-05-01

Despite the rapid expansion of interest in competency-based assessment, few descriptions of assessment systems specifically designed for a competency-based curriculum have been reported. The purpose of this article is to describe the design of a portfolio approach to a comprehensive, competency-based assessment system that is fully integrated with the curriculum to foster an educational environment focused on learning. The educational design goal of the Cleveland Clinic Lerner College of Medicine of Case Western Reserve University was to create an integrated educational program-curriculum and instructional methods, student assessment processes, and learning environment-to prepare medical students for success in careers as physician investigators. The first class in the five-year program matriculated in 2004. To graduate, a student must demonstrate mastery of nine competencies: research, medical knowledge, communication, professionalism, clinical skills, clinical reasoning, health care systems, personal development, and reflective practice. The portfolio provides a tool for collecting and managing multiple types of assessment evidence from multiple contexts and sources within the curriculum to document competence and promote reflective practice skills. This article describes how the portfolio was developed to provide both formative and summative assessment of student achievement in relation to the program's nine competencies.

Employability Skills Assessment Tool Development

ERIC Educational Resources Information Center

Rasul, Mohamad Sattar; Rauf, Rose Amnah Abd; Mansor, Azlin Norhaini; Puvanasvaran, A. P.

2012-01-01

Research nationally and internationally found that technical graduates are lacking in employability skills. As employability skills are crucial in outcome-based education, the main goal of this research is to develop an Employability Skill Assessment Tool to help students and lecturers produce competent graduates in employability skills needed by…

Teledermatology as an educational tool for teaching dermatology to residents and medical students.

PubMed

Boyers, Lindsay N; Schultz, Amanda; Baceviciene, Rasa; Blaney, Susan; Marvi, Natasha; Dellavalle, Robert P; Dunnick, Cory A

2015-04-01

Although teledermatology (TD) is regarded as a tool to improve patient access to specialty healthcare, little has been done to evaluate its role in medical education. We describe the TD program at the Denver (CO) Department of Veterans Affairs Medical Center and evaluate its use as an educational tool for teaching dermatology to dermatology residents and medical students. Dermatology residents manage TD consultations and review all cases with a faculty preceptor; medical students participate as observers when possible. This study assessed dermatology resident (n=14) and medical student (n=16) perceptions of TD and its usefulness in teaching six core clinical competencies. Both residents (79%) and medical students (88%) "strongly agree" or "agree" that TD is an important educational tool. In general, medical students were slightly more satisfied than residents across all of the core competencies assessed except for patient care. Medical students and residents were most satisfied with the competencies of practice-based learning and improvement and medical knowledge, whereas they were least satisfied with those of interpersonal and communication skills and professionalism. Overall, TD is valued as a teaching tool for dermatology in the areas of patient care, medical knowledge, practice-based learning and improvement, and systems-based practice.

Intercultural competence in medical education - essential to acquire, difficult to assess.

PubMed

Hamilton, John

2009-09-01

Determining student acquisition of intercultural competence (awareness, skills, knowledge and attitudes) is necessary in medical courses. However, addressing students' learning needs and developing effective tools to measure development of intercultural competence is challenging. Where this is done inadequately, skills may be overlooked or simplistic, one dimensional notions of culture be reinforced. This article examines aspects of the OSCE station development process, raising questions about how and when to assess acquisition of IC in undergraduate medical courses. It cautions against development of assessment tools which may lack authenticity and require students to engage in interactions which are unnatural. It argues for skills consistent with IC to be viewed as part of, and not separate from, the broad spectrum of skills which are a feature of any sensitive and appropriate doctor-patient interaction. Finally it advocates careful consideration of the optimum time to assess students' capacity to demonstrate IC in their interactions with patients, peers and staff, suggesting that this should come in later rather than earlier years, following theoretical, experiential and reflective learning.

National Field Testing of the "Mini" Version of the Transition Competence Battery for Adolescents and Young Adults Who Are Deaf.

ERIC Educational Resources Information Center

Bullis, Michael; Reiman, John W.; Davis, Cheryl; Reid, Chris

1997-01-01

A study involving 593 students (ages 15-29) with deafness investigated the validity of a shortened version of the Transition Competence Battery for Deaf Adolescents and Young Adults (TCB). Overall results suggest that the "Mini" TCB is a viable assessment tool that can be used in transition assessment. (Author/CR)

Modifying Defining Issues Test (DIT) as a Tool for Assessing Secondary Students' Social-Emotional Competencies

ERIC Educational Resources Information Center

Ee, Jessie

2014-01-01

This paper discusses the use of an alternative instrument to assess the social-emotional competence (SEC) of secondary school students in Singapore. The instrument was used in a larger study to explore an approach to infuse social-emotional learning in the curriculum for children in school. The design of this research instrument is based on the…

The validity of a professional competence tool for physiotherapy students in simulation-based clinical education: a Rasch analysis.

PubMed

Judd, Belinda K; Scanlan, Justin N; Alison, Jennifer A; Waters, Donna; Gordon, Christopher J

2016-08-05

Despite the recent widespread adoption of simulation in clinical education in physiotherapy, there is a lack of validated tools for assessment in this setting. The Assessment of Physiotherapy Practice (APP) is a comprehensive tool used in clinical placement settings in Australia to measure professional competence of physiotherapy students. The aim of the study was to evaluate the validity of the APP for student assessment in simulation settings. A total of 1260 APPs were collected, 971 from students in simulation and 289 from students in clinical placements. Rasch analysis was used to examine the construct validity of the APP tool in three different simulation assessment formats: longitudinal assessment over 1 week of simulation; longitudinal assessment over 2 weeks; and a short-form (25 min) assessment of a single simulation scenario. Comparison with APPs from 5 week clinical placements in hospital and clinic-based settings were also conducted. The APP demonstrated acceptable fit to the expectations of the Rasch model for the 1 and 2 week clinical simulations, exhibiting unidimensional properties that were able to distinguish different levels of student performance. For the short-form simulation, nine of the 20 items recorded greater than 25 % of scores as 'not-assessed' by clinical educators which impacted on the suitability of the APP tool in this simulation format. The APP was a valid assessment tool when used in longitudinal simulation formats. A revised APP may be required for assessment in short-form simulation scenarios.

Blinded evaluation of interrater reliability of an operative competency assessment tool for direct laryngoscopy and rigid bronchoscopy.

PubMed

Ishman, Stacey L; Benke, James R; Johnson, Kaalan Erik; Zur, Karen B; Jacobs, Ian N; Thorne, Marc C; Brown, David J; Lin, Sandra Y; Bhatti, Nasir; Deutsch, Ellen S

2012-10-01

OBJECTIVES To confirm interrater reliability using blinded evaluation of a skills-assessment instrument to assess the surgical performance of resident and fellow trainees performing pediatric direct laryngoscopy and rigid bronchoscopy in simulated models. DESIGN Prospective, paired, blinded observational validation study. SUBJECTS Paired observers from multiple institutions simultaneously evaluated residents and fellows who were performing surgery in an animal laboratory or using high-fidelity manikins. The evaluators had no previous affiliation with the residents and fellows and did not know their year of training. INTERVENTIONS One- and 2-page versions of an objective structured assessment of technical skills (OSATS) assessment instrument composed of global and a task-specific surgical items were used to evaluate surgical performance. RESULTS Fifty-two evaluations were completed by 17 attending evaluators. The instrument agreement for the 2-page assessment was 71.4% when measured as a binary variable (ie, competent vs not competent) (κ = 0.38; P = .08). Evaluation as a continuous variable revealed a 42.9% percentage agreement (κ = 0.18; P = .14). The intraclass correlation was 0.53, considered substantial/good interrater reliability (69% reliable). For the 1-page instrument, agreement was 77.4% when measured as a binary variable (κ = 0.53, P = .0015). Agreement when evaluated as a continuous measure was 71.0% (κ = 0.54, P < .001). The intraclass correlation was 0.73, considered high interrater reliability (85% reliable). CONCLUSIONS The OSATS assessment instrument is an effective tool for evaluating surgical performance among trainees with acceptable interrater reliability in a simulator setting. Reliability was good for both the 1- and 2-page OSATS checklists, and both serve as excellent tools to provide immediate formative feedback on operational competency.

An innovative approach to enhance dermatology competencies for advanced practice nurses: service–learning with a migrant farm worker health clinic.

PubMed

Downes, Elizabeth A; Connor, Ann; Howett, Maeve

2014-12-01

The purpose of this article is to describe a novel service–learning opportunity for graduate nursing students that promotes competency in dermatology. A hybrid service–learning course with online didactic content is described, along with tools for evaluation of dermatology competencies. Student evaluation of the course is discussed, and selected research articles are reviewed. Advanced practice nursing and medical education frequently does not adequately prepare primary care providers to be competent in the assessment and management of dermatologic conditions. Embedding dermatology content in a service–learning program can optimize the provision of care, strengthen competencies in dermatology and inter-professional care, and allow students to gain a deeper understanding of the population with which they work. The innovative service–learning program presented is a model for advanced practice nursing education. Tools for evaluating clinical competency and courses often need validation. Copyright 2014, SLACK Incorporated.

Adjudicative Competence

PubMed Central

Dawes, Sharron E.; Palmer, Barton W.; Jeste, Dilip V.

2008-01-01

Purpose of review Although the basic standards of adjudicative competence were specified by the U.S. Supreme Court in 1960, there remain a number of complex conceptual and practical issues in interpreting and applying these standards. In this report we provide a brief overview regarding the general concept of adjudicative competence and its assessment, as well as some highlights of recent empirical studies on this topic. Findings Most adjudicative competence assessments are conducted by psychiatrists or psychologists. There are no universal certification requirements, but some states are moving toward required certification of forensic expertise for those conducting such assessments. Recent data indicate inconsistencies in application of the existing standards even among forensic experts, but the recent publication of consensus guidelines may foster improvements in this arena. There are also ongoing efforts to develop and validate structured instruments to aid competency evaluations. Telemedicine-based competency interviews may facilitate evaluation by those with specific expertise for evaluation of complex cases. There is also interest in empirical development of educational methods to enhance adjudicative competence. Summary Adjudicative competence may be difficult to measure accurately, but the assessments and tools available are advancing. More research is needed on methods of enhancing decisional capacity among those with impaired competence. PMID:18650693

Toward an objective assessment of technical skills: a national survey of surgical program directors in Saudi Arabia.

PubMed

Alkhayal, Abdullah; Aldhukair, Shahla; Alselaim, Nahar; Aldekhayel, Salah; Alhabdan, Sultan; Altaweel, Waleed; Magzoub, Mohi Elden; Zamakhshary, Mohammed

2012-01-01

After almost a decade of implementing competency-based programs in postgraduate training programs, the assessment of technical skills remains more subjective than objective. National data on the assessment of technical skills during surgical training are lacking. We conducted this study to document the assessment tools for technical skills currently used in different surgical specialties, their relationship with remediation, the recommended tools from the program directors' perspective, and program directors' attitudes toward the available objective tools to assess technical skills. This study was a cross-sectional survey of surgical program directors (PDs). The survey was initially developed using a focus group and was then sent to 116 PDs. The survey contains demographic information about the program, the objective assessment tools used, and the reason for not using assessment tools. The last section discusses the recommended tools to be used from the PDs' perspective and the PDs' attitude and motivation to apply these tools in each program. The associations between the responses to the assessment questions and remediation were statistically evaluated. Seventy-one (61%) participants responded. Of the respondents, 59% mentioned using only nonstandardized, subjective, direct observation for technical skills assessment. Sixty percent use only summative evaluation, whereas 15% perform only formative evaluations of their residents, and the remaining 22% conduct both summative and formative evaluations of their residents' technical skills. Operative portfolios are kept by 53% of programs. The percentage of programs with mechanisms for remediation is 29% (19 of 65). The survey showed that surgical training programs use different tools to assess surgical skills competency. Having a clear remediation mechanism was highly associated with reporting remediation, which reflects the capability to detect struggling residents. Surgical training leadership should invest more in standardizing the assessment of surgical skills.

Do Health Professionals Need Additional Competencies for Stratified Cancer Prevention Based on Genetic Risk Profiling?

PubMed Central

Chowdhury, Susmita; Henneman, Lidewij; Dent, Tom; Hall, Alison; Burton, Alice; Pharoah, Paul; Pashayan, Nora; Burton, Hilary

2015-01-01

There is growing evidence that inclusion of genetic information about known common susceptibility variants may enable population risk-stratification and personalized prevention for common diseases including cancer. This would require the inclusion of genetic testing as an integral part of individual risk assessment of an asymptomatic individual. Front line health professionals would be expected to interact with and assist asymptomatic individuals through the risk stratification process. In that case, additional knowledge and skills may be needed. Current guidelines and frameworks for genetic competencies of non-specialist health professionals place an emphasis on rare inherited genetic diseases. For common diseases, health professionals do use risk assessment tools but such tools currently do not assess genetic susceptibility of individuals. In this article, we compare the skills and knowledge needed by non-genetic health professionals, if risk-stratified prevention is implemented, with existing competence recommendations from the UK, USA and Europe, in order to assess the gaps in current competences. We found that health professionals would benefit from understanding the contribution of common genetic variations in disease risk, the rationale for a risk-stratified prevention pathway, and the implications of using genomic information in risk-assessment and risk management of asymptomatic individuals for common disease prevention. PMID:26068647

Organizational cultural competence in community health and social service organizations: how to conduct a self-assessment.

PubMed

Olavarria, Marcela; Beaulac, Julie; Bélanger, Alexandre; Young, Marta; Aubry, Tim

2009-01-01

In an effort to address the significant socio-cultural changes in the population demographics of the United States (US) and Canada, organizations are increasingly seeking ways of improving their level of cultural competence. Evaluating organizational cultural competence is essential to address the needs of ethnic and cultural minorities. Yet, research related to organizational cultural competence is relatively new. The purpose of this paper is to review the extant literature with a specific focus on: (1) identifying the key standards that define culturally competent community health and social service organizations; and (2) outlining the core elements for evaluating cultural competence in a health and social service organization. Furthermore, issues related to choosing self-assessment tools and conducting an evaluation will be explored.

[Needs assessment for developing teaching competencies of medical educators].

PubMed

Si, Jihyun

2015-09-01

This study conducted a needs assessment for developing teaching competencies of medical educators by assessing their perceived ability to perform teaching competencies as well as their perceived importance of these competencies. Additionally, this study examined whether there were any differences in needs assessments scores among three faculty groups. Hundred and eighteen professors from Dong-A University College of Medicine were surveyed, and the data from 44 professors who answered all the questions were analyzed using IBM SPSS 21. The needs assessment tool measured participants' perceived ability to perform teaching competencies and perceived importance of these competencies. The Borich formula was used to calculate needs assessment scores. The most urgent needs for faculty development were identified for the teaching competencies of "diagnosis and reflection," followed by "test and feedback," and "facilitation." Additionally, two, out of 51, items with the highest needs assessment scores were "developing a thorough course syllabus" and "introducing students to the course syllabus on the first day of class." The assistant professor group scored significantly higher on educational needs related to "facilitation," "affection and concern for students," and "respect for diversity" competencies than the professor group. Furthermore, the educational needs scores for all the teaching competencies except "diagnosis and reflection," "global mindset," and "instructional management" were higher for the assistant professor group than the other two faculty groups. Thus, the educational needs assessment scores obtained in this study can be used as criteria for designing and developing faculty development programs for medical educators.

The Joint Commission has provided a tool to change your work force: are you paying attention?

PubMed

Decker, P J; Strader, M K

1998-03-01

Most health care managers wonder how to change employee "attitudes" so that their staff will be more accountable for patient satisfaction, cost reduction, and quality of care. Employees were trained to function in an industry where the power players were the physician and the administrator and now it is exceedingly difficult to get them to switch their attention to the patient and the payer in a market-driven economy. For hospital managers, the answer may be right at their fingertips: The Joint Commission on Accreditation of Healthcare Organizations' standards demanding that employee competence be objectively measured, proven, tracked & trended, improved, and age specific. A comprehensive competence assessment system can save the health care manager enormous work in measuring fewer things, focusing performance assessment on the 20 percent of things that are true problems, and helping to specifically define certain competencies such as customer focus and cost consciousness so that coaching, training, and giving performance feedback is easier. Developing a comprehensive competence assessment system is a powerful tool to change the culture of organizations. Consequently, it is important that managers be aware of those possibilities before they embark on developing "competencies" or before their organizations get too carried away on redesigning systems to satisfy standards.

Association between selected physical fitness parameters and esthetic competence in contemporary dancers.

PubMed

Angioi, Manuela; Metsios, George S; Twitchett, Emily; Koutedakis, Yiannis; Wyon, Matthew

2009-01-01

The physical demands imposed on contemporary dancers by choreographers and performance schedules make their physical fitness just as important to them as skill development. Nevertheless, it remains to be confirmed which physical fitness components are associated with aesthetic competence. The aim of this study was to: 1. replicate and test a novel aesthetic competence tool for reliability, and 2. investigate the association between selected physical fitness components and aesthetic competence by using this new tool. Seventeen volunteers underwent a series of physical fitness tests (body composition, flexibility, muscular power and endurance, and aerobic capacity) and aesthetic competence assessments (seven individual criteria commonly used by selected dance companies). Inter-rater reliability of the aesthetic competence tool was very high (r = 0.96). There were significant correlations between the aesthetic competence score and jump ability and push-ups (r = 0.55 and r = 0.55, respectively). Stepwise backward multiple regression analysis revealed that the best predictor of aesthetic competence was push-ups (R(2) = 0.30, p = 0.03). Univariate analyses also revealed that the interaction of push-ups and jump ability improved the prediction power of aesthetic competence (R(2) = 0.44, p = 0.004). It is concluded that upper body muscular endurance and jump ability best predict aesthetic competence of the present sample of contemporary dancers. Further research is required to investigate the contribution of other components of aesthetic competence, including upper body strength, lower body muscular endurance, general coordination, and static and dynamic balance.

Assessing Technical Competence in Surgical Trainees: A Systematic Review.

PubMed

Szasz, Peter; Louridas, Marisa; Harris, Kenneth A; Aggarwal, Rajesh; Grantcharov, Teodor P

2015-06-01

To systematically examine the literature describing the methods by which technical competence is assessed in surgical trainees. The last decade has witnessed an evolution away from time-based surgical education. In response, governing bodies worldwide have implemented competency-based education paradigms. The definition of competence, however, remains elusive, and the impact of these education initiatives in terms of assessment methods remains unclear. A systematic review examining the methods by which technical competence is assessed was conducted by searching MEDLINE, EMBASE, PsychINFO, and the Cochrane database of systematic reviews. Abstracts of retrieved studies were reviewed and those meeting inclusion criteria were selected for full review. Data were retrieved in a systematic manner, the validity and reliability of the assessment methods was evaluated, and quality was assessed using the Grading of Recommendations Assessment, Development and Evaluation classification. Of the 6814 studies identified, 85 studies involving 2369 surgical residents were included in this review. The methods used to assess technical competence were categorized into 5 groups; Likert scales (37), benchmarks (31), binary outcomes (11), novel tools (4), and surrogate outcomes (2). Their validity and reliability were mostly previously established. The overall Grading of Recommendations Assessment, Development and Evaluation for randomized controlled trials was high and low for the observational studies. The definition of technical competence continues to be debated within the medical literature. The methods used to evaluate technical competence predominantly include instruments that were originally created to assess technical skill. Very few studies identify standard setting approaches that differentiate competent versus noncompetent performers; subsequently, this has been identified as an area with great research potential.

Teledermatology as an Educational Tool for Teaching Dermatology to Residents and Medical Students

PubMed Central

Boyers, Lindsay N.; Schultz, Amanda; Baceviciene, Rasa; Blaney, Susan; Marvi, Natasha; Dellavalle, Robert P.

2015-01-01

Abstract Although teledermatology (TD) is regarded as a tool to improve patient access to specialty healthcare, little has been done to evaluate its role in medical education. We describe the TD program at the Denver (CO) Department of Veterans Affairs Medical Center and evaluate its use as an educational tool for teaching dermatology to dermatology residents and medical students. Dermatology residents manage TD consultations and review all cases with a faculty preceptor; medical students participate as observers when possible. This study assessed dermatology resident (n=14) and medical student (n=16) perceptions of TD and its usefulness in teaching six core clinical competencies. Both residents (79%) and medical students (88%) “strongly agree” or “agree” that TD is an important educational tool. In general, medical students were slightly more satisfied than residents across all of the core competencies assessed except for patient care. Medical students and residents were most satisfied with the competencies of practice-based learning and improvement and medical knowledge, whereas they were least satisfied with those of interpersonal and communication skills and professionalism. Overall, TD is valued as a teaching tool for dermatology in the areas of patient care, medical knowledge, practice-based learning and improvement, and systems-based practice. PMID:25635528

[Spanish validation of the MacArthur Competence Assessment Tool for Treatment interview to assess patients competence to consent treatment].

PubMed

Alvarez Marrodán, Ignacio; Baón Pérez, Beatriz; Navío Acosta, Mercedes; López-Antón, Raul; Lobo Escolar, Elena; Ventura Faci, Tirso

2014-09-09

To validate the MacArthur Competence Assessment Tool for Treatment (MacCAT-T) Spanish version, which assesses the mental capacity of patients to consent treatment, by examining 4 areas (Understanding, Appreciation, Reasoning and Expressing a choice). 160 subjects (80 Internal Medicine inpatients, 40 Psychiatric inpatients and 40 healthy controls). MacCAT-T, Mini-Mental Status Examination (MMSE). Feasibility study, reliability and validity calculations (against to gold standard of clinical expert). Mean duration of the MacCAT-T interview was 18min. Inter-rater reliability: Intraclass correlation coefficient for Understanding=0.98, Appreciation=0.97, Reasoning=0.98, Expressing a choice=0.91. Internal consistency (Cronbach's alpha): Understanding=0.87, for Appreciation=0.76, for Reasoning=0.86. Patients considered to be incapable (gold standard) scored lower in all the MacCAT-T areas. Poor performance on the MacCAT-T was related to cognitive impairment assessed by MMSE. Spanish version of the MacCAT-T is feasible, reliable, and valid for assessing the capacity of patients to consent treatment. Copyright © 2013 Elsevier España, S.L. All rights reserved.

The MacArthur Competence Assessment Tool-Criminal Adjudication: Factor structure, interrater reliability, and association with clinician opinion of competence in a forensic inpatient sample.

PubMed

Wood, Mary E; Anderson, Jaime L; Glassmire, David M

2017-06-01

Adjudicative competence is the most frequently referred evaluation in the forensic context, and it is because of this that periodic evaluation of competence assessment instruments is imperative. Among those instruments, the MacArthur Competence Assessment Tool-Criminal Adjudication (MacCAT-CA) has demonstrated adequate psychometric properties suggesting its utility in informing the forensic inquiry. The purpose of the current study was to further investigate the psychometric properties and ultimate utility of subscale scores using archival data from a sample of 103 male and female forensic patients who were hospitalized for competence restoration treatment. Results of the present study suggested adequate internal consistency and good model fit for the factor structure. Interrater reliability was evaluated by comparing the absolute agreement of scores derived from 2 independent research assistants for each of the subscales; 2 of the 3 subscales fell within the acceptable range given established interpretative benchmarks for forensic assessment. Of particular interest was that the Appreciation subscale, while heralding the lowest intraclass correlation coefficient, explained the largest proportion of variance in clinician opinion relative to the other 2 subscales. In other words, the most subjective subscale (as evidenced by the lowest intraclass correlation), explained the largest proportion of variance in ultimate opinion. The authors argue that, although these results are an important consideration in these assessments, they are neither surprising nor entirely problematic when considering the case-specific nature of the inquiries on the subscale, as well as the subjectivity of scoring criteria for each of the Appreciation items. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

Nurse competence scale: development and psychometric testing.

PubMed

Meretoja, Riitta; Isoaho, Hannu; Leino-Kilpi, Helena

2004-07-01

Self-assessment assists nurses to maintain and improve their practice by identifying their strengths and areas that may need to be further developed. Professional competence profiles encourage them to take an active part in the learning process of continuing education. Although competence recognition offers a way to motivate practising nurses to produce quality care, few measuring tools are available for this purpose. This paper describes the development and testing of the Nurse Competence Scale, an instrument with which the level of nurse competence can be assessed in different hospital work environments. The categories of the Nurse Competence Scale were derived from Benner's From Novice to Expert competency framework. A seven-step approach, including literature review and six expert groups, was used to identify and validate the indicators of nurse competence. After a pilot test, psychometric testing of the Nurse Competence Scale (content, construct and concurrent validity, and internal consistency) was undertaken with 498 nurses. The 73-item scale consists of seven categories, with responses on a visual analogy scale format. The frequency of using competencies was additionally tested with a four-point scale. Self-assessed overall scores indicated a high level of competence across categories. The Nurse Competence Scale data were normally distributed. The higher the frequency of using competencies, the higher was the self-assessed level of competence. Age and length of work experience had a positive but not very strong correlation with level of competence. According to the item analysis, the categories of the Nurse Competence Scale showed good internal consistency. The results provide strong evidence of the reliability and validity of the Nurse Competence Scale.

Enhancing an introductory Pharmacy Practice Experience at free medical clinics.

PubMed

Morello, Candis M; Singh, Renu F; Chen, Karen J; Best, Brookie M

2010-02-01

The aim of the study was to assess and improve first-year student pharmacists' satisfaction and learning experience in a Student-Run Free Medical Clinic Project (SFMCP) providing medical care to an underserved population. Two consecutive classes of first-year student pharmacists at the University of California San Diego (UCSD) Skaggs School of Pharmacy and Pharmaceutical Sciences participated in an Introductory Pharmacy Practice Experience (IPPE) at the UCSD SFMCP. This IPPE involved two inter-professional evening free clinics which provide medical care to an underserved population and opportunities for healthcare professional training and service. Year 1 students completed a self-assessment survey instrument and year 2 students completed the survey instrument plus a new competency checklist tool. Average scores from the self-assessment survey instrument were compared between years 1 and 2. Initial survey results showed that students felt the SFMCP was worthwhile; however, they did not experience enough involvement in the patient assistance programme or non-pharmacy-related clinic activities. After the competency checklist tool implementation, overall student pharmacist satisfaction of the SFMCP IPPE remained high (88%), participation in identified weak areas improved and students agreed that the tool helped focus their clinic experience. Areas of improvement were identified with the survey instrument and the competency checklist tool increased achievement of learning objectives. Overall, student pharmacists felt the SFMCP IPPE was a good learning experience. Practising pharmacists can employ these or similar tools in specific practice settings, to evaluate and help ensure that student pharmacists or interns are achieving applicable learning objectives.

A Dynamic Community of Discovery: Planning, Learning, and Change

ERIC Educational Resources Information Center

Gordon, Michelle; Ireland, Martha; Wong, Mina

2011-01-01

Ryerson University's Prior Learning and Competency Evaluation and Documentation (PLACED) program is funded by the Government of Ontario to engage internationally educated professionals (IEPs), employers, and regulatory/occupational bodies in the use of competency-based practices. In 2008, the authors created a self-assessment tool for IEPs that…

Evaluation of Professional Role Competency during Psychiatry Residency

ERIC Educational Resources Information Center

Grujich, Nikola N.; Razmy, Ajmal; Zaretsky, Ari; Styra, Rima G.; Sockalingam, Sanjeev

2012-01-01

Objective: The authors sought to determine psychiatry residents' perceptions on the current method of evaluating professional role competency and the use of multi-source feedback (MSF) as an assessment tool. Method: Authors disseminated a structured, anonymous survey to 128 University of Toronto psychiatry residents, evaluating the current mode of…

Preparing Professional School Counselors as Collaborators in Culturally Competent School Administration

ERIC Educational Resources Information Center

Nelson, Judith; Bustamante, Rebecca M.

2009-01-01

In collaboration with principals and other leadership team members, professional school counselors have ethical responsibilities in promoting culturally competent school environments. Pre-service training is the ideal time for school counselors and principals to develop the necessary background information, tools, and assessment skills to assist…

Integrating Prevention of Mother to Child HIV Transmission competencies into the nursing curriculum: Methodological lessons from a university-based undergraduate programme.

PubMed

Mbombo, Nomafrench; Bimerew, Million

2012-11-14

South Africa (SA) has the highest number of women infected with HIV and AIDS during pregnancy, which results in more than 70 000 infected babies being born each year AIDS is the major contributor to maternal and child morbidities and mortalities in the country. To combat this, the SA government has developed a national policy to prevent mother-to-child HIV transmission (PMTCT). However, for effective implementation of this policy, there is a dire need for a competent, skilled health worker to render the service. In response to this, the School of Nursing at the University of the Western Cape has integrated PMTCT competencies into the undergraduate Bachelor of Nursing Science curriculum. In this paper, we described teaching and learning approaches used to integrate PMTCT competencies, including the skills laboratory methodology and case-based learning, as well as a portfolio of evidence assessment tool. A quantitative descriptive design was used to analyse data collected from students in regard to assessment of PMTCT competencies achieved. The study used the conceptual framework of Lenburg's competency outcomes and performance assessment model, which focuses on competency development and assessment in a clinical environment. HIV competencies, including PMTCT, should be integrated both theoretically and at service delivery into other nursing and midwifery competencies, including assessment strategies. Provincial policies in provision of antiretrovirals by nurses and midwives become barriers to successful implementation of PMTCT, resulting in limited learning opportunities for students to practice PMTCT competencies. Further research is required to assess an attribute, affect, which is another prong for competencies.

A project to establish a skills competency matrix for EU nurses.

PubMed

Cowan, David T; Norman, Ian J; Coopamah, Vinoda P

Enhanced nurse workforce mobility in the European Union (EU) is seen as a remedy to shortages of nurses in some EU countries and a surplus in others. However, knowledge of differences in competence, culture, skill levels and working practices of nursing staff throughout EU countries is not fully documented because currently no tangible method exists to enable comparison. The European Healthcare Training and Accreditation Network (EHTAN) project intends to address this problem by establishing an assessment and evaluation methodology through the compilation of a skills competency matrix. To this end, subsequent to a review of documentation and literature on nursing competence definition and assessment, two versions of a nursing competence self-assessment questionnaire tool have been developed. The final competence matrix will be translated and disseminated for transnational use and it is hoped that this will inform EU and national policies on the training requirements of nurses and nursing mobility and facilitate the promotion of EU-wide recognition of nursing qualifications.

Assessing Implementation of Cultural Competency Content in the Curricula of Colleges of Pharmacy in the United States and Canada

PubMed Central

Onyoni, Esther Moraa

2007-01-01

Objectives To assess the presence of curricular and organizational content related to cultural competency within colleges of pharmacy in the United States and Canada. Methods Curriculum committee chairs (n = 87) and student leaders (n = 54) in colleges of pharmacy in the United States and Canada were surveyed via an e-mailed assessment tool. Results Forty-nine (56.3%) curriculum committee chairs and 27 (50%) student leaders returned usable responses. Respondents reported that cultural competency was mentioned in 61.2% of their mission statements, and half had made curricular changes with respect to diversity within the past 5 years. Almost 94% felt the necessity to add cultural competency topics to required courses in the curriculum, and 42.9% wanted to add a course specific to cultural competency into the curriculum. Conclusion Curriculum committee chairs recognize the need to add curricular content related to cultural competency, but not all of the respondents have implemented changes in their college's curriculum. PMID:17533433

Validity, Reliability and Acceptability of the Team Standardized Assessment of Clinical Encounter Report*

PubMed Central

Wong, Camilla L.; Norris, Mireille; Sinha, Samir S.; Zorzitto, Maria L.; Madala, Sushma; Hamid, Jemila S.

2016-01-01

Background The Team Standardized Assessment of a Clinical Encounter Report (StACER) was designed for use in Geriatric Medicine residency programs to evaluate Communicator and Collaborator competencies. Methods The Team StACER was completed by two geriatricians and interdisciplinary team members based on observations during a geriatric medicine team meeting. Postgraduate trainees were recruited from July 2010–November 2013. Inter-rater reliability between two geriatricians and between all team members was determined. Internal consistency of items for the constructs Communicator and Collaborator competencies was calculated. Raters completed a survey previously administered to Canadian geriatricians to assess face validity. Trainees completed a survey to determine the usefulness of this instrument as a feedback tool. Results Thirty postgraduate trainees participated. The prevalence-adjusted bias-adjusted kappa range inter-rater reliability for Communicator and Collaborator items were 0.87–1.00 and 0.86–1.00, respectively. The Cronbach’s alpha coefficient for Communicator and Collaborator items was 0.997 (95% CI: 0.993–1.00) and 0.997 (95% CI: 0.997–1.00), respectively. The instrument lacked discriminatory power, as all trainees scored “meets requirements” in the overall assessment. Niney-three per cent and 86% of trainees found feedback useful for developing Communicator and Collaborator competencies, respectively. Conclusions The Team StACER has adequate inter-rater reliability and internal consistency. Poor discriminatory power and face validity challenge the merit of using this evaluation tool. Trainees felt the tool provided useful feedback on Collaborator and Communicator competencies. PMID:28050222

Assessment of Surgical Skills and Competency.

PubMed

Bhatti, Nasir I

2017-10-01

Evaluation of surgical skills and competency are important aspects of the medical education process. Measurable and reproducible methods of assessment with objective feedback are essential components of surgical training. Objective Structured Assessment of Technical Skills (OSATS) is widely used across the medical specialties and otolaryngology-specific tools have been developed and validated for sinus and mastoid surgery. Although assessment of surgical skills can be time-consuming and requires human and financial resources, new evaluation methods and emerging technology may alleviate these barriers while also improving data collection practices. Copyright © 2017 Elsevier Inc. All rights reserved.

Competencies development and self-assessment in maintenance management e-training

NASA Astrophysics Data System (ADS)

Papathanassiou, Nikos; Pistofidis, Petros; Emmanouilidis, Christos

2013-10-01

The maintenance management function requires staff to possess a truly multidisciplinary set of skills. This includes competencies from engineering and information technology to health and safety, management and finance, while also taking into account the normative and legislative issues. This body of knowledge is rarely readily available within a single university course. The potential of e-learning in this field is significant, as it is a flexible and less costly alternative to conventional training. Furthermore, trainees can follow their own pace, as their available time is often a commodity. This article discusses the development of tools to support competencies development and self-assessment in maintenance management. Based on requirements arising from professional bodies' guidelines and a user survey, the developed tools implement a dedicated maintenance management training curriculum. The results from pilot testing on academic and industrial user groups are discussed and user evaluations are linked with specific e-learning design issues.

Considering Human Capital Theory in Assessment and Training: Mapping the Gap between Current Skills and the Needs of a Knowledge-Based Economy in Northeast Iowa

ERIC Educational Resources Information Center

Mihm-Herold, Wendy

2010-01-01

In light of the current economic downturn, thousands of Iowans are unemployed and this is the ideal time to build the skills of the workforce to compete in the knowledge-based economy so businesses and entrepreneurs can compete in a global economy. A tool for assessing the skills and knowledge of dislocated workers and students as well as…

Assessment of Technical Skills Competence in the Operating Room: A Systematic and Scoping Review.

PubMed

Fahim, Christine; Wagner, Natalie; Nousiainen, Markku T; Sonnadara, Ranil

2018-05-01

While academic accreditation bodies continue to promote competency-based medical education (CBME), the feasibility of conducting regular CBME assessments remains challenging. The purpose of this study was to identify evidence pertaining to the practical application of assessments that aim to measure technical competence for surgical trainees in a nonsimulated, operative setting. In August 2016, the authors systematically searched Medline, Embase, and the Cochrane Database of Systematic Reviews for English-language, peer-reviewed articles published in or after 1996. The title, abstract, and full text of identified articles were screened. Data regarding study characteristics, psychometric and measurement properties, implementation of assessment, competency definitions, and faculty training were extracted. The findings from the systematic review were supplemented by a scoping review to identify key strategies related to faculty uptake and implementation of CBME assessments. A total of 32 studies were included. The majority of studies reported reasonable scores of interrater reliability and internal consistency. Seven articles identified minimum scores required to establish competence. Twenty-five articles mentioned faculty training. Many of the faculty training interventions focused on timely completion of assessments or scale calibration. There are a number of diverse tools used to assess competence for intraoperative technical skills and a lack of consensus regarding the definition of technical competence within and across surgical specialties. Further work is required to identify when and how often trainees should be assessed and to identify strategies to train faculty to ensure timely and accurate assessment.

Educational Standards, Accountability, and Student Assessment: Legal and Administrative Considerations for Competency Testing.

ERIC Educational Resources Information Center

Hoegl, Juergen K.

Minimal competency testing of students can be an effective tool for improving educational quality if applied appropriately with adequate safeguards. The failure of academic standards through grade inflation, a policy of social promotion rather than academic promotion, and weak curricula has reduced public confidence in education and encouraged…

Accuracy of the MacArthur competence assessment tool for clinical research (MacCAT-CR) for measuring children's competence to consent to clinical research.

PubMed

Hein, Irma M; Troost, Pieter W; Lindeboom, Robert; Benninga, Marc A; Zwaan, C Michel; van Goudoever, Johannes B; Lindauer, Ramón J L

2014-12-01

An objective assessment of children's competence to consent to research participation is currently not possible. Age limits for asking children's consent vary considerably between countries, and, to our knowledge, the correlation between competence and children's age has never been systematically investigated. To test a standardized competence assessment instrument for children by modifying the MacArthur Competence Assessment Tool for Clinical Research (MacCAT-CR), to investigate its reliability and validity, and to examine the correlation of its assessment with age and estimate cutoff ages. This prospective study included children and adolescents aged 6 to 18 years in the inpatient and outpatient departments of allergology, gastroenterology, oncology, ophthalmology, and pulmonology from January 1, 2012, through January 1, 2014. Participants were eligible for clinical research studies, including observational studies and randomized clinical trials. Competence judgments by experts aware of the 4 relevant criteria-understanding, appreciation, reasoning, and choice-were used to establish the reference standard. The index test was the MacCAT-CR, which used a semistructured interview format. Interrater reliability, validity, and dimensionality of the MacCAT-CR and estimated cutoff ages for competence. Of 209 eligible patients, we included 161 (mean age, 10.6 years; 47.2% male). Good reproducibility of MacCAT-CR total and subscale scores was observed (intraclass correlation coefficient range, 0.68-0.92). We confirmed unidimensionality of the MacCAT-CR. By the reference standard, we judged 54 children (33.5%) to be incompetent; by the MacCAT-CR, 61 children (37.9%). Criterion-related validity of MacCAT-CR scores was supported by high overall accuracy in correctly classifying children as competent against the reference standard (area under the receiver operating characteristics curve, 0.78). Age was a good predictor of competence on the MacCAT-CR (area under the receiver operating characteristics curve, 0.90). In children younger than 9.6 years, competence was unlikely (sensitivity, 90%); in those older than 11.2 years, competence was probable (specificity, 90%). The optimal cutoff age was 10.4 years (sensitivity, 81%; specificity, 84%). The MacCAT-CR demonstrated strong psychometric properties. In children aged 9.6 to 11.2 years, consent may be justified when competence can be demonstrated in individual cases by the MacCAT-CR. The results contribute to a scientific underpinning of regulations for clinical research directed toward children.

Do we develop public health leaders?- association between public health competencies and emotional intelligence: a cross-sectional study

PubMed Central

2014-01-01

Background Professional development of public health leaders requires a form of instruction which is competency-based to help them develop the abilities to address complex and evolving demands of health care systems. Concurrently, emotional intelligence (EI) is a key to organisational success. Our aim was twofold: i) to assess the relationship between the level of self-assessed public health and EI competencies among Master of European Public Health (MEPH) students and graduates at Maastricht University, and; ii) to determine the relationship between different groups of public health competencies and specific EI skills. Methods A cross-sectional study was conducted including all recent MEPH graduates and students from 2009–2012, out of 67 eligible candidates N = 51 were contacted and N = 33 responded (11 males and 22 females; overall response: 64.7%).Two validated tools were employed: i) public health competencies self-assessment questionnaire, and; ii) Assessing Emotions Scale. Results Females scored higher than males in all seven domains of the self-assessed key public health competencies (NS) and emotional intelligence competences (P = 0.022). Overall, the mean value of public health competencies was the lowest in students with “staff” preferences and the highest among students with mixed job preferences (P < 0.001). There was evidence of a correlation between the overall public health competencies and the overall emotional intelligence competencies (r = 0.61, P < 0.001). Conclusions The study shows a positive correlation between public health specific competencies and EI attributes. It can contribute to the improvement of the educational content of PH curricula by rising awareness through self-assessment and supporting the identification of further educational needs related to leadership. PMID:24742091

Do we develop public health leaders?- association between public health competencies and emotional intelligence: a cross-sectional study.

PubMed

Czabanowska, Katarzyna; Malho, André; Schröder-Bäck, Peter; Popa, Daniela; Burazeri, Genc

2014-04-17

Professional development of public health leaders requires a form of instruction which is competency-based to help them develop the abilities to address complex and evolving demands of health care systems. Concurrently, emotional intelligence (EI) is a key to organisational success. Our aim was twofold: i) to assess the relationship between the level of self-assessed public health and EI competencies among Master of European Public Health (MEPH) students and graduates at Maastricht University, and; ii) to determine the relationship between different groups of public health competencies and specific EI skills. A cross-sectional study was conducted including all recent MEPH graduates and students from 2009-2012, out of 67 eligible candidates N = 51 were contacted and N = 33 responded (11 males and 22 females; overall response: 64.7%).Two validated tools were employed: i) public health competencies self-assessment questionnaire, and; ii) Assessing Emotions Scale. Females scored higher than males in all seven domains of the self-assessed key public health competencies (NS) and emotional intelligence competences (P = 0.022). Overall, the mean value of public health competencies was the lowest in students with "staff" preferences and the highest among students with mixed job preferences (P < 0.001). There was evidence of a correlation between the overall public health competencies and the overall emotional intelligence competencies (r = 0.61, P < 0.001). The study shows a positive correlation between public health specific competencies and EI attributes. It can contribute to the improvement of the educational content of PH curricula by rising awareness through self-assessment and supporting the identification of further educational needs related to leadership.

Using cloud-based mobile technology for assessment of competencies among medical students.

PubMed

Ferenchick, Gary S; Solomon, David

2013-01-01

Valid, direct observation of medical student competency in clinical settings remains challenging and limits the opportunity to promote performance-based student advancement. The rationale for direct observation is to ascertain that students have acquired the core clinical competencies needed to care for patients. Too often student observation results in highly variable evaluations which are skewed by factors other than the student's actual performance. Among the barriers to effective direct observation and assessment include the lack of effective tools and strategies for assuring that transparent standards are used for judging clinical competency in authentic clinical settings. We developed a web-based content management system under the name, Just in Time Medicine (JIT), to address many of these issues. The goals of JIT were fourfold: First, to create a self-service interface allowing faculty with average computing skills to author customizable content and criterion-based assessment tools displayable on internet enabled devices, including mobile devices; second, to create an assessment and feedback tool capable of capturing learner progress related to hundreds of clinical skills; third, to enable easy access and utilization of these tools by faculty for learner assessment in authentic clinical settings as a means of just in time faculty development; fourth, to create a permanent record of the trainees' observed skills useful for both learner and program evaluation. From July 2010 through October 2012, we implemented a JIT enabled clinical evaluation exercise (CEX) among 367 third year internal medicine students. Observers (attending physicians and residents) performed CEX assessments using JIT to guide and document their observations, record their time observing and providing feedback to the students, and their overall satisfaction. Inter-rater reliability and validity were assessed with 17 observers who viewed six videotaped student-patient encounters and by measuring the correlation between student CEX scores and their scores on subsequent standardized-patient OSCE exams. A total of 3567 CEXs were completed by 516 observers. The average number of evaluations per student was 9.7 (±1.8 SD) and the average number of CEXs completed per observer was 6.9 (±15.8 SD). Observers spent less than 10 min on 43-50% of the CEXs and 68.6% on feedback sessions. A majority of observers (92%) reported satisfaction with the CEX. Inter-rater reliability was measured at 0.69 among all observers viewing the videotapes and these ratings adequately discriminated competent from non-competent performance. The measured CEX grades correlated with subsequent student performance on an end-of-year OSCE. We conclude that the use of JIT is feasible in capturing discrete clinical performance data with a high degree of user satisfaction. Our embedded checklists had adequate inter-rater reliability and concurrent and predictive validity.

Research Priorities in the Utilization and Interpretation of Diagnostic Imaging: Education, Assessment, and Competency.

PubMed

Lewiss, Resa E; Chan, Wilma; Sheng, Alexander Y; Soto, Jorge; Castro, Alexandra; Meltzer, Andrew C; Cherney, Alan; Kumaravel, Manickam; Cody, Dianna; Chen, Esther H

2015-12-01

The appropriate selection and accurate interpretation of diagnostic imaging is a crucial skill for emergency practitioners. To date, the majority of the published literature and research on competency assessment comes from the subspecialty of point-of-care ultrasound. A group of radiologists, physicists, and emergency physicians convened at the 2015 Academic Emergency Medicine consensus conference to discuss and prioritize a research agenda related to education, assessment, and competency in ordering and interpreting diagnostic imaging. A set of questions for the continued development of an educational curriculum on diagnostic imaging for trainees and competency assessment using specific assessment methods based on current best practices was delineated. The research priorities were developed through an iterative consensus-driven process using a modified nominal group technique that culminated in an in-person breakout session. The four recommendations are: 1) develop a diagnostic imaging curriculum for emergency medicine (EM) residency training; 2) develop, study, and validate tools to assess competency in diagnostic imaging interpretation; 3) evaluate the role of simulation in education, assessment, and competency measures for diagnostic imaging; 4) study is needed regarding the American College of Radiology Appropriateness Criteria, an evidence-based peer-reviewed resource in determining the use of diagnostic imaging, to maximize its value in EM. In this article, the authors review the supporting reliability and validity evidence and make specific recommendations for future research on the education, competency, and assessment of learning diagnostic imaging. © 2015 by the Society for Academic Emergency Medicine.

Objective assessment of technique in laparoscopic colorectal surgery: what are the existing tools?

PubMed

Foster, J D; Francis, N K

2015-01-01

Assessment can improve the effectiveness of surgical training and enable valid judgments of competence. Laparoscopic colon resection surgery is now taught within surgical residency programs, and assessment tools are increasingly used to stimulate formative feedback and enhance learning. Formal assessment of technical performance in laparoscopic colon resection has been successfully applied at the specialist level in the English "LAPCO" National Training Program. Objective assessment tools need to be developed for training and assessment in laparoscopic rectal cancer resection surgery. Simulation may have a future role in assessment and accreditation in laparoscopic colorectal surgery; however, existing virtual reality models are not ready to be used for assessment of this advanced surgery.

A 2-year study of patient safety competency assessment in 29 clinical laboratories.

PubMed

Reed, Robyn C; Kim, Sara; Farquharson, Kara; Astion, Michael L

2008-06-01

Competency assessment is critical for laboratory operations and is mandated by the Clinical Laboratory Improvement Amendments of 1988. However, no previous reports describe methods for assessing competency in patient safety. We developed and implemented a Web-based tool to assess performance of 875 laboratory staff from 29 laboratories in patient safety. Question categories included workplace culture, categorizing error, prioritization of patient safety interventions, strength of specific interventions, and general patient safety concepts. The mean score was 85.0%, with individual scores ranging from 56% to 100% and scores by category from 81.3% to 88.6%. Of the most difficult questions (<72% correct), 6 were about intervention strength, 3 about categorizing error, 1 about workplace culture, and 1 about prioritization of interventions. Of the 13 questions about intervention strength, 6 (46%) were in the lowest quartile, suggesting that this may be a difficult topic for laboratory technologists. Computer-based competency assessments help laboratories identify topics for continuing education in patient safety.

Echo simulator with novel training and competency testing tools.

PubMed

Sheehan, Florence H; Otto, Catherine M; Freeman, Rosario V

2013-01-01

We developed and validated an echo simulator with three novel tools that facilitate training and enable quantitative and objective measurement of psychomotor as well as cognitive skill. First, the trainee can see original patient images - not synthetic or simulated images - that morph in real time as the mock transducer is manipulated on the mannequin. Second, augmented reality is used for Visual Guidance, a tool that assists the trainee in scanning by displaying the target organ in 3-dimensions (3D) together with the location of the current view plane and the plane of the anatomically correct view. Third, we introduce Image Matching, a tool that leverages the aptitude of the human brain for recognizing similarities and differences to help trainees learn to perform visual assessment of ultrasound images. Psychomotor competence is measured in terms of the view plane angle error. The construct validity of the simulator for competency testing was established by demonstrating its ability to discriminate novices vs. experts.

Assessing competency in practice-based learning: a foundation for milestones in learning portfolio entries.

PubMed

Webb, Travis P; Merkley, Taylor R; Wade, Thomas J; Simpson, Deborah; Yudkowsky, Rachel; Harris, Ilene

2014-01-01

Graduate medical education is undergoing a dramatic shift toward competency-based assessment of learners. Competency assessment requires clear definitions of competency and validated assessment methods. The purpose of this study is to identify criteria used by surgical educators to judge competence in Practice-Based Learning and Improvement (PBL&I) as demonstrated in learning portfolios. A total of 6 surgical learning and instructional portfolio entries served as documents to be assessed by 3 senior surgical educators. These faculty members were asked to rate and then identify criteria used to assess PBL&I competency. Individual interviews and group discussions were conducted, recorded, and transcribed to serve as the study dataset. Analysis was performed using qualitative methodology to identify themes for the purpose of defining competence in PBL&I. The assessment themes derived are presented with narrative examples to describe the progression of competency. The collaborative coding process resulted in identification of 7 themes associated with competency in PBL&I related to surgical learning and instructional portfolio entries: (1) self-awareness regarding effect of actions; (2) identification and thorough description of learning goals; (3) cases used as catalyst for reflection; (4) reconceptualization with appropriate use and critique of cited literature; (5) communication skills/completeness of entry template; (6) description of future behavioral change; and (7) engagement in process--identifies as personally relevant. The identified themes are consistent with and complement other criteria emerging from reflective practice literature and experiential learning theory. This study provides a foundation for further development of a tool for assessing learner portfolios consistent with the Accreditation Council for Graduate Medical Education's Next Accreditation System requirements. Copyright © 2014 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Establishing pass/fail criteria for bronchoscopy performance.

PubMed

Konge, Lars; Clementsen, Paul; Larsen, Klaus Richter; Arendrup, Henrik; Buchwald, Christian; Ringsted, Charlotte

2012-01-01

Several tools have been created to assess competence in bronchoscopy. However, educational guidelines still use an arbitrary number of performed procedures to decide when basic competency is acquired. The purpose of this study was to define pass/fail scores for two bronchoscopy assessment tools, and investigate how these scores relate to physicians' experience regarding the number of bronchoscopy procedures performed. We studied two assessment tools and used two standard setting methods to create cut scores: the contrasting-groups method and the extended Angoff method. In the first we compared bronchoscopy performance scores of 14 novices with the scores of 14 experienced consultants to find the score that best discriminated between the two groups. In the second we asked an expert group of 7 experienced bronchoscopists to judge how a borderline trainee would perform on each item of the test. Using the contrasting-groups method we found a standard that would fail all novices and pass all consultants. A clear pass related to prior experience of 75 procedures. The consequences of using the extended Angoff method were also acceptable: all trainees who had performed less than 50 bronchoscopies failed the test and all consultants passed. A clear pass related to 80 procedures. Our proposed pass/fail scores for these two methods seem appropriate in terms of consequences. Prior experience with the performance of 75 and 80 bronchoscopies, respectively, seemed to ensure basic competency. In the future objective assessment tools could become an important aid in the certification of physicians performing bronchoscopies. Copyright © 2011 S. Karger AG, Basel.

Broadening Perspectives on Clinical Performance Assessment: Rethinking the Nature of In-Training Assessment

ERIC Educational Resources Information Center

Govaerts, Marjan J. B.; van der Vleuten, Cees P. M.; Schuwirth, Lambert W. T.; Muijtjens, Arno M. M.

2007-01-01

Context: In-training assessment (ITA), defined as multiple assessments of performance in the setting of day-to-day practice, is an invaluable tool in assessment programmes which aim to assess professional competence in a comprehensive and valid way. Research on clinical performance ratings, however, consistently shows weaknesses concerning…

Continued Validation of the O-SCORE (Ottawa Surgical Competency Operating Room Evaluation): Use in the Simulated Environment.

PubMed

MacEwan, Matthew J; Dudek, Nancy L; Wood, Timothy J; Gofton, Wade T

2016-01-01

CONSTRUCT: The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) is a 9-item surgical evaluation tool designed to assess technical competence in surgical trainees using behavioral anchors. The initial development of the O-SCORE produced evidence for valid results. Further work is required to determine if the use of a single surgeon or an unblinded rater introduces bias. In addition, the relationship of the O-SCORE to other currently used technical assessment tools should be explored to provide validity evidence related to the relationship to other measures. We have designed this project to provide continued validity evidence for the O-SCORE related to these two issues. Nineteen residents and 2 staff Orthopedic Surgeons from the University of Ottawa volunteered to participate in a 2-part OSCE style station. Participants completed a written questionnaire followed by a videotaped 10-minute simulated open reduction and internal fixation of a midshaft radius fracture. Videos were rated individually by 2 blinded staff orthopedic surgeons using an Objective Structured Assessment of Technical Skills (OSATS) global rating scale, an OSATS checklist, and the O-SCORE in random order. O-SCORE results appeared sensitive to surgical training level even when raters were blinded. In addition, strong agreement between two independent observers using the O-SCORE suggests that the measure captures a performance easily recognized by surgical observers. Ratings on the O-SCORE also were strongly associated with global ratings on the currently most validated technical evaluation tool (OSATS). Collectively, these results suggest that the O-SCORE generates accurate, reproducible, and meaningful results when used in a randomized and blinded fashion, providing continued validity evidence for using this tool to evaluate surgical trainee competence. The O-SCORE was able to differentiate surgical trainee level using blinded raters providing further evidence of validity for the O-SCORE. There was strong agreement between two independent observers using the O-SCORE. Ratings on the O-SCORE also demonstrated equivalence to scores on the most validated technical evaluation tool (OSATS). These results suggest that the O-SCORE demonstrates accurate and reproducible results when used in a randomized and blinded fashion providing continued validity evidence for this tool in the evaluation of surgical competence in the trainees.

Competency checklists for strabismus surgery and retinopathy of prematurity examination.

PubMed

McClatchey, Scott K; Lane, R Gary; Kubis, Kenneth C; Boisvert, Chantal

2012-02-01

To evaluate two checklist tools that are designed to guide, document, and assess resident training in strabismus surgery and examination of infants at risk for retinopathy of prematurity (ROP). A panel of staff surgeons from several teaching institutions evaluated the checklists and provided constructive feedback. All former residents who had been trained via the use of these checklist tools were asked to take self-assessment surveys on competency in strabismus surgery and ROP examination. A Likert 5-point scale was used for all evaluations, with 1 being the lowest rating and 5 the highest rating. Six experts in strabismus and seven in ROP rated the checklists. Their comments were used to revise the checklists, which were sent to the same group for reevaluation. The mean Likert score for the final checklists was 4.9 of 5.0 for both checklists. Of 16 former residents, 9 responded to the self-assessments with a mean overall score of 4.1 (of 5.0) for strabismus surgery and 3.9 for ROP examination. These checklist tools can be used to assess the quality of a resident's training and experience in these specific ophthalmology skills. They are complementary to other curriculum and assessment tools and can serve to organize the educational experience while ensuring a uniformity of training. Published by Mosby, Inc.

Competency measurements: testing convergent validity for two measures.

PubMed

Cowin, Leanne S; Hengstberger-Sims, Cecily; Eagar, Sandy C; Gregory, Linda; Andrew, Sharon; Rolley, John

2008-11-01

This paper is a report of a study to investigate whether the Australian National Competency Standards for Registered Nurses demonstrate correlations with the Finnish Nurse Competency Scale. Competency assessment has become popular as a key regulatory requirement and performance indicator. The term competency, however, does not have a globally accepted definition and this has the potential to create controversy, ambiguity and confusion. Variations in meaning and definitions adopted in workplaces and educational settings will affect the interpretation of research findings and have implications for the nursing profession. A non-experimental cross-sectional survey design was used with a convenience sample of 116 new graduate nurses in 2005. The second version of the Australian National Competency Standards and the Nurse Competency Scale was used to elicit responses to self-assessed competency in the transitional year (first year as a Registered Nurse). Correlational analysis of self-assessed levels of competence revealed a relationship between the Australian National Competency Standards (ANCI) and the Nurse Competency Scale (NCS). The correlational relation between ANCI domains and NCS factors suggests that these scales are indeed used to measure related dimensions. A statistically significant relationship (r = 0.75) was found between the two competency measures. Although the finding of convergent validity is insufficient to establish construct validity for competency as used in both measures in this study, it is an important step towards this goal. Future studies on relationships between competencies must take into account the validity and reliability of the tools.

Relationship between financial competence and cognitive function in patients with schizophrenia.

PubMed

Niekawa, Nobuyuki; Sakuraba, Yukie; Uto, Hanae; Kumazawa, Yoshiko; Matsuda, Osamu

2007-10-01

The present study examined financial competence in patients with schizophrenia and the relationship between their financial competence and cognitive function. The subjects consisted of 25 patients with schizophrenia (10 inpatients and 15 outpatients) and 22 normal controls who were community-dwelling people with no psychiatric disorders or cognitive deficit. To assess the subjects' cognitive function and financial competence, they completed the Japanese version of the Neurobehavioral Cognitive Status Examination (COGNISTAT), which has 10 subtests, and the Financial Competency Assessment Tool (FCAT), which has six subordinate domains of financial competence. Patients with schizophrenia performed significantly worse than the controls in all scores on the FCAT. The financial scores that were significantly different between the patients and the normal controls were significantly positively correlated with the scores on several COGNISTAT subtests (e.g. comprehension). These results suggest that patients with schizophrenia have problems with financial competence and that these problems may be accounted for by deficits in several cognitive functions.

Using student writing assignments to assess critical thinking skills: a holistic approach.

PubMed

Niedringhaus, L K

2001-04-01

This work offers an example of one school's holistic approach to the evaluation of critical thinking by using student writing assignments. Faculty developed tools to assess achievement of critical thinking competencies, such as analysis, synthesis, insight, reflection, open mindedness, and depth, breadth, and appropriateness of clinical interventions. Faculty created a model for the development of program-specific critical thinking competencies, selected appropriate writing assignments that demonstrate critical thinking, and implemented a holistic assessment plan for data collection and analysis. Holistic assessment involves the identification of shared values and practices, and the use of concepts and language important to nursing.

Use of a Standardized Patient Exercise to Assess Core Competencies During Fellowship Training

PubMed Central

Barry, Curtis T.; Avissar, Uri; Asebrook, Maureen; Sostok, Michael A.; Sherman, Kenneth E.; Zucker, Stephen D.

2010-01-01

Background The Accreditation Council for Graduate Medical Education requires fellows in many specialties to demonstrate attainment of 6 core competencies, yet relatively few validated assessment tools currently exist. We present our initial experience with the design and implementation of a standardized patient (SP) exercise during gastroenterology fellowship that facilitates appraisal of all core clinical competencies. Methods Fellows evaluated an SP trained to portray an individual referred for evaluation of abnormal liver tests. The encounters were independently graded by the SP and a faculty preceptor for patient care, professionalism, and interpersonal and communication skills using quantitative checklist tools. Trainees' consultation notes were scored using predefined key elements (medical knowledge) and subjected to a coding audit (systems-based practice). Practice-based learning and improvement was addressed via verbal feedback from the SP and self-assessment of the videotaped encounter. Results Six trainees completed the exercise. Second-year fellows received significantly higher scores in medical knowledge (55.0 ± 4.2 [standard deviation], P  =  .05) and patient care skills (19.5 ± 0.7, P  =  .04) by a faculty evaluator as compared with first-year trainees (46.2 ± 2.3 and 14.7 ± 1.5, respectively). Scores correlated by Spearman rank (0.82, P  =  .03) with the results of the Gastroenterology Training Examination. Ratings of the fellows by the SP did not differ by level of training, nor did they correlate with faculty scores. Fellows viewed the exercise favorably, with most indicating they would alter their practice based on the experience. Conclusions An SP exercise is an efficient and effective tool for assessing core clinical competencies during fellowship training. PMID:21975896

Competence in advanced older people nursing: development of 'nursing older people--competence evaluation tool'.

PubMed

Bing-Jonsson, Pia Cecilie; Bjørk, Ida Torunn; Hofoss, Dag; Kirkevold, Marit; Foss, Christina

2015-03-01

Community care is characterised by a move from institutionalised to home-based care, a large patient population with comorbidities including cognitive failure, and nurses who struggle to keep up with their many competence demands. No study has examined the competence of nurses based on present demands, and an instrument for this purpose is lacking. We conducted a Delphi study based in Norway to develop the substantial content of a new competence measurement instrument. We sought to reach consensus regarding which nursing staff competence is most relevant to meet the current needs of older patients. A total of 42 experts participated in three consecutive panel investigations. Snowball sampling was used. The experts were clinicians, leaders, teachers, researchers and relatives of older people who required nursing. In Round 1, all experts were interviewed individually. These data were analysed using meaning coding and categorisation. In Rounds 2 and 3, the data were collected using electronic questionnaires and analysed quantitatively with SPSS. The experts agreed that health promotion as well as disease prevention, treatment, palliative care, ethics and regulation, assessment and taking action, covering basic needs, communication and documentation, responsibility and activeness, cooperation, and attitudes towards older people were the most relevant categories of competence. The experts showed clear consensus regarding the most relevant and current competence for nurses of older people. Assuming that older people in need of health care have the same requirements across cultures, this study's findings could be used as a basis for international studies. Those who nurse older people require competence that is complex and comprehensive. One way to evaluate nursing competence is through evaluation tools such as the Nursing Older People--Competence Evaluation tool. © 2014 John Wiley & Sons Ltd.

Rater Training to Support High-Stakes Simulation-Based Assessments

ERIC Educational Resources Information Center

Feldman, Moshe; Lazzara, Elizabeth H.; Vanderbilt, Allison A.; DiazGranados, Deborah

2012-01-01

Competency-based assessment and an emphasis on obtaining higher-level outcomes that reflect physicians' ability to demonstrate their skills has created a need for more advanced assessment practices. Simulation-based assessments provide medical education planners with tools to better evaluate the 6 Accreditation Council for Graduate Medical…

Development and Early Piloting of a CanMEDS Competency-Based Feedback Tool for Surgical Grand Rounds.

PubMed

Fahim, Christine; Bhandari, Mohit; Yang, Ilun; Sonnadara, Ranil

2016-01-01

Grand rounds offer an excellent opportunity for the evaluation of medical expertise, and other competencies, such as communication and professionalism. The purpose of this study was to develop a tool that would facilitate the provision of formative feedback for grand rounds to improve learning. The resulting CanMEDS-based evaluation tool was piloted in an academic surgical department. This study employed the use of a 3-phase, qualitatively-focused, embedded mixed methods approach. In Phase 1, an intrinsic case study was conducted to identify preliminary themes. These findings were crystallized using a quantitative survey. Following interpretation of these data, a grand rounds evaluation tool was developed in Phase 2. The tool was piloted in the Phase 3 focus group. This study was piloted at an academic surgical center among members of the Department of Surgery, McMaster University, Ontario, Canada. Purposive sampling was used for this study. A total of n = 7 individuals participated in the Phase 1 interviews, and n = 24 participants completed the Phase 1 survey. Participants included a representative sample of medical students, residents, fellows, and staff. The tool was piloted among n = 19 participants. The proposed evaluation tool contains 13 Likert-scale questions and 2 open-ended questions. The tool outlines specific questions to assess grand rounds presenters within the structure of the 7 CanMEDS competency domains. "Evaluation fatigue" was identified as a major barrier in the willingness to provide effective feedback. Further, a number of factors regarding the preferred content, structure, and format of surgical grand rounds were identified. This pilot study presents a CanMEDS-specific evaluation tool that can be applied to surgical grand rounds. With the increasing adoption of competency-based medical education, comprehensive evaluation of surgical activities is required. This form provides a template for the development of competency-based evaluation tools for medical and surgical learning activities. Copyright © 2016 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Derivation of a Performance Checklist for Ultrasound-Guided Arthrocentesis Using the Modified Delphi Method.

PubMed

Kunz, Derek; Pariyadath, Manoj; Wittler, Mary; Askew, Kim; Manthey, David; Hartman, Nicholas

2017-06-01

Arthrocentesis is an important skill for physicians in multiple specialties. Recent studies indicate a superior safety and performance profile for this procedure using ultrasound guidance for needle placement, and improving quality of care requires a valid measurement of competency using this modality. We endeavored to create a validated tool to assess the performance of this procedure using the modified Delphi technique and experts in multiple disciplines across the United States. We derived a 22-item checklist designed to assess competency for the completion of ultrasound-guided arthrocentesis, which demonstrated a Cronbach's alpha of 0.89, indicating an excellent degree of internal consistency. Although we were able to demonstrate content validity for this tool, further validity evidence should be acquired after the tool is used and studied in clinical and simulated contexts. © 2017 by the American Institute of Ultrasound in Medicine.

Impact of cap-assisted colonoscopy on the learning curve and quality in colonoscopy: a randomized controlled trial.

PubMed

Tang, Zhouwen; Zhang, Daniel S; Thrift, Aaron P; Patel, Kalpesh K

2018-03-01

Colonoscopy competency assessment in trainees traditionally has been informal. Comprehensive metrics such as the Assessment of Competency in Endoscopy (ACE) tool suggest that competency thresholds are higher than assumed. Cap-assisted colonoscopy (CAC) may improve competency, but data regarding novice trainees are lacking. We compared CAC versus standard colonoscopy (SC) performed by novice trainees in a randomized controlled trial. All colonoscopies performed by 3 gastroenterology fellows without prior experience were eligible for the study. Exclusion criteria included patient age <18 or >90 years, pregnancy, prior colon resection, diverticulitis, colon obstruction, severe hematochezia, referral for EMR, or a procedure done without patient sedation. Patients were randomized to either CAC or SC in a 1:1 fashion. The primary outcome was the independent cecal intubation rate (ICIR). Secondary outcomes were cecal intubation time, polyp detection rate, polyp miss rate, adenoma detection rate, ACE tool scores, and cumulative summation learning curves. A total of 203 colonoscopies were analyzed, 101 in CAC and 102 in SC. CAC resulted in a significantly higher cecal intubation rate, at 79.2% in CAC compared with 66.7% in SC (P = .04). Overall cecal intubation time was significantly shorter at 13.7 minutes for CAC versus 16.5 minutes for SC (P =.02). Cecal intubation time in the case of successful independent fellow intubation was not significantly different between CAC and SC (11.6 minutes vs 12.7 minutes; P = .29). Overall ACE tool motor and cognitive scores were higher with CAC. Learning curves for ICIR approached the competency threshold earlier with cap use but reached competency for only 1 fellow. The polyp detection rate, polyp miss rate, and adenoma detection rate were not significantly different between groups. CAC resulted in significant improvement in ICIR, overall ACE tool scores, and trend toward competency on learning curves when compared with SC in colonoscopy trainees without prior colonoscopy experience. (Clinical trial registration number: NCT02472730.). Copyright © 2018 American Society for Gastrointestinal Endoscopy. Published by Elsevier Inc. All rights reserved.

Reliability and validity of the Nurse Practitioners' Roles and Competencies Scale.

PubMed

Lin, Li-Chun; Lee, Sheuan; Ueng, Steve Wen-Neng; Tang, Woung-Ru

2016-01-01

The objective of this study was to test the reliability and construct validity of the Nurse Practitioners' Roles and Competencies Scale. The role of nurse practitioners has attracted international attention. The advanced nursing role played by nurse practitioners varies with national conditions and medical environments. To date, no suitable measurement tool has been available for assessing the roles and competencies of nurse practitioners in Asian countries. Secondary analysis of data from three studies related to nurse practitioners' role competencies. We analysed data from 563 valid questionnaires completed in three studies to identify the factor structure of the Nurse Practitioners' Roles and Competencies Scale. To this end, we performed exploratory factor analysis using principal component analysis extraction with varimax orthogonal rotation. The internal consistency reliabilities of the overall scale and its subscales were examined using Cronbach's alpha coefficient. The scale had six factors: professionalism, direct care, clinical research, practical guidance, medical assistance, as well as leadership and reform. These factors explained 67·5% of the total variance in nurse practitioners' role competencies. Cronbach's alpha coefficient for the overall scale was 0·98, and those of its subscales ranged from 0·83-0·97. The internal consistency reliability and construct validity of the Nurse Practitioners' Roles and Competencies Scale were good. The high internal consistency reliabilities suggest item redundancy, which should be minimised by using item response theory to enhance the applicability of this questionnaire for future academic and clinical studies. The Nurse Practitioners' Roles and Competencies Scale can be used as a tool for assessing the roles and competencies of nurse practitioners in Taiwan. Our findings can also serve as a reference for other Asian countries to develop the nurse practitioner role. © 2015 John Wiley & Sons Ltd.

Developing a model of competence in the operating theatre: psychometric validation of the perceived perioperative competence scale-revised.

PubMed

Gillespie, Brigid M; Polit, Denise F; Hamlin, Lois; Chaboyer, Wendy

2012-01-01

This paper describes the development and validation of the Revised Perioperative Competence Scale (PPCS-R). There is a lack of a psychometrically tested sound self-assessment tools to measure nurses' perceived competence in the operating room. Content validity was established by a panel of international experts and the original 98-item scale was pilot tested with 345 nurses in Queensland, Australia. Following the removal of several items, a national sample that included all 3209 nurses who were members of the Australian College of Operating Room Nurses was surveyed using the 94-item version. Psychometric testing assessed content validity using exploratory factor analysis, internal consistency using Cronbach's alpha, and construct validity using the "known groups" technique. During item reduction, several preliminary factor analyses were performed on two random halves of the sample (n=550). Usable data for psychometric assessment were obtained from 1122 nurses. The original 94-item scale was reduced to 40 items. The final factor analysis using the entire sample resulted in a 40 item six-factor solution. Cronbach's alpha for the 40-item scale was .96. Construct validation demonstrated significant differences (p<.0001) in perceived competence scores relative to years of operating room experience and receipt of specialty education. On the basis of these results, the psychometric properties of the PPCS-R were considered encouraging. Further testing of the tool in different samples of operating room nurses is necessary to enable cross-cultural comparisons. Copyright © 2011 Elsevier Ltd. All rights reserved.

Self-assessed efficacy of a clinical musculoskeletal anatomy workshop: A preliminary survey.

PubMed

Saavedra, Miguel Ángel; Navarro-Zarza, José E; Alvarez-Nemegyei, José; Canoso, Juan J; Kalish, Robert A; Villaseñor-Ovies, Pablo; Hernández-Díaz, Cristina

2015-01-01

To survey the efficacy of a practical workshop on clinical musculoskeletal anatomy held in five American countries. A self-assessment competence questionnaire sent to participants 1-3 months after the workshop. Results were compared to the results of a practical, instructor-assessed, pre-workshop test. The response rate of participants was 76.4%. The overall, self-assessed competence score for anatomical items that had been included in the pre-test was 76.9 (scale 0-100) as compared to an overall score of 48.1 in the practical, pre-workshop test (p<0.001). For items that were addressed in the workshop, but not included in the pre-test, self-assessed competence was rated at 62.9. Differences in anatomical knowledge between individuals from different countries and professional groups noted in the practical pre-test were no longer present in the post-test self-assessment. From this preliminary data and supporting evidence from the literature we believe that our anatomy workshop provides an effective didactic tool for increasing competence in musculoskeletal anatomy. Copyright © 2014 Elsevier España, S.L.U. All rights reserved.

Patient safety competencies in undergraduate nursing students: a rapid evidence assessment.

PubMed

Bianchi, Monica; Bressan, Valentina; Cadorin, Lucia; Pagnucci, Nicola; Tolotti, Angela; Valcarenghi, Dario; Watson, Roger; Bagnasco, Annamaria; Sasso, Loredana

2016-12-01

To identify patient safety competencies, and determine the clinical learning environments that facilitate the development of patient safety competencies in nursing students. Patient safety in nursing education is of key importance for health professional environments, settings and care systems. To be effective, safe nursing practice requires a good integration between increasing knowledge and the different clinical practice settings. Nurse educators have the responsibility to develop effective learning processes and ensure patient safety. Rapid Evidence Assessment. MEDLINE, CINAHL, SCOPUS and ERIC were searched, yielding 500 citations published between 1 January 2004-30 September 2014. Following the Rapid Evidence Assessment process, 17 studies were included in this review. Hawker's (2002) quality assessment tool was used to assess the quality of the selected studies. Undergraduate nursing students need to develop competencies to ensure patient safety. The quality of the pedagogical atmosphere in the clinical setting has an important impact on the students' overall level of competence. Active student engagement in clinical processes stimulates their critical reasoning, improves interpersonal communication and facilitates adequate supervision and feedback. Few studies describe the nursing students' patient safety competencies and exactly what they need to learn. In addition, studies describe only briefly which clinical learning environments facilitate the development of patient safety competencies in nursing students. Further research is needed to identify additional pedagogical strategies and the specific characteristics of the clinical learning environments that encourage the development of nursing students' patient safety competencies. © 2016 John Wiley & Sons Ltd.

Feasibility and Outcomes of Implementing a Portfolio Assessment System Alongside a Traditional Grading System.

PubMed

O'Brien, Celia Laird; Sanguino, Sandra M; Thomas, John X; Green, Marianne M

2016-11-01

Portfolios are a powerful tool to collect and evaluate evidence of medical students' competence across time. However, comprehensive portfolio assessment systems that are implemented alongside traditional graded curricula at medical schools in the United States have not been described in the literature. This study describes the development and implementation of a longitudinal competency-based electronic portfolio system alongside a graded curriculum at a relatively large U.S. medical school. In 2009, the authors developed a portfolio system that served as a repository for all student assessments organized by competency domain. Five competencies were selected for a preclerkship summative portfolio review. Students submitted reflections on their performance. In 2014, four clinical faculty members participated in standard-setting activities and used expert judgment and holistic review to rate students' competency achievement as "progressing toward competence," "progressing toward competence with some concern," or "progressing toward competence pending remediation." Follow-up surveys measured students' and faculty members' perceptions of the process. Faculty evaluated 156 portfolios and showed high levels of agreement in their ratings. The majority of students achieved the "progressing toward competence" benchmark in all competency areas. However, 31 students received at least one concerning rating, which was not reflected in their course grades. Students' perceptions of the system's ability to foster self-assessment were mixed. The portfolio review process allowed faculty to identify students with a concerning rating in a behavioral competency who would not have been identified in a traditional grading system. Identification of these students allows for intervention and early remediation.

Preliminary Turkish study of psychiatric in-patients' competence to make treatment decisions.

PubMed

Aydin Er, Rahime; Sehiralti, Mine; Aker, Ahmet Tamer

2013-03-01

Competence is a prerequisite for informed consent. Patients who are found to be competent are entitled to accept or refuse the proposed treatment. In recent years, there has been an increased interest in studies examining competence for treatment in psychiatric patients. In this study, we aimed to investigate the decision-making competencies of inpatients with a range of psychiatric diseases. This study was carried out at the psychiatry clinic of Kocaeli University Hospital in Turkey from June 2007 to February 2008. Decision-making competence was assessed in 83 patients using the MacArthur Competence Assessment Tool-Treatment (MacCAT-T). The study groups consisted of patients with mood (39.8%), psychotic (27.7%) and anxiety disorders (18.1%), and alcohol/substance addiction (14.5%). There was a significant relation between decision-making competence and demographic and clinical characteristics. Appreciation of the given information was more impaired in psychotic disorder patients than in other patients, but understanding and reasoning of the given information was similar in all groups. These results reveal the importance of evaluating decision-making competencies of psychiatric patients before any treatment or intervention is carried out to ascertain their ability to give informed consent to treatment. Institutional and national policies need to be determined and put into practice relating to the assessment and management of competence in patients with psychiatric disorders. Copyright © 2012 Wiley Publishing Asia Pty Ltd.

Working in disadvantaged communities: What additional competencies do we need?

PubMed Central

Harris, Elizabeth; Harris, Mark F; Madden, Lynne; Wise, Marilyn; Sainsbury, Peter; MacDonald, John; Gill, Betty

2009-01-01

Background Residents of socioeconomically disadvantaged locations are more likely to have poor health than residents of socioeconomically advantaged locations and this has been comprehensively mapped in Australian cities. These inequalities present a challenge for the public health workers based in or responsible for improving the health of people living in disadvantaged localities. The purpose of this study was to develop a generic workforce needs assessment tool and to use it to identify the competencies needed by the public health workforce to work effectively in disadvantaged communities. Methods A two-step mixed method process was used to identify the workforce needs. In step 1 a generic workforce needs assessment tool was developed and applied in three NSW Area Health Services using focus groups, key stakeholder interviews and a staff survey. In step 2 the findings of this needs assessment process were mapped against the existing National Health Training Package (HLT07) competencies, gaps were identified, additional competencies described and modules of training developed to fill identified gaps. Results There was a high level of agreement among the AHS staff on the nature of the problems to be addressed but less confidence indentifying the work to be done. Processes for needs assessments, community consultations and adapting mainstream programs to local needs were frequently mentioned as points of intervention. Recruiting and retaining experienced staff to work in these communities and ensuring their safety were major concerns. Workforce skill development needs were seen in two ways: higher order planning/epidemiological skills and more effective working relationships with communities and other sectors. Organisational barriers to effective practice were high levels of annual compulsory training, balancing state and national priorities with local needs and giving equal attention to the population groups that are easy to reach and to those that are difficult to engage. A number of additional competency areas were identified and three training modules developed. Conclusion The generic workforce needs assessment tool was easy to use and interpret. It appears that the public health workforce involved in this study has a high level of understanding of the relationship between the social determinants and health. However there is a skill gap in identifying and undertaking effective intervention. PMID:19393091

A systematic review of clinical assessment for undergraduate nursing students.

PubMed

Wu, Xi Vivien; Enskär, Karin; Lee, Cindy Ching Siang; Wang, Wenru

2015-02-01

Consolidated clinical practicum prepares pre-registration nursing students to function as beginning practitioners. The clinical competencies of final-year nursing students provide a key indication of professional standards of practice and patient safety. Thus, clinical assessment of nursing students is a crucial issue for educators and administrators. The aim of this systematic review was to explore the clinical competency assessment for undergraduate nursing students. PubMed, CINAHL, ScienceDirect, Web of Science, and EBSCO were systematically searched from January 2000 to December 2013. The systematic review was in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Published quantitative and qualitative studies that examined clinical assessment practices and tools used in clinical nursing education were retrieved. Quality assessment, data extraction, and analysis were completed on all included studies. This review screened 2073 titles, abstracts and full-text records, resulting in 33 included studies. Two reviewers assessed the quality of the included studies. Fourteen quantitative and qualitative studies were identified for this evaluation. The evidence was ordered into emergent themes; the overarching themes were current practices in clinical assessment, issues of learning and assessment, development of assessment tools, and reliability and validity of assessment tools. There is a need to develop a holistic clinical assessment tool with reasonable level of validity and reliability. Clinical assessment is a robust activity and requires collaboration between clinical partners and academia to enhance the clinical experiences of students, the professional development of preceptors, and the clinical credibility of academics. Copyright © 2014 Elsevier Ltd. All rights reserved.

The accountability of clinical education: its definition and assessment.

PubMed

Murray, E; Gruppen, L; Catton, P; Hays, R; Woolliscroft, J O

2000-10-01

Medical education is not exempt from increasing societal expectations of accountability. Competition for financial resources requires medical educators to demonstrate cost-effective educational practice; health care practitioners, the products of medical education programmes, must meet increasing standards of professionalism; the culture of evidence-based medicine demands an evaluation of the effect educational programmes have on health care and service delivery. Educators cannot demonstrate that graduates possess the required attributes, or that their programmes have the desired impact on health care without appropriate assessment tools and measures of outcome. To determine to what extent currently available assessment approaches can measure potentially relevant medical education outcomes addressing practitioner performance, health care delivery and population health, in order to highlight areas in need of research and development. Illustrative publications about desirable professional behaviour were synthesized to obtain examples of required competencies and health outcomes. A MEDLINE search for available assessment tools and measures of health outcome was performed. There are extensive tools for assessing clinical skills and knowledge. Some work has been done on the use of professional judgement for assessing professional behaviours; scholarship; and multiprofessional team working; but much more is needed. Very little literature exists on assessing group attributes of professionals, such as clinical governance, evidence-based practice and workforce allocation, and even less on examining individual patient or population health indices. The challenge facing medical educators is to develop new tools, many of which will rely on professional judgement, for assessing these broader competencies and outcomes.

Use of web based systems to support postgraduate medical education.

PubMed

Tochel, C; Beggs, K; Haig, A; Roberts, J; Scott, H; Walker, K; Watson, M

2011-12-01

BACKGROUND To meet the demands of delivering the Foundation programme across a geographically diverse country, two web based systems (ePortfolio and eLearning) were developed to promote accessibility to training material and assessment tools on standardised platforms. This study evaluated the use of both tools throughout an entire academic year. METHODS All Scottish Foundation trainees' online learning and assessment data in 2007/08 were analysed, providing a national breakdown of post specialty, completion rates of mandatory assessments (including summary analysis of anonymised scores), and trainees' use of non-mandatory learning tools. Independent verification of competence data was sought from Deaneries. RESULTS There were high levels of engagement with both the ePortfolio (75-97% assessment completion) and eLearning systems (89-98% induction course completion), and the majority of trainees completed all required elements. There was extensive use of ePortfolio beyond mandatory levels for recording of learning events, including almost 20 000 personal learning records submitted by second year trainees. There was evidence that ePortfolio was used to record achievement of clinical competence rather than to track improvements towards competence (median workplace based assessment scores were 'high' or 'very high'). Online learning modules received positive feedback and its flexible format suited the trainees' working environment. External verification of formal assessment data revealed good correlation with locally stored outcomes, both indicating approximately 99% programme completion rates. CONCLUSIONS Core components of the Foundation programme have been delivered successfully to thousands of trainees across Scotland using web based systems to deliver and support education and assessment. There is great potential for further exploration of this carefully managed, rich dataset at individual, regional, and national levels to inform the future of medical education.

Exploring the Role of First Impressions in Rater-Based Assessments

ERIC Educational Resources Information Center

Wood, Timothy J.

2014-01-01

Medical education relies heavily on assessment formats that require raters to assess the competence and skills of learners. Unfortunately, there are often inconsistencies and variability in the scores raters assign. To ensure the scores from these assessment tools have validity, it is important to understand the underlying cognitive processes that…

[Current state of competence assessment in nursing].

PubMed

Darmann-Finck, Ingrid; Reuschenbach, Bernd

2013-01-01

Competency measurement is central to the optimisation of outcome oriented educational processes in nursing, similar to the concept of evidence based practice. The classification of measurement tools provides the basis for describing the current state of research and development in relation to competence measurement in nursing science, and any gaps are identified. The article concludes with questioning the importance of outcome oriented quality orientation in order to achieve an increase in quality during training. Further methodological developments and qualitative studies are needed to examine the context specific processes of interaction and learning, beyond competence diagnostics. Copyright © 2012. Published by Elsevier GmbH.

Development and Trialling of a Graduated Descriptors Tool for Australian Pharmacy Students

ERIC Educational Resources Information Center

Stupans, Ieva; Owen, Susanne; McKauge, Leigh; Pont, Lisa; Ryan, Greg; Woulfe, Jim

2012-01-01

Profession-derived competency standards are key determinants for curriculum and assessment in many professional university programmes. An Australian Learning and Teaching Council funded project used a participatory action research approach to enable the collaborative development of a graduated (or incremental) descriptors tool related to…

Evaluation of insight training of ambulance drivers in Sweden using DART, a new e-learning tool.

PubMed

Albertsson, Pontus; Sundström, Anna

2011-12-01

The aim of the study was to evaluate whether a new e-learning tool for insight training of ambulance drivers can have an effect on drivers' driving behaviors, perceived driving competence, competence to assess risks, self-reflection, and safety attitudes. A quasi-experimental study design, with participants nonrandomly assigned into a control and intervention group, was used. The intervention group participated in the insight-training course and the control group did not. Both groups completed a self- and peer assessment online questionnaire before and after the training. The main finding is that the ambulance drivers assessed themselves through the instruments after the training, with the e-learning tool Driver Access Recording Tool (DART), as safer drivers in the areas of speed adaptation, closing up, and overtaking. In the answers from the group-based evaluation, the ambulance drivers responded that they were more reflective/analytical, had increased their risk awareness, and had changed their driving behaviors. After insight training, the ambulance drivers in this study assessed themselves as safer drivers in several important areas, including speed adaptation, closing up, and overtaking. In future training of ambulance drivers there should be more focus on insight training instead of previous training focusing on maneuvering capabilities.

Creation of a Tool for Assessing Knowledge in Evidence-Based Decision-Making in Practicing Health Care Providers.

PubMed

Spurr, Kathy; Dechman, Gail; Lackie, Kelly; Gilbert, Robert

2016-01-01

Evidence-based decision-making (EBDM) is the process health care providers (HCPs) use to identify and appraise potential evidence. It supports the integration of best research evidence with clinical expertise and patient values into the decision-making process for patient care. Competence in this process is essential to delivery of optimal care. There is no objective tool that assesses EBDM across HCP groups. This research aimed to develop a content valid tool to assess knowledge of the principles of evidence-based medicine and the EBDM process, for use with all HCPs. A Delphi process was used in the creation of the tool. Pilot testing established its content validity with the added benefit of evaluating HCPs' knowledge of EBDM. Descriptive statistics and multivariate mixed models were used to evaluate individual survey responses in total, as well as within each EBDM component. The tool consisted of 26 multiple-choice questions. A total of 12,884 HCPs in Nova Scotia were invited to participate in the web-based validation study, yielding 818 (6.3%) participants, 471 of whom completed all questions. The mean overall score was 68%. Knowledge in one component, integration of evidence with clinical expertise and patient preferences, was identified as needing development across all HCPs surveyed. A content valid tool for assessing HCP EBDM knowledge was created and can be used to support the development of continuing education programs to enhance EBDM competency.

Gaze-Based Assistive Technology - Usefulness in Clinical Assessments.

PubMed

Wandin, Helena

2017-01-01

Gaze-based assistive technology was used in informal clinical assessments. Excerpts of medical journals were analyzed by directed content analysis using a model of communicative competence. The results of this pilot study indicate that gaze-based assistive technology is a useful tool in communication assessments that can generate clinically relevant information.

Ready or Not? Assessing Change Readiness for Implementation of the Geospatial Technology Competency Model[c

ERIC Educational Resources Information Center

Annulis, Heather M.; Gaudet, Cyndi H.

2007-01-01

A shortage of a qualified and skilled workforce exists to meet the demands of the geospatial industry (NASA, 2002). Solving today's workforce issues requires new and innovative methods and techniques for this high growth, high technology industry. One tool to support workforce development is a competency model which can be used to build a…

Otolaryngology residency education: a scoping review on the shift towards competency-based medical education.

PubMed

Wagner, N; Fahim, C; Dunn, K; Reid, D; Sonnadara, R R

2017-06-01

Residency training programmes worldwide are experiencing a shift from the traditional time-based curriculum to competency-based medical education (CBME), due to changes in the healthcare system that have impacted clinical learning opportunities. Otolaryngology-Head and Neck Surgery (OTL-HNS) programmes are one of the first North American surgical specialties to adopt the new CBME curriculum. The purpose of this scoping review is to examine the literature pertaining to CBME in OTL-HNS programmes worldwide, to identify the tools that have been developed and identify potential barriers to the implementation of CBME. Four online databases, OVID MEDLINE (R) from 1946 to 5 August 2015, EMBASE 1974 to 5 August 2015, Cochrane and CINAHL databases up to 5 August 2015, were searched using key words related to OTL-HNS and CBME. Two researchers independently reviewed the literature in a systematic manner and met to discuss and address any discrepancies at each step of the review process. Of the 207 publications identified in the initial search, 31 were included in this scoping review. Two key themes emerged from the literature: first, OTL-HNS programmes reported a need for new assessment tools that assess competency and also provide the learner with formative feedback. Second, although varieties of tools assessing both technical and non-technical skills have been developed, implementation of such tools has been met with some challenges. These challenges include a lack of faculty support, inadequate administrative support and a lack of knowledge on how to start the transition to CBME. This scoping review suggests that task-specific checklists, entrustment scales, evaluation portfolios from multiple assessments and faculty training sessions are key aspects to incorporate as OTL-HNS training programmes shift towards a CBME curriculum. © 2016 John Wiley & Sons Ltd.

Addressing mental health disparities through clinical competence not just cultural competence: the need for assessment of sociocultural issues in the delivery of evidence-based psychosocial rehabilitation services.

PubMed

Yamada, Ann-Marie; Brekke, John S

2008-12-01

Recognition of ethnic/racial disparities in mental health services has not directly resulted in the development of culturally responsive psychosocial interventions. There remains a fundamental need for assessment of sociocultural issues that have been linked with the expectations, needs, and goals of culturally diverse consumers with severe and persistent mental illness. The authors posit that embedding the assessment of sociocultural issues into psychosocial rehabilitation practice is one step in designing culturally relevant empirically supported practices. It becomes a foundation on which practitioners can examine the relevance of their interventions to the diversity encountered in everyday practice. This paper provides an overview of the need for culturally and clinically relevant assessment practices and asserts that by improving the assessment of sociocultural issues the clinical competence of service providers is enhanced. The authors offer a conceptual framework for linking clinical assessment of sociocultural issues to consumer outcomes and introduce an assessment tool adapted to facilitate the process in psychosocial rehabilitation settings. Emphasizing competent clinical assessment skills will ultimately offer a strategy to address disparities in treatment outcomes for understudied populations of culturally diverse consumers with severe and persistent mental illness.

Development and implementation of a competency-based clinical evaluation tool for midwifery education.

PubMed

Woeber, Kate

2018-07-01

The learning goals and evaluation strategies of competency-based midwifery programs must be explicit and well-defined. In the US, didactic learning is evaluated through a standardized certification examination, but standardized clinical competence evaluation is lacking. The Midwifery Competency Assessment Tool (MCAT) has been adapted from the International Confederation of Midwives' (ICM) "Essential Competencies" and from the American College of Nurse-Midwives' (ACNM) "Core Competencies", with student self-evaluation based on Benner's Novice-to-Expert theory. The MCAT allows for the measurement and monitoring of competence development in all domains of full-scope practice over the course of the midwifery program. Strengths of the MCAT are that it provides clear learning goals and performance evaluations for students, ensures and communicates content mapping across a curriculum, and highlights strengths and gaps in clinical opportunities at individual clinical sites and for entire programs. Challenges of the MCAT lie in balancing the number of competency items to be measured with the tedium of form completion, in ensuring the accuracy of student self-evaluation, and in determining "adequate" competence achievement when particular clinical opportunities are limited. Use of the MCAT with competency-based clinical education may facilitate a more standardized approach to clinical evaluation, as well as a more strategic approach to clinical site development and use. Copyright © 2018 Elsevier Ltd. All rights reserved.

Assessing Motor Skill Competency in Elementary School Students: A Three-Year Study.

PubMed

Chen, Weiyun; Mason, Steve; Hypnar, Andrew; Bennett, Austin

2016-03-01

This study was to examine how well fourth- and fifth-grade students demonstrated motor skill competency assessed with selected PE Metrics assessment rubrics (2009). Fourth- and fifth-grade students (n = 1,346-1,926) were assessed on their performance of three manipulative skills using the PE Metrics Assessment Rubrics during the pre-intervention year, the post-intervention year 1, and the post-intervention year 3. Descriptive statistics, independent t-test, ANOVA, and follow-up comparisons were conducted for data analysis. The results indicated that the post-intervention year 2 cohort performed significantly more competent than the pre-intervention cohort and the post-intervention year 1 cohort on the three manipulative skill assessments. The post-intervention year 1 cohort significantly outperformed the pre-intervention cohort on the soccer dribbling, passing, and receiving and the striking skill assessments, but not on the throwing skill assessment. Although the boys in the three cohorts performed significantly better than the girls on all three skills, the girls showed substantial improvement on the overhand throwing and the soccer skills from baseline to the post-intervention year 1 and the post-intervention year 2. However, the girls, in particular, need to improve striking skill. The CTACH PE was conducive to improving fourth- and fifth-grade students' motor skill competency in the three manipulative skills. This study suggest that PE Metrics assessment rubrics are feasible tools for PE teachers to assess levels of students' demonstration of motor skill competency during a regular PE lesson. Key pointsCATCH PE is an empirically-evidenced quality PE curricular that is conducive to improving students' manipulative skill competency.Boys significantly outperformed than girls in all three manipulative skills.Girls need to improve motor skill competency in striking skill. PE Metrics are feasible assessment rubrics that can be easily used by trained physical education teachers to assess students' manipulative skill competency.

Assessing Motor Skill Competency in Elementary School Students: A Three-Year Study

PubMed Central

Chen, Weiyun; Mason, Steve; Hypnar, Andrew; Bennett, Austin

2016-01-01

This study was to examine how well fourth- and fifth-grade students demonstrated motor skill competency assessed with selected PE Metrics assessment rubrics (2009). Fourth- and fifth-grade students (n = 1,346-1,926) were assessed on their performance of three manipulative skills using the PE Metrics Assessment Rubrics during the pre-intervention year, the post-intervention year 1, and the post-intervention year 3. Descriptive statistics, independent t-test, ANOVA, and follow-up comparisons were conducted for data analysis. The results indicated that the post-intervention year 2 cohort performed significantly more competent than the pre-intervention cohort and the post-intervention year 1 cohort on the three manipulative skill assessments. The post-intervention year 1 cohort significantly outperformed the pre-intervention cohort on the soccer dribbling, passing, and receiving and the striking skill assessments, but not on the throwing skill assessment. Although the boys in the three cohorts performed significantly better than the girls on all three skills, the girls showed substantial improvement on the overhand throwing and the soccer skills from baseline to the post-intervention year 1 and the post-intervention year 2. However, the girls, in particular, need to improve striking skill. The CTACH PE was conducive to improving fourth- and fifth-grade students’ motor skill competency in the three manipulative skills. This study suggest that PE Metrics assessment rubrics are feasible tools for PE teachers to assess levels of students’ demonstration of motor skill competency during a regular PE lesson. Key points CATCH PE is an empirically-evidenced quality PE curricular that is conducive to improving students’ manipulative skill competency. Boys significantly outperformed than girls in all three manipulative skills. Girls need to improve motor skill competency in striking skill. PE Metrics are feasible assessment rubrics that can be easily used by trained physical education teachers to assess students’ manipulative skill competency. PMID:26957932

Development of a competency framework for the nutrition in emergencies sector.

PubMed

Meeker, Jessica; Perry, Abigail; Dolan, Carmel; Emary, Colleen; Golden, Kate; Abla, Caroline; Walsh, Anne; Maclaine, Ali; Seal, Andrew

2014-03-01

There is a recognised need to strengthen capacity in the nutrition in emergencies sector and for greater clarity on the role of emergency nutritionists and the skills they require. Competency frameworks are an important tool for human resource development and have been developed for several other humanitarian sectors. We therefore developed a technical competency framework for practitioners in nutrition in emergencies. Existing competency frameworks were reviewed and interviews conducted to explore methods used in developing competency frameworks for other sectors. Competencies were identified through interviews with field experts, feedback from course trainees, academic course content and job specifications. Competencies were then categorised and behavioural indicators developed for each. The draft framework was then reviewed by members of the Global Nutrition Cluster and modified in an iterative process. Global. Not applicable. A wide range of competencies were identified as essential for nutritionists working in emergencies, covering technical skills and general core competencies. The proposed framework contains twenty competency areas with 161 behavioural indicators categorised into three levels, corresponding to the requirements of progressively more senior roles. Many of the competencies are common across development and emergency nutrition. The proposed technical competency framework should prove to be a valuable tool in creating standards within the sector and promoting effective capacity strengthening and professionalisation. Continued research is needed to validate the framework, optimise methods for assessment, develop approaches to integrate it within the sector and measure its impact on performance.

Constructing a competency-based bariatric surgery fellowship training curriculum.

PubMed

McBride, Corrigan L; Rosenthal, Raul J; Brethauer, Stacy; DeMaria, Eric; Kelly, John J; Morton, John M; Lo Menzo, Emanuele; Moore, Rachel; Pomp, Alfons; Nguyen, Ninh T

2017-03-01

Bariatric fellowship training after general surgery has historically been time based and competence was determined at completion based on a minimum number of cases during the fellowship. Graduate medical education is moving toward competency-based medical education where learners are evaluated during the course of their training and competence assessment occurs throughout. The Executive Council of the American Society of Metabolic and Bariatric Surgery (ASMBS) at the direction of the American Board of Surgery wanted to transition the bariatric surgery fellowship curriculum from its traditional format to a competency-based curriculum using competency-based medical education principles. The ASMBS Education and Training Committee established a task force of 9 members to create a new curriculum and all of the necessary evaluation tools to support the curriculum, and initiate a pilot program. A competency-based curriculum consisting of 6 modules with cognitive and technical milestones, and the innovative evaluation tools needed to evaluate the learners, was created. A pilot program consisting of 10 programs and 19 fellows has been undertaken for the 2016-2017 academic year. The Education Committee of the ASMBS is leading the charge in curriculum development for competency-based medical education for bariatric fellowship. Copyright © 2017 American Society for Bariatric Surgery. Published by Elsevier Inc. All rights reserved.

Recent interview-based measures of competency to stand trial: a critical review augmented with research data.

PubMed

Rogers, R; Grandjean, N; Tillbrook, C E; Vitacco, M J; Sewell, K W

2001-01-01

Forensic experts are frequently asked to conduct competency-to-stand trial evaluations and address the substantive prongs propounded in Dusky v. United States (1960). In understanding its application to competency evaluations, alternative conceptualizations of Dusky are critically examined. With Dusky providing the conceptual framework, three interview-based competency measures are reviewed: the Georgia Court Competency Test (GCCT), the MacArthur Competence Assessment Tool-Criminal Adjudication (Mac-CAT-CA), and the Evaluation of Competency to Stand Trial-Revised (ECST-R). This review has a twin focus on reliability of each measure and its correspondence to Dusky prongs. The current review is augmented by new factor analytic data on the MacCAT-CA and ECST-R. The article concludes with specific recommendations for competency evaluations. Copyright 2001 John Wiley & Sons, Ltd.

Adopting a blended learning approach to teaching evidence based medicine: a mixed methods study

PubMed Central

2013-01-01

Background Evidence Based Medicine (EBM) is a core unit delivered across many medical schools. Few studies have investigated the most effective method of teaching a course in EBM to medical students. The objective of this study was to identify whether a blended-learning approach to teaching EBM is more effective a didactic-based approach at increasing medical student competency in EBM. Methods A mixed-methods study was conducted consisting of a controlled trial and focus groups with second year graduate medical students. Students received the EBM course delivered using either a didactic approach (DID) to learning EBM or a blended-learning approach (BL). Student competency in EBM was assessed using the Berlin tool and a criterion-based assessment task, with student perceptions on the interventions assessed qualitatively. Results A total of 61 students (85.9%) participated in the study. Competency in EBM did not differ between the groups when assessed using the Berlin tool (p = 0.29). Students using the BL approach performed significantly better in one of the criterion-based assessment tasks (p = 0.01) and reported significantly higher self-perceived competence in critical appraisal skills. Qualitative analysis identified that students had a preference for the EBM course to be delivered using the BL approach. Conclusions Implementing a blended-learning approach to EBM teaching promotes greater student appreciation of EBM principles within the clinical setting. Integrating a variety of teaching modalities and approaches can increase student self-confidence and assist in bridging the gap between the theory and practice of EBM. PMID:24341502

Development of the competency scale for primary care managers in Thailand: Scale development.

PubMed

Kitreerawutiwong, Keerati; Sriruecha, Chanaphol; Laohasiriwong, Wongsa

2015-12-09

The complexity of the primary care system requires a competent manager to achieve high-quality healthcare. The existing literature in the field yields little evidence of the tools to assess the competency of primary care administrators. This study aimed to develop and examine the psychometric properties of the competency scale for primary care managers in Thailand. The scale was developed using in-depth interviews and focus group discussions among policy makers, managers, practitioners, village health volunteers, and clients. The specific dimensions were extracted from 35 participants. 123 items were generated from the evidence and qualitative data. Content validity was established through the evaluation of seven experts and the original 123 items were reduced to 84 items. The pilot testing was conducted on a simple random sample of 487 primary care managers. Item analysis, reliability testing, and exploratory factor analysis were applied to establish the scale's reliability and construct validity. Exploratory factor analysis identified nine dimensions with 48 items using a five-point Likert scale. Each dimension accounted for greater than 58.61% of the total variance. The scale had strong content validity (Indices = 0.85). Each dimension of Cronbach's alpha ranged from 0.70 to 0.88. Based on these analyses, this instrument demonstrated sound psychometric properties and therefore is considered an effective tool for assessment of the primary care manager competencies. The results can be used to improve competency requirements of primary care managers, with implications for health service management workforce development.

Diversity in emergency medicine education: expanding the horizon.

PubMed

Bowman, Steven H; Moreno-Walton, Lisa; Ezenkwele, Ugo A; Heron, Sheryl L

2011-10-01

An emergency medicine (EM)-based curriculum on diversity, inclusion, and cultural competency can also serve as a mechanism to introduce topics on health care disparities. Although the objectives of such curricula and the potential benefits to EM trainees are apparent, there are relatively few resources available for EM program directors to use to develop these specialized curricula. The object of this article is to 1) broadly discuss the current state of curricula of diversity, inclusion, and cultural competency in EM training programs; 2) identify tools and disseminate strategies to embed issues of disparities in health care in the creation of the curriculum; and 3) provide resources for program directors to develop their own curricula. A group of EM program directors with an interest in cultural competency distributed a preworkshop survey through the Council of Emergency Medicine Residency Directors (CORD) e-mail list to EM program directors to assess the current state of diversity and cultural competency training in EM programs. Approximately 50 members attended a workshop during the 2011 CORD Academic Assembly as part of the Best Practices track, where the results of the survey were disseminated and discussed. In addition to the objectives listed above, the presenters reviewed the literature regarding the rationale for a cultural competency curriculum and its relationship to addressing health care disparities, the relationship to unconscious physician bias, and the Tool for Assessing Cultural Competence Training (TACCT) model for curriculum development. © 2011 by the Society for Academic Emergency Medicine.

Introducing a design exigency to promote student learning through assessment: A case study.

PubMed

Grealish, Laurie A; Shaw, Julie M

2018-02-01

Assessment technologies are often used to classify student and newly qualified nurse performance as 'pass' or 'fail', with little attention to how these decisions are achieved. Examining the design exigencies of classification technologies, such as performance assessment technologies, provides opportunities to explore flexibility and change in the process of using those technologies. Evaluate an established assessment technology for nursing performance as a classification system. A case study analysis that is focused on the assessment approach and a priori design exigencies of performance assessment technology, in this case the Australian Nursing Standards Assessment Tool 2016. Nurse assessors are required to draw upon their expertise to judge performance, but that judgement is described as a source of bias, creating confusion. The definition of satisfactory performance is 'ready to enter practice'. To pass, the performance on each criterion must be at least satisfactory, indicating to the student that no further improvement is required. The Australian Nursing Standards Assessment Tool 2016 does not have a third 'other' category, which is usually found in classification systems. Introducing a 'not yet competent' category and creating a two-part, mixed methods assessment process can improve the Australian Nursing Standards Assessment Tool 2016 assessment technology. Using a standards approach in the first part, judgement is valued and can generate learning opportunities across a program. Using a measurement approach in the second part, student performance can be 'not yet competent' but still meet criteria for year level performance and a graded pass. Subjecting the Australian Nursing Standards Assessment Tool 2016 assessment technology to analysis as a classification system provides opportunities for innovation in design. This design innovation has the potential to support students who move between programs and clinicians who assess students from different universities. Copyright © 2017 Elsevier Ltd. All rights reserved.

Towards development and validation of an intraoperative assessment tool for robot-assisted radical prostatectomy training: results of a Delphi study.

PubMed

Morris, Christopher; Hoogenes, Jen; Shayegan, Bobby; Matsumoto, Edward D

2017-01-01

As urology training shifts toward competency-based frameworks, the need for tools for high stakes assessment of trainees is crucial. Validated assessment metrics are lacking for many robot-assisted radical prostatectomy (RARP). As it is quickly becoming the gold standard for treatment of localized prostate cancer, the development and validation of a RARP assessment tool for training is timely. We recruited 13 expert RARP surgeons from the United States and Canada to serve as our Delphi panel. Using an initial inventory developed via a modified Delphi process with urology residents, fellows, and staff at our institution, panelists iteratively rated each step and sub-step on a 5-point Likert scale of agreement for inclusion in the final assessment tool. Qualitative feedback was elicited for each item to determine proper step placement, wording, and suggestions. Panelist's responses were compiled and the inventory was edited through three iterations, after which 100% consensus was achieved. The initial inventory steps were decreased by 13% and a skip pattern was incorporated. The final RARP stepwise inventory was comprised of 13 critical steps with 52 sub-steps. There was no attrition throughout the Delphi process. Our Delphi study resulted in a comprehensive inventory of intraoperative RARP steps with excellent consensus. This final inventory will be used to develop a valid and psychometrically sound intraoperative assessment tool for use during RARP training and evaluation, with the aim of increasing competency of all trainees. Copyright® by the International Brazilian Journal of Urology.

Towards development and validation of an intraoperative assessment tool for robot-assisted radical prostatectomy training: results of a Delphi study

PubMed Central

Morris, Christopher; Hoogenes, Jen; Shayegan, Bobby; Matsumoto, Edward D.

2017-01-01

ABSTRACT Introduction As urology training shifts toward competency-based frameworks, the need for tools for high stakes assessment of trainees is crucial. Validated assessment metrics are lacking for many robot-assisted radical prostatectomy (RARP). As it is quickly becoming the gold standard for treatment of localized prostate cancer, the development and validation of a RARP assessment tool for training is timely. Materials and methods We recruited 13 expert RARP surgeons from the United States and Canada to serve as our Delphi panel. Using an initial inventory developed via a modified Delphi process with urology residents, fellows, and staff at our institution, panelists iteratively rated each step and sub-step on a 5-point Likert scale of agreement for inclusion in the final assessment tool. Qualitative feedback was elicited for each item to determine proper step placement, wording, and suggestions. Results Panelist’s responses were compiled and the inventory was edited through three iterations, after which 100% consensus was achieved. The initial inventory steps were decreased by 13% and a skip pattern was incorporated. The final RARP stepwise inventory was comprised of 13 critical steps with 52 sub-steps. There was no attrition throughout the Delphi process. Conclusions Our Delphi study resulted in a comprehensive inventory of intraoperative RARP steps with excellent consensus. This final inventory will be used to develop a valid and psychometrically sound intraoperative assessment tool for use during RARP training and evaluation, with the aim of increasing competency of all trainees. PMID:28379668

Development and validation of a child health workforce competence framework.

PubMed

Smith, Lynda; Hawkins, Jean; McCrum, Anita

2011-05-01

Providing high quality, effective services is fundamental to the delivery of key health outcomes for children and young people. This requires a competent workforce. This paper reports on the development of a validated competence framework tool for the children and young people's health workforce. The framework brings together policy, strategic agendas and existing workforce competences. The framework will contribute to the improvement of children's physical and mental wellbeing by identifying competences required to provide proactive services that respond to children and young people with acute, continuing and complex needs. It details five core competences for the workforce, the functions that underpin them and levels of competence required to deliver a particular service. The framework will be of value to commissioners to inform contracting, to providers to ensure services are delivered by a workforce with relevant competences to meet identified needs, and to the workforce to assess existing capabilities and identify gaps in competence.

[Clinical competence evaluation using the Objective Structured Clinical Examination (OSCE) in medical internship at UNAM].

PubMed

Trejo Mejía, Juan Andrés; Martínez González, Adrián; Méndez Ramírez, Ignacio; Morales López, Sara; Ruiz Pérez, Leobardo C; Sánchez Mendiola, Melchor

2014-01-01

The Objective Structured Clinical Examination (OSCE) is a widely used measurement tool to assess clinical competence in the health sciences. There is little published evidence of its use in Mexican medical schools. To assess clinical competence in medical students with an OSCE, before and after the Medical Internship. Prospective cohort study, pre- post-test research design. The assessed population was medical students at UNAM Faculty of Medicine in Mexico in their Internship year. The instrument was an 18-stations OSCE, three stations per academic area of the Internship curriculum. We assessed the clinical competence of 278 students in a pretest OSCE when starting the Internship year, and tested them 10 months later with an equivalent post-test OSCE. The sample of students was 30.4% of the total Internship population. Test reliability with Cronbach's alpha was 0.62 in the pre-test and 0.64 in the post-test. The global mean score in the pretest OSCE was 55.6 ± 6.6 and in the post-test 63.2 ± 5.7 (p < 0.001), with a Cohen's d of 1.2. The clinical competence of medical students measured with an OSCE is higher after the medical internship year. This difference suggests that the internship can influence the development of clinical competence in medical students.

Assessment of postgraduate dental students using mini-clinical examination tool in periodontology and implantology

PubMed Central

Rathod, Surekha Ramrao; Kolte, Abhay; Shori, Tony; Kher, Vishal

2017-01-01

Introduction: Mini-clinical examination (mini-CEX) is a new assessment tool that observes the student using a standard rating form. The aim of this study was to evaluate the feasibility and usefulness of the mini-CEX as an assessment and feedback tool in the postgraduate setting in periodontology. Materials and Methods: Eight postgraduate students and two evaluators were included in this study carried out for 4 months during which the students were made to appear for four encounters evaluated on a standardized nine-point Likert scale. Feedback was obtained from the students about this assessment after the fourth encounter. Results: Sixty-three percent of the students felt that mini-CEX is better than the conventional assessment tools. Seventy-five percent of the students felt that this type of mini-CEX assessment helped improve the student–teacher relationship and student–patient relationship. Sixty-three percent of the students were satisfied with this assessment pattern and were willing to face more encounters as it helped them improve their competencies. Seventy-five percent of the students agreed that they felt anxious on being observed while taking cases. Conclusion: The training and assessment of a wide range of procedures make dentistry unique. Good communication skills and counseling can allay patient's fear and anxiety. This structured way of assessment of clinical skills and feedback provides good clinical care and helps improve the quality of the resulting information which would induce confidence, improve clinical competencies, and alleviate the fear of examination among the students. PMID:29491581

Development, validity and reliability testing of the East Midlands Evaluation Tool (EMET) for measuring impacts on trainees' confidence and competence following end of life care training.

PubMed

Whittaker, B; Parry, R; Bird, L; Watson, S; Faull, C

2017-02-02

To develop, test and validate a versatile questionnaire, the East Midlands Evaluation Tool (EMET), for measuring effects of end of life care training events on trainees' self-reported confidence and competence. A paper-based questionnaire was designed on the basis of the English Department of Health's core competences for end of life care, with sections for completion pretraining, immediately post-training and also for longer term follow-up. Preliminary versions were field tested at 55 training events delivered by 13 organisations to 1793 trainees working in diverse health and social care backgrounds. Iterative rounds of development aimed to maximise relevance to events and trainees. Internal consistency was assessed by calculating interitem correlations on questionnaire responses during field testing. Content validity was assessed via qualitative content analysis of (1) responses to questionnaires completed by field tester trainers and (2) field notes from a workshop with a separate cohort of experienced trainers. Test-retest reliability was assessed via repeat administration to a cohort of student nurses. The EMET comprises 27 items with Likert-scaled responses supplemented with questions seeking free-text responses. It measures changes in self-assessed confidence and competence on 5 subscales: communication skills; assessment and care planning; symptom management; advance care planning; overarching values and knowledge. Test-retest reliability was found to be good, as was internal consistency: the questions successfully assess different aspects of the same underlying concept. The EMET provides a time-efficient, reliable and flexible means of evaluating effects of training on self-reported confidence and competence in the key elements of end of life care. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

Development and Psychometric Testing of the Caregiver Communication Competence Scale in Patients With Dementia.

PubMed

Chao, Hui-Chen; Yang, Ya-Ping; Huang, Mei-Chih; Wang, Jing-Jy

2016-01-01

Appropriate communication skills are essential for understanding patient needs, particularly those of patients with dementia. Assessing health care providers' competence in communicating with patients with dementia is critical for planning a communication education program. However, no formally established scale can be used. The purpose of the current study was to develop a valid and reliable instrument for determining the communication competence of health care providers with patients with dementia. Through use of a literature review and previous clinical experience, an initial 28-item scale was developed to assess the frequency of use of each item by health care providers. Fourteen items were extracted and three factors were distinguished. Results indicated that the internal consistency reliability of the 14-item scale was 0.84. Favorable convergent and discriminant validities were reached. The communication competence scale provides administrators or educators with a useful tool for assessing communication competence of health care providers when interacting with patients with dementia so a suitable education program can be planned and implemented. Copyright 2016, SLACK Incorporated.

[Assessment of competence in pediatric postgraduate education: implementation of a pediatric version of the Mini-Cex].

PubMed

Urman, Gabriela; Folgueral, Silvana; Gasparri, Mercedes; López, Diana; Urman, Jorge; Grosman, Arnoldo; Alves de Lima, Alberto

2011-12-01

Faced with the increased challenge of assessing competences in young doctors, the purpose of the study was to evaluate the implementation of a pediatric version of the Mini-Cex in pediatric trainees as well as the level of satisfaction of teachers and students with the new assessment tool. From July 2007 to August 2009, 54 pediatric trainees were periodically monitored in a variety of clinical settings by 50 teachers. The competences evaluated included medical interviewing, physical examination and counseling skills, humanistic qualities/ professionalism, clinical judgment, organization and overall clinical competence. The feasibility of this study was defined as an average 4 observations per participant, and observations in all clinical rotations. During the study, 388 observations were carried over 54 students (average of 7.18 observations per student); 57% took place in ambulatory settings, 60% were of low complexity and 85% involved healthy children programmed consultations. The ratings for specific competences had little variation; the focus related to the setting. Used in a variety of settings, with different patient problems, the method was well accepted by both students and teachers.

Status of Competency-Based Medical Education in Endoscopy Training: A Nationwide Survey of US ACGME-Accredited Gastroenterology Training Programs.

PubMed

Patel, S G; Keswani, R; Elta, G; Saini, S; Menard-Katcher, P; Del Valle, J; Hosford, L; Myers, A; Ahnen, D; Schoenfeld, P; Wani, S

2015-07-01

The Accreditation Council for Graduate Medical Education (ACGME) emphasizes the importance of medical trainees meeting specific performance benchmarks and demonstrating readiness for unsupervised practice. The aim of this study was to examine the readiness of Gastroenterology (GI) fellowship programs for competency-based evaluation in endoscopic procedural training. ACGME-accredited GI program directors (PDs) and GI trainees nationwide completed an online survey of domains relevant to endoscopy training and competency assessment. Participants were queried about current methods and perceived quality of endoscopy training and assessment of competence. Participants were also queried about factors deemed important in endoscopy competence assessment. Five-point Likert items were analyzed as continuous variables by an independent t-test and χ(2)-test was used for comparison of proportions. Survey response rate was 64% (94/148) for PDs and 47% (546/1,167) for trainees. Twenty-three percent of surveyed PDs reported that they do not have a formal endoscopy curriculum. PDs placed less importance (1—very important to 5—very unimportant) on endoscopy volume (1.57 vs. 1.18, P<0.001), adenoma detection rate (2.00 vs. 1.53, P<0.001), and withdrawal times (1.96 vs. 1.68, P=0.009) in determining endoscopy competence compared with trainees. A majority of PDs report that competence is assessed by procedure volume (85%) and teaching attending evaluations (96%). Only a minority of programs use skills assessment tools (30%) or specific quality metrics (28%). Specific competencies are mostly assessed by individual teaching attending feedback as opposed to official documentation or feedback from a PD. PDs rate the overall quality of their endoscopy training and assessment of competence as better than overall ratings by trainees. Although the majority of PDs and trainees nationwide believe that measuring specific metrics is important in determining endoscopy competence, most programs still rely on procedure volume and subjective attending evaluations to determine overall competence. As medical training transitions from an apprenticeship model to competency-based education, there is a need for improved endoscopy curricula which are better suited to demonstrate readiness for unsupervised practice.

Reading Assessment: Principles and Practices for Elementary Teachers. Second Edition

ERIC Educational Resources Information Center

Barrentine, Shelby J., Ed.; Stokes, Sandra M., Ed.

2005-01-01

How do teachers respond to the competing pressures of school accountability, high-stakes testing, classroom assessment and instruction? This updated collection of articles from The Reading Teacher can help. Readers will find tools for: (1) Building school assessment policies; (2) Helping students succeed on high-stakes tests; (3) Using assessment…

Competencies: From Deconstruction to Reconstruction and Back Again, Lessons Learned

PubMed Central

Leach, David C.

2008-01-01

I address the potential impact of the Association of Schools of Public Health’s development of a competency model for the graduate Master of Public Health. I reflect on the model in relation to the Accreditation Council for Graduate Medical Education’s adoption of a competency-based model for medical education. Six lessons learned by the Accreditation Council for Graduate Medical Education that the Association of Schools of Public Health might consider in moving forward are how learning outcomes can be enhanced by using competency models, the effect of competency development processes in “creating a common language” among educators, the benefits and challenges of numerous competencies within a model, the usefulness of the Dreyfus model for progressive competency development, the need for multiple assessment tools used over time, and the value of learning portfolios. PMID:18633085

Competency Maps: an Effective Model to Integrate Professional Competencies Across a STEM Curriculum

NASA Astrophysics Data System (ADS)

Sánchez Carracedo, Fermín; Soler, Antonia; Martín, Carme; López, David; Ageno, Alicia; Cabré, Jose; Garcia, Jordi; Aranda, Joan; Gibert, Karina

2018-05-01

Curricula designed in the context of the European Higher Education Area need to be based on both domain-specific and professional competencies. Whereas universities have had extensive experience in developing students' domain-specific competencies, fostering professional competencies poses a new challenge we need to face. This paper presents a model to globally develop professional competencies in a STEM (science, technology, engineering, and mathematics) degree program, and assesses the results of its implementation after 4 years. The model is based on the use of competency maps, in which each competency is defined in terms of competency units. Each competency unit is described by a set of expected learning outcomes at three domain levels. This model allows careful analysis, revision, and iteration for an effective integration of professional competencies in domain-specific subjects. A global competency map is also designed, including all the professional competency learning outcomes to be achieved throughout the degree. This map becomes a useful tool for curriculum designers and coordinators. The results were obtained from four sources: (1) students' grades (classes graduated from 2013 to 2016, the first 4 years of the new Bachelor's Degree in Informatics Engineering at the Barcelona School of Informatics); (2) students' surveys (answered by students when they finished the degree); (3) the government employment survey, where former students evaluate their satisfaction of the received training in the light of their work experience; and (4) the Everis Foundation University-Enterprise Ranking, answered by over 2000 employers evaluating their satisfaction regarding their employees' university training, where the Barcelona School of Informatics scores first in the national ranking. The results show that competency maps are a good tool for developing professional competencies in a STEM degree.

Assessment of Competence in EVAR Procedures: A Novel Rating Scale Developed by the Delphi Technique.

PubMed

Strøm, M; Lönn, L; Bech, B; Schroeder, T V; Konge, L

2017-07-01

To develop a procedure specific global rating scale for assessment of operator competence in endovascular aortic repair (EVAR). A Delphi approach was used to achieve expert consensus. A panel of 32 international experts (median 300 EVAR procedures, range 200-3000) from vascular surgery (n = 21) and radiology (n = 11) was established. The first Delphi round was based on a review of endovascular skills assessment papers, stent graft instructions for use, and structured interviews. It led to a primary pool of 83 items that were formulated as global rating scale items with tentative anchors. Iterative Delphi rounds were executed. The panellists rated the importance of each item on a 5 point Likert scale. Consensus was defined as 80% of the panel rating an item 4 or 5 in the primary round and 90% in subsequent rounds. Consensus on the final assessment tool was defined as Cronbach's alpha > .8 after a minimum of three rounds. Thirty-two of 35 invited experts participated. Three rounds of surveys were completed with a completion rate of 100% in the first two rounds and 91% in round three. The 83 primary assessment items were supplemented with five items suggested by the panel and reduced to seven pivotal assessment items that reached consensus, Cronbach's alpha = 0.82. The seven item rating scale covers key elements of competence in EVAR stent placement and deployment. Each item has well defined grades with explicit anchors at unacceptable, acceptable, and superior performance on a 5 point Likert scale. The Delphi methodology allowed for international consensus on a new procedure specific global rating scale for assessment of competence in EVAR. The resulting scale, EndoVascular Aortic Repair Assessment of Technical Expertise (EVARATE), represents key elements in the procedure. EVARATE constitutes an assessment tool for providing structured feedback to endovascular operators in training. Copyright © 2017 European Society for Vascular Surgery. Published by Elsevier Ltd. All rights reserved.

Addressing Mental Health Disparities through Clinical Competence Not Just Cultural Competence: The Need for Assessment of Sociocultural Issues in the Delivery of Evidence-Based Psychosocial Rehabilitation Services

PubMed Central

Yamada, Ann-Marie; Brekke, John S

2008-01-01

Recognition of ethnic/racial disparities in mental health services has not directly resulted in the development of culturally responsive psychosocial interventions. There remains a fundamental need for assessment of sociocultural issues that have been linked with the expectations, needs, and goals of culturally diverse consumers with severe and persistent mental illness. The authors posit that embedding the assessment of sociocultural issues into psychosocial rehabilitation practice is one step in designing culturally relevant empirically supported practices. It becomes a foundation on which practitioners can examine the relevance of their interventions to the diversity encountered in everyday practice. This paper provides an overview of the need for culturally and clinically relevant assessment practices and asserts that by improving the assessment of sociocultural issues the clinical competence of service providers is enhanced. The authors offer a conceptual framework for linking clinical assessment of sociocultural issues to consumer outcomes and introduce an assessment tool adapted to facilitate the process in psychosocial rehabilitation settings. Emphasizing competent clinical assessment skills will ultimately offer a strategy to address disparities in treatment outcomes for understudied populations of culturally diverse consumers with severe and persistent mental illness. PMID:18778881

Predicting Airport Screening Officers' Visual Search Competency With a Rapid Assessment.

PubMed

Mitroff, Stephen R; Ericson, Justin M; Sharpe, Benjamin

2018-03-01

Objective The study's objective was to assess a new personnel selection and assessment tool for aviation security screeners. A mobile app was modified to create a tool, and the question was whether it could predict professional screeners' on-job performance. Background A variety of professions (airport security, radiology, the military, etc.) rely on visual search performance-being able to detect targets. Given the importance of such professions, it is necessary to maximize performance, and one means to do so is to select individuals who excel at visual search. A critical question is whether it is possible to predict search competency within a professional search environment. Method Professional searchers from the USA Transportation Security Administration (TSA) completed a rapid assessment on a tablet-based X-ray simulator (XRAY Screener, derived from the mobile technology app Airport Scanner; Kedlin Company). The assessment contained 72 trials that were simulated X-ray images of bags. Participants searched for prohibited items and tapped on them with their finger. Results Performance on the assessment significantly related to on-job performance measures for the TSA officers such that those who were better XRAY Screener performers were both more accurate and faster at the actual airport checkpoint. Conclusion XRAY Screener successfully predicted on-job performance for professional aviation security officers. While questions remain about the underlying cognitive mechanisms, this quick assessment was found to significantly predict on-job success for a task that relies on visual search performance. Application It may be possible to quickly assess an individual's visual search competency, which could help organizations select new hires and assess their current workforce.

Assessing competence in communication and interpersonal skills: the Kalamazoo II report.

PubMed

Duffy, F Daniel; Gordon, Geoffrey H; Whelan, Gerald; Cole-Kelly, Kathy; Frankel, Richard; Buffone, Natalie; Lofton, Stephanie; Wallace, MaryAnne; Goode, Leslie; Langdon, Lynn

2004-06-01

Accreditation of residency programs and certification of physicians requires assessment of competence in communication and interpersonal skills. Residency and continuing medical education program directors seek ways to teach and evaluate these competencies. This report summarizes the methods and tools used by educators, evaluators, and researchers in the field of physician-patient communication as determined by the participants in the "Kalamazoo II" conference held in April 2002. Communication and interpersonal skills form an integrated competence with two distinct parts. Communication skills are the performance of specific tasks and behaviors such as obtaining a medical history, explaining a diagnosis and prognosis, giving therapeutic instructions, and counseling. Interpersonal skills are inherently relational and process oriented; they are the effect communication has on another person such as relieving anxiety or establishing a trusting relationship. This report reviews three methods for assessment of communication and interpersonal skills: (1) checklists of observed behaviors during interactions with real or simulated patients; (2) surveys of patients' experience in clinical interactions; and (3) examinations using oral, essay, or multiple-choice response questions. These methods are incorporated into educational programs to assess learning needs, create learning opportunities, or guide feedback for learning. The same assessment tools, when administered in a standardized way, rated by an evaluator other than the teacher, and using a predetermined passing score, become a summative evaluation. The report summarizes the experience of using these methods in a variety of educational and evaluation programs and presents an extensive bibliography of literature on the topic. Professional conversation between patients and doctors shapes diagnosis, initiates therapy, and establishes a caring relationship. The degree to which these activities are successful depends, in large part, on the communication and interpersonal skills of the physician. This report focuses on how the physician's competence in professional conversation with patients might be measured. Valid, reliable, and practical measures can guide professional formation, determine readiness for independent practice, and deepen understanding of the communication itself.

Perspective: Toward a competency framework for faculty.

PubMed

Milner, Robert J; Gusic, Maryellen E; Thorndyke, Luanne E

2011-10-01

Today, faculty in academic medicine face challenges in all three mission areas--research, education, and patient care--and require a broad set of competencies to survive in this changing environment. To support faculty and to design assessments that match new expectations, the authors argue that it is essential to capture the full scope of skills, knowledge, and behaviors necessary for a successful faculty member. Thus, it is timely to explore and define competencies for faculty in academic medicine. The authors describe three approaches to identifying faculty competencies. Each reveals diverse but overlapping sets of competency domains, reflecting the breadth of activities expected of today's faculty. To organize these competencies into a coherent framework, the authors propose a model based on a typology of competency. A key feature of the model is the division between occupational competencies, which are largely role-specific, and personal competencies, which are necessary for all faculty. A competency framework also must be developmental, to reflect the growth in skills, knowledge, and behaviors from trainee to expert and to allow for an individual's changing roles over a career. Such a competency framework will inform professional development activities and require assessment of competence. The generation of competencies also will reveal areas of faculty practice that are poorly measured, requiring new tools to be incorporated into existing processes of faculty evaluation. The authors provide general principles to guide the identification of a competency framework for faculty and invite the academic medicine community to engage in further discussion.

Cross-Cultural Education in U.S. Medical Schools: Development of an Assessment Tool.

ERIC Educational Resources Information Center

Dolhun, Eduardo Pena; Munoz, Claudia; Grumbach, Kevin

2003-01-01

Medical schools were invited to provide written and Web-based materials related to implementing cross-cultural competency in their curricula. A tool was developed to measure teaching methods, skill sets, and eight content areas in cross-cultural education. Most programs emphasized teaching general themes, such as the doctor-patient relationship,…

Teamwork Skills Assessment for Cooperative Learning

ERIC Educational Resources Information Center

Strom, Paris S.; Strom, Robert D.

2011-01-01

Teamwork skills are required at work, but teacher efforts in many countries to track achievement within this context have been hindered by lack of assessment tools and input from students. The Teamwork Skills Inventory relies on peer and self-evaluation to establish accountability, identify competencies, and detect learning needs. Twenty-five…

Teaching and assessing competence in cataract surgery.

PubMed

Henderson, Bonnie An; Ali, Rasha

2007-02-01

To review recent literature regarding innovative techniques, methods of teaching and assessing competence and skill in cataract surgery. The need for assessment of surgical competency and the requirement of wet lab facilities in ophthalmic training programs are being increasingly emphasized. Authors have proposed the use of standardized forms to collect objective and subjective data regarding the residents' surgical performance. Investigators have reported methods to improve visualization of cadaver and animal eyes for the wet lab, including the use of capsular dyes. The discussion of virtual reality as a teaching tool for surgical programs continues. Studies have proven that residents trained on a laparoscopic simulator outperformed nontrained residents during actual surgery for both surgical times and numbers of errors. Besides virtual reality systems, a program is being developed to separate the cognitive portion from the physical aspects of surgery. Another program couples surgical videos with three-dimensional animation to enhance the trainees' topographical understanding. Proper assessment of surgical competency is becoming an important focus of training programs. The use of surgical data forms may assist in standardizing objective assessments. Virtual reality, cognitive curriculum and animation video programs can be helpful in improving residents' surgical performance.

Evaluation of the organizational cultural competence of a community health center: a multimethod approach.

PubMed

Cherner, Rebecca; Olavarria, Marcela; Young, Marta; Aubry, Tim; Marchant, Christina

2014-09-01

Cultural competence is an important component of client-centered care in health promotion and community health services, especially considering the changing demographics of North America. Although a number of tools for evaluating cultural competence have been developed, few studies have reported on the results of organizational cultural competence evaluations in health care or social services settings. This article aims to fill this gap by providing a description of a cultural competence evaluation of a community health center serving a diverse population. Data collection included reviewing documents, and surveying staff, management, and the Board of Directors. The organization fully met 28 of 53 standards of cultural competence, partially met 21 standards, and did not meet 2 standards, and 2 standards could not be assessed due to missing information. The advantages and lessons learned from this organizational cultural competence evaluation are discussed. © 2014 Society for Public Health Education.

Examining the effectiveness of an intervention designed for the restoration of competency to stand trial.

PubMed

Mueller, Crystal; Wylie, A Michael

2007-01-01

This study examined the effectiveness of the Fitness Game, an intervention created for the restoration of competency to stand trial. Of 28 participants who completed either the experimental or control intervention at Hawai'i State Hospital (HSH), it was found that both groups had significant pretest to posttest improvements on Understanding and Appreciation subsets of the MacArthur Competence Assessment Tool-Criminal Adjudication (MacCAT-CA). However, there was no statistically significant difference between the groups on competency measures, indicating that the Fitness Game was not significantly more effective at restoring competency than non-legal programming. In other words, individuals committed to a psychiatric hospital for care and treatment were as likely to improve as those receiving additional specialized competency restoration treatment. Copyright (c) 2007 John Wiley & Sons, Ltd.

The two-week pediatric surgery rotation: is it time wasted?

PubMed

Dutta, S; Wales, P W; Fecteau, A

2004-05-01

With increasing medical school emphasis on generalist training and decreasing enrollment in surgical residency, the authors assessed the adequacy of a 2-week pediatric surgery rotation on meeting the learning and competency objectives outlined in The Canadian Association of Pediatric Surgeons' Self-Directed Evaluation Tool. A prospective survey was conducted of 39 clinical clerks. An anonymous self-assessment scale measuring competency objectives (medical and psychosocial) was administered pre-and postrotation. Also, exposure to pediatric surgical conditions from a list of "essential" and "nonessential" learning objectives was measured. Statistical analysis was performed using paired t test with significance at.05 level. Response rate was 77% and 54% for the competency and learning objectives, respectively. Students reported improvement in medical (P <.00001; 95% CI, 1.30, 1.90) and psychosocial (P =.00036; 95% CI 0.64, 1.28) competency objectives after the rotation. Almost all "essential" learning objectives were met. Overall, students reported an increased awareness of the breadth of pediatric surgical practice (P <.0001; 95% CI 2.06, 3.18). A 2-week rotation in pediatric surgery appears adequate in fulfilling most competency and learning objectives, but discussion is needed about how to best assess student competency, which topics are considered essential, and the long-term effect on recruitment to the profession.

Training on the DSM-5 Cultural Formulation Interview improves cultural competence in general psychiatry residents: A pilot study.

PubMed

Mills, Stacia; Xiao, Anna Q; Wolitzky-Taylor, Kate; Lim, Russell; Lu, Francis G

2017-04-01

The objective of this study was to assess whether a 1-hour didactic session on the DSM-5 Cultural Formulation Interview (CFI) improves the cultural competence of general psychiatry residents. The main hypothesis was that teaching adult psychiatry residents a 1-hour session on the CFI would improve cultural competence. The exploratory hypothesis was that trainees with more experience in cultural diversity would have a greater increase in cultural competency scores. Psychiatry residents at a metropolitan, county hospital completed demographics and preintervention questionnaires, were exposed to a 1-hour session on the CFI, and were given a postintervention questionnaire. The questionnaire was an adapted version of the validated Cultural Competence Assessment Tool . Paired samples t tests compared pre- to posttest change. Hierarchical linear regression assessed whether pretraining characteristics predicted posttest scores. The mean change of total pre- and posttest scores was significant ( p = .002), as was the mean change in subscales Nonverbal Communications ( p < .001) and Cultural Knowledge ( p = .002). Demographic characteristics did not predict higher posttest scores (when covarying for pretest scores). Psychiatry residents' cultural competence scores improved irrespective of previous experience in cultural diversity. More research is needed to further explore the implications of the improved scores in clinical practice.

Iranian Expert Opinion about Necessary Criteria for Hospitals Management Performance Assessments

PubMed Central

Dadgar, Elham; Janati, Ali; Tabrizi, Jafar Sadegh; Asghari-Jafarabadi, Mohammad; Barati, Omid

2012-01-01

Background: Managers in the hospital should have enough managerial skill to be coordinated with the complex environment. Defining a competency framework assessment for hospital man-agement will help to establish core competencies for hospital managers. The aim of this study was to develop concrete and suitable performance assessment criteria using expert's view. Methods: In this qualitative study in total, 20 professionals participated in the interview and Fo¬cus Group Discussions (FGD). Two of informants were interviewed and 18 professionals par¬ticipants in three focus group discussions. Discussions and interviews were well planned, the FGD environments were suitable and after interviews completion the notes were checked with participant for completeness. Thematic analysis method was used for the analysis of qualitative data. Results: Findings from 3 FGDs and 2 semi structured interviews done with 20 professionals were categorized accordance to themes. The findings were classified in 7 major and 41 sub themes. The major themes include competency related to planning, organization and staff per-formance management, leadership, information management, and clinical governance and per-formance indicators. Conclusion: All participants had hospital administration experience; so their explanation impor¬tant in identifying the criteria and developing hospital managers’ performance assessment tool. In addition to professional perspectives and studies done in other countries, in order to design this kind of tools, it is necessary to adopt the obtained findings to the local hospital conditions. PMID:24688938

Validity aspects of the patient feedback questionnaire on consultation skills (PFC), a promising learning instrument in medical education.

PubMed

Reinders, Marcel E; Blankenstein, Annette H; Knol, Dirk L; de Vet, Henrica C W; van Marwijk, Harm W J

2009-08-01

A focus on the communicator competency is considered to be an important requirement to help physicians to acquire consultation skills. A feedback questionnaire, in which patients assess consultation skills might be a useful learning tool. An existing questionnaire on patient perception of patient-centeredness (PPPC) was adapted to cover the 'communicator' items in the competency profile. We assessed the face and content validity, the construct validity and the internal consistency of this new patient feedback on consultation skills (PFC) questionnaire. We assessed the face validity of the PFC by interviewing patients and general practice trainees (GPTs) during the developmental process. The content validity was determined by experts (n=10). First-year GPTs (23) collected 222 PFCs, from which the data were used to assess the construct validity (factor analysis), internal consistency, response rates and ceiling effects. The PFC adequately covers the corresponding 'communicator' competency (face and content validity). Factor analysis showed a one-dimensional construct. The internal consistency was high (Cronbach's alpha 0.89). For the single items, the response rate varied from 89.2% to 100%; the maximum score (ceiling effect) varied from 45.5% to 89.2%. The PFC appears to be a valid, internally consistent instrument. The PFC may be a valuable learning tool with which GPTs, other physicians and medical students can acquire feedback from patients regarding their consultation skills.

The MacArthur Competence Assessment Tool-Fitness to Plead: a preliminary evaluation of a research instrument for assessing fitness to plead in England and Wales.

PubMed

Akinkunmi, Akintunde A

2002-01-01

This study concerns the preliminary evaluation of an instrument, the MacArthur Competence Assessment Tool-Fitness to Plead (MacCAT-FP), for assessing competence to stand trial, according to the legal requirements in England and Wales. The purposes of the study were to ascertain whether the instrument can be administered practically to groups of prisoners, both those with mental illness and those without; to examine its internal consistency and interrater reliability; to determine whether it can distinguish between fit and unfit individuals, as judged by expert forensic psychiatrists; and to discover whether it is sensitive to changes over time. The instrument was administered to two groups of remanded prisoners: those transferred to psychiatric units for treatment and those without mental illness. In addition to the MacCAT-FP, scales measuring symptom severity and IQ were administered to all participants. Results suggest that the instrument is practical, with good internal consistency and interrater reliability. The correlation of scores with psychiatrists' opinions as to fitness was 0.77. Scores of unfit patients were significantly different from those of fit individuals. The instrument was able to detect significant differences in scores over time. It is suggested that once the instrument has been further validated for research purposes, it may be developed for clinical application.

An argument for using additional bedside tools, such as bedside ultrasound, for volume status assessment in hospitalized medical patients: a needs assessment survey.

PubMed

Low, David; Vlasschaert, Meghan; Novak, Kerri; Chee, Alex; Ma, Irene W Y

2014-11-01

The frequency at which housestaff need to assess volume status on medical inpatients is unknown. In this brief report, we invited 39 housestaff, over 13 randomly selected dates, to complete a 25-item survey. Participants (n = 31, 79%) logged a total of 455 hours, reporting 197 pages or telephone requests received regarding medical inpatients. Of these, 41 pages (21%) required a volume status assessment. Participants reported their volume status assessment competency to be moderate (median score = 3, interquartile range = 3 to 4, where 1 = not competent to perform independently and 6 = above average competence). In 9 of the 41 assessments (22%), at least 1 barrier was reported in determining volume status. The most commonly reported barriers were conflicting physical examination findings (n = 8, 20%) and suboptimal patient examination (n = 5, 12%). Over 20% of pages regarding admitted medical patients required volume status assessments by medical housestaff. Despite moderate self-reported competence in the ability to assess volume status, barriers such as conflicting physical examination findings and suboptimal patient examinations were present in up to 20% of assessments. Therefore, we urge educators to consider incorporating bedside ultrasound training for volume status into the internal medicine curriculum. © 2014 Society of Hospital Medicine.

The Teamwork Mini-Clinical Evaluation Exercise (T-MEX): a workplace-based assessment focusing on collaborative competencies in health care.

PubMed

Olupeliyawa, Asela M; O'Sullivan, Anthony J; Hughes, Chris; Balasooriya, Chinthaka D

2014-02-01

Teamwork is an important and challenging area of learning during the transition from medical graduate to intern. This preliminary investigation examined the psychometric and logistic properties of the Teamwork Mini-Clinical Evaluation Exercise (T-MEX) for the workplace-based assessment of key competencies in working with health care teams. The authors designed the T-MEX for direct observation and assessment of six collaborative behaviors in seven clinical situations important for teamwork, feedback, and reflection. In 2010, they tested it on University of New South Wales senior medical students during their last six-week clinical term to investigate its overall utility, including validity and reliability. Assessors rated students in different situations on the extent to which they met expectations for interns for each collaborative behavior. Both assessors and students rated the tool's usefulness and feasibility. Assessment forms for 88 observed encounters were submitted by 25 students. The T-MEX was suited to a broad range of collaborative clinical practice situations, as evidenced by the encounter types and the behaviors assessed by health care team members. The internal structure of the behavior ratings indicated construct validity. A generalizability study found that eight encounters were adequate for high-stakes measurement purposes. The mean times for observation and feedback and the participants' perceptions suggested usefulness for feedback and feasibility in busy clinical settings. Findings suggest that the T-MEX has good utility for assessing trainee competence in working with health care teams. It fills a gap within the suite of existing tools for workplace-based assessment of professional attributes.

Competency-Based Education for the Molecular Genetic Pathology Fellow

PubMed Central

Talbert, Michael L.; Dunn, S. Terence; Hunt, Jennifer; Hillyard, David R.; Mirza, Imran; Nowak, Jan A.; Van Deerlin, Vivianna; Vnencak-Jones, Cindy L.

2009-01-01

The following report represents guidelines for competency-based fellowship training in Molecular Genetic Pathology (MGP) developed by the Association for Molecular Pathology Training and Education Committee and Directors of MGP Programs in the United States. The goals of the effort were to describe each of the Accreditation Council for Graduate Medical Education competencies as they apply to MGP fellowship training, provide a summary of goals and objectives, and recommend assessment tools. These guidelines are particularly pertinent to MGP training, which is a relatively new specialty that operates within a rapidly changing scientific and technological arena. It is hoped that this document will provide additional material for directors of existing MGP programs to consider for improvement of program objectives and enhancement of evaluation tools already in place. In addition, the guidelines should provide a valuable framework for the development of new MGP programs. PMID:19797613

A model for future healthcare leadership. CHA's competency study provides a practical tool for leadership development.

PubMed

Nygren, D J; Ukeritis, M D; Hickman, J L

1994-06-01

The Catholic Health Association's (CHA's) study "Transformational Leadership for the Healing Ministry: Competencies for the Future" is a powerful tool for the identification and development of leaders in Catholic healthcare. The study can help executives measure their own performance against a standard of excellence and establish goals to improve their performance. Trustees can use the study to establish policies for identification, assessment, development, and career planning for senior executives. Sponsors might consider the competencies as they intensify collaboration in ministry with lay colleagues by encouraging leadership development or as they participate with trustees in the selection of executives. The model presented in CHA's study is dynamic and adaptable to the leadership needs of various organizations. It should not yield a homogenized view of the "ideal" leader in the Catholic ministry. Nor should it encourage elitism or invidious comparisons between leaders or organizations.

Development and Validation of a Parent Report Measure for Assessing Social-Emotional Competencies of Children and Adolescents

ERIC Educational Resources Information Center

Merrell, Kenneth W.; Felver-Gant, Josh C.; Tom, Karalyn M.

2011-01-01

Based on the premises that strength-based assessment of children and adolescents is an important emerging area, and that additional tools for this purpose are needed, this study details development and validation efforts on a new strength-based assessment: the Social-Emotional Assets and Resilience Scale, parent form (SEARS-P). Following careful…

Teachers' Perceptions of a Fundamental Movement Skill (FMS) Assessment Battery in a School Setting

ERIC Educational Resources Information Center

Lander, Natalie; Morgan, Philip J.; Salmon, Jo; Barnett, Lisa M.

2016-01-01

Fundamental movement skills (FMS) competence is low in adolescent girls. An assessment tool for teachers is needed to monitor FMS in this demographic. The present study explored whether the Canadian Agility and Movement Skill Assessment (CAMSA) is feasible for use by physical education (PE) teachers of Australian Year 7 girls in a school setting.…

The Use of Authentic Assessment to Report Accountability Data on Young Children's Language, Literacy and Pre-Math Competency

ERIC Educational Resources Information Center

Gao, Xin; Grisham-Brown, Jennifer

2011-01-01

This validity study examined the validity of Assessment, Evaluation, and Programming System, 2nd Edition (AEPS®), a curriculum-based, authentic assessment for infants and young children. The primary purposes were to: a) examine whether the AEPS® is a concurrently valid tool for measuring young children's language, literacy and pre-math skills for…

Quality Management and Information Brokerage.

ERIC Educational Resources Information Center

van Halm, Johan

1995-01-01

To compete effectively, information brokers need to adopt management and marketing tools; Total Quality Management can upgrade an organization's performance by using customer feedback of its services. SERVQUAL identifies gaps in service by assessing quality expectations versus quality experiences. (AEF)

Prospective Science Teachers' Perception Related to Formative Assessment Approaches in Turkey

ERIC Educational Resources Information Center

Yasar, M. Diyaddin

2017-01-01

In this study, it was aimed to investigate the perceptions and competences of prospective science teachers about formative assessment approaches. Qualitative case study methodology was used in the study. Research group consisted of 17 senior students of science education. As a data collection tool, a semi-structured "Formative Assessment…

Exploring the Validity of a Second Language Intercultural Pragmatics Assessment Tool

ERIC Educational Resources Information Center

Timpe-Laughlin, Veronika; Choi, Ikkyu

2017-01-01

Pragmatics has been a key component of language competence frameworks. While the majority of second/foreign language (L2) pragmatics tests have targeted productive skills, the assessment of receptive pragmatic skills remains a developing field. This study explores validation evidence for a test of receptive L2 pragmatic ability called the American…

Initial Steps in Creating a Developmentally Valid Tool for Observing/Assessing Rope Jumping

ERIC Educational Resources Information Center

Roberton, Mary Ann; Thompson, Gregory; Langendorfer, Stephen J.

2017-01-01

Background: Valid motor development sequences show the various behaviors that children display as they progress toward competence in specific motor skills. Teachers can use these sequences to observe informally or formally assess their students. While longitudinal study is ultimately required to validate developmental sequences, there are earlier,…

Self-Assessing Social and Emotional Instruction and Competencies: A Tool for Teachers

ERIC Educational Resources Information Center

Yoder, Nicholas

2014-01-01

Social and emotional learning (SEL) is crucial for improved educational attainment. As teachers help students achieve new college and career readiness standards, they need to use teaching practices that promote student social and emotional learning in the classroom. Our new resource, "Self-Assessing Social and Emotional Instruction and…

Development and Initial Testing of a Structured Clinical Observation Tool to Assess Pharmacotherapy Competence

ERIC Educational Resources Information Center

Young, John Q.; Lieu, Sandra; O'Sullivan, Patricia; Tong, Lowell

2011-01-01

Objective: The authors developed and tested the feasibility and utility of a new direct-observation instrument to assess trainee performance of a medication management session. Methods: The Psychopharmacotherapy-Structured Clinical Observation (P-SCO) instrument was developed based on multiple sources of expertise and then implemented in 4…

Using Community Health Assessment to Teach and Explore Health Status Disparities

ERIC Educational Resources Information Center

Sullivan, Marianne; Levine, Jack

2014-01-01

Introduction: Community health assessment (CHA) is a useful tool for identifying health status disparities at the community level. Developing the skills of master's level public health students to conduct CHA addresses a number of the Association of Schools of Public Health Core competencies for graduate public health education. Teaching…

Assessment of Social Information Processing in early childhood: development and initial validation of the Schultz Test of Emotion Processing-Preliminary Version.

PubMed

Schultz, David; Ambike, Archana; Logie, Sean Kevin; Bohner, Katherine E; Stapleton, Laura M; Vanderwalde, Holly; Min, Christopher B; Betkowski, Jennifer A

2010-07-01

Crick and Dodge's (Psychological Bulletin 115:74-101, 1994) social information processing model has proven very useful in guiding research focused on aggressive and peer-rejected children's social-cognitive functioning. Its application to early childhood, however, has been much more limited. The present study responds to this gap by developing and validating a video-based assessment tool appropriate for early childhood, the Schultz Test of Emotion Processing-Preliminary Version (STEP-P). One hundred twenty-five Head Start preschool children participated in the study. More socially competent children more frequently attributed sadness to the victims of provocation and labeled aggressive behaviors as both morally unacceptable and less likely to lead to positive outcomes. More socially competent girls labeled others' emotions more accurately. More disruptive children more frequently produced physically aggressive solutions to social provocations, and more disruptive boys less frequently interpreted social provocations as accidental. The STEP-P holds promise as an assessment tool that assesses knowledge structures related to the SIP model in early childhood.

Impact of Student vs Faculty Facilitators on Motivational Interviewing Student Outcomes.

PubMed

Widder-Prewett, Rebecca; Draime, Juanita A; Cameron, Ginger; Anderson, Douglas; Pinkerton, Mark; Chen, Aleda M H

2017-08-01

Objective. To determine the impact of student or faculty facilitation on student self-assessed attitudes, confidence, and competence in motivational interviewing (MI) skills; actual competence; and evaluation of facilitator performance. Methods. Second-year pharmacy (P2) students were randomly assigned to a student or faculty facilitator for a four-hour, small-group practice of MI skills. MI skills were assessed in a simulated patient encounter with the mMITI (modified Motivational Interviewing Treatment Integrity) tool. Students completed a pre-post, 6-point, Likert-type assessment addressing the research objectives. Differences were assessed using a Mann-Whitney U test. Results. Student (N=44) post-test attitudes, confidence, perceived or actual competence, and evaluations of facilitator performance were not different for faculty- and student-facilitated groups. Conclusion. Using pharmacy students as small-group facilitators did not affect student performance and were viewed as equally favorable. Using pharmacy students as facilitators can lessen faculty workload and provide an outlet for students to develop communication and facilitation skills that will be needed in future practice.

Milestones: Critical Elements in Clinical Informatics Fellowship Programs

PubMed Central

Lehmann, Christoph U.; Munger, Benson

2016-01-01

Summary Background Milestones refer to points along a continuum of a competency from novice to expert. Resident and fellow assessment and program evaluation processes adopted by the ACGME include the mandate that programs report the educational progress of residents and fellows twice annually utilizing Milestones developed by a specialty specific ACGME working group of experts. Milestones in clinical training programs are largely unmapped to specific assessment tools. Residents and fellows are mainly assessed using locally derived assessment instruments. These assessments are then reviewed by the Clinical Competency Committee which assigns and reports trainee ratings using the specialty specific reporting Milestones. Methods and Results The challenge and opportunity facing the nascent specialty of Clinical Informatics is how to optimally utilize this framework across a growing number of accredited fellowships. The authors review how a mapped milestone framework, in which each required sub-competency is mapped to a single milestone assessment grid, can enable the use of milestones for multiple uses including individualized learning plans, fellow assessments, and program evaluation. Furthermore, such a mapped strategy will foster the ability to compare fellow progress within and between Clinical Informatics Fellowships in a structured and reliable fashion. Clinical Informatics currently has far less variability across programs and thus could easily utilize a more tightly defined set of milestones with a clear mapping to sub-competencies. This approach would enable greater standardization of assessment instruments and processes across programs while allowing for variability in how those sub-competencies are taught. Conclusions A mapped strategy for Milestones offers significant advantages for Clinical Informatics programs. PMID:27081414

Competency Assessment in Family Medicine Residency: Observations, Knowledge-Based Examinations, and Advancement.

PubMed

Mainous, Arch G; Fang, Bo; Peterson, Lars E

2017-12-01

The Family Medicine (FM) Milestones are competency-based assessments of residents in key dimensions relevant to practice in the specialty. Residency programs use the milestones in semiannual reviews of resident performance from the time of entry into the program to graduation. Using a national sample, we investigated the relationship of FM competency-based assessments to resident progress and the complementarity of milestones with knowledge-based assessments in FM residencies. We used midyear and end-of-year milestone ratings for all FM residents in Accreditation Council for Graduate Medical Education-accredited programs during academic years 2014-2015 and 2015-2016. The milestones contain 22 items across 6 competencies. We created a summative index across the milestones. The American Board of Family Medicine database provided resident demographics and in-training examination (ITE) scores. We linked information to the milestone data. The sample encompassed 6630 FM residents. The summative milestone index increased, on average, for each cohort (postgraduate year 1 [PGY-1] to PGY-2 and PGY-2 to PGY-3) at each assessment. The correlation between the milestone index that excluded the medical knowledge milestone and ITE scores was r  = .195 ( P  < .001) for PGY-1 to PGY-2 cohort and r  = .254 ( P  < .001) for PGY-2 to PGY-3 cohort. For both cohorts, ITE scores and composite milestone assessments were higher for residents who advanced than for those who did not. Competency-based assessment using the milestones for FM residents seems to be a viable multidimensional tool to assess the successful progression of residents.

The role of assessment in competency-based medical education.

PubMed

Holmboe, Eric S; Sherbino, Jonathan; Long, Donlin M; Swing, Susan R; Frank, Jason R

2010-01-01

Competency-based medical education (CBME), by definition, necessitates a robust and multifaceted assessment system. Assessment and the judgments or evaluations that arise from it are important at the level of the trainee, the program, and the public. When designing an assessment system for CBME, medical education leaders must attend to the context of the multiple settings where clinical training occurs. CBME further requires assessment processes that are more continuous and frequent, criterion-based, developmental, work-based where possible, use assessment methods and tools that meet minimum requirements for quality, use both quantitative and qualitative measures and methods, and involve the wisdom of group process in making judgments about trainee progress. Like all changes in medical education, CBME is a work in progress. Given the importance of assessment and evaluation for CBME, the medical education community will need more collaborative research to address several major challenges in assessment, including "best practices" in the context of systems and institutional culture and how to best to train faculty to be better evaluators. Finally, we must remember that expertise, not competence, is the ultimate goal. CBME does not end with graduation from a training program, but should represent a career that includes ongoing assessment.

Electrodiagnostic medicine skills competency in physical medicine and rehabilitation residents: a method for development and assessment.

PubMed

Brown, David; Cuccurullo, Sara; Lee, Joseph; Petagna, Ann; Strax, Thomas

2008-08-01

This project sought to create an educational module including evaluation methodology to instruct physical medicine and rehabilitation (PM&R) residents in electrodiagnostic evaluation of patients with neuromuscular problems, and to verify acquired competencies in those electrodiagnostic skills through objective evaluation methodology. Sixteen residents were trained by board-certified neuromuscular and electrodiagnostic medicine physicians through technical training, lectures, and review of self-assessment examination (SAE) concepts from the American Academy of Physical Medicine & Rehabilitation syllabus provided in the Archives of Physical Medicine and Rehabilitation. After delivery of the educational module, knowledge acquisition and skill attainment were measured in (1) clinical skill in diagnostic procedures via a procedure checklist, (2) diagnosis and ability to design a patient-care management plan via chart simulated recall (CSR) exams, (3) physician/patient interaction via patient surveys, (4) physician/staff interaction via 360-degree global ratings, and (5) ability to write a comprehensive patient-care report and to document a patient-care management plan in accordance with Medicare guidelines via written patient reports. Assessment tools developed for this program address the basic competencies outlined by the Accreditation Council for Graduate Medical Education (ACGME). To test the success of the standardized educational module, data were collected on an ongoing basis. Objective measures compared resident SAE scores in electrodiagnostics (EDX) before and after institution of the comprehensive EDX competency module in a PM&R residency program. Fifteen of 16 residents (94%) successfully demonstrated proficiency in every segment of the evaluation element of the educational module by the end of their PGY-4 electrodiagnostic rotation. The resident who did not initially pass underwent remedial coursework and passed on the second attempt. Furthermore, the residents' proficiency as demonstrated by the evaluation after implementation of the standardized educational module positively correlated to an increase in resident SAE scores in EDX compared with resident scores before implementation of the educational module. Resident proficiency in EDX medicine skills and knowledge was objectively verified after completion of the standardized educational module. Validation of the assessment tools is evidenced by collected data correlating with significantly improved SAE scores and American Association of Neuromuscular and Electrodiagnostic Medicine (AANEM) exam scores, as outlined in the result section. In addition, the clinical development tool (procedure checklist) was validated by residents being individually observed performing skills and deemed competent by an AANEM-certified physician. The standardized educational module and evaluation methodology provide a potential framework for the definition of baseline competency in the clinical skill area of EDX.

Toward competency-based curriculum: Application of workplace-based assessment tools in the National Saudi Arabian Anesthesia Training Program.

PubMed

Boker, Ama

2016-01-01

The anesthesia training program of the Saudi Commission for health specialties has introduced a developed competency-based anesthesia residency program starting from 2015 with the utilization of the workplace-based assessment (WBA) tools, namely mini-clinical exercises (mini-CEX), direct observation of procedural skills (DOPS), and case-based discussion (CBD). This work aimed to describe the process of development of anesthesia-specific list of mini-CEX, DOPS, and CBD tools within the Saudi Arabian Anesthesia Training Programs. To introduce the main concepts of formative WBA tools and to develop anesthesia-specific applications for each of the selected WBA tools, four 1-day workshops were held at the level of major training committees at eastern (Dammam), western (Jeddah), and central (Riyadh) regions in the Kingdom were conducted. Sixty-seven faculties participated in these workshops. After conduction of the four workshops, the anesthesia-specific applications setting of mini-CEX, DOPS, and CBD tools among the 5-year levels were fully described. The level of the appropriate consultation skills was divided according to the case complexity adopted from the American Society of Anesthesiologists physical classification for adult and obstetric and pediatric patient as well as the type of the targeted anesthetic procedure. WBA anesthesia-specific lists of mini-CEX, DOPS, and CBD forms were easily incorporated first into guidelines to help the first stage of implementation of formative assessment in the Saudi Arabian Anesthesia Residency Program, and this can be helpful to replicate such program within other various training programs in Saudi Arabia and abroad.

The use of standardized patients in the plastic surgery residency curriculum: teaching core competencies with objective structured clinical examinations.

PubMed

Davis, Drew; Lee, Gordon

2011-07-01

As of 2006, the Accreditation Council for Graduate Medical Education had defined six "core competencies" of residency education: interpersonal communication skills, medical knowledge, patient care, professionalism, practice-based learning and improvement, and systems-based practice. Objective structured clinical examinations using standardized patients are becoming effective educational tools, and the authors developed a novel use of the examinations in plastic surgery residency education that assesses all six competencies. Six plastic surgery residents, two each from postgraduate years 4, 5, and 6, participated in the plastic surgery-specific objective structured clinical examination that focused on melanoma. The examination included a 30-minute videotaped encounter with a standardized patient actor and a postencounter written exercise. The residents were scored on their performance in all six core competencies by the standardized patients and faculty experts on a three-point scale (1 = novice, 2 = moderately skilled, and 3 = proficient). Resident performance was averaged for each postgraduate year, stratified according to core competency, and scored from a total of 100 percent. Residents overall scored well in interpersonal communications skills (84 percent), patient care (83 percent), professionalism (86 percent), and practice-based learning (84 percent). Scores in medical knowledge showed a positive correlation with level of training (86 percent). All residents scored comparatively lower in systems-based practice (65 percent). The residents reported unanimously that the objective structured clinical examination was realistic and educational. The objective structured clinical examination provided comprehensive and meaningful feedback and identified areas of strengths and weakness for the residents and for the teaching program. The examination is an effective assessment tool for the core competencies and a valuable adjunct to residency training.

Developing an Evaluation Tool for Assessing Clinical Ethics Consultation Skills in Simulation Based Education: The ACES Project.

PubMed

Wasson, Katherine; Parsi, Kayhan; McCarthy, Michael; Siddall, Viva Jo; Kuczewski, Mark

2016-06-01

The American Society for Bioethics and Humanities has created a quality attestation (QA) process for clinical ethics consultants; the pilot phase of reviewing portfolios has begun. One aspect of the QA process which is particularly challenging is assessing the interpersonal skills of individual clinical ethics consultants. We propose that using case simulation to evaluate clinical ethics consultants is an approach that can meet this need provided clear standards for assessment are identified. To this end, we developed the Assessing Clinical Ethics Skills (ACES) tool, which identifies and specifies specific behaviors that a clinical ethics consultant should demonstrate in an ethics case simulation. The aim is for the clinical ethics consultant or student to use a videotaped case simulation, along with the ACES tool scored by a trained rater, to demonstrate their competence as part of their QA portfolio. The development and piloting of the tool is described.

Informed consent instead of assent is appropriate in children from the age of twelve: Policy implications of new findings on children's competence to consent to clinical research.

PubMed

Hein, Irma M; De Vries, Martine C; Troost, Pieter W; Meynen, Gerben; Van Goudoever, Johannes B; Lindauer, Ramón J L

2015-11-09

For many decades, the debate on children's competence to give informed consent in medical settings concentrated on ethical and legal aspects, with little empirical underpinnings. Recently, data from empirical research became available to advance the discussion. It was shown that children's competence to consent to clinical research could be accurately assessed by the modified MacArthur Competence Assessment Tool for Clinical Research. Age limits for children to be deemed competent to decide on research participation have been studied: generally children of 11.2 years and above were decision-making competent, while children of 9.6 years and younger were not. Age was pointed out to be the key determining factor in children's competence. In this article we reflect on policy implications of these findings, considering legal, ethical, developmental and clinical perspectives. Although assessment of children's competence has a normative character, ethics, law and clinical practice can benefit from research data. The findings may help to do justice to the capacities children possess and challenges they may face when deciding about treatment and research options. We discuss advantages and drawbacks of standardized competence assessment in children on a case-by-case basis compared to application of a fixed age limit, and conclude that a selective implementation of case-by-case competence assessment in specific populations is preferable. We recommend the implementation of age limits based on empirical evidence. Furthermore, we elaborate on a suitable model for informed consent involving children and parents that would do justice to developmental aspects of children and the specific characteristics of the parent-child dyad. Previous research outcomes showed that children's medical decision-making capacities could be operationalized into a standardized assessment instrument. Recommendations for policies include a dual consent procedure, including both child as well as parents, for children from the age of 12 until they reach majority. For children between 10 and 12 years of age, and in case of children older than 12 years in special research populations of mentally compromised patients, we suggest a case-by-case assessment of children's competence to consent. Since such a dual consent procedure is fundamentally different from a procedure of parental permission and child assent, and would imply a considerable shift regarding some current legislations, practical implications are elaborated.

Using competences and competence tools in workforce development.

PubMed

Green, Tess; Dickerson, Claire; Blass, Eddie

The NHS Knowledge and Skills Framework (KSF) has been a driving force in the move to competence-based workforce development in the NHS. Skills for Health has developed national workforce competences that aim to improve behavioural performance, and in turn increase productivity. This article describes five projects established to test Skills for Health national workforce competences, electronic tools and products in different settings in the NHS. Competences and competence tools were used to redesign services, develop job roles, identify skills gaps and develop learning programmes. Reported benefits of the projects included increased clarity and a structured, consistent and standardized approach to workforce development. Findings from the evaluation of the tools were positive in terms of their overall usefulness and provision of related training/support. Reported constraints of using the competences and tools included issues relating to their availability, content and organization. It is recognized that a highly skilled and flexible workforce is important to the delivery of high-quality health care. These projects suggest that Skills for Health competences can be used as a 'common currency' in workforce development in the UK health sector. This would support the need to adapt rapidly to changing service needs.

A user's guide to coping with estuarine management bureaucracy: An Estuarine Planning Support System (EPSS) tool.

PubMed

Lonsdale, Jemma; Nicholson, Rose; Weston, Keith; Elliott, Michael; Birchenough, Andrew; Sühring, Roxana

2018-02-01

Estuaries are amongst the most socio-economically and ecologically important environments however, due to competing and conflicting demands, management is often challenging with a complex legislative framework managed by multiple agencies. To facilitate the understanding of this legislative framework, we have developed a GISbased Estuarine Planning Support System tool. The tool integrates the requirements of the relevant legislation and provides a basis for assessing the current environmental state of an estuary as well as informing and assessing new plans to ensure a healthy estuarine state. The tool ensures that the information is easily accessible for regulators, managers, developers and the public. The tool is intended to be adaptable, but is assessed using the Humber Estuary, United Kingdom as a case study area. The successful application of the tool for complex socio-economic and environmental systems demonstrates that the tool can efficiently guide users through the complex requirements needed to support sustainable development. Crown Copyright © 2017. Published by Elsevier Ltd. All rights reserved.

Assessing Embedded Geospatial Student Learning Outcomes

ERIC Educational Resources Information Center

Carr, John David

2012-01-01

Geospatial tools and technologies have become core competencies for natural resource professionals due to the monitoring, modeling, and mapping capabilities they provide. To prepare students with needed background, geospatial instructional activities were integrated across Forest Management; Natural Resources; Fisheries, Wildlife, &…

Motivational interviewing workshop in a virtual world: learning as avatars.

PubMed

Shershneva, Marianna; Kim, Ji-Hye; Kear, Cynthia; Heyden, Robin; Heyden, Neil; Lee, Jay; Mitchell, Suzanne

2014-04-01

Limited research has been done to understand outcomes of continuing medical education offered in three-dimensional, immersive virtual worlds. We studied a case of a virtual world workshop on motivational interviewing (MI) applied to smoking cessation counseling and its educational impact. To facilitate content development and evaluation, we specified desired MI competencies. The workshop consisted of three sessions, which included lectures, practice with standardized patients, and chat interactions. Data were collected from 13 primary care physicians and residents through workshop observation, and pre- and 3-month post-workshop telephone/Skype interviews and interactions with standardized patients. Interactions with standardized patients were assessed by an expert using a validated MI tool and by standardized patients using a tool developed for this study. For 11 participants who attended two or three sessions, we conducted paired-samples t tests comparing mean differences between the competency scores before and after the workshop. Expert assessment showed significant improvement on six of seven MI competencies. All participants reported learning new knowledge and skills, and nine described incorporating new learning into their clinical practice. Practicing MI with standardized patients and/or observing others' practice appeared to be the most helpful workshop component. The evaluated workshop had positive impact on participants' competencies and practice as related to MI applied to smoking cessation counseling. Our findings support further exploration of three-dimensional virtual worlds as learning environments for continuing medical education.

Student Self-Assessment and Faculty Assessment of Performance in an Interprofessional Error Disclosure Simulation Training Program.

PubMed

Poirier, Therese I; Pailden, Junvie; Jhala, Ray; Ronald, Katie; Wilhelm, Miranda; Fan, Jingyang

2017-04-01

Objectives. To conduct a prospective evaluation for effectiveness of an error disclosure assessment tool and video recordings to enhance student learning and metacognitive skills while assessing the IPEC competencies. Design. The instruments for assessing performance (planning, communication, process, and team dynamics) in interprofessional error disclosure were developed. Student self-assessment of performance before and after viewing the recordings of their encounters were obtained. Faculty used a similar instrument to conduct real-time assessments. An instrument to assess achievement of the Interprofessional Education Collaborative (IPEC) core competencies was developed. Qualitative data was reviewed to determine student and faculty perceptions of the simulation. Assessment. The interprofessional simulation training involved a total of 233 students (50 dental, 109 nursing and 74 pharmacy). Use of video recordings made a significant difference in student self-assessment for communication and process categories of error disclosure. No differences in student self-assessments were noted among the different professions. There were differences among the family member affects for planning and communication for both pre-video and post-video data. There were significant differences between student self-assessment and faculty assessment for all paired comparisons, except communication in student post-video self-assessment. Students' perceptions of achievement of the IPEC core competencies were positive. Conclusion. The use of assessment instruments and video recordings may have enhanced students' metacognitive skills for assessing performance in interprofessional error disclosure. The simulation training was effective in enhancing perceptions on achievement of IPEC core competencies. This enhanced assessment process appeared to enhance learning about the skills needed for interprofessional error disclosure.

Student Self-Assessment and Faculty Assessment of Performance in an Interprofessional Error Disclosure Simulation Training Program

PubMed Central

Pailden, Junvie; Jhala, Ray; Ronald, Katie; Wilhelm, Miranda; Fan, Jingyang

2017-01-01

Objectives. To conduct a prospective evaluation for effectiveness of an error disclosure assessment tool and video recordings to enhance student learning and metacognitive skills while assessing the IPEC competencies. Design. The instruments for assessing performance (planning, communication, process, and team dynamics) in interprofessional error disclosure were developed. Student self-assessment of performance before and after viewing the recordings of their encounters were obtained. Faculty used a similar instrument to conduct real-time assessments. An instrument to assess achievement of the Interprofessional Education Collaborative (IPEC) core competencies was developed. Qualitative data was reviewed to determine student and faculty perceptions of the simulation. Assessment. The interprofessional simulation training involved a total of 233 students (50 dental, 109 nursing and 74 pharmacy). Use of video recordings made a significant difference in student self-assessment for communication and process categories of error disclosure. No differences in student self-assessments were noted among the different professions. There were differences among the family member affects for planning and communication for both pre-video and post-video data. There were significant differences between student self-assessment and faculty assessment for all paired comparisons, except communication in student post-video self-assessment. Students’ perceptions of achievement of the IPEC core competencies were positive. Conclusion. The use of assessment instruments and video recordings may have enhanced students’ metacognitive skills for assessing performance in interprofessional error disclosure. The simulation training was effective in enhancing perceptions on achievement of IPEC core competencies. This enhanced assessment process appeared to enhance learning about the skills needed for interprofessional error disclosure. PMID:28496274

Workplace based assessment: a step to promote competency based postgraduate training.

PubMed

Singh, Tejinder; Modi, Jyoti Nath

2013-06-08

There has been an increasing emphasis on defining outcomes of medical education in terms of performance of trainees. This is a step beyond the description of outcomes in terms of competence that encompasses mostly potential abilities rather than the actual performance. The contextual adaptations and behavior judgments of the trainees are best assessed by a program of in-training assessment. Workplace based assessment (WPBA) is one of the modalities, which assesses the trainee in authentic settings. Though Postgraduate (PG) medical training in India is said to be competency-based, most institutions do not have any formative or in-training assessment program for the same. The two cardinal elements of WPBA are direct observation and conducted in work place in addition to provision of feedback to the trainee. The WPBA conforms to the highest (Level 4: Does) of Millers pyramid and also has the potential to assess at all four levels. Some of the tools used for WPBA are: Logbooks, Clinical Encounter Cards (CEC), mini-Clinical Evaluation Exercise (mini-CEX), Case based discussions, Direct Observation of Procedural Skills (DOPS), Multisource feedback (peers, co-workers, seniors, patients) etc. These can be documented in the form of a portfolio that provides a longitudinal view of experiences and progress of the trainee. The WPBA scores high on validity and educational impact by virtue of being based on direct observation in real situation and contextual feedback. The feasibility and acceptability is enhanced by making appropriate choices of tools, advance planning, building of mutual trust, and training of assessors. Given the established benefits of WPBA in shaping clinical learning, there is an imminent need for including this mode of assessment in our clinical training programs especially PG training.

Competence to Complete Psychiatric Advance Directives: Effects of Facilitated Decision Making

PubMed Central

Swanson, Jeffrey W.; Appelbaum, Paul S.; Swartz, Marvin S.; Ferron, Joelle; Van Dorn, Richard A.; Wagner, H. Ryan

2013-01-01

Psychiatric advance directives (PADs) statutes presume competence to complete these documents, but the range and dimensions of decisional competence among people who actually complete PADs is unknown. This study examines clinical and neuropsychological correlates of performance on a measure to assess competence to complete PADs and investigates the effects of a facilitated PAD intervention on decisional capacity. N = 469 adults with psychotic disorders were interviewed at baseline and then randomly assigned to either a control group in which they received written materials about PADs or to an intervention group in which they were offered an opportunity to meet individually with a trained facilitator to create a PAD. At baseline, domains on the Decisional Competence Assessment Tool for PADs (DCAT-PAD) were most strongly associated with IQ, verbal memory, abstract thinking, and psychiatric symptoms. At one-month follow-up, participants in the intervention group showed more improvement on the DCAT-PAD than controls, particularly among participants with pre-morbid IQ estimates below the median of 100. The results suggest that PAD facilitation is an effective method to boost competence of cognitively-impaired clients to write PADs and make treatment decisions within PADs, thereby maximizing the chances their advance directives will be valid. PMID:17294136

Development and evaluation of a web-based breast cancer cultural competency course for primary healthcare providers.

PubMed

Palmer, Richard C; Samson, Raquel; Triantis, Maria; Mullan, Irene D

2011-08-15

To develop and evaluate a continuing medical education (CME) course aimed at improving healthcare provider knowledge about breast cancer health disparities and the importance of cross-cultural communication in provider-patient interactions about breast cancer screening. An interactive web-based CME course was developed and contained information about breast cancer disparities, the role of culture in healthcare decision making, and demonstrated a model of cross-cultural communication. A single group pre-/post-test design was used to assess knowledge changes. Data on user satisfaction was also collected. In all, 132 participants registered for the CME with 103 completing both assessments. Differences between pre-/post-test show a significant increase in knowledge (70% vs. 94%; p < .001). Ninety-five percent of participants agreed that the web based training was an appropriate tool to train healthcare providers about cultural competency and health disparities. There was an overall high level of satisfaction among all users. Users felt that learning objectives were met and the web-based format was appropriate and easy to use and suggests that web-based CME formats are an appropriate tool to teach cultural competency skills. However, more information is needed to understand how the CME impacted practice behaviors.

A Systematic Review of Tools Used to Assess Team Leadership in Health Care Action Teams.

PubMed

Rosenman, Elizabeth D; Ilgen, Jonathan S; Shandro, Jamie R; Harper, Amy L; Fernandez, Rosemarie

2015-10-01

To summarize the characteristics of tools used to assess leadership in health care action (HCA) teams. HCA teams are interdisciplinary teams performing complex, critical tasks under high-pressure conditions. The authors conducted a systematic review of the PubMed/MEDLINE, CINAHL, ERIC, EMBASE, PsycINFO, and Web of Science databases, key journals, and review articles published through March 2012 for English-language articles that applied leadership assessment tools to HCA teams in all specialties. Pairs of reviewers assessed identified articles for inclusion and exclusion criteria and abstracted data on study characteristics, tool characteristics, and validity evidence. Of the 9,913 abstracts screened, 83 studies were included. They described 61 team leadership assessment tools. Forty-nine tools (80%) provided behaviors, skills, or characteristics to define leadership. Forty-four tools (72%) assessed leadership as one component of a larger assessment, 13 tools (21%) identified leadership as the primary focus of the assessment, and 4 (7%) assessed leadership style. Fifty-three studies (64%) assessed leadership at the team level; 29 (35%) did so at the individual level. Assessments of simulated (n = 55) and live (n = 30) patient care events were performed. Validity evidence included content validity (n = 75), internal structure (n = 61), relationship to other variables (n = 44), and response process (n = 15). Leadership assessment tools applied to HCA teams are heterogeneous in content and application. Comparisons between tools are limited by study variability. A systematic approach to team leadership tool development, evaluation, and implementation will strengthen understanding of this important competency.

Competency Assessment in Senior Emergency Medicine Residents for Core Ultrasound Skills.

PubMed

Schmidt, Jessica N; Kendall, John; Smalley, Courtney

2015-11-01

Quality resident education in point-of-care ultrasound (POC US) is becoming increasingly important in emergency medicine (EM); however, the best methods to evaluate competency in graduating residents has not been established. We sought to design and implement a rigorous assessment of image acquisition and interpretation in POC US in a cohort of graduating residents at our institution. We evaluated nine senior residents in both image acquisition and image interpretation for five core US skills (focused assessment with sonography for trauma (FAST), aorta, echocardiogram (ECHO), pelvic, central line placement). Image acquisition, using an observed clinical skills exam (OSCE) directed assessment with a standardized patient model. Image interpretation was measured with a multiple-choice exam including normal and pathologic images. Residents performed well on image acquisition for core skills with an average score of 85.7% for core skills and 74% including advanced skills (ovaries, advanced ECHO, advanced aorta). Residents scored well but slightly lower on image interpretation with an average score of 76%. Senior residents performed well on core POC US skills as evaluated with a rigorous assessment tool. This tool may be developed further for other EM programs to use for graduating resident evaluation.

A systematic review of the literature about competence and poor insight.

PubMed

Ruissen, A M; Widdershoven, G A M; Meynen, G; Abma, T A; van Balkom, A J L M

2012-02-01

  Formally, incompetence implies that a patient cannot meet the legal requirements for informed consent. Our aim was to review the scientific literature on the relationship between competence and insight in patients with psychiatric disorders, how competence and insight are connected in these patients and whether there are differences in competence and insight among patients with different disorders.   A search in PubMed/Medline was performed. Articles were assessed on relevance criteria by two independent reviewers. Study design, population, variables, and outcomes were extracted.   Seven articles were included on studies of psychiatric inpatients and outpatients and of psychotic and non-psychotic patients. All studies used the MacArthur Competence Assessment Tool (MacCAT). All studies but one found a strong correlation between poor insight and incompetence. Psychotic patients with poor insight are very likely to be incompetent, and psychotic patients with adequate insight are generally competent. One well-executed study showed that in non-psychotic disorders, however, another relationship emerges; competence and insight do not completely overlap in these patients.   Most incompetent psychotic patients have poor insight, but non-psychotic patients with adequate insight were incompetent in a substantial number of cases. In sum: non-psychotic patients with adequate insight can be incompetent. © 2011 John Wiley & Sons A/S.

[Construction of competency model of 'excellent doctor' in Chinese medicine].

PubMed

Jin, Aning; Tian, Yongquan; Zhao, Taiyang

2014-05-01

To evaluate outstanding and ordinary persons from personal characteristics using competency as the important criteria, which is the future direction of medical education reform. We carried on a behavior event interview about famous doctors of old traditional Chinese medicine, compiled competency dictionary, proceed control prediction test. SPSS and AMOS were used to be data analysis tools on statistics. We adopted the model of peer assessment and contrast to carry out empirical research. This project has carried on exploratory factor analysis and confirmatory factor analysis, established a "5A" competency model which include moral ability, thinking ability, communication ability, learning and practical ability. Competency model of "excellent doctor" in Chinese medicine has been validated, with good reliability and validity, and embodies the characteristics of traditional Chinese medicine personnel training, with theoretical and practical significance for excellence in medicine physician training.

The Capacity to Consent to Research among Older Adults

ERIC Educational Resources Information Center

Lee, Minhong

2010-01-01

The aim of this study was to develop and validate an assessment tool for research consent competence in older participants. A four-item instrument was developed to assess the capacity of the older adults to consent to research. Data were obtained from 203 nursing home residents from two facilities and 201 community-dwelling older adults in a…

Standards as a Tool for Teaching and Assessing Cross-Curricular Writing

ERIC Educational Resources Information Center

Evensen, Lars Sigfred; Berge, Kjell Lars; Thygesen, Ragnar; Matre, Synnove; Solheim, Randi

2016-01-01

The Berge et al. article in this volume presents the functional construct of writing that underlies summative and formative assessment of writing as a key competency in Norway. A functional construct implies that specific acts of writing and their purposes constrain what is a relevant selection among the semiotic resources that writing generally…

Developing a Scale for Innovation Management at Schools: A Study of Validity and Reliability

ERIC Educational Resources Information Center

Bulbul, Tuncer

2012-01-01

The purpose of this study is to develop a valid and reliable assessment tool for use in determining the competency beliefs of school administrators about innovation management. The scale applied to a study group of 216 school administrators, after work Centered on assessing intelligibility and specialized opinion. Exploratory and confirmatory…

Listening for Competence through Documentation: Assessing Children with Language Delays Using Digital Video

ERIC Educational Resources Information Center

Suarez, Stephanie Cox; Daniels, Karen J.

2009-01-01

This case study uses documentation as a tool for formative assessment to interpret the learning of twin boys with significantly delayed language skills. Reggio-inspired documentation (the act of collecting, interpreting, and reflecting on traces of learning from video, images, and observation notes) focused on the unfolding of the boys' nonverbal…

The Challenge of Assessing Creative Problem Solving in Client-Based Marketing Development Projects: A SOLO Taxonomy Approach

ERIC Educational Resources Information Center

Jaskari, Minna-Maarit

2013-01-01

Creativity and marketing imagination are essential core competencies for marketers. Therefore, higher marketing education emphasizes creativity in several ways. However, assessing creativity and creative problem solving is challenging and tools for this purpose have not been developed in the context of marketing education. To address this gap, we…

Perspectives on procedure-based assessments: a thematic analysis of semistructured interviews with 10 UK surgical trainees.

PubMed

Shalhoub, Joseph; Marshall, Dominic C; Ippolito, Kate

2017-03-24

The introduction of competency-based training has necessitated development and implementation of accompanying mechanisms for assessment. Procedure-based assessments (PBAs) are an example of workplace-based assessments that are used to examine focal competencies in the workplace. The primary objective was to understand surgical trainees' perspective on the value of PBA. Semistructured interviews with 10 surgical trainees individually interviewed to explore their views. Interviews were audio-recorded and transcribed; following this, they were open and axial coded. Thematic analysis was then performed. Semistructured interviews yielded several topical and recurring themes. In trainees' experience, the use of PBAs as a summative tool limits their educational value. Trainees reported a lack of support from seniors and variation in the usefulness of the tool based on stage of training. Concerns related to the validity of PBAs for evaluating trainees' performance with reports of 'gaming' the system and trainees completing their own assessments. Trainees did identify the significant value of PBAs when used correctly. Benefits included the identification of additional learning opportunities, standardisation of assessment and their role in providing a measure of progress. The UK surgical trainees interviewed identified both limitations and benefits to PBAs; however, we would argue based on their responses and our experience that their use as a summative tool limits their formative use as an educational opportunity. PBAs should either be used exclusively to support learning or solely as a summative tool; if so, further work is needed to audit, validate and standardise them for this purpose. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

Simulation-based ongoing professional practice evaluation in psychiatry: a novel tool for performance assessment.

PubMed

Gorrindo, Tristan; Goldfarb, Elizabeth; Birnbaum, Robert J; Chevalier, Lydia; Meller, Benjamin; Alpert, Jonathan; Herman, John; Weiss, Anthony

2013-07-01

Ongoing professional practice evaluation (OPPE) activities consist of a quantitative, competency-based evaluation of clinical performance. Hospitals must design assessments that measure clinical competencies, are scalable, and minimize impact on the clinician's daily routines. A psychiatry department at a large academic medical center designed and implemented an interactive Web-based psychiatric simulation focusing on violence risk assessment as a tool for a departmentwide OPPE. Of 412 invited clinicians in a large psychiatry department, 410 completed an online simulation in April-May 2012. Participants received scheduled e-mail reminders with instructions describing how to access the simulation. Using the Computer Simulation Assessment Tool, participants viewed an introductory video and were then asked to conduct a risk assessment, acting as a clinician in the encounter by selecting actions from a series of drop-down menus. Each action was paired with a corresponding video segment of a clinical encounter with a standardized patient. Participants were scored on the basis of their actions within the simulation (Measure 1) and by their responses to the open-ended questions in which they were asked to integrate the information from the simulation in a summative manner (Measure 2). Of the 410 clinicians, 381 (92.9%) passed Measure 1,359 (87.6%) passed Measure 2, and 5 (1.2%) failed both measures. Seventy-five (18.3%) participants were referred for focused professional practice evaluation (FPPE) after failing either Measure 1, Measure 2, or both. Overall, Web-based simulation and e-mail engagement tools were a scalable and efficient way to assess a large number of clinicians in OPPE and to identify those who required FPPE.

Use of a television crime-drama series to promote legal understanding in mentally ill, incompetent defendants: a pilot study.

PubMed

Montgomery, John; Brooks, Martin H

2005-03-01

Videotaped material is used for educational purposes in many areas of medicine. In forensic facilities, programs designed to restore competency to stand trial (CST) in incompetent, mentally ill defendants have utilized videotaped courtroom proceedings as learning tools. This pilot study reviewed the progress of incompetent defendants (N = 15) who participated in a program that utilized videotaped segments of the television crime-drama "Law & Order", among other techniques, to promote CST in individuals deemed unfit to stand trial. The authors hypothesized that participation in at least one cycle of the Competency Restoration Group (CRG)'s curriculum would be associated with improvement in the areas of understanding, reasoning and appreciation. In order to assess whether the group was beneficial to the patient's treatment goal of competency restoration, patients were screened using the MacArthur Competence Assessment Tool-Criminal Adjudication (MacCAT-CA) prior to starting the group and after completing a cycle of the group's curriculum. The Wilcoxon signed ranks test was employed to analyze the results from the pre- and post-group MacCAT-CA testing. The tests yielded significant (p < 0.005) post-test differences in the hypothesized direction for each of the three subsections: Understanding, Reasoning, and Appreciation as well as a significant post test improvement in the total MacCAT-CA scores. These results suggest that a didactic program, using a popular crime drama series, can be effective in facilitating learning in competency restoration programs. Limitations of this study include its lack of a control group and small population.

Occupational therapists' capacity-building needs related to older driver screening, assessment, and intervention: a Canadawide survey.

PubMed

Korner-Bitensky, Nicol; Menon, Anita; von Zweck, Claudia; Van Benthem, Kathy

2010-01-01

Older driver safety is a growing concern. We identified capacity-building needs of occupational therapists related to older driver screening, assessment, and intervention. A Canadawide survey was undertaken involving 133 occupational therapists working with an older clientele. A standardized questionnaire elicited information regarding (1) actual practices related to older driver screening, assessment, and intervention; (2) perceived competence; and (3) need for continuing education. Occupational therapists were twice as likely to use screening tools rather than in-depth assessments (n = 79 vs. n = 37). Only 25 occupational therapists offered on-road assessment, and even fewer offered retraining (n = 11). Occupational therapists more often felt very competent in domains related to screening as opposed to assessment, and most were interested in continuing education. Driving services offered were primarily related to screening compared with assessment or intervention. Occupational therapists would benefit from driving-related professional training aimed at enhancing professional capacity in this arena.

The patient assessment questionnaire: a new instrument for evaluating the interpersonal skills of vocational dental practitioners.

PubMed

Hurst, Y K; Prescott-Clements, L E; Rennie, J S

2004-10-23

This paper describes a pilot study aimed at evaluating a new instrument, the patient assessment questionnaire (PAQ), which uses patient ratings for the assessment of communication skills and professionalism in vocational practitioners (VDPs). The PAQ was developed as part of an assessment system designed to address all round competence. Acohort of 99 VDPs took part in the study. Questionnaires were distributed to consecutive patients in the general dental service at two time points in the training year. Data from the pilot study was analysed to determine whether the PAQ fulfilled the criteria for robust assessment. Results provide evidence of high levels of reliability, validity and feasibility of the PAQ instrument. All indications to date suggest that the PAQ will prove to be a valuable assessment tool. It is currently being evaluated as part of the system used to assess the all round competence of dental graduates undertaking vocational training in Scotland.

Ways to Write a Milestone: Approaches to Operationalizing the Development of Competence in Graduate Medical Education.

PubMed

Leep Hunderfund, Andrea N; Reed, Darcy A; Starr, Stephanie R; Havyer, Rachel D; Lang, Tara R; Norby, Suzanne M

2017-09-01

To identify approaches to operationalizing the development of competence in Accreditation Council for Graduate Medical Education (ACGME) milestones. The authors reviewed all 25 "Milestone Project" documents available on the ACGME Web site on September 11, 2013, using an iterative process to identify approaches to operationalizing the development of competence in the milestones associated with each of 601 subcompetencies. Fifteen approaches were identified. Ten focused on attributes and activities of the learner, such as their ability to perform different, increasingly difficult tasks (304/601; 51%), perform a task better and faster (171/601; 45%), or perform a task more consistently (123/601; 20%). Two approaches focused on context, inferring competence from performing a task in increasingly difficult situations (236/601; 29%) or an expanding scope of engagement (169/601; 28%). Two used socially defined indicators of competence such as progression from "learning" to "teaching," "leading," or "role modeling" (271/601; 45%). One approach focused on the supervisor's role, inferring competence from a decreasing need for supervision or assistance (151/601; 25%). Multiple approaches were often combined within a single set of milestones (mean 3.9, SD 1.6). Initial ACGME milestones operationalize the development of competence in many ways. These findings offer insights into how physicians understand and assess the developmental progression of competence and an opportunity to consider how different approaches may affect the validity of milestone-based assessments. The results of this analysis can inform the work of educators developing or revising milestones, interpreting milestone data, or creating assessment tools to inform milestone-based performance measures.

Key factors in children's competence to consent to clinical research.

PubMed

Hein, Irma M; Troost, Pieter W; Lindeboom, Robert; Benninga, Marc A; Zwaan, C Michel; van Goudoever, Johannes B; Lindauer, Ramón J L

2015-10-24

Although law is established on a strong presumption that persons younger than a certain age are not competent to consent, statutory age limits for asking children's consent to clinical research differ widely internationally. From a clinical perspective, competence is assumed to involve many factors including the developmental stage, the influence of parents and peers, and life experience. We examined potential determining factors for children's competence to consent to clinical research and to what extent they explain the variation in competence judgments. From January 1, 2012 through January 1, 2014, pediatric patients aged 6 to 18 years, eligible for clinical research studies were enrolled prospectively at various in- and outpatient pediatric departments. Children's competence to consent was assessed by MacArthur Competence Assessment Tool for Clinical Research. Potential determining child variables included age, gender, intelligence, disease experience, ethnicity and socio-economic status (SES). We used logistic regression analysis and change in explained variance in competence judgments to quantify the contribution of a child variable to the total explained variance. Contextual factors included risk and complexity of the decision to participate, parental competence judgment and the child's or parents decision to participate. Out of 209 eligible patients, 161 were included (mean age, 10.6 years, 47.2 % male). Age, SES, intelligence, ethnicity, complexity, parental competence judgment and trial participation were univariately associated with competence (P < 0.05). Total explained variance in competence judgments was 71.5 %. Only age and intelligence significantly and independently explained the variance in competence judgments, explaining 56.6 % and 12.7 % of the total variance respectively. SES, male gender, disease experience and ethnicity each explained less than 1 % of the variance in competence judgments. Contextual factors together explained an extra 2.8 % (P > 0.05). Age is the factor that explaines most of to the variance in children's competence to consent, followed by intelligence. Experience with disease did not affect competence in this study, nor did other variables. Development and use of a standardized instrument for assessing children's competence to consent in drug trials: Are legally established age limits valid?, NTR3918.

Validation of the one pass measure for motivational interviewing competence.

PubMed

McMaster, Fiona; Resnicow, Ken

2015-04-01

This paper examines the psychometric properties of the OnePass coding system: a new, user-friendly tool for evaluating practitioner competence in motivational interviewing (MI). We provide data on reliability and validity with the current gold-standard: Motivational Interviewing Treatment Integrity tool (MITI). We compared scores from 27 videotaped MI sessions performed by student counselors trained in MI and simulated patients using both OnePass and MITI, with three different raters for each tool. Reliability was estimated using intra-class coefficients (ICCs), and validity was assessed using Pearson's r. OnePass had high levels of inter-rater reliability with 19/23 items found from substantial to almost perfect agreement. Taking the pair of scores with the highest inter-rater reliability on the MITI, the concurrent validity between the two measures ranged from moderate to high. Validity was highest for evocation, autonomy, direction and empathy. OnePass appears to have good inter-rater reliability while capturing similar dimensions of MI as the MITI. Despite the moderate concurrent validity with the MITI, the OnePass shows promise in evaluating both traditional and novel interpretations of MI. OnePass may be a useful tool for developing and improving practitioner competence in MI where access to MITI coders is limited. Copyright © 2015. Published by Elsevier Ireland Ltd.

Measurement of Clinical Performance of Nurses: A Literature Review.

ERIC Educational Resources Information Center

Robb, Yvonne; Fleming, Valerie; Dietert, Christine

2002-01-01

A research review (n=12) yielded a number of tools for assessing nurses' clinical competence but none that is universally accepted. The review did identify methods that could be used to develop a useful instrument. (Contains 23 references.) (SK)

The Entrepreneurial Mind.

ERIC Educational Resources Information Center

Timmons, Jeffry A.

Focusing on the knowledge and tools necessary for starting new entrepreneurial ventures, this book draws on information from the venture capital industry to describe the characteristics of entrepreneurs and entrepreneurial managers and includes self-assessment and management competency inventories. Following an introduction describing the elements…

Reliability and Validity of the Turkish Version of the Moral Competence Scale for Public Health Nurses: A Methodological Study.

PubMed

Yildiz, Esra; Güdücü Tüfekci, Fatma

Moral competencies must be improved in nursing area practice. To evaluate the moral competence seems necessary for nurses. The aims of this study are to adapt and evaluate the psychometric properties of the moral competence questionnaire for public health nurses in Turkey. The moral competence scale was translated into Turkish by a skilled translator, after which it was back-translated into English by another translator. We then administered the Turkish version of the moral competence scale to 138 public health nurses working in family and public health centers in Erzurum, a city in eastern Turkey. We analyzed the data using factor analysis and Cronbach's α. Three factors were extracted, which together explained a total of 67.50% of the variance. The Cronbach's α values were .83, .91, .87, and .88 for factors 1, 2, and 3 and for the whole scale, respectively. The Turkish version of the moral competence scale for public health nurses is a valid and reliable assessment tool.

Creating More Effective Mentors: Mentoring the Mentor.

PubMed

Gandhi, Monica; Johnson, Mallory

2016-09-01

Given the diversity of those affected by HIV, increasing diversity in the HIV biomedical research workforce is imperative. A growing body of empirical and experimental evidence supports the importance of strong mentorship in the development and success of trainees and early career investigators in academic research settings, especially for mentees of diversity. Often missing from this discussion is the need for robust mentoring training programs to ensure that mentors are trained in best practices on the tools and techniques of mentoring. Recent experimental evidence shows improvement in mentor and mentee perceptions of mentor competency after structured and formalized training on best practices in mentoring. We developed a 2-day "Mentoring the Mentors" workshop at UCSF to train mid-level and senior HIV researchers from around the country [recruited mainly from Centers for AIDS Research (CFARs)] on best practices, tools and techniques of effective mentoring. The workshop content was designed using principles of Social Cognitive Career Theory (SCCT) and included training specifically geared towards working with early career investigators from underrepresented groups, including sessions on unconscious bias, microaggressions, and diversity supplements. The workshop has been held three times (September 2012, October 2013 and May 2015) with plans for annual training. Mentoring competency was measured using a validated tool before and after each workshop. Mentoring competency skills in six domains of mentoring-specifically effective communication, aligning expectations, assessing understanding, fostering independence, addressing diversity and promoting development-all improved as assessed by a validated measurement tool for participants pre- and -post the "Mentoring the Mentors" training workshops. Qualitative assessments indicated a greater awareness of the micro-insults and unconscious bias experienced by mentees of diversity and a commitment to improve awareness and mitigate these effects via the mentor-mentee relationship. Our "Mentoring the Mentors" workshop for HIV researchers/mentors offers a formal and structured curriculum on best practices, tools and techniques of effective mentoring, and methods to mitigate unconscious bias in the mentoring relationship. We found quantitative and qualitative improvements in mentoring skills as assessed by self-report by participants after each workshop and plan additional programs with longitudinal longer-term assessments focused on objective mentee outcomes (grants, papers, academic retention). Mentoring training can improve mentoring skills and is likely to improve outcomes for optimally-mentored mentees.

Creating more effective mentors: Mentoring the mentor

PubMed Central

Gandhi, Monica; Johnson, Mallory

2016-01-01

Introduction Given the diversity of those affected by HIV, increasing diversity in the HIV biomedical research workforce is imperative. A growing body of empirical and experimental evidence supports the importance of strong mentorship in the development and success of trainees and early career investigators in academic research settings, especially for mentees of diversity. Often missing from this discussion is the need for robust mentoring training programs to ensure that mentors are trained in best practices on the tools and techniques of mentoring. Recent experimental evidence shows improvement in mentor and mentee perceptions of mentor’s competency after structured and formalized training on best practices in mentoring. Methods We developed a 2-day “Mentoring the Mentors” workshop at UCSF to train mid-level and senior HIV researchers from around the country (recruited mainly from Centers for AIDS Research (CFARs)) on best practices, tools and techniques of effective mentoring. The workshop content was designed using principles of Social Cognitive Career Theory (SCCT) and included training specific to working with early career investigators from underrepresented groups, including training on unconscious bias, microaggressions, and diversity supplements. The workshop has been held 3 times (September 2012, October 2013 and May 2015) with plans for annual training. Mentoring competency was measured using a validated tool before and after each workshop. Results Mentoring competency skills in six domains of mentoring -specifically effective communication, aligning expectations, assessing understanding, fostering independence, addressing diversity and promoting development - all improved as assessed by a validated measurement tool for participants pre- and-post the “Mentoring the Mentors” training workshops. Qualitative assessments indicated a greater awareness of the micro-insults and unconscious bias experienced by mentees of diversity and a commitment to improve awareness and mitigate these effects via the mentor-mentee relationship. Discussion Our “Mentoring the Mentors” workshop for HIV researchers/mentors offers a formal and structured curriculum on best practices, tools and techniques of effective mentoring, and methods to mitigate unconscious bias in the mentoring relationship and at the institutional level with mentees of diversity. We found quantitative and qualitative improvements in mentoring skills as assessed by self-report by participants after each workshop and plan additional programs with longitudinal longer-term assessments focused on objective mentee outcomes (grants, papers, academic retention). Mentoring training can improve mentoring skills and are likely to improve outcomes for optimally-mentored mentees. PMID:27039092

The assessment of pathologists/laboratory medicine physicians through a multisource feedback tool.

PubMed

Lockyer, Jocelyn M; Violato, Claudio; Fidler, Herta; Alakija, Pauline

2009-08-01

There is increasing interest in ensuring that physicians demonstrate the full range of Accreditation Council for Graduate Medical Education competencies. To determine whether it is possible to develop a feasible and reliable multisource feedback instrument for pathologists and laboratory medicine physicians. Surveys with 39, 30, and 22 items were developed to assess individual physicians by 8 peers, 8 referring physicians, and 8 coworkers (eg, technologists, secretaries), respectively, using 5-point scales and an unable-to-assess category. Physicians completed a self-assessment survey. Items addressed key competencies related to clinical competence, collaboration, professionalism, and communication. Data from 101 pathologists and laboratory medicine physicians were analyzed. The mean number of respondents per physician was 7.6, 7.4, and 7.6 for peers, referring physicians, and coworkers, respectively. The reliability of the internal consistency, measured by Cronbach alpha, was > or = .95 for the full scale of all instruments. Analysis indicated that the medical peer, referring physician, and coworker instruments achieved a generalizability coefficient of .78, .81, and .81, respectively. Factor analysis showed 4 factors on the peer questionnaire accounted for 68.8% of the total variance: reports and clinical competency, collaboration, educational leadership, and professional behavior. For the referring physician survey, 3 factors accounted for 66.9% of the variance: professionalism, reports, and clinical competency. Two factors on the coworker questionnaire accounted for 59.9% of the total variance: communication and professionalism. It is feasible to assess this group of physicians using multisource feedback with instruments that are reliable.

Educating and Training the Future Adolescent Health Workforce.

PubMed

Kokotailo, Patricia K; Baltag, Valentina; Sawyer, Susan M

2018-05-01

Unprecedented attention is now focused on adolescents with growing appreciation of their disease burden and of the opportunities of investing in adolescent health. New investments are required to build the technical capacity for policy, programming, research, and clinical care across the world, especially in resource-poor settings where most adolescents live. Strategies to educate and train the future workforce are needed. Competency-based education and training is the standard of education in preservice (undergraduate and postgraduate) health education and medical specialty training. Yet competency is difficult to quantify and standardize, as are the processes that underpin competency-based education and training. The primary objective of this review was to identify how quality education in adolescent health and medicine is determined. This information was used to inform the development of a conceptual framework for institutions teaching adolescent health, which can be used to assess the quality of teaching and learning and to monitor the implementation of these adolescent health competencies. Specific teaching modalities and assessment tools that have been used to teach adolescent health are described to exemplify how an educational program can be delivered and assessed. This framework is a step toward the development of a more adolescent-competent health workforce. Copyright © 2017 The Society for Adolescent Health and Medicine. Published by Elsevier Inc. All rights reserved.

Information Literacy: Instrument Development to Measure Competencies and Knowledge Among Nursing Educators, Nursing Administrators, and Nursing Clinicians: A Pilot Study

PubMed Central

Pierce, Susan; Pravikoff, Diane; Tanner, Annelle

2003-01-01

This poster describes a pilot study conducted to establish validity and reliability of an instrument that will be used in a nationwide needs assessment, implemented to identify gaps in Information Literacy skills, competencies, and knowledge among key nursing groups nationally. Data and information gathered using the tool will guide the profession in developing appropriate education and continuing education programs to close identified gaps and enhance nurses’ readiness for Evidence-Based Practice (EBP). PMID:14728475

Social competence: evaluation of assertiveness in Spanish adolescents.

PubMed

Castedo, Antonio López; Juste, Margarita Pino; Alonso, José Domínguez

2015-02-01

Relations between assertiveness in adolescents' social behavior and demographic variables were assessed in 4,943 Spanish adolescents, ages 12 to 17 years, enrolled in 32 schools for Compulsory Secondary Education. Province of residence, school size, age, grade, and academic focus were statistically significant sources of variance in assertiveness scores. All effects were small. Patterns in responses indicate the items should be reviewed to improve the measure for adolescents, and as a tool for addressing teens' social competence in real life situations.

Competencies in Organizational E-Learning: Concepts and Tools

ERIC Educational Resources Information Center

Sicilia, Miguel-Angel, Ed.

2007-01-01

"Competencies in Organizational E-Learning: Concepts and Tools" provides a comprehensive view of the way competencies can be used to drive organizational e-learning, including the main conceptual elements, competency gap analysis, advanced related computing topics, the application of semantic Web technologies, and the integration of competencies…

ED leadership competency matrix: an administrative management tool.

PubMed

Propp, Douglas A; Glickman, Seth; Uehara, Dennis T

2003-10-01

A successful ED relies on its leaders to master and demonstrate core competencies to be effective in the many arenas in which they interact and are responsible. A unique matrix model for the assessment of an ED leadership's key administrative skill sets is presented. The model incorporates capabilities related to the individual's cognitive aptitude, experience, acquired technical skills, behavioral characteristics, as well as the ability to manage relationships effectively. Based on the personnel inventory using the matrix, focused evaluation, development, and recruitment of ED key leaders occurs. This dynamic tool has provided a unique perspective for the evaluation and enhancement of overall ED leadership performance. It is hoped that incorporation of such a model will similarly improve the accomplishments of EDs at other institutions.

Significance of Objective Structured Clinical Examinations to Plastic Surgery Residency Training.

PubMed

Simmons, Brian J; Zoghbi, Yasmina; Askari, Morad; Birnbach, David J; Shekhter, Ilya; Thaller, Seth R

2017-09-01

Objective structured clinical examinations (OSCEs) have proven to be a powerful tool. They possess more than a 30-year track record in assessing the competency of medical students, residents, and fellows. Objective structured clinical examinations have been used successfully in a variety of medical specialties, including surgery. They have recently found their way into the subspecialty of plastic surgery. This article uses a systematic review of the available literature on OSCEs and their recent use in plastic surgery. It incorporates survey results assessing program directors' views on the use of OSCEs. Approximately 40% of programs surveyed use OSCEs to assess the Accreditation Council for Graduate Medical Education core competencies. We found that 40% use OSCEs to evaluate specific plastic surgery milestones. Objective structured clinical examinations are usually performed annually. They cost anywhere between $100 and more than $1000 per resident. Four milestones giving residents the most difficulties on OSCEs were congenital anomalies, noncancer breast surgery, breast reconstruction, and practice-based learning and improvement. It was determined that challenges with milestones were due to lack of adequate general knowledge and surgical ward patient care, as well as deficits in professionalism and system-based problems. Programs were able to remediate weakness found by OSCEs using a variety of methods. Objective structured clinical examinations offer a unique tool to objectively assess the proficiency of residents in key areas of the Accreditation Council for Graduate Medical Education core competencies. In addition, they can be used to assess the specific milestones that plastic surgery residents must meet. This allows programs to identify and improve identified areas of weakness.

Assessing Photosynthesis by Fluorescence Imaging

ERIC Educational Resources Information Center

Saura, Pedro; Quiles, Maria Jose

2011-01-01

This practical paper describes a novel fluorescence imaging experiment to study the three processes of photochemistry, fluorescence and thermal energy dissipation, which compete during the dissipation of excitation energy in photosynthesis. The technique represents a non-invasive tool for revealing and understanding the spatial heterogeneity in…

Effects of state organizational structure and forensic examiner training on pretrial competence assessments.

PubMed

Edens, J F; Poythress, N G; Nicholson, R A; Otto, R K

1999-05-01

States differ widely in their delivery of pretrial forensic evaluation services, in terms of organizational structure and training requirements of forensic examiners. It was hypothesized that defendants adjudicated incompetent to proceed in states using community-based, private-practitioner systems would show less impairment on a competence assessment measure, the MacArthur Competence Assessment Tool-Criminal Adjudication (MacCAT-CA), than defendants adjudicated incompetent in states using traditional, inpatient systems. It also was hypothesized that mean MacCAT-CA scores for incompetent defendants from states requiring forensic training/certification would be lower than for defendants from states lacking such requirements. Results indicated significant differences across the four types of service delivery systems examined. However, planned comparisons revealed no differences between a state using a traditional, inpatient model and a state employing a community-based, private-practitioner model. Analyses examining the effects of mandatory forensic training failed to support the hypothesis that training requirements result in the adoption of higher thresholds for determining incompetence.

Development and use of mathematical models and software frameworks for integrated analysis of agricultural systems and associated water use impacts

USGS Publications Warehouse

Fowler, K. R.; Jenkins, E.W.; Parno, M.; Chrispell, J.C.; Colón, A. I.; Hanson, Randall T.

2016-01-01

The development of appropriate water management strategies requires, in part, a methodology for quantifying and evaluating the impact of water policy decisions on regional stakeholders. In this work, we describe the framework we are developing to enhance the body of resources available to policy makers, farmers, and other community members in their e orts to understand, quantify, and assess the often competing objectives water consumers have with respect to usage. The foundation for the framework is the construction of a simulation-based optimization software tool using two existing software packages. In particular, we couple a robust optimization software suite (DAKOTA) with the USGS MF-OWHM water management simulation tool to provide a flexible software environment that will enable the evaluation of one or multiple (possibly competing) user-defined (or stakeholder) objectives. We introduce the individual software components and outline the communication strategy we defined for the coupled development. We present numerical results for case studies related to crop portfolio management with several defined objectives. The objectives are not optimally satisfied for any single user class, demonstrating the capability of the software tool to aid in the evaluation of a variety of competing interests.

An exploratory study of an assessment tool derived from the competencies of the interprofessional education collaborative.

PubMed

Dow, Alan W; DiazGranados, Deborah; Mazmanian, Paul E; Retchin, Sheldon M

2014-07-01

Linking the outcomes from interprofessional education to improvements in patient care has been hampered by educational assessments that primarily measure the short-term benefits of specific curricular interventions. Competencies, recently published by the Interprofessional Education Collaborative (IPEC), elaborate overarching goals for interprofessional education by specifying desired outcomes for graduating health professions students. The competencies define a transition point between the prescribed and structured educational experience of a professional degree program and the more self-directed, patient-oriented learning associated with professional practice. Drawing on the IPEC competencies for validity, we created a 42-item questionnaire to assess outcomes related to collaborative practice at the degree program level. To establish the usability and psychometric properties of the questionnaire, it was administered to all the students on a health science campus at a large urban university in the mid-Atlantic of the United States. The student responses (n = 481) defined four components aligned in part with the four domains of the IPEC competencies. In addition, the results demonstrated differences in scores by domain that can be used to structure future curricula. These findings suggest a questionnaire based on the IPEC competencies might provide a measure to assess programmatic outcomes related to interprofessional education. We discuss directions for future research, such as a comparison of results within and between institutions, and how these results could provide valuable insights about the effect of different curricular approaches to interprofessional education and the success of various educational programs at preparing students for collaborative practice.

AN EXPLORATORY STUDY OF AN ASSESSMENT TOOL DERIVED FROM THE COMPETENCIES OF THE INTERPROFESSIONAL EDUCATION COLLABORATIVE

PubMed Central

Dow, Alan W.; DiazGranados, Deborah; Mazmanian, Paul E.; Retchin, Sheldon M.

2015-01-01

Linking the outcomes from interprofessional education to improvements in patient care has been hampered by educational assessments that primarily measure the short-term benefits of specific curricular interventions. Competencies, recently published by the Interprofessional Education Collaborative (IPEC), elaborate overarching goals for interprofessional education by specifying desired outcomes for graduating health professions students. The competencies define a transition point between the prescribed and structured educational experience of a professional degree program and the more self-directed, patient-oriented learning associated with professional practice. Drawing on the IPEC competencies for validity, we created a 42-item questionnaire to assess outcomes related to collaborative practice at the degree program level. To establish the usability and psychometric properties of the questionnaire, it was administered to all the students on a health science campus at a large urban university in the mid-Atlantic of the United States. The student responses (n = 481) defined four components aligned in part with the four domains of the IPEC competencies. In addition, the results demonstrated differences in scores by domain that can be used to structure future curricula. These findings suggest a questionnaire based on the IPEC competencies might provide a measure to assess programmatic outcomes related to interprofessional education. We discuss directions for future research, such as a comparison of results within and between institutions, and how these results could provide valuable insights about the effect of different curricular approaches to interprofessional education and the success of various educational programs at preparing students for collaborative practice. PMID:24593327

Exploring Design Features to Enhance Computer-Based Assessment: Learners' Views on Using a Confidence-Indicator Tool and Computer-Based Feedback

ERIC Educational Resources Information Center

Nix, Ingrid; Wyllie, Ali

2011-01-01

Many institutions encourage formative computer-based assessment (CBA), yet competing priorities mean that learners are necessarily selective about what they engage in. So how can we motivate them to engage? Can we facilitate learners to take more control of shaping their learning experience? To explore this, the Learning with Interactive…

A Self Rating Scale as a Pre and Post Assessment Tool for Use with Instructional Modules.

ERIC Educational Resources Information Center

Gotts, Sandra Harris

This article describes a self rating pre- and post-assessment instrument that has been developed at the Central Michigan University (CMU). Nine instructional modules have been developed and are being used in science methods courses at CMU. Each module focuses on an identified area of competency for elementary science teachers and contains a…

The Systems SOAP Note: A Systems Learning Tool.

PubMed

Mitsuishi, Fumi; Young, John Q; Leary, Mark; Dilley, James; Mangurian, Christina

2016-02-01

Systems-based practice (SBP) is the only Accreditation Council for Graduate Medical Education (ACGME) competency concerned with public health and is relatively neglected in residency curricula. A tool was developed and pilot-tested to improve SBP learning on inpatient psychiatry rotations. A four-step approach was used: (1) literature review, (2) expert consultation, (3) tool development, and (4) pilot testing on four cases and evaluation for completion time and preliminary efficacy. Out of 51 SBP articles, six (12%) focused on psychiatric residency programs, and none had a practical SBP learning tool. The "systems SOAP (subjective, objective, assessment, plan) note" (S-SOAP) was structured after a clinical SOAP note and was easy to use (mean completion time = 60 min), and residents self-reported more insight into systems issues. The S-SOAP tool was effectively integrated into clinical experience and provided insight into systemic complexities. Future research should assess SBP knowledge acquisition after the use of such tools.

Impact of Student vs Faculty Facilitators on Motivational Interviewing Student Outcomes

PubMed Central

Widder-Prewett, Rebecca; Cameron, Ginger; Anderson, Douglas; Pinkerton, Mark; Chen, Aleda M. H.

2017-01-01

Objective. To determine the impact of student or faculty facilitation on student self-assessed attitudes, confidence, and competence in motivational interviewing (MI) skills; actual competence; and evaluation of facilitator performance. Methods. Second-year pharmacy (P2) students were randomly assigned to a student or faculty facilitator for a four-hour, small-group practice of MI skills. MI skills were assessed in a simulated patient encounter with the mMITI (modified Motivational Interviewing Treatment Integrity) tool. Students completed a pre-post, 6-point, Likert-type assessment addressing the research objectives. Differences were assessed using a Mann-Whitney U test. Results. Student (N=44) post-test attitudes, confidence, perceived or actual competence, and evaluations of facilitator performance were not different for faculty- and student-facilitated groups. Conclusion. Using pharmacy students as small-group facilitators did not affect student performance and were viewed as equally favorable. Using pharmacy students as facilitators can lessen faculty workload and provide an outlet for students to develop communication and facilitation skills that will be needed in future practice. PMID:28970608

Toolbox of assessment tools of technical skills in otolaryngology-head and neck surgery: A systematic review.

PubMed

Labbé, Mathilde; Young, Meredith; Nguyen, Lily H P

2017-10-08

To support the development of programs of assessment of technical skills in the operating room (OR), we systematically reviewed the literature to identify assessment tools specific to otolaryngology-head and neck surgery (OTL-HNS) core procedures and summarized their characteristics. We systematically searched Embase, MEDLINE, PubMed, and Cochrane to identify and report on assessment tools that can be used to assess residents' technical surgical skills in the operating room for OTL-HNS core procedures. Of the 736 unique titles retrieved, 16 articles met inclusion criteria, covering 11 different procedures (in otology, rhinology, laryngology, head and neck, and general otolaryngology). The tools were composed of a task-specific checklist and/or global rating scale and were developed in the OR, on human cadavers, or in a simulation setting. Our study reports on published tools for assessing technical skills for OTL-HNS residents during core procedures conducted in the OR. These assessment tools could facilitate the provision of timely feedback to trainees including specific goals for improvement. However, the paucity of publications suggests little agreement on how to best perform work-based direct-observation assessment for core surgical procedures in OTL-HNS. The sparsity of tools specific to OTL-HNS may become a barrier to a fluid transition to competency-based medical education. Laryngoscope, 2017. © 2017 The American Laryngological, Rhinological and Otological Society, Inc.

Competency-based assessment in surgeon-performed head and neck ultrasonography: A validity study.

PubMed

Todsen, Tobias; Melchiors, Jacob; Charabi, Birgitte; Henriksen, Birthe; Ringsted, Charlotte; Konge, Lars; von Buchwald, Christian

2018-06-01

Head and neck ultrasonography (HNUS) increasingly is used as a point-of-care diagnostic tool by otolaryngologists. However, ultrasonography (US) is a very operator-dependent image modality. Hence, this study aimed to explore the diagnostic accuracy of surgeon-performed HNUS and to establish validity evidence for an objective structured assessment of ultrasound skills (OSAUS) used for competency-based assessment. A prospective experimental study. Six otolaryngologists and 11 US novices were included in a standardized test setup for which they had to perform focused HNUS of eight patients suspected for different head and neck lesions. Their diagnostic accuracy was calculated based on the US reports, and two blinded raters assessed the video-recorded US performance using the OSAUS scale. The otolaryngologists obtained a high diagnostic accuracy on 88% (range 63%-100%) compared to the US novices on 38% (range 0-63%); P < 0.001. The OSAUS score demonstrated good inter-case reliability (0.85) and inter-rater reliability (0.76), and significant discrimination between otolaryngologist and US novices; P < 0.001. A strong correlation between the OSAUS score and the diagnostic accuracy was found (Spearman's ρ, 0.85; P < P 0.001), and a pass/fail score was established at 2.8. Strong validity evidence supported the use of the OSAUS scale to assess HNUS competence with good reliability, significant discrimination between US competence levels, and a strong correlation of assessment score to diagnostic accuracy. An OSAUS pass/fail score was established and could be used for competence-based assessment in surgeon-performed HNUS. NA. Laryngoscope, 128:1346-1352, 2018. © 2017 The American Laryngological, Rhinological and Otological Society, Inc.

Relationships between academic performance of medical students and their workplace performance as junior doctors.

PubMed

Carr, Sandra E; Celenza, Antonio; Puddey, Ian B; Lake, Fiona

2014-07-30

Little recent published evidence explores the relationship between academic performance in medical school and performance as a junior doctor. Although many forms of assessment are used to demonstrate a medical student's knowledge or competence, these measures may not reliably predict performance in clinical practice following graduation. This descriptive cohort study explores the relationship between academic performance of medical students and workplace performance as junior doctors, including the influence of age, gender, ethnicity, clinical attachment, assessment type and summary score measures (grade point average) on performance in the workplace as measured by the Junior Doctor Assessment Tool. There were two hundred participants. There were significant correlations between performance as a Junior Doctor (combined overall score) and the grade point average (r = 0.229, P = 0.002), the score from the Year 6 Emergency Medicine attachment (r = 0.361, P < 0.001) and the Written Examination in Year 6 (r = 0.178, P = 0.014). There was no significant effect of any individual method of assessment in medical school, gender or ethnicity on the overall combined score of performance of the junior doctor. Performance on integrated assessments from medical school is correlated to performance as a practicing physician as measured by the Junior Doctor Assessment Tool. These findings support the value of combining undergraduate assessment scores to assess competence and predict future performance.

Motivational Interviewing Workshop in a Virtual World: Learning as Avatars

PubMed Central

Shershneva, Marianna; Kim, Ji-Hye; Kear, Cynthia; Heyden, Robin; Heyden, Neil; Lee, Jay; Mitchell, Suzanne

2015-01-01

Background Limited research has been done to understand outcomes of continuing medical education offered in three-dimensional, immersive virtual worlds. Objectives We studied a case of a virtual world workshop on motivational interviewing (MI) applied to smoking cessation counseling and its educational impact. Methods To facilitate content development and evaluation, we specified desired MI competencies. The workshop consisted of three sessions, which included lectures, practice with standardized patients, and chat interactions. Data were collected from 13 primary care physicians and residents through workshop observation, and pre- and three-month post-workshop telephone/Skype interviews and interactions with standardized patients. Interactions with standardized patients were assessed by an expert using a validated MI tool, and by standardized patients using a tool developed for this study. For 11 participants who attended two or three sessions, we conducted paired-samples t-tests comparing mean differences between the competency scores pre- and post-event. Results Expert assessment showed significant improvement on six of seven MI competencies (p< .05). All participants reported learning new knowledge and skills, and nine described incorporating new learning into their clinical practice. Practicing MI with standardized patients and/or observing others' practice appeared to be the most helpful workshop component. Conclusions The evaluated workshop had positive impact on participants' competencies and practice as related to MI applied to smoking cessation counseling. Our findings support further exploration of three-dimensional virtual worlds as learning environments for continuing medical education. PMID:24788420

Informatics Approach to Improving Surgical Skills Training

ERIC Educational Resources Information Center

Islam, Gazi

2013-01-01

Surgery as a profession requires significant training to improve both clinical decision making and psychomotor proficiency. In the medical knowledge domain, tools have been developed, validated, and accepted for evaluation of surgeons' competencies. However, assessment of the psychomotor skills still relies on the Halstedian model of…

Cultural Competency Training in Emergency Medicine.

PubMed

Mechanic, Oren J; Dubosh, Nicole M; Rosen, Carlo L; Landry, Alden M

2017-09-01

The Emergency Department is widely regarded as the epicenter of medical care for diverse and largely disparate types of patients. Physicians must be aware of the cultural diversity of their patient population to appropriately address their medical needs. A better understanding of residency preparedness in cultural competency can lead to better training opportunities and patient care. The objective of this study was to assess residency and faculty exposure to formal cultural competency programs and assess future needs for diversity education. A short survey was sent to all 168 Accreditation Council for Graduate Medical Education program directors through the Council of Emergency Medicine Residency Directors listserv. The survey included drop-down options in addition to open-ended input. Descriptive and bivariate analyses were used to analyze data. The response rate was 43.5% (73/168). Of the 68.5% (50/73) of residency programs that include cultural competency education, 90% (45/50) utilized structured didactics. Of these programs, 86.0% (43/50) included race and ethnicity education, whereas only 40.0% (20/50) included education on patients with limited English proficiency. Resident comfort with cultural competency was unmeasured by most programs (83.6%: 61/73). Of all respondents, 93.2% (68/73) were interested in a universal open-source cultural competency curriculum. The majority of the programs in our sample have formal resident didactics on cultural competency. Some faculty members also receive cultural competency training. There are gaps, however, in types of cultural competency training, and many programs have expressed interest in a universal open-source tool to improve cultural competency for Emergency Medicine residents. Copyright © 2017 Elsevier Inc. All rights reserved.

Assessing resident performance and training of colonoscopy in a general surgery training program.

PubMed

Hope, William W; Hooks, W Borden; Kilbourne, S Nicole; Adams, Ashley; Kotwall, Cyrus A; Clancy, Thomas V

2013-05-01

Recently, the adequacy of endoscopy training in general surgery residency programs has been questioned. Efforts to improve resident endoscopic training and to judge competency are ongoing but not well studied. We assessed resident performance using two assessment tools in colonoscopy in a general surgery residency program. Prospectively collected data were reviewed from consecutive colonoscopies by a single surgeon: September 2008 to June 2011. Colonoscopies performed without residents were excluded. Data included patient demographics, procedural data, and outcomes. Following the colonoscopy, residents were graded by the attending surgeon using up two different assessment tools. Descriptive statistics were calculated and outcomes were compared. Colonoscopies were performed by residents in 100 patients. Average age was 52 (range, 22-79) years. Females made up 66 % of patients, and 63 % were Caucasian. Postgraduate level (PG-Y) 3 level residents performed 72 % of colonoscopies. The average resident participation was 73 % of the procedure. Biopsies were performed in 35 %; adenomatous polyps were found in 17 % and invasive cancer in 1 %. Bowel preparation was deemed good in 76 % of patients. Colonoscopy was completed in 90 % of patients. Reasons for incomplete exam were technical (7 patients), inability to pass a stricture (2 patients), and poor prep (1 patient). For completed full colonoscopies, the average time to reach the cecum was 22 min, and withdrawal time was 13 min. Resident assessments were made in 89 of the colonoscopies using 2 separate assessment tools. There were no mortalities; the morbidity rate was 3 %. Morbidities included a perforation related to a biopsy requiring surgery and partial colectomy, a postpolypectomy bleed requiring repeat colonoscopy with clipping of the bleeding vessel, and a patient with transient bradycardia requiring atropine during the procedure. Using objective assessment tools, overall resident skill and knowledge in performing colonoscopy appears to improve based on increasing PG-Y level, although this was not evident with all categories measured. Methods to assess competency continue to evolve and should be the focus of future research.

The application of flow competence evaluations to the assessment of flood-flow velocities and stresses

NASA Technical Reports Server (NTRS)

Komar, Paul D.

1987-01-01

The concept of flow competence is generally employed to evaluate the velocities, discharges, and bottom stresses of river floods inferred from the size of the largest sediment particles transported. Flow competence has become an important tool for evaluating the hydraulics of exceptional floods on Earth, including those which eroded the Channeled Scabland of eastern Washington, and has potential for similar evaluations of the floods which carved the outflow channels on Mars. For the most part, flow-competence evaluations were empirical, based on data compiled from a variety of sources including major terrestrial floods caused by natural processes or dam failures. Such flow-competence relationships would appear to provide a straight-forward assessment of flood-flow stresses and velocities based on the maximum size of gravel and boulders transported. However, a re-examination of the data base and comparisons with measurements of selective entrainment and transport of gravel in rivers open to question such evaluations. Analyses of the forces acting on the grain during entrainment by pivoting, rolling, or sliding, an approach which focuses more on the physical processes than the purely empirical relationships can be demonstrated. These derived equations require further testing by flume and field measurements before being applied to flow-competence evaluations. Such tests are now underway.

Rater Training to Support High-Stakes Simulation-Based Assessments

PubMed Central

Feldman, Moshe; Lazzara, Elizabeth H.; Vanderbilt, Allison A.; DiazGranados, Deborah

2013-01-01

Competency-based assessment and an emphasis on obtaining higher-level outcomes that reflect physicians’ ability to demonstrate their skills has created a need for more advanced assessment practices. Simulation-based assessments provide medical education planners with tools to better evaluate the 6 Accreditation Council for Graduate Medical Education (ACGME) and American Board of Medical Specialties (ABMS) core competencies by affording physicians opportunities to demonstrate their skills within a standardized and replicable testing environment, thus filling a gap in the current state of assessment for regulating the practice of medicine. Observational performance assessments derived from simulated clinical tasks and scenarios enable stronger inferences about the skill level a physician may possess, but also introduce the potential of rater errors into the assessment process. This article reviews the use of simulation-based assessments for certification, credentialing, initial licensure, and relicensing decisions and describes rater training strategies that may be used to reduce rater errors, increase rating accuracy, and enhance the validity of simulation-based observational performance assessments. PMID:23280532

Cognitive skills assessment during robot-assisted surgery: separating the wheat from the chaff.

PubMed

Guru, Khurshid A; Esfahani, Ehsan T; Raza, Syed J; Bhat, Rohit; Wang, Katy; Hammond, Yana; Wilding, Gregory; Peabody, James O; Chowriappa, Ashirwad J

2015-01-01

To investigate the utility of cognitive assessment during robot-assisted surgery (RAS) to define skills in terms of cognitive engagement, mental workload, and mental state; while objectively differentiating between novice and expert surgeons. In all, 10 surgeons with varying operative experience were assigned to beginner (BG), combined competent and proficient (CPG), and expert (EG) groups based on the Dreyfus model. The participants performed tasks for basic, intermediate and advanced skills on the da Vinci Surgical System. Participant performance was assessed using both tool-based and cognitive metrics. Tool-based metrics showed significant differences between the BG vs CPG and the BG vs EG, in basic skills. While performing intermediate skills, there were significant differences only on the instrument-to-instrument collisions between the BG vs CPG (2.0 vs 0.2, P = 0.028), and the BG vs EG (2.0 vs 0.1, P = 0.018). There were no significant differences between the CPG and EG for both basic and intermediate skills. However, using cognitive metrics, there were significant differences between all groups for the basic and intermediate skills. In advanced skills, there were no significant differences between the CPG and the EG except time (1116 vs 599.6 s), using tool-based metrics. However, cognitive metrics revealed significant differences between both groups. Cognitive assessment of surgeons may aid in defining levels of expertise performing complex surgical tasks once competence is achieved. Cognitive assessment may be used as an adjunct to the traditional methods for skill assessment during RAS. © 2014 The Authors. BJU International © 2014 BJU International.

Internal Medicine Residency Program Directors' Views of the Core Entrustable Professional Activities for Entering Residency: An Opportunity to Enhance Communication of Competency Along the Continuum.

PubMed

Angus, Steven V; Vu, T Robert; Willett, Lisa L; Call, Stephanie; Halvorsen, Andrew J; Chaudhry, Saima

2017-06-01

To examine internal medicine (IM) residency program directors' (PDs') perspectives on the Core Entrustable Professional Activities for Entering Residency (Core EPAs)-introduced into undergraduate medical education to further competency-based assessment-and on communicating competency-based information during transitions. A spring 2015 Association of Program Directors in Internal Medicine survey asked PDs of U.S. IM residency programs for their perspectives on which Core EPAs new interns must or should possess on day 1, which are most essential, and which have the largest gap between expected and observed performance. Their views and preferences were also requested regarding communicating competency-based information at transitions from medical school to residency and residency to fellowship/employment. The response rate was 57% (204/361 programs). The majority of PDs felt new interns must/should possess 12 of the 13 Core EPAs. PDs' rankings of Core EPAs by relative importance were more varied than their rankings by the largest gaps in performance. Although preferred timing varied, most PDs (82%) considered it important for medical schools to communicate Core EPA-based information to PDs; nearly three-quarters (71%) would prefer a checklist format. Many (60%) would be willing to provide competency-based evaluations to fellowship directors/employers. Most (> 80%) agreed that there should be a bidirectional communication mechanism for programs/employers to provide feedback on competency assessments. The gaps identified in Core EPA performance may help guide medical schools' curricular and assessment tool design. Sharing competency-based information at transitions along the medical education continuum could help ensure production of competent, practice-ready physicians.

Moving Towards Culturally Competent Health Systems: Organizational and Market Factors

PubMed Central

Weech-Maldonado, Robert; Elliott, Marc; Pradhan, Rohit; Schiller, Cameron; Dreachslin, Janice; Hays, Ron D.

2012-01-01

Cultural competency has been proposed as an organizational strategy to address racial/ethnic disparities in the health care system; disparities are a long-standing policy challenge whose relevance is only increasing with the increasing population diversity of the US and across the world. Using an integrative conceptual framework based on the resource dependency and institutional theories, we examine the relationship between organizational and market factors and hospitals’ degree of cultural competency. Our sample consists of 119 hospitals located in the state of California (US) and is constructed using the following datasets for the year 2006: Cultural Competency Assessment Tool of Hospitals (CCATH) Survey, California’s Office of Statewide Health Planning & Development’s Hospital Inpatient Discharges and Annual Hospital Financial Data, American Hospital Association’s Annual Survey, and the Area Resource File. The dependent variable consists of the degree of hospital cultural competency, as assessed by the CCATH overall score. Organizational variables include ownership status, teaching hospital, payer mix, size, system membership, financial performance, and the proportion of inpatient racial/ethnic minorities. Market characteristics included hospital competition, the proportion of racial/ethnic minorities in the area, metropolitan area, and per capita income. Regression analyses were conducted to assess the relationship between the CCATH overall score and organizational and market variables. Our results show that hospitals which are not-for-profit, serve a more diverse inpatient population, and are located in more competitive and affluent markets exhibit a higher degree of cultural competency. Our results underscore the importance of both institutional and competitive market pressures in guiding hospital behavior. For instance, while not-for-profit may adopt innovative/progressive policies like cultural competency simply as a function of their organizational goals, linking cultural competency with organizational performance may be essential to attract more profit driven hospitals. PMID:22647564

Moving towards culturally competent health systems: organizational and market factors.

PubMed

Weech-Maldonado, Robert; Elliott, Marc N; Pradhan, Rohit; Schiller, Cameron; Dreachslin, Janice; Hays, Ron D

2012-09-01

Cultural competency has been proposed as an organizational strategy to address racial/ethnic disparities in the healthcare system; disparities are a long-standing policy challenge whose relevance is only increasing with the increasing population diversity of the US and across the world. Using an integrative conceptual framework based on the resource dependency and institutional theories, we examine the relationship between organizational and market factors and hospitals' degree of cultural competency. Our sample consists of 119 hospitals located in the state of California (US) and is constructed using the following datasets for the year 2006: Cultural Competency Assessment Tool of Hospitals (CCATH) Survey, California's Office of Statewide Health Planning & Development's Hospital Inpatient Discharges and Annual Hospital Financial Data, American Hospital Association's Annual Survey, and the Area Resource File. The dependent variable consists of the degree of hospital cultural competency, as assessed by the CCATH overall score. Organizational variables include ownership status, teaching hospital, payer mix, size, system membership, financial performance, and the proportion of inpatient racial/ethnic minorities. Market characteristics included hospital competition, the proportion of racial/ethnic minorities in the area, metropolitan area, and per capita income. Regression analyses were conducted to assess the relationship between the CCATH overall score and organizational and market variables. Our results show that hospitals which are not-for-profit, serve a more diverse inpatient population, and are located in more competitive and affluent markets exhibit a higher degree of cultural competency. Our results underscore the importance of both institutional and competitive market pressures in guiding hospital behavior. For instance, while not-for-profit may adopt innovative/progressive policies like cultural competency simply as a function of their organizational goals, linking cultural competency with organizational performance may be essential to attract more profit driven hospitals. Copyright © 2012 Elsevier Ltd. All rights reserved.

Validation of a scenario-based assessment of critical thinking using an externally validated tool.

PubMed

Buur, Jennifer L; Schmidt, Peggy; Smylie, Dean; Irizarry, Kris; Crocker, Carlos; Tyler, John; Barr, Margaret

2012-01-01

With medical education transitioning from knowledge-based curricula to competency-based curricula, critical thinking skills have emerged as a major competency. While there are validated external instruments for assessing critical thinking, many educators have created their own custom assessments of critical thinking. However, the face validity of these assessments has not been challenged. The purpose of this study was to compare results from a custom assessment of critical thinking with the results from a validated external instrument of critical thinking. Students from the College of Veterinary Medicine at Western University of Health Sciences were administered a custom assessment of critical thinking (ACT) examination and the externally validated instrument, California Critical Thinking Skills Test (CCTST), in the spring of 2011. Total scores and sub-scores from each exam were analyzed for significant correlations using Pearson correlation coefficients. Significant correlations between ACT Blooms 2 and deductive reasoning and total ACT score and deductive reasoning were demonstrated with correlation coefficients of 0.24 and 0.22, respectively. No other statistically significant correlations were found. The lack of significant correlation between the two examinations illustrates the need in medical education to externally validate internal custom assessments. Ultimately, the development and validation of custom assessments of non-knowledge-based competencies will produce higher quality medical professionals.

Student reflections on learning cross-cultural skills through a 'cultural competence' OSCE.

PubMed

Miller, Elizabeth; Green, Alexander R

2007-05-01

Medical schools use OSCEs (objective structured clinical examinations) to assess students' clinical knowledge and skills, but the use of OSCEs in the teaching and assessment of cross-cultural care has not been well described. To examine medical students' reflections on a cultural competence OSCE station as an educational experience. Students at Harvard Medical School in Boston completed a 'cultural competence' OSCE station (about a patient with uncontrolled hypertension and medication non-adherence). Individual semi-structured interviews were conducted with a convenience sample of twenty-two second year medical students, which were recorded, transcribed, and analysed. Students' reflections on what they learned as the essence of the case encompassed three categories: (1) eliciting the patient's perspective on their illness; (2) examining how and why patients take their medications and inquiring about alternative therapies; and (3) exploring the range of social and cultural factors associated with medication non-adherence. A cultural competence OSCE station that focuses on eliciting patients' perspectives and exploring medication non-adherence can serve as a unique and valuable teaching tool. The cultural competence OSCE station may be one pedagogic method for incorporating cross-cultural care into medical school curricula.

Delphi Method Validation of a Procedural Performance Checklist for Insertion of an Ultrasound-Guided Internal Jugular Central Line.

PubMed

Hartman, Nicholas; Wittler, Mary; Askew, Kim; Manthey, David

2016-01-01

Placement of ultrasound-guided central lines is a critical skill for physicians in several specialties. Improving the quality of care delivered surrounding this procedure demands rigorous measurement of competency, and validated tools to assess performance are essential. Using the iterative, modified Delphi technique and experts in multiple disciplines across the United States, the study team created a 30-item checklist designed to assess competency in the placement of ultrasound-guided internal jugular central lines. Cronbach α was .94, indicating an excellent degree of internal consistency. Further validation of this checklist will require its implementation in simulated and clinical environments. © The Author(s) 2014.

Ares Project Technology Assessment: Approach and Tools

NASA Technical Reports Server (NTRS)

Hueter, Uwe; Tyson, Richard

2010-01-01

Technology assessments provide a status of the development maturity of specific technologies. Along with benefit analysis, the risks the project assumes can be quantified. Normally due to budget constraints, the competing technologies are prioritized and decisions are made which ones to fund. A detailed technology development plan is produced for the selected technologies to provide a roadmap to reach the desired maturity by the project s critical design review. Technology assessments can be conducted for both technology only tasks or for product development programs. This paper is primarily biased toward the product development programs. The paper discusses the Ares Project s approach to technology assessment. System benefit analysis, risk assessment, technology prioritization, and technology readiness assessment are addressed. A description of the technology readiness level tool being used is provided.

Digital Storytelling: A Tool for Identifying and Developing Cultural Competence with Preservice Teachers in an Introduction to Middle Level Education Course

ERIC Educational Resources Information Center

Ruppert, Nancy; Adcock, Lee T.; Crave, Jared

2017-01-01

Using five themes associated with a diversity intensive undergraduate course, preservice teachers in an upper level introduction to middle grade course described their knowledge of cultural competence using digital storytelling as the tool. Findings suggest digital storytelling provides a tool to explore and describe how cultural competence is…

An assessment of workplace programmes designed to control inhalation risks using respiratory protective equipment.

PubMed

Bell, Nikki; Vaughan, Nicholas P; Morris, Len; Griffin, Peter

2012-04-01

Few studies have assessed respiratory protective equipment (RPE) failures at the organizational level despite evidence to suggest that compliance with good practice may be low. The aim of this study was to develop an understanding of what current RPE programmes look like across industry and how this compares with good practice. Twenty cross-industry site visits were conducted with companies that had RPE programmes in place. Visits involved management interviews to explore current RPE systems and procedures and the decision making underpinning these. Observations of RPE operatives were included followed by short interviews to discuss the behaviours observed. Post-site assessments jointly undertaken by an RPE scientist and psychologist produced ratings for each site on six critical aspects of RPE programmes (knowledge/awareness, selection, use, training/information, supervision, and storage/cleaning/maintenance). Overall ratings for theoretical competence (i.e. management knowledge of RPE) and practical control (i.e. actual RPE practice on the shop floor) were also given. Qualitative analysis was performed on all interview data. The performance of RPE programmes varied across industry. Fewer than half the companies visited were considered to have an acceptable level of theoretical competence and practical control. Four distinct groups emerged from the 20 sites studied, ranging from Learners (low theoretical competence and practical control--four sites), Developers (acceptable theoretical competence and low practical control--five sites), and Fortuitous (low theoretical competence and acceptable practical control--two sites), to Proficient (acceptable theoretical competence and practical control--nine sites). None of the companies visited were achieving optimal control through the use of RPE. Widespread inadequacies were found with programme implementation, particularly training, supervision, and maintenance. Our taxonomy based on the four groups (Learners, Developers, Fortuitous, and Proficient) provided a useful expert-informed tool for explaining the variation in performance of RPE programmes across industry. Although further research and development are required, this taxonomy offers a useful starting point for the development of practical tools that may assist managers in making the much-needed improvements to all facets of programme implementation, particularly training, supervision, and maintenance.

Critical thinking competence and disposition of clinical nurses in a medical center.

PubMed

Feng, Rung-Chuang; Chen, Mei-Jung; Chen, Mei-Chuan; Pai, Yu-Chu

2010-06-01

Critical thinking is essential in nursing practice. Promoting critical thinking competence in clinical nurses is an important way to improve problem solving and decision-making competence to further improve the quality of patient care. However, using an adequate tool to test nurses' critical thinking competence and disposition may provide the reference criteria for clinical nurse characterization, training planning, and resource allocation for human resource management. The purpose of this study was to measure the critical thinking competence and critical thinking disposition of clinical nurses as well as to explore the related factors of critical thinking competence. Clinical nurses from four different clinical ladders selected from one medical center were stratified randomly. All qualified subjects who submitted valid questionnaires were included in the study. A Taiwan version of the modified Watson-Glaser Critical Thinking Appraisal and Critical Thinking Disposition Inventory was developed to measure the critical thinking competence and critical thinking disposition of clinical nurses. Validity was evaluated using the professional content test (content validity index = .93). Reliability was assessed with a Cronbach's alpha coefficient of .85. Data were analyzed using the SPSS for Windows (Version 12.0; SPSS Inc., Chicago, IL). Results showed that competence of interpretation was the highest critical thinking competence factor. Inference was the lowest, and reflective thinking as a critical thinking disposition was more positive. In addition, age, years of nursing experience, and experiences in other hospitals significantly influenced critical thinking competence (p < .05). Factors of age, years of experience, and nurses clinical ladder were shown to affect critical thinking disposition scores. Clinical ladder N4 nurses had the highest scores in both competence and disposition. A significant relationship was found between critical thinking competence and disposition scores, with 29.3% of the variance in critical thinking competence potentially explained by total years of nurse hospital experience. Clinical ladder and age were predictive factors for critical thinking disposition. Commonality was 27.9%. Nursing experience and clinical ladders positively affect critical thinking competence and disposition. Issues of critical thinking competence increasingly need to be measured. Therefore, appropriate tools for nursing professions should be further developed and explored for specific areas of practice.

Development of a Competency Mapping Tool for Undergraduate Professional Degree Programmes, Using Mechanical Engineering as a Case Study

ERIC Educational Resources Information Center

Holmes, David W.; Sheehan, Madoc; Birks, Melanie; Smithson, John

2018-01-01

Mapping the curriculum of a professional degree to the associated competency standard ensures graduates have the competence to perform as professionals. Existing approaches to competence mapping vary greatly in depth, complexity, and effectiveness, and a standardised approach remains elusive. This paper describes a new mapping software tool that…

Family Medicine Residency Program Directors Attitudes and Knowledge of Family Medicine CAM Competencies

PubMed Central

Gardiner, Paula; Filippelli, Amanda C.; Lebensohn, Patricia; Bonakdar, Robert

2013-01-01

Context Little is known about the incorporation of integrative medicine (IM) and complementary and alternative medicine (CAM) into family medicine residency programs. Objective The Society for Teachers of Family Medicine (STFM) approved a set of CAM/IM competencies for family medicine residencies. We hope to evaluate with an online survey tool, whether residency programs are implementing such competencies into their curriculum. We also hope to assess the knowledge and attitudes of Residency Directors (RDs) on the CAM/IM competencies. Design A survey was distributed by the CAFM (Council of Academic Family Medicine) Educational Research Alliance to RDs via email. The survey was distributed to 431 RDs. Of those who received it, 212 responded for a response rate of 49.1%. Questions assessed the knowledge and attitudes of CAM/IM competencies and incorporation of CAM/IM into residency curriculum. Results Forty-five percent of RDs were aware of the competencies. In term of RD attitudes, 58% reported that CAM/IM is an important component of residents' curriculum yet, 60% report not having specific learning objectives for CAM/IM in their residency curriculum. Among all programs, barriers to CAM/IM implementation included: time in residents' schedules (77%); faculty training (75%); access to CAM experts (43%); lack of reimbursement (43%), and financial resources (29%). Conclusions While many RDs are aware of the STFM CAM/IM competencies and acknowledge their role in residence education, there are many barriers preventing residencies to implementing the STFM CAM/IM competencies. PMID:24021471

Family medicine residency program directors attitudes and knowledge of family medicine CAM competencies.

PubMed

Gardiner, Paula; Filippelli, Amanda C; Lebensohn, Patricia; Bonakdar, Robert

2013-01-01

Little is known about the incorporation of integrative medicine (IM) and complementary and alternative medicine (CAM) into family medicine residency programs. The Society for Teachers of Family Medicine (STFM) approved a set of CAM/IM competencies for family medicine residencies. We hope to evaluate whether residency programs are implementing such competencies into their curriculum using an online survey tool. We also hope to assess the knowledge and attitudes of Residency Directors (RDs) on the CAM/IM competencies. A survey was distributed by the Council of Academic Family Medicine (CAFM) Educational Research Alliance to RDs via e-mail. The survey was distributed to 431 RDs. Of those who received it, 212 responded, giving a response rate of 49.1%. Questions assessed the knowledge and attitudes of CAM/IM competencies and incorporation of CAM/IM into the residency curriculum. Forty-five percent of RDs were aware of the competencies. In terms of RD attitudes, 58% reported that CAM/IM is an important component of residents' curriculum; yet, 60% report not having specific learning objectives for CAM/IM in their residency curriculum. Among all programs, barriers to CAM/IM implementation included time in residents' schedules (77%); faculty training (75%); access to CAM experts (43%); lack of reimbursement (43%); and financial resources (29%). While many RDs are aware of the STFM CAM/IM competencies and acknowledge their role in residence education, there are many barriers that prevent residencies from implementing the STFM CAM/IM competencies. © 2013 Elsevier Inc. All rights reserved.

Variation in learning curves and competence for ERCP among advanced endoscopy trainees by using cumulative sum analysis.

PubMed

Wani, Sachin; Hall, Matthew; Wang, Andrew Y; DiMaio, Christopher J; Muthusamy, V Raman; Keswani, Rajesh N; Brauer, Brian C; Easler, Jeffrey J; Yen, Roy D; El Hajj, Ihab; Fukami, Norio; Ghassemi, Kourosh F; Gonzalez, Susana; Hosford, Lindsay; Hollander, Thomas G; Wilson, Robert; Kushnir, Vladimir M; Ahmad, Jawad; Murad, Faris; Prabhu, Anoop; Watson, Rabindra R; Strand, Daniel S; Amateau, Stuart K; Attwell, Augustin; Shah, Raj J; Early, Dayna; Edmundowicz, Steven A; Mullady, Daniel

2016-04-01

There are limited data on learning curves and competence in ERCP. By using a standardized data collection tool, we aimed to prospectively define learning curves and measure competence among advanced endoscopy trainees (AETs) by using cumulative sum (CUSUM) analysis. AETs were evaluated by attending endoscopists starting with the 26th hands-on ERCP examination and then every ERCP examination during the 12-month training period. A standardized ERCP competency assessment tool (using a 4-point scoring system) was used to grade the examination. CUSUM analysis was applied to produce learning curves for individual technical and cognitive components of ERCP performance (success defined as a score of 1, acceptable and unacceptable failures [p1] of 10% and 20%, respectively). Sensitivity analyses varying p1 and by using a less-stringent definition of success were performed. Five AETs were included with a total of 1049 graded ERCPs (mean ± SD, 209.8 ± 91.6/AET). The majority of cases were performed for a biliary indication (80%). The overall and native papilla allowed cannulation times were 3.1 ± 3.6 and 5.7 ± 4, respectively. Overall learning curves demonstrated substantial variability for individual technical and cognitive endpoints. Although nearly all AETs achieved competence in overall cannulation, none achieved competence for cannulation in cases with a native papilla. Sensitivity analyses increased the proportion of AETs who achieved competence. This study demonstrates that there is substantial variability in ERCP learning curves among AETs. A specific case volume does not ensure competence, especially for native papilla cannulation. Copyright © 2016 American Society for Gastrointestinal Endoscopy. Published by Elsevier Inc. All rights reserved.

A model of four hierarchical levels to train Chinese residents' teaching skills for "practice-based learning and improvement" competency.

PubMed

Yang, Ying-Ying; Yang, Ling-Yu; Hsu, Hui-Chi; Huang, Chia-Chang; Huang, Chin-Chou; Kirby, Ralph; Cheng, Hao Min; Chang, Ching-Chi; Chuang, Chiao-Lin; Liang, Jen-Feng; Lin, Chun-Chi; Lee, Wei-Shin; Ho, Shung-Tai; Lee, Fa-Yauh

2015-01-01

The current study focused on validating a protocol for training and auditing the resident's practice-based learning and improvement (PBLI) and quality improvement (QI) competencies for primary care. Twelve second-year (R2), 12 first-year (R1) and 12 postgraduate year-1 residents were enrolled into group A, B and C, respectively, as trainees. After three training protocols had been completed, a writing test, self-assessed questionnaire and mini-OSTE and end-of-rotation assessment were used in auditing the PBLI competency, performance and teaching ability of trainees. Baseline expert-assessed PBLI and QI knowledge application tool writing scores were low for the R1 and R2 residents. After three training protocols, PBLI and QI proficiencies, performance and teaching abilities were improved to similar levels cross the three training levels of residents based on the expert-assessed writing test-audited assessments and on the faculty and standardized clerk-assessed end-of-rotation-/mini-OSTE-audited assessments. The different four-level hierarchical protocols used to teach group A, B and C were equally beneficial and fitted their needs; namely the different levels of the trainees. Specifically, each level was able to augment their PBLI and QI proficiency. This educational intervention helps medical institutions to train residents as PBLI instructors.

The Implementation of Pharmacy Competence Teaching in Estonia

PubMed Central

Volmer, Daisy; Sepp, Kristiina; Veski, Peep; Raal, Ain

2017-01-01

Background: The PHAR-QA, “Quality Assurance in European Pharmacy Education and Training”, project has produced the European Pharmacy Competence Framework (EPCF). The aim of this study was to evaluate the existing pharmacy programme at the University of Tartu, using the EPCF. Methods: A qualitative assessment of the pharmacy programme by a convenience sample (n = 14) representing different pharmacy stakeholders in Estonia. EPCF competency levels were determined by using a five-point scale tool adopted from the Dutch competency standards framework. Mean scores of competency levels given by academia and other pharmacy stakeholders were compared. Results: Medical and social sciences, pharmaceutical technology, and pharmacy internship were more frequent subject areas contributing to EPCF competencies. In almost all domains, the competency level was seen higher by academia than by other pharmacy stakeholders. Despite on-board theoretical knowledge, the competency level at graduation could be insufficient for independent professional practice. Other pharmacy stakeholders would improve practical implementation of theoretical knowledge, especially to increase patient care competencies. Conclusions: The EPCF was utilized to evaluate professional competencies of entry-level pharmacists who have completed a traditional pharmacy curriculum. More efficient training methods and involvement of practicing specialists were suggested to reduce the gaps of the existing pharmacy programme. Applicability of competence teaching in Estonia requires more research and collaborative communication within the pharmacy sector. PMID:28970430

Educating medical students as competent users of health information technologies: the MSOP data.

PubMed

McGowan, Julie J; Passiment, Morgan; Hoffman, Helene M

2007-01-01

As more health information technologies become part of the health care environment, the need for physicians with medical informatics competencies is growing. In 2006, a survey was created to determine the degree to which the Association of American Medical College's Medical School Objectives Project (MSOP) medical informatics competencies had been incorporated into medical school curricula in the United States. a web-based tool was used to create the survey; medical education deans or their designees were requested to complete the survey. Analysis focused on the clinician, researcher, and manager roles of physicians. Seventy usable surveys were returned. Many of the objectives were stated in the schools' respective curricula and the competencies were being evaluated. However, only a few schools taught and assessed the medical informatics objectives that required interaction with health information. To insure that physicians have the knowledge, skills, and attitudes to effectively and efficiently interact with today's health information technologies, more medical informatics concepts need to be included and assessed in all undergraduate medical education curricula in the United States.

Can Hospital Cultural Competency Reduce Disparities in Patient Experiences with Care?

PubMed Central

Weech-Maldonado, Robert; Elliott, Marc N.; Pradhan, Rohit; Schiller, Cameron; Hall, Allyson; Hays, Ron D.

2013-01-01

Background Cultural competency has been espoused as an organizational strategy to reduce health disparities in care. Objective To examine the relationship between hospital cultural competency and inpatient experiences with care. Research Design The first model predicted Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) scores from hospital random effects, plus fixed effects for hospital cultural competency, individual race/ethnicity/language, and case-mix variables. The second model tested if the association between a hospital’s cultural competency and HCAHPS scores differed for minority and non-Hispanic white patients. Subjects The National CAHPS® Benchmarking Database’s (NCBD) HCAHPS Surveys and the Cultural Competency Assessment Tool of Hospitals (CCATH) Surveys for California hospitals were merged, resulting in 66 hospitals and 19,583 HCAHPS respondents in 2006. Measures Dependent variables include ten HCAHPS measures: six composites (communication with doctors, communication with nurses, staff responsiveness, pain control, communication about medications, and discharge information), two individual items (cleanliness, and quietness of patient rooms), and two global items (overall hospital rating, and whether patient would recommend hospital). Results Hospitals with greater cultural competency have better HCAHPS scores for doctor communication, hospital rating, and hospital recommendation. Furthermore, HCAHPS scores for minorities were higher at hospitals with greater cultural competency on four other dimensions: nurse communication, staff responsiveness, quiet room, and pain control. Conclusions Greater hospital cultural competency may improve overall patient experiences, but may particularly benefit minorities in their interactions with nurses and hospital staff. Such effort may not only serve longstanding goals of reducing racial/ethnic disparities in inpatient experience, but may also contribute to general quality improvement. PMID:23064277

Multicomponent Musculoskeletal Movement Assessment Tools: A Systematic Review and Critical Appraisal of Their Development and Applicability to Professional Practice.

PubMed

Bennett, Hunter; Davison, Kade; Arnold, John; Slattery, Flynn; Martin, Max; Norton, Kevin

2017-10-01

Multicomponent movement assessment tools have become commonplace to measure movement quality, proposing to indicate injury risk and performance capabilities. Despite popular use, there has been no attempt to compare the components of each tool reported in the literature, the processes in which they were developed, or the underpinning rationale for their included content. As such, the objective of this systematic review was to provide a comprehensive summary of current movement assessment tools and appraise the evidence supporting their development. A systematic literature search was performed using PRISMA guidelines to identify multicomponent movement assessment tools. Commonalities between tools and the evidence provided to support the content of each tool was identified. Each tool underwent critical appraisal to identify the rigor in which it was developed, and its applicability to professional practice. Eleven tools were identified, of which 5 provided evidence to support their content as assessments of movement quality. One assessment tool (Soccer Injury Movement Screen [SIMS]) received an overall score of above 65% on critical appraisal, with a further 2 tools (Movement Competency Screen [MCS] and modified 4 movement screen [M4-MS]) scoring above 60%. Only the MCS provided clear justification for its developmental process. The remaining 8 tools scored between 40 and 60%. On appraisal, the MCS, M4-MS, and SIMS seem to provide the most practical value for assessing movement quality as they provide the strongest reports of developmental rigor and an identifiable evidence base. In addition, considering the evidence provided, these tools may have the strongest potential for identifying performance capabilities and guiding exercise prescription in athletic and sport-specific populations.

Developing and testing a measurement tool for assessing predictors of breakfast consumption based on a health promotion model.

PubMed

Dehdari, Tahereh; Rahimi, Tahereh; Aryaeian, Naheed; Gohari, Mahmood Reza; Esfeh, Jabiz Modaresi

2014-01-01

To develop an instrument for measuring Health Promotion Model constructs in terms of breakfast consumption, and to identify the constructs that were predictors of breakfast consumption among Iranian female students. A questionnaire on Health Promotion Model variables was developed and potential predictors of breakfast consumption were assessed using this tool. One hundred female students, mean age 13 years (SD ± 1.2 years). Two middle schools from moderate-income areas in Qom, Iran. Health Promotion Model variables were assessed using a 58-item questionnaire. Breakfast consumption was also measured. Internal consistency (Cronbach alpha), content validity index, content validity ratio, multiple linear regression using stepwise method, and Pearson correlation. Content validity index and content validity ratio scores of the developed scale items were 0.89 and 0.93, respectively. Internal consistencies (range, .74-.91) of subscales were acceptable. Prior related behaviors, perceived barriers, self-efficacy, and competing demand and preferences were 4 constructs that could predict 63% variance of breakfast frequency per week among subjects. The instrument developed in this study may be a useful tool for researchers to explore factors affecting breakfast consumption among students. Students with a high level of self-efficacy, more prior related behavior, fewer perceived barriers, and fewer competing demands were most likely to regularly consume breakfast. Copyright © 2014 Society for Nutrition Education and Behavior. Published by Elsevier Inc. All rights reserved.

Development, validation, and utility of an instrument to assess core competencies in the Leadership Education in Neurodevelopmental and Related Disabilities (LEND) program.

PubMed

Leff, Stephen S; Baum, Katherine T; Bevans, Katherine B; Blum, Nathan J

2015-02-01

To describe the development and psychometric evaluation of the Core Competency Measure (CCM), an instrument designed to assess professional competencies as defined by the Maternal Child Health Bureau (MCHB) and targeted by Leadership Education in Neurodevelopmental and Related Disabilities (LEND) programs. The CCM is a 44-item self-report measure comprised of six subscales to assess clinical, interdisciplinary, family-centered/cultural, community, research, and advocacy/policy competencies. The CCM was developed in an iterative fashion through participatory action research, and then nine cohorts of LEND trainees (N = 144) from 14 different disciplines completed the CCM during the first week of the training program. A 6-factor confirmatory factor analysis model was fit to data from the 44 original items. After three items were removed, the model adequately fit the data (comparative fit indices = .93, root mean error of approximation = .06) with all factor loadings exceeding .55. The measure was determined to be quite reliable as adequate internal consistency and test-retest reliability were found for each subscale. The instrument's construct validity was supported by expected differences in self-rated competencies among fellows representing various disciplines, and the convergent validity was supported by the pattern of inter-correlations between subscale scores. The CCM appears to be a reliable and valid measure of MCHB core competencies for our sample of LEND trainees. It provides an assessment of key training areas addressed by the LEND program. Although the measure was developed within only one LEND Program, with additional research it has the potential to serve as a standardized tool to evaluate the strengths and limitations of MCHB training, both within and between programs.

Industrial Sponsor Perspective on Leveraging Capstone Design Projects to Enhance Their Business

ERIC Educational Resources Information Center

Weissbach, Robert S.; Snyder, Joseph W.; Evans, Edward R., Jr.; Carucci, James R., Jr.

2017-01-01

Capstone design projects have become commonplace among engineering and engineering technology programs. These projects are valuable tools when assessing students, as they require students to work in teams, communicate effectively, and demonstrate technical competency. The use of industrial sponsors enhances these projects by giving these projects…

Integrating Reading and Technology: The Development of "Pamanpintermu"

ERIC Educational Resources Information Center

Arifani, Yudhi; Rosyid, Harunur

2016-01-01

Reading as one of English skills has paramount features in shaping EFL English competence. Referring to the importance for reading, it is inevitable that teaching method, assessments tools, reading material and activities have indispensable tasks to attain EFL learners' reading objectives. This study is intended to develop integrated reading…

Teaching Social Skills in a Virtual Environment: An Exploratory Study

ERIC Educational Resources Information Center

Baker, Jason; Parks-Savage, Agatha; Rehfuss, Mark

2009-01-01

This article reports on an exploratory study which examines the use of virtual environment technology as a tool to teach elementary school children social skills. Small group interventions were assessed to determine how the participants were measurably different on 7 different dependent variables: problem behaviors, academic competence,…

Reliability of the ECHOWS Tool for Assessment of Patient Interviewing Skills.

PubMed

Boissonnault, Jill S; Evans, Kerrie; Tuttle, Neil; Hetzel, Scott J; Boissonnault, William G

2016-04-01

History taking is an important component of patient/client management. Assessment of student history-taking competency can be achieved via a standardized tool. The ECHOWS tool has been shown to be valid with modest intrarater reliability in a previous study but did not demonstrate sufficient power to definitively prove its stability. The purposes of this study were: (1) to assess the reliability of the ECHOWS tool for student assessment of patient interviewing skills and (2) to determine whether the tool discerns between novice and experienced skill levels. A reliability and construct validity assessment was conducted. Three faculty members from the United States and Australia scored videotaped histories from standardized patients taken by students and experienced clinicians from each of these countries. The tapes were scored twice, 3 to 6 weeks apart. Reliability was assessed using interclass correlation coefficients (ICCs) and repeated measures. Analysis of variance models assessed the ability of the tool to discern between novice and experienced skill levels. The ECHOWS tool showed excellent intrarater reliability (ICC [3,1]=.74-.89) and good interrater reliability (ICC [2,1]=.55) as a whole. The summary of performance (S) section showed poor interrater reliability (ICC [2,1]=.27). There was no statistical difference in performance on the tool between novice and experienced clinicians. A possible ceiling effect may occur when standardized patients are not coached to provide complex and obtuse responses to interviewer questions. Variation in familiarity with the ECHOWS tool and in use of the online training may have influenced scoring of the S section. The ECHOWS tool demonstrates excellent intrarater reliability and moderate interrater reliability. Sufficient training with the tool prior to student assessment is recommended. The S section must evolve in order to provide a more discerning measure of interviewing skills. © 2016 American Physical Therapy Association.

Evaluation of perceived and actual competency in a family medicine objective structured clinical examination

PubMed Central

Graves, Lisa; Lalla, Leonora; Young, Meredith

2017-01-01

Abstract Objective To examine the relationship between objective assessment of performance and self-rated competence immediately before and after participation in a required summative family medicine clerkship objective structured clinical examination (OSCE). Design Learners rated their competence (on a 7-point Likert scale) before and after an OSCE along 3 dimensions: general, specific, and professional competencies relevant to family medicine. Setting McGill University in Montreal, Que. Participants All 168 third-year clinical clerks completing their mandatory family medicine rotation in 2010 to 2011 were invited to participate. Main outcome measures Self-ratings of competence and objective performance scores were compared, and were examined to determine if OSCEs could be a “corrective” tool for self-rating perceived competence (ie, if the experience of undergoing an assessment might assist learners in recalibrating their understanding of their own performance). Results A total of 140 (83%) of the third-year clinical clerks participated. Participating in an OSCE decreased learners’ ratings of perceived competence (pre-OSCE score = 4.9, post-OSCE score = 4.7; F1,3192 = 4.2; P < .05). Learners’ mean self-rated competence for all categories of behaviour (before and after) showed no relationship to OSCE performance (r < 0.12 and P > .08 for all), nor did ratings of station-relevant competence (before and after) (r < 0.19 and P > .09 for all). Ratings of competence before and after the OSCE were correlated for individual students (r > 0.40 and P < .001 for all). Conclusion After the OSCE, students’ self-ratings of perceived competence had decreased, and these ratings had little relationship to actual performance, regardless of the specificity of the rated competency. Discordance between perceived and actual competence is neither novel nor unique to family medicine. However, this discordance is an important consideration for the development of competency-based curricula. PMID:28404722

Evaluation of perceived and actual competency in a family medicine objective structured clinical examination.

PubMed

Graves, Lisa; Lalla, Leonora; Young, Meredith

2017-04-01

To examine the relationship between objective assessment of performance and self-rated competence immediately before and after participation in a required summative family medicine clerkship objective structured clinical examination (OSCE). Learners rated their competence (on a 7-point Likert scale) before and after an OSCE along 3 dimensions: general, specific, and professional competencies relevant to family medicine. McGill University in Montreal, Que. All 168 third-year clinical clerks completing their mandatory family medicine rotation in 2010 to 2011 were invited to participate. Self-ratings of competence and objective performance scores were compared, and were examined to determine if OSCEs could be a "corrective" tool for self-rating perceived competence (ie, if the experience of undergoing an assessment might assist learners in recalibrating their understanding of their own performance). A total of 140 (83%) of the third-year clinical clerks participated. Participating in an OSCE decreased learners' ratings of perceived competence (pre-OSCE score = 4.9, post-OSCE score = 4.7; F 1,3192 = 4.2; P  < .05). Learners' mean self-rated competence for all categories of behaviour (before and after) showed no relationship to OSCE performance ( r < 0.12 and P > .08 for all), nor did ratings of station-relevant competence (before and after) ( r < 0.19 and P > .09 for all). Ratings of competence before and after the OSCE were correlated for individual students ( r > 0.40 and P < .001 for all). After the OSCE, students' self-ratings of perceived competence had decreased, and these ratings had little relationship to actual performance, regardless of the specificity of the rated competency. Discordance between perceived and actual competence is neither novel nor unique to family medicine. However, this discordance is an important consideration for the development of competency-based curricula. Copyright© the College of Family Physicians of Canada.

Assessment of technical and nontechnical skills in surgical residents.

PubMed

Ponton-Carss, Alicia; Kortbeek, John B; Ma, Irene W Y

2016-11-01

Surgical competence encompasses both technical and nontechnical skills. This study seeks to evaluate the validity evidence for a comprehensive surgical skills examination and to examine the relationship between technical and nontechnical skills. Six examination stations assessing both technical and nontechnical skills, conducted yearly for surgical trainees (n = 120) between 2010 and 2014 are included. The assessment tools demonstrated acceptable internal consistency. Interstation reliability for technical skills was low (alpha = .39). Interstation reliability for the nontechnical skills was lower (alpha range -.05 to .31). Nontechnical skills domains were strongly correlated, ranging from r = .65, P < .001 to .86, P < .001. The associations between nontechnical and technical skills were inconsistent, ranging from poor (r = -.06; P = .54) to moderate (r = .45; P < .001). Multiple samplings of integrated technical and nontechnical skills are necessary to assess overall surgical competency. Copyright © 2016 Elsevier Inc. All rights reserved.

Assessment of pharmacy students' communication competence using the Roter Interaction Analysis System during objective structured clinical examinations.

PubMed

Kubota, Yoshie; Yano, Yoshitaka; Seki, Susumu; Takada, Kaori; Sakuma, Mio; Morimoto, Takeshi; Akaike, Akinori; Hiraide, Atsushi

2011-04-11

To determine the value of using the Roter Interaction Analysis System during objective structured clinical examinations (OSCEs) to assess pharmacy students' communication competence. As pharmacy students completed a clinical OSCE involving an interview with a simulated patient, 3 experts used a global rating scale to assess students' overall performance in the interview, and both the student's and patient's languages were coded using the Roter Interaction Analysis System (RIAS). The coders recorded the number of utterances (ie, units of spoken language) in each RIAS category. Correlations between the raters' scores and the number and types of utterances were examined. There was a significant correlation between students' global rating scores on the OSCE and the number of utterances in the RIAS socio-emotional category but not the RIAS business category. The RIAS proved to be a useful tool for assessing the socio-emotional aspect of students' interview skills.

Distributed learning or medical tourism? A Canadian residency program's experience in global health.

PubMed

Kelly, Kate; McCarthy, Anne; McLean, Laurie

2015-01-01

Global health experiences (GHEs) are becoming increasingly prevalent in surgical residency education. Although it may seem intuitive that participation in GHEs develops CanMEDS competencies, this has not been studied in depth in surgery. The purpose of this study is (1) to explore if and how otolaryngology-head and neck surgery (OHNS) resident participation in GHEs facilitates the development of CanMEDS competencies and (2) to develop an OHNS GHE tool to facilitate the integration of CanMEDS into GHE participation and evaluation. An online survey explored the GHEs of current and past OHNS residents in Canada. Based on the data collected and a literature review, a foundational tool was then created to (1) enable OHNS residents to structure their GHEs into CanMEDS-related learning objectives and (2) enable OHNS program directors to more effectively evaluate residents' GHEs with respect to CanMEDS competencies. Participants' GHEs varied widely. These experiences often contributed informally to the development of several CanMEDS competencies. However, few residents had concrete objectives, rarely were CanMEDS roles clearly incorporated, and most residents were not formally evaluated during their experience. Residents felt they achieved greater learning when predeparture objectives and postexperience reflections were integrated into their GHEs. Although GHEs vary widely, they can serve as valuable forums for developing CanMEDS competencies among participating residents. Without clear objectives that adhere to the CanMEDS framework or formal assessment methods however, residents in GHEs risk becoming medical tourists. The use of an objective and evaluation tool may facilitate the creation of predeparture learning objectives, encourage self-reflection on their GHE, and better enable program directors to evaluate residents participating in GHEs. Copyright © 2015 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

State of the Science of Spirituality and Palliative Care Research Part II: Screening, Assessment, and Interventions.

PubMed

Balboni, Tracy A; Fitchett, George; Handzo, George F; Johnson, Kimberly S; Koenig, Harold G; Pargament, Kenneth I; Puchalski, Christina M; Sinclair, Shane; Taylor, Elizabeth J; Steinhauser, Karen E

2017-09-01

The State of the Science in Spirituality and Palliative Care was convened to address the current landscape of research at the intersection of spirituality and palliative care and to identify critical next steps to advance this field of inquiry. Part II of the SOS-SPC report addresses the state of extant research and identifies critical research priorities pertaining to the following questions: 1) How do we assess spirituality? 2) How do we intervene on spirituality in palliative care? And 3) How do we train health professionals to address spirituality in palliative care? Findings from this report point to the need for screening and assessment tools that are rigorously developed, clinically relevant, and adapted to a diversity of clinical and cultural settings. Chaplaincy research is needed to form professional spiritual care provision in a variety of settings, and outcomes assessed to ascertain impact on key patient, family, and clinical staff outcomes. Intervention research requires rigorous conceptualization and assessments. Intervention development must be attentive to clinical feasibility, incorporate perspectives and needs of patients, families, and clinicians, and be targeted to diverse populations with spiritual needs. Finally, spiritual care competencies for various clinical care team members should be refined. Reflecting those competencies, training curricula and evaluation tools should be developed, and the impact of education on patient, family, and clinician outcomes should be systematically assessed. Published by Elsevier Inc.

Revision of the Competency Standards for Occupational Therapy Driver Assessors: An overview of the evidence for the inclusion of cognitive and perceptual assessments within fitness-to-drive evaluations.

PubMed

Fields, Sally M; Unsworth, Carolyn A

2017-08-01

Determination of fitness-to-drive after illness or injury is a complex process typically requiring a comprehensive driving assessment, including off-road and on-road assessment components. The competency standards for occupational therapy driver assessors (Victoria, Australia) define the requirements for performance of a comprehensive driving assessment, and we are currently revising these. Assessment of cognitive and perceptual skills forms an important part of the off-road assessment. The aim of this systematic review of systematic reviews (known as an overview) is to identify what evidence exists for including assessment of cognitive and perceptual skills within fitness-to-drive evaluations to inform revision of the competency standards. Five electronic databases (MEDLINE, CINAHL, PsycINFO, The Cochrane Library, OT Seeker) were systematically searched. Systematic review articles were appraised by two authors for eligibility. Methodological quality was independently assessed using the AMSTAR tool. Narrative analysis was conducted to summarise the content of eligible reviews. A total of 1228 results were retrieved. Fourteen reviews met the inclusion criteria. Reviews indicated that the components of cognition and perception most frequently identified as being predictive of fitness-to-drive were executive function (n = 13), processing speed (n = 12), visuospatial skills, attention, memory and mental flexibility (n = 11). Components less indicative were perception, concentration (n = 10), praxis (n = 9), language (n = 7) and neglect (n = 6). This overview of systematic reviews supports the inclusion of assessment of a range of cognitive and perceptual skills as key elements in a comprehensive driver assessment and therefore should be included in the revised competency standards for occupational therapy driver assessors. © 2017 Occupational Therapy Australia.

Health education and competency scale: Development and testing.

PubMed

Hwang, Huei-Lih; Kuo, Mei-Ling; Tu, Chin-Tang

2018-02-01

To develop a tool for measuring competency in conducting health education and to evaluate its psychometric properties in a population of entry-level nurses. Until now, no generic instrument has been developed specifically for measuring competency in health education, which is an essential competency for nurses. Existing scales are either insufficient for psychometric evaluation or are designed specifically for senior nurses. To evaluate curricula and courses designed for entry-level nurses, educators require an instrument for measuring improvement in core competency from baseline to determine whether the minimum level of ability has been achieved. Item development for the survey instrument used for data collection in this study was based on the results of a literature review. The self-evaluated Health Education Competency Scale developed in this study was used to survey 457 nursing students at two nursing schools and 165 clinical nurses at a medical centre in south Taiwan in 2016. The participants were randomly divided into two equal groups. One group was analysed by exploratory factor analysis with varimax rotation, and one group was analysed by confirmatory factor analysis. Factor analysis yielded a four-factor (assessment, pedagogy, motivation and empowerment) solution (18 items) that accounted for 75.9% of the variance. The total scale and subscales had good reliabilities and construct validity coefficients. For measuring competency in entry-level nurses, the Health Education Competency Scale had a good data fit and sound psychometric properties. The proposed scale can be used to assess health education competency for college nursing students and practising nurses. Furthermore, it can provide educators with valuable insight into the minimum competencies required for entry-level nurses to deliver quality health care to clients and can guide them in the practice of client-based teaching. © 2017 John Wiley & Sons Ltd.

Reliable and valid assessment of Lichtenstein hernia repair skills.

PubMed

Carlsen, C G; Lindorff-Larsen, K; Funch-Jensen, P; Lund, L; Charles, P; Konge, L

2014-08-01

Lichtenstein hernia repair is a common surgical procedure and one of the first procedures performed by a surgical trainee. However, formal assessment tools developed for this procedure are few and sparsely validated. The aim of this study was to determine the reliability and validity of an assessment tool designed to measure surgical skills in Lichtenstein hernia repair. Key issues were identified through a focus group interview. On this basis, an assessment tool with eight items was designed. Ten surgeons and surgical trainees were video recorded while performing Lichtenstein hernia repair, (four experts, three intermediates, and three novices). The videos were blindly and individually assessed by three raters (surgical consultants) using the assessment tool. Based on these assessments, validity and reliability were explored. The internal consistency of the items was high (Cronbach's alpha = 0.97). The inter-rater reliability was very good with an intra-class correlation coefficient (ICC) = 0.93. Generalizability analysis showed a coefficient above 0.8 even with one rater. The coefficient improved to 0.92 if three raters were used. One-way analysis of variance found a significant difference between the three groups which indicates construct validity, p < 0.001. Lichtenstein hernia repair skills can be assessed blindly by a single rater in a reliable and valid fashion with the new procedure-specific assessment tool. We recommend this tool for future assessment of trainees performing Lichtenstein hernia repair to ensure that the objectives of competency-based surgical training are met.

Simulation in bronchoscopy: current and future perspectives.

PubMed

Nilsson, Philip Mørkeberg; Naur, Therese Maria Henriette; Clementsen, Paul Frost; Konge, Lars

2017-01-01

To provide an overview of current literature that informs how to approach simulation practice of bronchoscopy and discuss how findings from other simulation research can help inform the use of simulation in bronchoscopy training. We conducted a literature search on simulation training of bronchoscopy and divided relevant studies in three categories: 1) structuring simulation training in bronchoscopy, 2) assessment of competence in bronchoscopy training, and 3) development of cheap alternatives for bronchoscopy simulation. Bronchoscopy simulation is effective, and the training should be structured as distributed practice with mastery learning criteria (ie, training until a certain level of competence is achieved). Dyad practice (training in pairs) is possible and may increase utility of available simulators. Trainee performance should be assessed with assessment tools with established validity. Three-dimensional printing is a promising new technology opening possibilities for developing cheap simulators with innovative features.

Visual Literacy in Preservice Teachers: a Case Study in Biology

NASA Astrophysics Data System (ADS)

Ruiz-Gallardo, José Reyes; García Fernández, Beatriz; Mateos Jiménez, Antonio

2017-07-01

In this study, we explore the competence of preservice teachers (n = 161) in labelling and creating new cross-sectional human diagrams, based on anatomy knowledge depicted in longitudinal sections. Using educational standards to assess visual literacy and ad hoc open questions, results indicate limited skills for both tasks. However, their competence is particularly poor creating diagrams, where shortcomings were observed not only in visual literacy but in content knowledge. We discuss the misconceptions detected during these assessments. Visual literacy training should be strengthened for these students, as it is a skill that is especially important for future teachers to use in learning, assessing, and reflecting on content in science education. This is particularly important in preservice teachers since they should be fluent in the use of visual teaching tools in teaching anatomy and other content in the biology curriculum.

The Feasibility and Acceptability of Using a Portfolio to Assess Professional Competence

PubMed Central

Tuekam, Rosine

2011-01-01

ABSTRACT Purpose: Little is known about physical therapists' views on the use of portfolios to evaluate professional competence. The purpose of this study was to gather the opinions of physical therapists on the feasibility and acceptability of a portfolio prepared to demonstrate evidence of clinical specialization through reported activities and accomplishments related to professional development, leadership, and research. Methods: Twenty-nine Canadian physical therapists practising in the neurosciences area were given 8 weeks to prepare a professional portfolio. Participants submitted the portfolio along with a survey addressing the preparation of the portfolio and its role as an assessment tool. Qualitative content analysis was used to interpret the participants' comments. Results: Participants reported that maintaining organized records facilitated the preparation of their portfolio. They experienced pride when reviewing their completed portfolios, which summarized their professional activities and highlighted their achievements. Concerns were noted about the veracity of self-reported records and the ability of the documentation to provide a comprehensive view of the full scope of the professional competencies required for clinical specialization (e.g., clinical skills). Conclusion: The study's findings support the feasibility and acceptability of a portfolio review to assess professional competence and clinical specialization in physical therapy and have implications for both physical therapists and professional agencies. PMID:22210983

The feasibility and acceptability of using a portfolio to assess professional competence.

PubMed

Miller, Patricia A; Tuekam, Rosine

2011-01-01

Little is known about physical therapists' views on the use of portfolios to evaluate professional competence. The purpose of this study was to gather the opinions of physical therapists on the feasibility and acceptability of a portfolio prepared to demonstrate evidence of clinical specialization through reported activities and accomplishments related to professional development, leadership, and research. Twenty-nine Canadian physical therapists practising in the neurosciences area were given 8 weeks to prepare a professional portfolio. Participants submitted the portfolio along with a survey addressing the preparation of the portfolio and its role as an assessment tool. Qualitative content analysis was used to interpret the participants' comments. Participants reported that maintaining organized records facilitated the preparation of their portfolio. They experienced pride when reviewing their completed portfolios, which summarized their professional activities and highlighted their achievements. Concerns were noted about the veracity of self-reported records and the ability of the documentation to provide a comprehensive view of the full scope of the professional competencies required for clinical specialization (e.g., clinical skills). The study's findings support the feasibility and acceptability of a portfolio review to assess professional competence and clinical specialization in physical therapy and have implications for both physical therapists and professional agencies.

Social development of children born very preterm: a systematic review.

PubMed

Ritchie, Kirsten; Bora, Samudragupta; Woodward, Lianne J

2015-10-01

To review systematically studies examining the development of social competence in children born very preterm (VPT) (gestation <33 wks) and identify neonatal and family predictors. Peer-reviewed original articles were extracted from PubMed and PsycINFO following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Selection criteria included children born VPT and comparison children born at term, sample born after 1990, and children assessed between 0 and 17 years on at least one measure of social competence spanning social adjustment, performance, and/or social skills. Twenty-three studies were included. Seven focused on social competence and another 16 examined social competence within a range of outcomes. Study quality was low. Limitations included reliance on single informant data, cross-sectional measurement, use of brief screening tools, absence of child or peer report, and no conceptual model. In terms of social adjustment, 16 out of 21 studies found children born VPT had more peer problems and social withdrawal. Findings of social performance were mixed, with some studies suggesting differences in prosocial behavior (4/14) and others not. Social skills were assessed in four studies and showed children born VPT had poorer skills than children born at term. Predictors of social competence included gestational age, neonatal brain abnormalities, and family socio-economic status. Children born VPT have poorer social competence. These difficulties emerge early and persist throughout childhood. © 2015 Mac Keith Press.

A 2-year study of Gram stain competency assessment in 40 clinical laboratories.

PubMed

Goodyear, Nancy; Kim, Sara; Reeves, Mary; Astion, Michael L

2006-01-01

We used a computer-based competency assessment tool for Gram stain interpretation to assess the performance of 278 laboratory staff from 40 laboratories on 40 multiple-choice questions. We report test reliability, mean scores, median, item difficulty, discrimination, and analysis of the highest- and lowest-scoring questions. The questions were reliable (KR-20 coefficient, 0.80). Overall mean score was 88% (range, 63%-98%). When categorized by cell type, the means were host cells, 93%; other cells (eg, yeast), 92%; gram-positive, 90%; and gram-negative, 88%. When categorized by type of interpretation, the means were other (eg, underdecolorization), 92%; identify by structure (eg, bacterial morphologic features), 91%; and identify by name (eg, genus and species), 87%. Of the 6 highest-scoring questions (mean scores, > or = 99%) 5 were identify by structure and 1 was identify by name. Of the 6 lowest-scoring questions (mean scores, < 75%) 5 were gram-negative and 1 was host cells. By type of interpretation, 2 were identify by structure and 4 were identify by name. Computer-based Gram stain competency assessment examinations are reliable. Our analysis helps laboratories identify areas for continuing education in Gram stain interpretation and will direct future revisions of the tests.

The cognitive based approach of capacity assessment in psychiatry: a philosophical critique of the MacCAT-T.

PubMed

Breden, Torsten Marcus; Vollmann, Jochen

2004-12-01

This article gives a brief introduction to the MacArthur Competence Assessment Tool-Treatment (MacCAT-T) and critically examines its theoretical presuppositions. On the basis of empirical, methodological and ethical critique it is emphasised that the cognitive bias that underlies the MacCAT-T assessment needs to be modified. On the one hand it has to be admitted that the operationalisation of competence in terms of value-free categories, e.g. rational decision abilities, guarantees objectivity to a great extent; but on the other hand it bears severe problems. Firstly, the cognitive focus is in itself a normative convention in the process of anthropological value-attribution. Secondly, it misses the complexity of the decision process in real life. It is therefore suggested that values, emotions and other biographic and context specific aspects should be considered when interpreting the cognitive standards according to the MacArthur model. To fill the gap between cognitive and non-cognitive approaches the phenomenological theory of personal constructs is briefly introduced. In conclusion some main demands for further research to develop a multi-step model of competence assessment are outlined.

Rating scale for the assessment of competence in ultrasound-guided peripheral vascular access - a Delphi Consensus Study.

PubMed

Primdahl, Stine C; Todsen, Tobias; Clemmesen, Louise; Knudsen, Lars; Weile, Jesper

2016-09-21

Peripheral vascular access is vital for treatment and diagnostics of hospitalized patients. Ultrasound-guided vascular access (UGVA) is superior to the landmark technique. To ensure competence-based education, an assessment tool of UGVA competence is needed. We aimed to develop a global rating scale (RS) for assessment of UGVA competence based on opinions on the content from ultrasound experts in a modified Delphi consensus study. We included experts from anesthesiology, emergency medicine and radiology across university hospitals in Denmark. Nine elements were drafted based on existing literature and recommendations from international societies. In a multi-round survey, the experts rated the elements on a five-point Likert scale according to importance, and suggested missing elements. The final Delphi round occurred when >80% of the experts rated all elements ≥4 on the Likert scale. Sixteen experts consented to participate in the study, one withdrew consent prior to the first Delphi round, and 14 completed all three Delphi rounds. In the first Delphi round the experts excluded one element from the scale and changed the content of two elements. In the second Delphi round, the experts excluded one element from the scale. In the third Delphi round, consensus was obtained on the eight elements: preparation of utensils, ergonomics, preparation of the ultrasound device, identification of blood vessels, anatomy, hygiene, coordination of the needle, and completion of the procedure. We developed an RS for assessment of UGVA competence based on opinions of ultrasound experts through a modified Delphi consensus study.

Continuing Education Course to Attain Collaborative Comprehensive Medication Review Competencies

PubMed Central

Tuomainen, Lea; Ovaskainen, Harri; Peura, Sirpa; Sevón-Vilkman, Nina; Tanskanen, Paavo; Airaksinen, Marja S.A.

2009-01-01

Objective To implement a long-term continuing education course for pharmacy practitioners to acquire competency in and accreditation for conducting collaborative comprehensive medication reviews (CMRs). Design A 1½- year curriculum for practicing pharmacists that combined distance learning (using e-learning tools) and face-to-face learning was created. The training consisted of 5 modules: (1) Multidisciplinary Collaboration; (2) Clinical Pharmacy and Pharmacotherapy; (3) Rational Pharmacotherapy; (4) CMR Tools; and (5) Optional Studies. Assessment The curriculum and participants' learning were evaluated using essays and learning diaries. At the end of the course, students submitted portfolios and completed an Internet-based survey instrument. Almost all respondents (92%) indicated their educational needs had been met by the course and 68% indicated they would conduct CMRs in their practice. The most important factors facilitating learning were working with peers and in small groups. Factors preventing learning were mostly related to time constraints. Conclusion Comprehensive medication review competencies were established by a 1½- year continuing education curriculum that combined different teaching methods and experiential learning. Peer support was greatly appreciated as a facilitator of learning by course participants. PMID:19885077

Diagnostic flexible pharyngo-laryngoscopy: development of a procedure specific assessment tool using a Delphi methodology.

PubMed

Melchiors, Jacob; Henriksen, Mikael Johannes Vuokko; Dikkers, Frederik G; Gavilán, Javier; Noordzij, J Pieter; Fried, Marvin P; Novakovic, Daniel; Fagan, Johannes; Charabi, Birgitte W; Konge, Lars; von Buchwald, Christian

2018-05-01

Proper training and assessment of skill in flexible pharyngo-laryngoscopy are central in the education of otorhinolaryngologists. To facilitate an evidence-based approach to curriculum development in this field, a structured analysis of what constitutes flexible pharyngo-laryngoscopy is necessary. Our aim was to develop an assessment tool based on this analysis. We conducted an international Delphi study involving experts from twelve countries in five continents. Utilizing reiterative assessment, the panel defined the procedure and reached consensus (defined as 80% agreement) on the phrasing of an assessment tool. FIFTY PANELISTS COMPLETED THE DELPHI PROCESS. THE MEDIAN AGE OF THE PANELISTS WAS 44 YEARS (RANGE 33-64 YEARS). MEDIAN EXPERIENCE IN OTORHINOLARYNGOLOGY WAS 15 YEARS (RANGE 6-35 YEARS). TWENTY-FIVE WERE SPECIALIZED IN LARYNGOLOGY, 16 WERE HEAD AND NECK SURGEONS, AND NINE WERE GENERAL OTORHINOLARYNGOLOGISTS. AN ASSESSMENT TOOL WAS CREATED CONSISTING OF TWELVE DISTINCT ITEMS.: Conclusion The gathering of validity evidence for assessment of core procedural skills within Otorhinolaryngology is central to the development of a competence-based education. The use of an international Delphi panel allows for the creation of an assessment tool which is widely applicable and valid. This work allows for an informed approach to technical skills training for flexible pharyngo-laryngoscopy and as further validity evidence is gathered allows for a valid assessment of clinical performance within this important skillset.

Teaching and assessing resident competence in practice-based learning and improvement.

PubMed

Ogrinc, Greg; Headrick, Linda A; Morrison, Laura J; Foster, Tina

2004-05-01

We designed, implemented, and evaluated a 4-week practice-based learning and improvement (PBLI) elective. Eleven internal medicine residents from 2 separate residency programs participated in the PBLI elective and 22 other residents comprised a comparison group. Residents in each group had similar pretest Quality Improvement Knowledge Application Tool scores; but after the PBLI elective, participant scores were significantly higher. Also, participants' self-assessed ratings of PBLI skills increased after the rotation and remained elevated 6 months afterward. In this curriculum, residents completed a project to improve patient care and demonstrated their knowledge on an evaluation tool in a way that was superior to nonparticipants.

Teaching and Assessing Resident Competence in Practice-based Learning and Improvement

PubMed Central

Ogrinc, Greg; Headrick, Linda A; Morrison, Laura J; Foster, Tina

2004-01-01

We designed, implemented, and evaluated a 4-week practice-based learning and improvement (PBLI) elective. Eleven internal medicine residents from 2 separate residency programs participated in the PBLI elective and 22 other residents comprised a comparison group. Residents in each group had similar pretest Quality Improvement Knowledge Application Tool scores; but after the PBLI elective, participant scores were significantly higher. Also, participants’ self-assessed ratings of PBLI skills increased after the rotation and remained elevated 6 months afterward. In this curriculum, residents completed a project to improve patient care and demonstrated their knowledge on an evaluation tool in a way that was superior to nonparticipants. PMID:15109311

Reliability, Validity, and Factor Structure of the Current Assessment Practice Evaluation-Revised (CAPER) in a National Sample.

PubMed

Lyon, Aaron R; Pullmann, Michael D; Dorsey, Shannon; Martin, Prerna; Grigore, Alexandra A; Becker, Emily M; Jensen-Doss, Amanda

2018-05-11

Measurement-based care (MBC) is an increasingly popular, evidence-based practice, but there are no tools with established psychometrics to evaluate clinician use of MBC practices in mental health service delivery. The current study evaluated the reliability, validity, and factor structure of scores generated from a brief, standardized tool to measure MBC practices, the Current Assessment Practice Evaluation-Revised (CAPER). Survey data from a national sample of 479 mental health clinicians were used to conduct exploratory and confirmatory factor analyses, as well as reliability and validity analyses (e.g., relationships between CAPER subscales and clinician MBC attitudes). Analyses revealed competing two- and three-factor models. Regardless of the model used, scores from CAPER subscales demonstrated good reliability and convergent and divergent validity with MBC attitudes in the expected directions. The CAPER appears to be a psychometrically sound tool for assessing clinician MBC practices. Future directions for development and application of the tool are discussed.

Replication-Competent Influenza A Viruses Expressing Reporter Genes.

PubMed

Breen, Michael; Nogales, Aitor; Baker, Steven F; Martínez-Sobrido, Luis

2016-06-23

Influenza A viruses (IAV) cause annual seasonal human respiratory disease epidemics. In addition, IAV have been implicated in occasional pandemics with inordinate health and economic consequences. Studying IAV, in vitro or in vivo, requires the use of laborious secondary methodologies to identify virus-infected cells. To circumvent this requirement, replication-competent IAV expressing an easily traceable reporter protein can be used. Here we discuss the development and applications of recombinant replication-competent IAV harboring diverse fluorescent or bioluminescent reporter genes in different locations of the viral genome. These viruses have been employed for in vitro and in vivo studies, such as the screening of neutralizing antibodies or antiviral compounds, the identification of host factors involved in viral replication, cell tropism, the development of vaccines, or the assessment of viral infection dynamics. In summary, reporter-expressing, replicating-competent IAV represent a powerful tool for the study of IAV both in vitro and in vivo.

Replication-Competent Influenza A Viruses Expressing Reporter Genes

PubMed Central

Breen, Michael; Nogales, Aitor; Baker, Steven F.; Martínez-Sobrido, Luis

2016-01-01

Influenza A viruses (IAV) cause annual seasonal human respiratory disease epidemics. In addition, IAV have been implicated in occasional pandemics with inordinate health and economic consequences. Studying IAV, in vitro or in vivo, requires the use of laborious secondary methodologies to identify virus-infected cells. To circumvent this requirement, replication-competent IAV expressing an easily traceable reporter protein can be used. Here we discuss the development and applications of recombinant replication-competent IAV harboring diverse fluorescent or bioluminescent reporter genes in different locations of the viral genome. These viruses have been employed for in vitro and in vivo studies, such as the screening of neutralizing antibodies or antiviral compounds, the identification of host factors involved in viral replication, cell tropism, the development of vaccines, or the assessment of viral infection dynamics. In summary, reporter-expressing, replicating-competent IAV represent a powerful tool for the study of IAV both in vitro and in vivo. PMID:27347991

Tracking Patient Encounters and Clinical Skills to Determine Competency in Ambulatory Care Advanced Pharmacy Practice Experiences

PubMed Central

Pereira, Chrystian R.; Harris, Ila M.; Moon, Jean Y.; Westberg, Sarah M.; Kolar, Claire

2016-01-01

Objective. To determine if the amount of exposure to patient encounters and clinical skills correlates to student clinical competency on ambulatory care advanced pharmacy practice experiences (APPEs). Design. Students in ambulatory care APPEs tracked the number of patients encountered by medical condition and the number of patient care skills performed. At the end of the APPE, preceptors evaluated students’ competency for each medical condition and skill, referencing the Dreyfus model for skill acquisition. Assessment. Data was collected from September 2012 through August 2014. Forty-six responses from a student tracking tool were matched to preceptor ratings. Students rated as competent saw more patients and performed more skills overall. Preceptors noted minimal impact on workload. Conclusions. Increased exposure to patient encounters and skills performed had a positive association with higher Dreyfus stage, which may represent a starting point in the conversation for more thoughtful design of ambulatory care APPEs. PMID:26941440

Teaching cultural competence using an exemplar from literary journalism.

PubMed

Anderson, Kathryn L

2004-06-01

Fadiman's work of literary journalism, The Spirit Catches You and You Fall Down, was used as a case study to teach transcultural and other nursing concepts to undergraduate nursing students. Campinha-Bacote's model of cultural competence was used to organize transcultural nursing concepts in the course. Before and after the course, students completed assessments consisting of two cultural attitude questionnaires and a paper describing a personal experience with adherence and failure to adhere by a Mexican American client. After reading Fadiman's book and completing several short writing assignments examining key course concepts, student scores on the questionnaires were mostly unchanged. However, students demonstrated growth in cultural awareness and skill in their "after" papers. Results suggest that valid, reliable tools are needed to detect changes in cultural competence. Qualitative data suggest that students can begin the process of becoming culturally competent through the creative use of literature in nursing education.

How prepared are transition-age deaf and hard of hearing students for adult living? Results of the transition competence battery.

PubMed

Luft, Pamela; Huff, Kelly

2011-01-01

One result of the Individuals With Disabilities Education Act and related legislation is that most deaf and hard of hearing (DHH) students attend local public schools. Although such placements may provide greater access to general education classrooms and curriculum, DHH students' specialized needs are less likely to be addressed. Using the Transition Competence Battery (TCB; Reiman, Bullis, & Davis, 1993), the researchers examined the transition strengths and needs of 53 middle and high school DHH students attending public schools. It was found that the students had substantial transition competence deficits and that none reached the recommended competence levels on more than 4 of the 6 subtests. The TCB is an important transition tool that fulfills requirements under the 2004 Individuals With Disabilities Education Improvement Act Amendments to use age-appropriate and results-oriented transition assessments that document program and intervention outcomes.

Competence to make treatment decisions in anorexia nervosa: thinking processes and values

PubMed Central

Tan, Dr. Jacinta O. A.; Hope, Professor Tony; Stewart, Dr. Anne; Fitzpatrick, Professor Raymond

2005-01-01

This paper explores the ethical and conceptual implications of the findings from an empirical study of decision-making capacity in anorexia nervosa. In the study, ten female patients aged 13 to 21 years with a diagnosis of anorexia nervosa, and eight sets of parents, took part in semi-structured interviews. The purpose of the interviews was to identify aspects of thinking that might be relevant to the issue of competence to refuse treatment. All the patient participants were also tested using the MacCAT-T test of competence. This is a formalised, structured interviewer-administered test of competence, which is a widely accepted clinical tool for determining capacity. The young women also completed five brief self-administered questionnaires to assess their levels of psychopathology. The issues identified from the interviews are described under two headings: difficulties with thought processing, and changes in values. The results suggest that competence to refuse treatment may be compromised in people with anorexia nervosa in ways that are not captured by traditional legal approaches or current standardised tests of competence. PMID:18066393

Procedure-specific assessment tool for flexible pharyngo-laryngoscopy: gathering validity evidence and setting pass-fail standards.

PubMed

Melchiors, Jacob; Petersen, K; Todsen, T; Bohr, A; Konge, Lars; von Buchwald, Christian

2018-06-01

The attainment of specific identifiable competencies is the primary measure of progress in the modern medical education system. The system, therefore, requires a method for accurately assessing competence to be feasible. Evidence of validity needs to be gathered before an assessment tool can be implemented in the training and assessment of physicians. This evidence of validity must according to the contemporary theory on validity be gathered from specific sources in a structured and rigorous manner. The flexible pharyngo-laryngoscopy (FPL) is central to the otorhinolaryngologist. We aim to evaluate the flexible pharyngo-laryngoscopy assessment tool (FLEXPAT) created in a previous study and to establish a pass-fail level for proficiency. Eighteen physicians with different levels of experience (novices, intermediates, and experienced) were recruited to the study. Each performed an FPL on two patients. These procedures were video recorded, blinded, and assessed by two specialists. The score was expressed as the percentage of a possible max score. Cronbach's α was used to analyze internal consistency of the data, and a generalizability analysis was performed. The scores of the three different groups were explored, and a pass-fail level was determined using the contrasting groups' standard setting method. Internal consistency was strong with a Cronbach's α of 0.86. We found a generalizability coefficient of 0.72 sufficient for moderate stakes assessment. We found a significant difference between the novice and experienced groups (p < 0.001) and strong correlation between experience and score (Pearson's r = 0.75). The pass/fail level was established at 72% of the maximum score. Applying this pass-fail level in the test population resulted in half of the intermediary group receiving a failing score. We gathered validity evidence for the FLEXPAT according to the contemporary framework as described by Messick. Our results support a claim of validity and are comparable to other studies exploring clinical assessment tools. The high rate of physicians underperforming in the intermediary group demonstrates the need for continued educational intervention. Based on our work, we recommend the use of the FLEXPAT in clinical assessment of FPL and the application of a pass-fail level of 72% for proficiency.

Competing Values Framework: A useful tool to define the predominant culture in a maternity setting in Australia.

PubMed

Adams, Catherine; Dawson, Angela; Foureur, Maralyn

2017-04-01

To identify the predominant culture of an organisation which could then assess readiness for change. An exploratory design using the Competing Values Framework (CVF) as a self-administered survey tool. The Maternity Unit in one Australian metropolitan tertiary referral hospital. All 120 clinicians (100 midwives and 20 obstetricians) employed in the maternity service were invited to participate; 26% responded. The identification of the predominant culture of an organisation to assess readiness for change prior to the implementation of a new policy. The predominant culture of this maternity unit, as described by those who responded to the survey, was one of hierarchy with a focus on rules and regulations and less focus on innovation, flexibility and teamwork. These results suggest that this unit did not have readiness to change. There is value in undertaking preparatory work to gain a better understanding of the characteristics of an organisation prior to designing and implementing change. This understanding can influence additional preliminary work that may be required to increase the readiness for change and therefore increase the opportunity for successful change. The CVF is a useful tool to identify the predominant culture and characteristics of an organisation that could influence the success of change. Copyright © 2016 Australian College of Midwives. All rights reserved.

The National Clinical Assessment Tool for Medical Students in the Emergency Department (NCAT-EM)

PubMed Central

Jung, Julianna; Franzen, Douglas; Lawson, Luan; Manthey, David; Tews, Matthew; Dubosh, Nicole; Fisher, Jonathan; Haughey, Marianne; House, Joseph B.; Trainor, Arleigh; Wald, David A.; Hiller, Katherine

2018-01-01

Introduction Clinical assessment of medical students in emergency medicine (EM) clerkships is a highly variable process that presents unique challenges and opportunities. Currently, clerkship directors use institution-specific tools with unproven validity and reliability that may or may not address competencies valued most highly in the EM setting. Standardization of assessment practices and development of a common, valid, specialty-specific tool would benefit EM educators and students. Methods A two-day national consensus conference was held in March 2016 in the Clerkship Directors in Emergency Medicine (CDEM) track at the Council of Residency Directors in Emergency Medicine (CORD) Academic Assembly in Nashville, TN. The goal of this conference was to standardize assessment practices and to create a national clinical assessment tool for use in EM clerkships across the country. Conference leaders synthesized the literature, articulated major themes and questions pertinent to clinical assessment of students in EM, clarified the issues, and outlined the consensus-building process prior to consensus-building activities. Results The first day of the conference was dedicated to developing consensus on these key themes in clinical assessment. The second day of the conference was dedicated to discussing and voting on proposed domains to be included in the national clinical assessment tool. A modified Delphi process was initiated after the conference to reconcile questions and items that did not reach an a priori level of consensus. Conclusion The final tool, the National Clinical Assessment Tool for Medical Students in Emergency Medicine (NCAT-EM) is presented here. PMID:29383058

Questionnaire-based evaluation of everyday competence in older adults.

PubMed

Kalisch, Tobias; Richter, Julia; Lenz, Melanie; Kattenstroth, Jan-Christoph; Kolankowska, Izabela; Tegenthoff, Martin; Dinse, Hubert R

2011-01-01

Gerontological research aims at understanding factors that are crucial for mediating "successful aging". This term denotes the absence of significant disease and disabilities, maintenance of high levels of physical and cognitive function, and preservation of social and productive activities. Preservation of an active lifestyle is considered an effective means through which everyday competence can be attained. In this context, it is crucial to obtain ratings of modern day older adults' everyday competence by means of appropriate assessments. Here, we introduce the Everyday Competence Questionnaire (ECQ), designed to assess healthy older adults' everyday competence. The ECQ includes 17 items, covering housekeeping, leisure activities, sports, daily routines, manual skills, subjective well-being, and general linguistic usage. The ECQ was administered to a population of 158 healthy subjects aged 60-91 years, who were divided into groups on the basis of their physical activity. These groups were community-dwelling subjects, those living independently and having a sedentary lifestyle, those living independently but characterized by a general lifestyle without any noteworthy physical activity, and those living independently and exercising regularly. Age, gender, and education levels were balanced between the groups. Using the ECQ, we could identify and distinguish different everyday competence levels between the groups tested: Subjects characterized by an active lifestyle outperformed all other groups. Subjects characterized by a general lifestyle showed higher everyday competence than those with a sedentary lifestyle or subjects who needed care. Furthermore, the ECQ data showed a significant positive correlation between individual physical activity and everyday competence. The ECQ is a novel tool for the questionnaire-based evaluation of everyday competence among healthy subjects. By including leisure activities, it considers the changed living conditions of modern-day older adults.

Online Tobacco Cessation Training and Competency Assessment for Complementary and Alternative Medicine (CAM) Practitioners: Protocol for the CAM Reach Web Study

PubMed Central

Howerter, Amy; Eaves, Emery R; Hall, John R; Buller, David B; Gordon, Judith S

2016-01-01

Background Complementary and alternative medicine (CAM) practitioners, such as chiropractors, acupuncturists, and massage therapists, are a growing presence in the US health care landscape and already provide health and wellness care to significant numbers of patients who use tobacco. For decades, conventional biomedical practitioners have received training to provide evidence-based tobacco cessation brief interventions (BIs) and referrals to cessation services as part of routine clinical care, whereas CAM practitioners have been largely overlooked for BI training. Web-based training has clear potential to meet large-scale training dissemination needs. However, despite the exploding use of Web-based training for health professionals, Web-based evaluation of clinical skills competency remains underdeveloped. Objective In pursuit of a long-term goal of helping CAM practitioners integrate evidence-based practices from US Public Health Service Tobacco Dependence Treatment Guideline into routine clinical care, this pilot protocol aims to develop and test a Web-based tobacco cessation training program tailored for CAM practitioners. Methods In preparation for a larger trial to examine the effect of training on CAM practitioner clinical practice behaviors around tobacco cessation, this developmental study will (1) adapt an existing in-person tobacco cessation BI training program that is specifically tailored for CAM therapists for delivery via the Internet; (2) develop a novel, Web-based tool to assess CAM practitioner competence in tobacco cessation BI skills, and conduct a pilot validation study comparing the competency assessment tool to live video role plays with a standardized patient; (3) pilot test the Web-based training with 120 CAM practitioners (40 acupuncturists, 40 chiropractors, 40 massage therapists) for usability, accessibility, acceptability, and effects on practitioner knowledge, self-efficacy, and competency with tobacco cessation; and (4) conduct qualitative and quantitative formative research on factors influencing practitioner tobacco cessation clinical behaviors (eg, practice environment, peer social influence, and insurance reimbursement). Results Web-training and competency assessment tool development and study enrollment and training activities are complete (N=203 practitioners enrolled). Training completion rates were lower than expected (36.9%, 75/203), necessitating over enrollment to ensure a sufficient number of training completers. Follow-up data collection is in progress. Data analysis will begin immediately after data collection is complete. Conclusions To realize CAM practitioners’ potential to promote tobacco cessation and use of evidence-based treatments, there is a need to know more about the facilitative and inhibitory factors influencing CAM practitioner tobacco intervention behaviors (eg, social influence and insurance reimbursement). Given marked differences between conventional and CAM practitioners, extant knowledge about factors influencing conventional practitioner adoption of tobacco cessation behaviors cannot be confidently extrapolated to CAM practitioners. The potential impact of this study is to expand tobacco cessation and health promotion infrastructure in a new group of health practitioners who can help combat the continuing epidemic of tobacco use. PMID:26740468

Reliable assessment of laparoscopic performance in the operating room using videotape analysis.

PubMed

Chang, Lily; Hogle, Nancy J; Moore, Brianna B; Graham, Mark J; Sinanan, Mika N; Bailey, Robert; Fowler, Dennis L

2007-06-01

The Global Operative Assessment of Laparoscopic Skills (GOALS) is a valid assessment tool for objectively evaluating the technical performance of laparoscopic skills in surgery residents. We hypothesized that GOALS would reliably differentiate between an experienced (expert) and an inexperienced (novice) laparoscopic surgeon (construct validity) based on a blinded videotape review of a laparoscopic cholecystectomy procedure. Ten board-certified surgeons actively engaged in the practice and teaching of laparoscopy reviewed and evaluated the videotaped operative performance of one novice and one expert laparoscopic surgeon using GOALS. Each reviewer recorded a score for both the expert and the novice videotape reviews in each of the 5 domains in GOALS (depth perception, bimanual dexterity, efficiency, tissue handling, and overall competence). The scores for the expert and the novice were compared and statistically analyzed using single-factor analysis of variance (ANOVA). The expert scored significantly higher than the novice did in the domains of depth perception (p = .005), bimanual dexterity (p = .001), efficiency (p = .001), and overall competence ( p = .001). Interrater reliability for the reviewers of the novice tape was Cronbach alpha = .93 and the expert tape was Cronbach alpha = .87. There was no difference between the two for tissue handling. The Global Operative Assessment of Laparoscopic Skills is a valid, objective assessment tool for evaluating technical surgical performance when used to blindly evaluate an intraoperative videotape recording of a laparoscopic procedure.

Motivational interviewing in health care: results of a brief training in endocrinology.

PubMed

Bean, Melanie K; Biskobing, Diane; Francis, Gary L; Wickham, Edmond

2012-09-01

Despite the importance of lifestyle change in disease management and the growing evidence supporting motivational interviewing (MI) as an effective counseling method to promote behavioral change, to date there are few published reports about MI training in graduate medical education. The study aimed to pilot the feasibility and effectiveness of a brief MI training intervention for endocrinology fellows and other providers. We used a pretest/posttest design to evaluate a brief MI training for 5 endocrinology fellows and 9 other providers. All participants completed subjective assessments of perceived confidence and beliefs about behavioral counseling at pretest and posttest. Objective assessment of MI was conducted using fellows' audiotaped patient encounters, which were coded using a validated tool for adherence to MI before and after the training. Paired t tests examined changes in objective and subjective assessments. The training intervention was well received and feasible in the endocrinology setting. At posttest, participants reported increased endorsement of the MI spirit and improved confidence in MI skills. Objective assessment revealed relative improvements in MI skills across several domains. However, most domains, as assessed by a validated tool, did not reach competency level after the training intervention. Although more intensive training may be needed to develop MI competence, the results of our pilot study suggest that brief, targeted MI training has short-term efficacy and is well received by endocrinology fellows and other providers.

Advancing Competency-Based Medical Education: A Charter for Clinician-Educators.

PubMed

Carraccio, Carol; Englander, Robert; Van Melle, Elaine; Ten Cate, Olle; Lockyer, Jocelyn; Chan, Ming-Ka; Frank, Jason R; Snell, Linda S

2016-05-01

The International Competency-Based Medical Education (ICBME) Collaborators have been working since 2009 to promote understanding of competency-based medical education (CBME) and accelerate its uptake worldwide. This article presents a charter, supported by a literature-based rationale, which is meant to provide a shared mental model of CBME that will serve as a path forward in its widespread implementation.At a 2013 summit, the ICBME Collaborators laid the groundwork for this charter. Here, the fundamental principles of CBME and professional responsibilities of medical educators in its implementation process are described. The authors outline three fundamental principles: (1) Medical education must be based on the health needs of the populations served; (2) the primary focus of education and training should be the desired outcomes for learners rather than the structure and process of the educational system; and (3) the formation of a physician should be seamless across the continuum of education, training, and practice.Building on these principles, medical educators must demonstrate commitment to teaching, assessing, and role modeling the range of identified competencies. In the clinical setting, they must provide supervision that balances patient safety with the professional development of learners, being transparent with stakeholders about level of supervision needed. They must use effective and efficient assessment strategies and tools for basing transition decisions on competence rather than time in training, empowering learners to be active participants in their learning and assessment. Finally, advancing CBME requires program evaluation and research, faculty development, and a collaborative approach to realize its full potential.

Developing asynchronous online interprofessional education.

PubMed

Sanborn, Heidi

2016-09-01

For many health programmes, developing interprofessional education (IPE) has been a challenge. Evidence on the best method for design and implementation of IPE has been slow to emerge, with little research on how to best incorporate IPE in the asynchronous online learning environment. This leaves online programmes with no clear guidance when embarking upon an initiative to integrate IPE into the curriculum. One tool that can be effective at guiding the incorporation of IPE across all learning platforms is the Interprofessional Education Collaborative (IPEC) competencies. A project was designed to integrate the nationally defined IPEC competencies throughout an asynchronous, online baccalaureate nursing completion programme. A programme-wide review led to targeted revision of course and unit-level objectives, learning experiences, and assessments based on the IPEC framework. As a result of this effort, the programme curriculum now provides interprofessional learning activities across all courses. This report provides a method for using the IPEC competencies to incorporate IPE within various asynchronous learning assessments, assuring students learn about, with, and from other professions.

Assessing undergraduate nursing students' knowledge, attitudes, and cultural competence in caring for lesbian, gay, bisexual, and transgender patients.

PubMed

Strong, Kristy L; Folse, Victoria N

2015-01-01

Lesbian, gay, bisexual, and transgender (LGBT) patients experience barriers to health care that include fear of discrimination, as well as insensitivity and lack of knowledge about LGBT-specific health needs among providers. This study examined the effectiveness of an educational intervention designed to improve knowledge and attitudes of baccalaureate nursing students regarding LGBT patient care. Education focused on key terminology, health disparities, medical needs of transgender patients, and culturally sensitive communication skills for competent LGBT patient care. Knowledge level and attitudes were evaluated before and after the intervention using a survey based on a modified Attitudes Toward Lesbians and Gay Men Scale and two assessment tools developed for this study. A statistically significant increase in positive attitudes and knowledge level was found immediately after the intervention. Findings from this study support the inclusion of education related to LGBT patient health care in undergraduate nursing curricula to promote cultural competence and sensitivity. Copyright 2015, SLACK Incorporated.

Internationalization Assessment in Schools: Theoretical Contributions and Practical Implications

ERIC Educational Resources Information Center

Yemini, Miri

2012-01-01

Cosmopolitan, international capital has become an integral ingredient in the set of competencies considered to provide a competitive edge and to be required for affective citizenship in the 21st century. Recently, internationalization of education has become a more common phenomenon in local schools around the world, serving as a tool to provide…

Assessing Disease Class-Specific Diagnostic Ability: A Practical Adaptive Test Approach.

ERIC Educational Resources Information Center

Papa, Frank J.; Schumacker, Randall E.

Measures of the robustness of disease class-specific diagnostic concepts could play a central role in training programs designed to assure the development of diagnostic competence. In the pilot study, the authors used disease/sign-symptom conditional probability estimates, Monte Carlo procedures, and artificial intelligence (AI) tools to create…

The Bottomless File Box: Electronic Portfolios for Learning and Evaluation Purposes

ERIC Educational Resources Information Center

Jarrott, Shannon; Gambrel, Laura Eubanks

2011-01-01

Portfolios have been used for the past three decades in higher education for assessment of student competency and also as a reflection tool to assist student learning. Electronic portfolios, or ePortfolios, have additional benefits compared to paper portfolios in that they are easily accessible, portable, and sharable, and they are more…

Relationship between body condition score at calving and reproductive performance in young postpartum cows grazing native range

USDA-ARS?s Scientific Manuscript database

Body condition score is used as a management tool to predict competency of reproduction in beef cows. Therefore, a retrospective study was performed to evaluate association of BCS at calving with subsequent pregnancy rate, days to first estrus, nutrient status (assessed by blood metabolites), and c...

Cultural competent patient-centered nursing care.

PubMed

Darnell, Linda K; Hickson, Shondell V

2015-03-01

This article provides a theoretic framework for culturally diverse practice, provides a model for developing cultural competency, and provides best-practice guidelines for conducting a cultural assessment on patients to identify their diverse needs to integrate into a patient-centered plan of care. The role of ethics is discussed to empower mutual respect, equality, and trust building in patients to promote positive health care outcomes. Cultural diversity tool kits from the National League for Nursing and the American Association of Colleges of Nursing are reviewed to provide educational resources to the front line nurse. Copyright © 2015 Elsevier Inc. All rights reserved.

Anesthesiologists' learning curves for bedside qualitative ultrasound assessment of gastric content: a cohort study.

PubMed

Arzola, Cristian; Carvalho, Jose C A; Cubillos, Javier; Ye, Xiang Y; Perlas, Anahi

2013-08-01

Focused assessment of the gastric antrum by ultrasound is a feasible tool to evaluate the quality of the stomach content. We aimed to determine the amount of training an anesthesiologist would need to achieve competence in the bedside ultrasound technique for qualitative assessment of gastric content. Six anesthesiologists underwent a teaching intervention followed by a formative assessment; then learning curves were constructed. Participants received didactic teaching (reading material, picture library, and lecture) and an interactive hands-on workshop on live models directed by an expert sonographer. The participants were instructed on how to perform a systematic qualitative assessment to diagnose one of three distinct categories of gastric content (empty, clear fluid, solid) in healthy volunteers. Individual learning curves were constructed using the cumulative sum method, and competence was defined as a 90% success rate in a series of ultrasound examinations. A predictive model was further developed based on the entire cohort performance to determine the number of cases required to achieve a 95% success rate. Each anesthesiologist performed 30 ultrasound examinations (a total of 180 assessments), and three of the six participants achieved competence. The average number of cases required to achieve 90% and 95% success rates was estimated to be 24 and 33, respectively. With appropriate training and supervision, it is estimated that anesthesiologists will achieve a 95% success rate in bedside qualitative ultrasound assessment after performing approximately 33 examinations.

Evaluation of a Cultural Competence Intervention with Implications for the Nurse-Patient Encounter

NASA Astrophysics Data System (ADS)

Bradford, Althea Betty

A short-term intervention on participants' knowledge of cultural competence was provided to 38 students in a baccalaureate nursing program at Winston-Salem State University (WSSU). The study examined the effectiveness of this intervention. Although WSSU is a Historically Black University, the majority of students in this program were White. Six tools were used for data collection. The Cultural Competence Survey consisted of 19 Likert Scale items that also gave participants an opportunity to elaborate on each response. Four tools allowed participants to provide written answers to prompts related to cultural competence. The final tool made it possible for the investigator to record impressions and reflections regarding various aspects of the study. Results showed that the students are familiar with cultural competence and want to avoid stereotypical behavior in their nurse-patient encounters. The study suggests a need for education in cultural competence in three areas: 1) accepting that cultural competence is a lifelong endeavor, 2) understanding patients from a holistic perspective, and 3) recognizing that all people have biases; however, the competent nurse is self-aware and has been educated to recognize biased behavior.

[Assessment in Medical Education].

PubMed

Ramírez, Martha Delgado; Gómez-Restrepo, Carlos

2012-01-01

The assessment of medical education is fundamental for proper feedback and evaulation of students. The overall purpose of the evaluation is numerically and formatively revised. The suggested evaluation processes is described taking into account the Miller competence pyramid as a tool for designing evaluations and concepts of validity and reproducibility in assessments. The utility of student and the teacher is also raised. The issue of assessment should leave the knowledge parameters that are usually used. Copyright © 2012 Asociación Colombiana de Psiquiatría. Publicado por Elsevier España. All rights reserved.

Assessing clinical competency in the health sciences

NASA Astrophysics Data System (ADS)

Panzarella, Karen Joanne

To test the success of integrated curricula in schools of health sciences, meaningful measurements of student performance are required to assess clinical competency. This research project analyzed a new performance assessment tool, the Integrated Standardized Patient Examination (ISPE), for assessing clinical competency: specifically, to assess Doctor of Physical Therapy (DPT) students' clinical competence as the ability to integrate basic science knowledge with clinical communication skills. Thirty-four DPT students performed two ISPE cases, one of a patient who sustained a stroke and the other a patient with a herniated lumbar disc. Cases were portrayed by standardized patients (SPs) in a simulated clinical setting. Each case was scored by an expert evaluator in the exam room and then by one investigator and the students themselves via videotape. The SPs scored each student on an overall encounter rubric. Written feedback was obtained from all participants in the study. Acceptable reliability was demonstrated via inter-rater agreement as well as inter-rater correlations on items that used a dichotomous scale, whereas the items requiring the use of the 4-point rubric were somewhat less reliable. For the entire scale both cases had a significant correlation between the Expert-Investigator pair of raters, for the CVA case r = .547, p < .05 and for the HD case r = .700, p < .01. The SPs scored students higher than the other raters. Students' self-assessments were most closely aligned with the investigator. Effects were apparent due to case. Content validity was gathered in the process of developing cases and patient scenarios that were used in this study. Construct validity was obtained from the survey results analyzed from the experts and students. Future studies should examine the effect of rater training upon the reliability. Criterion or predictive validity could be further studied by comparing students' performances on the ISPE with other independent estimates of students' competence. The unique integration questions of the ISPE were judged to have good content validity from experts and students, suggestive that integration, a most crucial element of clinical competence, while done in the mind of the student, can be practiced, learned and assessed.

Development of a Systems Engineering Competency Model Tool for the Aviation and Missile Research, Development, And Engineering Center (AMRDEC)

DTIC Science & Technology

2017-06-01

The Naval Postgraduate School has developed a competency model for the systems engineering profession and is implementing a tool to support high...stakes human resource functions for the U.S. Army. A systems engineering career competency model (SECCM), recently developed by the Navy and verified by...the Office of Personnel Management (OPM), defines the critical competencies for successful performance as a systems engineer at each general schedule

Cultural consultation as a model for training multidisciplinary mental healthcare professionals in cultural competence skills: preliminary results.

PubMed

Owiti, J A; Ajaz, A; Ascoli, M; de Jongh, B; Palinski, A; Bhui, K S

2014-01-01

Lack of cultural competence in care contributes to poor experiences and outcomes from care for migrants and racial and ethnic minorities. As a result, health and social care organizations currently promote cultural competence of their workforce as a means of addressing persistent poor experiences and outcomes. At present, there are unsystematic and diverse ways of promoting cultural competence, and their impact on clinician skills and patient outcomes is unknown. We developed and implemented an innovative model, cultural consultation service (CCS), to promote cultural competence of clinicians and directly improve on patient experiences and outcomes from care. CCS model is an adaptation of the McGill model, which uses ethnographic methodology and medical anthropological knowledge. The method and approach not only contributes both to a broader conceptual and dynamic understanding of culture, but also to learning of cultural competence skills by healthcare professionals. The CCS model demonstrates that multidisciplinary workforce can acquire cultural competence skills better through the clinical encounter, as this promotes integration of learning into day-to-day practice. Results indicate that clinicians developed a broader and patient-centred understanding of culture, and gained skills in narrative-based assessment method, management of complexity of care, competing assumptions and expectations, and clinical cultural formulation. Cultural competence is defined as a set of skills, attitudes and practices that enable the healthcare professionals to deliver high-quality interventions to patients from diverse cultural backgrounds. Improving on the cultural competence skills of the workforce has been promoted as a way of reducing ethnic and racial inequalities in service outcomes. Currently, diverse models for training in cultural competence exist, mostly with no evidence of effect. We established an innovative narrative-based cultural consultation service in an inner-city area to work with community mental health services to improve on patients' outcomes and clinicians' cultural competence skills. We targeted 94 clinicians in four mental health service teams in the community. After initial training sessions, we used a cultural consultation model to facilitate 'in vivo' learning. During cultural consultation, we used an ethnographic interview method to assess patients in the presence of referring clinicians. Clinicians' self-reported measure of cultural competence using the Tool for Assessing Cultural Competence Training (n = 28, at follow-up) and evaluation forms (n = 16) filled at the end of each cultural consultation showed improvement in cultural competence skills. We conclude that cultural consultation model is an innovative way of training clinicians in cultural competence skills through a dynamic interactive process of learning within real clinical encounters. © 2013 John Wiley & Sons Ltd.

Active Learning through a Debate Series in a First-Year Pharmacy Self-Care Course

PubMed Central

Collins, Christine; Danison, Ryan; Lewis, Michelle

2015-01-01

Objective. To evaluate the usefulness of formal debates in the pharmacy classroom as a way to learn course material and as a tool for developing competency in essential skills including critical thinking, communication, public speaking, research methods, and teamwork. Design. Debates were incorporated into a self-care course, where students were assigned different debate topics focused on controversial issues. Quantitative analysis was completed to assess debate style learning, knowledge about the subjects presented, and the impact on necessary skills. Assessment. Quizzes given before and after debates showed up to a 36% improvement in grades and up to a 31% change in opinions on the topic. Students assessed themselves as more competent in the skill sets at the completion of the debate series. Conclusion. Incorporation of debates into didactic style courses offers students an opportunity to improve upon skills that will help them succeed as pharmacists. PMID:25861106

Validity of portfolio assessment: which qualities determine ratings?

PubMed

Driessen, Erik W; Overeem, Karlijn; van Tartwijk, Jan; van der Vleuten, Cees P M; Muijtjens, Arno M M

2006-09-01

The portfolio is becoming increasingly accepted as a valuable tool for learning and assessment. The validity of portfolio assessment, however, may suffer from bias due to irrelevant qualities, such as lay-out and writing style. We examined the possible effects of such qualities in a portfolio programme aimed at stimulating Year 1 medical students to reflect on their professional and personal development. In later curricular years, this portfolio is also used to judge clinical competence. We developed an instrument, the Portfolio Analysis Scoring Inventory, to examine the impact of form and content aspects on portfolio assessment. The Inventory consists of 15 items derived from interviews with experienced mentors, the literature, and the criteria for reflective competence used in the regular portfolio assessment procedure. Forty portfolios, selected from 231 portfolios for which ratings from the regular assessment procedure were available, were rated by 2 researchers, independently, using the Inventory. Regression analysis was used to estimate the correlation between the ratings from the regular assessment and those resulting from the Inventory items. Inter-rater agreement ranged from 0.46 to 0.87. The strongest predictor of the variance in the regular ratings was 'quality of reflection' (R 0.80; R2 66%). No further items accounted for a significant proportion of variance. Irrelevant items, such as writing style and lay-out, had negligible effects. The absence of an impact of irrelevant criteria appears to support the validity of the portfolio assessment procedure. Further studies should examine the portfolio's validity for the assessment of clinical competence.

Researcher readiness for participating in community-engaged dissemination and implementation research: a conceptual framework of core competencies.

PubMed

Shea, Christopher M; Young, Tiffany L; Powell, Byron J; Rohweder, Catherine; Enga, Zoe K; Scott, Jennifer E; Carter-Edwards, Lori; Corbie-Smith, Giselle

2017-09-01

Participating in community-engaged dissemination and implementation (CEDI) research is challenging for a variety of reasons. Currently, there is not specific guidance or a tool available for researchers to assess their readiness to conduct CEDI research. We propose a conceptual framework that identifies detailed competencies for researchers participating in CEDI and maps these competencies to domains. The framework is a necessary step toward developing a CEDI research readiness survey that measures a researcher's attitudes, willingness, and self-reported ability for acquiring the knowledge and performing the behaviors necessary for effective community engagement. The conceptual framework for CEDI competencies was developed by a team of eight faculty and staff affiliated with a university's Clinical and Translational Science Award (CTSA). The authors developed CEDI competencies by identifying the attitudes, knowledge, and behaviors necessary for carrying out commonly accepted CE principles. After collectively developing an initial list of competencies, team members individually mapped each competency to a single domain that provided the best fit. Following the individual mapping, the group held two sessions in which the sorting preferences were shared and discrepancies were discussed until consensus was reached. During this discussion, modifications to wording of competencies and domains were made as needed. The team then engaged five community stakeholders to review and modify the competencies and domains. The CEDI framework consists of 40 competencies organized into nine domains: perceived value of CE in D&I research, introspection and openness, knowledge of community characteristics, appreciation for stakeholder's experience with and attitudes toward research, preparing the partnership for collaborative decision-making, collaborative planning for the research design and goals, communication effectiveness, equitable distribution of resources and credit, and sustaining the partnership. Delineation of CEDI competencies advances the broader CE principles and D&I research goals found in the literature and facilitates development of readiness assessments tied to specific training resources for researchers interested in conducting CEDI research.

Implementing goals for non-cognitive outcomes within a basic science course.

PubMed

Derstine, Pamela L

2002-09-01

An essential principle of competency-based education (CBE) is use of observable outcomes with assessments as judgments of competence based on defined criteria. Faculty are accustomed to using learning objectives as the defining criteria for knowledge, assessing students using written exams. Faculty are less familiar with how the principles of CBE are applied to other competencies. We recently adopted school-wide goals and objectives, modeled after the ACGME Outcomes Project. The present objective was to give faculty first-hand experience in CBE within a basic science course, including both cognitive and non-cognitive outcomes. The format for the learner-centered, first-year Cell and Molecular Biology course was previously described.(1) Course goals were that students: (1) gain an understanding of the principles and concepts of cell and molecular biology, (2) develop an appreciation for how these principles and concepts are important to medicine, (3) demonstrate an ability to think critically using these principles and concepts. Goal 1 was measured by written exams. We assumed goals 2 and 3 were met through small-group problem-solving sessions, and outcomes were not assessed. The revised 2001 course goals were to prepare students for medical knowledge and lifelong learning and communication and professionalism. The goals for medical knowledge and lifelong learning were to: (1) demonstrate ability to use principles and concepts of cell biology, molecular biology, and genetics to analyze medically relevant data, solve problems, make predictions, and determine a course of action; (2) effectively use information technology to search, evaluate, and critically review scientific evidence related to principles and concepts covered in the course; (3) use appropriate techniques to teach peers in a conference setting. The goals for communication and professionalism were to: (1) use appropriate skills and attitudes to collaborate effectively with peers and faculty to accomplish learning goals; (2) maintain a personal learning portfolio to develop habits of reflective learning, broaden understanding of content beyond recall, and enhance communication with faculty; (3) demonstrate personal integrity in meeting course requirements and in interactions with peers and faculty throughout the course. Goals for medical knowledge and lifelong learning were assessed by written exams and by separate tools utilizing four-point Likert scales (novice, advanced beginner, proficient, distinguished) with specific observable criteria for a written research paper and a group PowerPoint presentation. Faculty and student assessments generated a number that was combined with exam grades for a lettered competency grade. A 19-item, five-point Likert scale was used by students to self- and peer-assess goals for communication and professionalism. Small-group faculty facilitators used the tool to give formative feedback midcourse, summative feedback at course conclusion, and competency grades. The tools may be viewed at: . Faculty achieved enhanced understanding of students, assisted by descriptive criteria, while suggesting improvements in forms. Better agreement on criteria definitions and consistency in form use is needed. Students developed understanding and improved communication/professionalism skills, based on repeated exposures to criteria and feedback. It remains to be seen whether the skills are used/developed in other courses. A majority of students did not use the learning portfolio as envisioned. Better design and implementation of school-wide rather than course-specific reflective portfolios may increase use and integrate learning in all courses with all six competencies.

Development of the implant surgical technique and assessment rating system

PubMed Central

Park, Jung-Chul; Hwang, Ji-Wan; Lee, Jung-Seok; Jung, Ui-Won; Choi, Seong-Ho; Cho, Kyoo-Sung; Chai, Jung-Kiu

2012-01-01

Purpose There has been no attempt to establish an objective implant surgical evaluation protocol to assess residents' surgical competence and improve their surgical outcomes. The present study presents a newly developed assessment and rating system and simulation model that can assist the teaching staffs to evaluate the surgical events and surgical skills of residents objectively. Methods Articles published in peer-reviewed English journals were selected using several scientific databases and subsequently reviewed regarding surgical competence and assessment tools. Particularly, medical journals reporting rating and evaluation protocols for various types of medical surgeries were thoroughly analyzed. Based on these studies, an implant surgical technique assessment and rating system (iSTAR) has been developed. Also, a specialized dental typodont was developed for the valid and reliable assessment of surgery. Results The iSTAR consists of two parts including surgical information and task-specific checklists. Specialized simulation model was subsequently produced and can be used in combination with iSTAR. Conclusions The assessment and rating system provided may serve as a reference guide for teaching staffs to evaluate the residents' implant surgical techniques. PMID:22413071

Assessing medical students' performance in core competencies using multiple admission programs for colleges and universities: from the perspective of multi-source feedback.

PubMed

Fang, Ji-Tseng; Ko, Yu-Shien; Chien, Chu-Chun; Yu, Kuang-Hui

2013-01-01

Since 1994, Taiwanese medical universities have employed the multiple application method comprising "recommendations and screening" and "admission application." The purpose of this study is to examine whether medical students admitted using different admission programs gave different performances. To evaluate the six core competencies for medical students proposed by Accreditation Council for Graduate Medical Education (ACGME), this study employed various assessment tools, including student opinion feedback, multi-source feedback (MSF), course grades, and examination results.MSF contains self-assessment scale, peer assessment scale, nursing staff assessment scale, visiting staff assessment scale, and chief resident assessment scale. In the subscales, the CronbachÊs alpha were higher than 0.90, indicating good reliability. Research participants consisted of 182 students from the School of Medicine at Chang Gung University. Regarding studentsÊ average grade for the medical ethics course, the performance of students who were enrolled through school recommendations exceeded that of students who were enrolled through the National College University Entrance Examination (NCUEE) p = 0.011), and all considered "teamwork" as the most important. Different entry pipelines of students in the "communication," "work attitude," "medical knowledge," and "teamwork" assessment scales showed no significant difference. The improvement rate of the students who were enrolled through the school recommendations was better than that of the students who were enrolled through the N CUEE in the "professional skills," "medical core competencies," "communication," and "teamwork" projects of self-assessment and peer assessment scales. However, the students who were enrolled through the NCUEE were better in the "professional skills," "medical core competencies," "communication," and "teamwork" projects of the visiting staff assessment scale and the chief resident assessment scale. Collectively, the performance of the students enrolled through recommendations was slightly better than that of the students enrolled through the NCUEE, although statistical significance was found in certain parts of the grades only.

Development and validation of a collaborative behaviors objective assessment tool for end-of-life communication.

PubMed

Blackhall, Leslie J; Erickson, Jeanne; Brashers, Valentina; Owen, John; Thomas, Shannon

2014-01-01

Helping families make end-of-life decisions requires close collaboration between physicians and nurses. However, medical and nursing students have little formal training in how to collaborate in this task, and few instruments are available to measure collaborative behaviors. The objective of this project was to develop and validate observational assessment tools to measure specific interprofessional competencies in medical and nursing students related to end-of-life discussions. A literature search for evidence-based guidelines and competencies and focus groups with an expert panel of nurses and physicians were used to outline best collaborative practice behaviors for nurses and physicians in an end-of-life decision making simulation. The panel used these practice-behavior checklists to rate videotaped student scenarios and then refined the checklists for validity and clarity until the tools had acceptable inter-rater reliability. The setting was a workshop teaching end-of-life communication to third-year nursing and medical students. Inter-rater reliability was measured using percent agreement and kappa; internal consistency was measured using Cronbach's alpha. Collaborative behaviors objective assessment tools (CBOATs) for nursing and medical students were developed. For the medical CBOAT we found 85% agreement between raters, with an overall kappa of 0.744 and Cronbach's alpha of 0.806. For the nursing CBOAT there was 81% agreement, with a kappa of 0.686 and Cronbach's alpha of 0.845. Development of an end-of-life CBOAT clarified the important collaborative behaviors needed by physician and nurse. The resulting instruments provide a helpful guide for teaching interprofessional sessions related to the end of life and measuring student outcomes using an objective strategy.

Adult Bronchoscopy Training

PubMed Central

Wahidi, Momen M.; Read, Charles A.; Buckley, John D.; Addrizzo-Harris, Doreen J.; Shah, Pallav L.; Herth, Felix J. F.; de Hoyos Parra, Alberto; Ornelas, Joseph; Yarmus, Lonny; Silvestri, Gerard A.

2015-01-01

BACKGROUND: The determination of competency of trainees in programs performing bronchoscopy is quite variable. Some programs provide didactic lectures with hands-on supervision, other programs incorporate advanced simulation centers, whereas others have a checklist approach. Although no single method has been proven best, the variability alone suggests that outcomes are variable. Program directors and certifying bodies need guidance to create standards for training programs. Little well-developed literature on the topic exists. METHODS: To provide credible and trustworthy guidance, rigorous methodology has been applied to create this bronchoscopy consensus training statement. All panelists were vetted and approved by the CHEST Guidelines Oversight Committee. Each topic group drafted questions in a PICO (population, intervention, comparator, outcome) format. MEDLINE data through PubMed and the Cochrane Library were systematically searched. Manual searches also supplemented the searches. All gathered references were screened for consideration based on inclusion criteria, and all statements were designated as an Ungraded Consensus-Based Statement. RESULTS: We suggest that professional societies move from a volume-based certification system to skill acquisition and knowledge-based competency assessment for trainees. Bronchoscopy training programs should incorporate multiple tools, including simulation. We suggest that ongoing quality and process improvement systems be introduced and that certifying agencies move from a volume-based certification system to skill acquisition and knowledge-based competency assessment for trainees. We also suggest that assessment of skill maintenance and improvement in practice be evaluated regularly with ongoing quality and process improvement systems after initial skill acquisition. CONCLUSIONS: The current methods used for bronchoscopy competency in training programs are variable. We suggest that professional societies and certifying agencies move from a volume- based certification system to a standardized skill acquisition and knowledge-based competency assessment for pulmonary and thoracic surgery trainees. PMID:25674901

The national portfolio of learning for postgraduate family medicine training in South Africa: experiences of registrars and supervisors in clinical practice

PubMed Central

2013-01-01

Background In South Africa the submission of a portfolio of learning has become a national requirement for assessment of family medicine training. A national portfolio has been developed, validated and implemented. The aim of this study was to explore registrars’ and supervisors’ experience regarding the portfolio’s educational impact, acceptability, and perceived usefulness for assessment of competence. Methods Semi-structured interviews were conducted with 17 purposively selected registrars and supervisors from all eight South African training programmes. Results The portfolio primarily had an educational impact through making explicit the expectations of registrars and supervisors in the workplace. This impact was tempered by a lack of engagement in the process by registrars and supervisors who also lacked essential skills in reflection, feedback and assessment. The acceptability of the portfolio was limited by service delivery demands, incongruence between the clinical context and educational requirements, design of the logbook and easy availability of the associated tools. The use of the portfolio for formative assessment was strongly supported and appreciated, but was not always happening and in some cases registrars had even organised peer assessment. Respondents were unclear as to how the portfolio would be used for summative assessment. Conclusions The learning portfolio had a significant educational impact in shaping work-place based supervision and training and providing formative assessment. Its acceptability and usefulness as a learning tool should increase over time as supervisors and registrars become more competent in its use. There is a need to clarify how it will be used in summative assessment. PMID:24207009

The national portfolio of learning for postgraduate family medicine training in South Africa: experiences of registrars and supervisors in clinical practice.

PubMed

Jenkins, Louis; Mash, Bob; Derese, Anselme

2013-11-08

In South Africa the submission of a portfolio of learning has become a national requirement for assessment of family medicine training. A national portfolio has been developed, validated and implemented. The aim of this study was to explore registrars' and supervisors' experience regarding the portfolio's educational impact, acceptability, and perceived usefulness for assessment of competence. Semi-structured interviews were conducted with 17 purposively selected registrars and supervisors from all eight South African training programmes. The portfolio primarily had an educational impact through making explicit the expectations of registrars and supervisors in the workplace. This impact was tempered by a lack of engagement in the process by registrars and supervisors who also lacked essential skills in reflection, feedback and assessment. The acceptability of the portfolio was limited by service delivery demands, incongruence between the clinical context and educational requirements, design of the logbook and easy availability of the associated tools. The use of the portfolio for formative assessment was strongly supported and appreciated, but was not always happening and in some cases registrars had even organised peer assessment. Respondents were unclear as to how the portfolio would be used for summative assessment. The learning portfolio had a significant educational impact in shaping work-place based supervision and training and providing formative assessment. Its acceptability and usefulness as a learning tool should increase over time as supervisors and registrars become more competent in its use. There is a need to clarify how it will be used in summative assessment.

What are the necessary practice competencies for two providers: dermal fillers and botulinum toxin type A injections?

PubMed

Spear, Marcia

2010-01-01

There has been a steady increase in the number of individuals who undergo dermal fillers and botulinum toxin Type A injections. The majority of these procedures are performed by nurse providers. The purpose of this study was to collect national data on the current practice among nursing providers within the American Society of Plastic Surgical Nurses (ASPSN). The goal was to utilize the national data and develop a document of the necessary competencies to guide the practice of providers of dermal fillers and botulinum toxin Type A injections. A survey tool was developed and validated for content by expert nursing providers among the membership of the ASPSN and disseminated via e-mail to the membership of the ASPSN. In addition, data from investigator training, mentoring, and evidence from a review of the literature were also incorporated into the competency document utilizing the Competency Outcomes and Performance Assessment (COPA) model. Common core issues became apparent that included contraindications for the use of botulinum toxin Type A and dermal fillers, postprocedure complications as well as strategies in terms of managing complications. The data also revealed that there is no common method providers are taught to assess the aesthetic patient and a lack of a collaborative relationship in current practice. Overwhelmingly, the respondents supported the need for defined practice competencies. A competency document to guide the practice of providers of dermal fillers and botulinum toxin Type A has been developed for completion of this DNP project.

A survey tool for measuring evidence-based decision making capacity in public health agencies.

PubMed

Jacobs, Julie A; Clayton, Paula F; Dove, Cassandra; Funchess, Tanya; Jones, Ellen; Perveen, Ghazala; Skidmore, Brandon; Sutton, Victor; Worthington, Sarah; Baker, Elizabeth A; Deshpande, Anjali D; Brownson, Ross C

2012-03-09

While increasing attention is placed on using evidence-based decision making (EBDM) to improve public health, there is little research assessing the current EBDM capacity of the public health workforce. Public health agencies serve a wide range of populations with varying levels of resources. Our survey tool allows an individual agency to collect data that reflects its unique workforce. Health department leaders and academic researchers collaboratively developed and conducted cross-sectional surveys in Kansas and Mississippi (USA) to assess EBDM capacity. Surveys were delivered to state- and local-level practitioners and community partners working in chronic disease control and prevention. The core component of the surveys was adopted from a previously tested instrument and measured gaps (importance versus availability) in competencies for EBDM in chronic disease. Other survey questions addressed expectations and incentives for using EBDM, self-efficacy in three EBDM skills, and estimates of EBDM within the agency. In both states, participants identified communication with policymakers, use of economic evaluation, and translation of research to practice as top competency gaps. Self-efficacy in developing evidence-based chronic disease control programs was lower than in finding or using data. Public health practitioners estimated that approximately two-thirds of programs in their agency were evidence-based. Mississippi participants indicated that health department leaders' expectations for the use of EBDM was approximately twice that of co-workers' expectations and that the use of EBDM could be increased with training and leadership prioritization. The assessment of EBDM capacity in Kansas and Mississippi built upon previous nationwide findings to identify top gaps in core competencies for EBDM in chronic disease and to estimate a percentage of programs in U.S. health departments that are evidence-based. The survey can serve as a valuable tool for other health departments and non-governmental organizations to assess EBDM capacity within their own workforce and to assist in the identification of approaches that will enhance the uptake of EBDM processes in public health programming and policymaking. Localized survey findings can provide direction for focusing workforce training programs and can indicate the types of incentives and policies that could affect the culture of EBDM in the workplace.

Your District's Got Talent: Technology Tools to Help Keep Good Employees

ERIC Educational Resources Information Center

Turk-Peterson, Cheryl

2009-01-01

Even in these extraordinary times of state budget shortfalls, furlough programs, and layoffs, school districts must still fill positions and compete for top talent among Generation X (those born between 1965 and 1980) and Generation Y (those born in 1981 or later). Every day, school district leaders are challenged to cultivate, assess, and retain…

Transformation Rubric for Engaged Learning: A Tool and Method for Measuring Life-Changing Experiences

ERIC Educational Resources Information Center

Springfield, Emily; Gwozdek, Anne; Smiler, Andrew P.

2015-01-01

ePortfolios and other engaged learning experiences can have extensive impact on students in many facets of their lives, such as subject-area learning, skill and competence development, perspectives on "how the world works," and even students' own identities, confidence, and needs. Assessing these various impacts can be a challenge for…

Errare Humanum Est! A Socio-Psychological Approach to a "Climbing Mount Fuji" PISA Question

ERIC Educational Resources Information Center

Selleri, Patrizia; Carugati, Felice

2018-01-01

There is a consensus that the items proposed by the Program for International Student Assessment (PISA) program allow us to focus on the outcomes of the processes of appropriation and transformation of learning tools at the end of compulsory schooling, particularly regarding the key competencies for lifelong learning and citizenship in digital…

Evaluating the Validity of the Nisonger Child Behavior Rating Form--Parent Version

ERIC Educational Resources Information Center

Norris, Megan; Lecavalier, Luc

2011-01-01

Youth with intellectual and developmental disabilities (IDD) experience high rates of emotional and behavior problems. The Nisonger Child Behavior Rating Form (NCBRF) is one of the few tools developed to assess these problems in this population. It consists of a 10-item Social Competence section and a 66-item Problem Behavior section. The goal of…

Multi-Cultural Competency-Based Vocational Curricula. Machine Trades. Multi-Cultural Competency-Based Vocational/Technical Curricula Series.

ERIC Educational Resources Information Center

Hepburn, Larry; Shin, Masako

This document, one of eight in a multi-cultural competency-based vocational/technical curricula series, is on machine trades. This program is designed to run 36 weeks and cover 6 instructional areas: use of measuring tools; benchwork/tool bit grinding; lathe work; milling work; precision grinding; and combination machine work. A duty-task index…

Fidelity in complex behaviour change interventions: a standardised approach to evaluate intervention integrity

PubMed Central

Mars, Tom; Ellard, David; Carnes, Dawn; Homer, Kate; Underwood, Martin; Taylor, Stephanie J C

2013-01-01

Objectives The aim of this study was to (1) demonstrate the development and testing of tools and procedures designed to monitor and assess the integrity of a complex intervention for chronic pain (COping with persistent Pain, Effectiveness Research into Self-management (COPERS) course); and (2) make recommendations based on our experiences. Design Fidelity assessment of a two-arm randomised controlled trial intervention, assessing the adherence and competence of the facilitators delivering the intervention. Setting The intervention was delivered in the community in two centres in the UK: one inner city and one a mix of rural and urban locations. Participants 403 people with chronic musculoskeletal pain were enrolled in the intervention arm and 300 attended the self-management course. Thirty lay and healthcare professionals were trained and 24 delivered the courses (2 per course). We ran 31 courses for up to 16 people per course and all were audio recorded. Interventions The course was run over three and a half days; facilitators delivered a semistructured manualised course. Outcomes We designed three measures to evaluate fidelity assessing adherence to the manual, competence and overall impression. Results We evaluated a random sample of four components from each course (n=122). The evaluation forms were reliable and had good face validity. There were high levels of adherence in the delivery: overall adherence was two (maximum 2, IQR 1.67–2.00), facilitator competence exhibited more variability, and overall competence was 1.5 (maximum 2, IQR 1.25–2.00). Overall impression was three (maximum 4, IQR 2.00–3.00). Conclusions Monitoring and assessing adherence and competence at the point of intervention delivery can be realised most efficiently by embedding the principles of fidelity measurement within the design stage of complex interventions and the training and assessment of those delivering the intervention. More work is necessary to ensure that more robust systems of fidelity evaluation accompany the growth of complex interventions. Trial Registration ISRCTN No ISRCTN24426731. PMID:24240140

Topography measurements and applications in ballistics and tool mark identifications*

PubMed Central

Vorburger, T V; Song, J; Petraco, N

2016-01-01

The application of surface topography measurement methods to the field of firearm and toolmark analysis is fairly new. The field has been boosted by the development of a number of competing optical methods, which has improved the speed and accuracy of surface topography acquisitions. We describe here some of these measurement methods as well as several analytical methods for assessing similarities and differences among pairs of surfaces. We also provide a few examples of research results to identify cartridge cases originating from the same firearm or tool marks produced by the same tool. Physical standards and issues of traceability are also discussed. PMID:27182440

A model for communication skills assessment across the undergraduate curriculum.

PubMed

Rider, Elizabeth A; Hinrichs, Margaret M; Lown, Beth A

2006-08-01

Physicians' interpersonal and communication skills have a significant impact on patient care and correlate with improved healthcare outcomes. Some studies suggest, however, that communication skills decline during the four years of medical school. Regulatory and other medical organizations, recognizing the importance of interpersonal and communication skills in the practice of medicine, now require competence in communication skills. Two challenges exist: to select a framework of interpersonal and communication skills to teach across undergraduate medical education, and to develop and implement a uniform model for the assessment of these skills. The authors describe a process and model for developing and institutionalizing the assessment of communication skills across the undergraduate curriculum. Consensus was built regarding communication skill competencies by working with course leaders and examination directors, a uniform framework of competencies was selected to both teach and assess communication skills, and the framework was implemented across the Harvard Medical School undergraduate curriculum. The authors adapted an assessment framework based on the Bayer-Fetzer Kalamazoo Consensus Statement adapted a patient and added and satisfaction tool to bring patients' perspectives into the assessment of the learners. The core communication competencies and evaluation instruments were implemented in school-wide courses and assessment exercises including the first-year Patient-Doctor I Clinical Assessment, second-year Objective Structured Clinical Exam (OSCE), third-year Patient-Doctor III Clinical Assessment, fourth-year Comprehensive Clinical Practice Examination and the Core Medicine Clerkships. Faculty were offered workshops and interactive web-based teaching to become familiar with the framework, and students used the framework with repeated opportunities for faculty feedback on these skills. A model is offered for educational leaders and others who are involved in designing assessment in communication skills. By presenting an approach for implementation, the authors hope to provide guidance for the successful integration of communication skills assessment in undergraduate medical education.

A review on adult pragmatic assessments

PubMed Central

Sobhani Rad, Davood

2014-01-01

Pragmatics is defined as appropriate use of language either to comprehend ideas or to interact in social situations effectively. Pragmatic competence, which is processed in the right hemisphere, comprises a number of interrelated skills that manifest in a range of adaptive behaviors. Due to the widespread influence of language in communication, studying pragmatic profiles, by developing appropriate tools, is of importance. Here, a range of pragmatic theories and assessment instruments available for use in adult patients is reviewed. PMID:25422728

Development and pilot testing the Family Conference Rating Scale: A tool aimed to assess interprofessional patient-centred communication and collaboration competencies.

PubMed

Dojeiji, Sue; Byszewski, Anna; Wood, Tim

2015-01-01

There is a paucity of evidence-based literature on the essential communication and collaboration skills to guide health care teams in conducting and assessing their performance in the Family Conference (FC). The authors developed and collected validity evidence for a rating scale of team FC performance, the Family Conference Rating Scale (FCRS). In phase 1, essential FC communication and collaboration skills were identified through a review of existing communication tools and literature on team functioning; a draft 34-item scale was developed. In phase 2, the scale was narrowed to a 6-category, 9-point scale with descriptors of expected behaviours through an iterative process: testing of the scale on 10 FC transcripts by two experts, soliciting feedback from a focus group of seven health care providers, and testing by non-experts on 49 live FCs. In phase 3, scores on the revised scale were validated by 10 health care providers from different disciplines by rating three videos of FCs of variable quality. Raters were able to detect inter-video variation in FC quality. The reliability of the FCRS was 0.95 and the inter-rater reliability, 0.68. The FCRS may enhance the ability of health professions educators to teach and assess interprofessional patient-centred communication and collaboration competencies.

Assessment of Pharmacy Students' Communication Competence Using the Roter Interaction Analysis System During Objective Structured Clinical Examinations

PubMed Central

Kubota, Yoshie; Seki, Susumu; Takada, Kaori; Sakuma, Mio; Morimoto, Takeshi; Akaike, Akinori; Hiraide, Atsushi

2011-01-01

Objective To determine the value of using the Roter Interaction Analysis System during objective structured clinical examinations (OSCEs) to assess pharmacy students' communication competence. Methods As pharmacy students completed a clinical OSCE involving an interview with a simulated patient, 3 experts used a global rating scale to assess students' overall performance in the interview, and both the student's and patient's languages were coded using the Roter Interaction Analysis System (RIAS). The coders recorded the number of utterances (ie, units of spoken language) in each RIAS category. Correlations between the raters' scores and the number and types of utterances were examined. Results There was a significant correlation between students' global rating scores on the OSCE and the number of utterances in the RIAS socio-emotional category but not the RIAS business category. Conclusions The RIAS proved to be a useful tool for assessing the socio-emotional aspect of students' interview skills. PMID:21655397

Immune competence assessment in marine medaka (Orzyias melastigma)-a holistic approach for immunotoxicology.

PubMed

Ye, Roy R; Peterson, Drew R; Seemann, Frauke; Kitamura, Shin-Ichi; Lee, J S; Lau, Terrance C K; Tsui, Stephen K W; Au, Doris W T

2017-12-01

Many anthropogenic pollutants in coastal marine environments can induce immune impairments in wild fish and reduce their survival fitness. There is a pressing need to establish sensitive and high throughput in vivo tools to systematically evaluate the immunosuppressive effects of contaminants in marine teleosts. This study reviewed a battery of in vivo immune function detection technologies established for different biological hierarchies at molecular (immune function pathways and genes by next generation sequencing (NGS)), cellular (leukocytes profiles by flow cytometry), tissues/organ system (whole adult histo-array), and organism (host resistance assays (HRAs)) levels, to assess the immune competence of marine medaka Oryzias melastigma. This approach enables a holistic assessment of fish immune competence under different chemical exposure or environmental scenarios. The data obtained will also be useful to unravel the underlying immunotoxic mechanisms. Intriguingly, NGS analysis of hepatic immune gene expression profiles (male > female) are in support of the bacterial HRA findings, in which infection-induced mortality was consistently higher in females than in males. As such, reproductive stages and gender-specific responses must be taken into consideration when assessing the risk of immunotoxicants in the aquatic environment. The distinct phenotypic sexual dimorphism and short generation time (3 months) of marine medaka offer additional advantages for sex-related immunotoxicological investigation.

Leadership competencies in the context of health services.

PubMed

Jahrami, Haitham; Marnoch, Gordon; Gray, Ann Marie

2008-05-01

In a rapidly changing health-care environment, clinicians are increasingly called upon to assume complex leadership responsibilities. The research was undertaken to develop an understanding of the limits to the conceptual and methodological basis of leadership competency modelling in health services context. Data were collected from all of the clinicians in a Psychiatric Hospital, Bahrain using a researcher-developed questionnaire. Data were gathered to critically assess the validity of the competency-based approach to leadership on the basis of subjects' capacity to discriminate in terms of importance and accomplishment between the items featured in a research tool containing a comprehensive list of 124 leadership competencies. The results of the analyses indicate a weak identification with the competencies in the sense of revealing low levels of discriminatory sophistication on the part of subjects. The study design was limited to participants working in single hospital; therefore, the conclusions made cannot yet be regarded categorically as generalizable. Leadership selection, development and education activities may not achieve their ultimate outcomes due to the subject identification problem associated with the competence approach. It might be necessary to reconsider the efficiency of human resource activities that rely solely on the competency approach. The conceptual basis of leadership competence in health services has been previously neglected. This research casts doubt on competency approaches to leadership if based on subject identification with pre-defined items.

Perinatal nursing education for single-room maternity care: an evaluation of a competency-based model.

PubMed

Janssen, Patricia A; Keen, Lois; Soolsma, Jetty; Seymour, Laurie C; Harris, Susan J; Klein, Michael C; Reime, Birgit

2005-01-01

To evaluate the success of a competency-based nursing orientation programme for a single-room maternity care unit by measuring improvement in self-reported competency after six months. Single-room maternity care has challenged obstetrical nurses to provide comprehensive nursing care during all phases of the in-hospital birth experience. In this model, nurses provide intrapartum, postpartum and newborn care in one room. To date, an evaluation of nursing education for single-room maternity care has not been published. A prospective cohort design comparing self-reported competencies prior to starting work in the single-room maternity care and six months after. Nurses completed a competency-based education programme in which they could select from a menu of learning methods and content areas according to their individual needs. Learning methods included classroom lectures, self-paced learning packages, and preceptorships in the clinical area. Competencies were measured by a standardized perinatal self-efficacy tool and a tool developed by the authors for this study, the Single-Room Maternity Care Competency Tool. A paired analysis was undertaken to take into account the paired (before and after) nature of the design. Scores on the perinatal self-efficacy scale and the single-room maternity care competency tool were improved. These differences were statistically significant. Improvements in perinatal and single-room maternity care-specific competencies suggest that our education programme was successful in preparing nurses for their new role in the single-room maternity care setting. This conclusion is supported by reported increases in nursing and patient satisfaction in the single-room maternity care compared with the traditional labour/delivery and postpartum settings. An education programme tailored to the learning needs of experienced clinical nurses contributes to improvements in nursing competencies and patient care.

Institutional capacity for health systems research in East and Central Africa schools of public health: enhancing capacity to design and implement teaching programs.

PubMed

Nangami, Mabel N; Rugema, Lawrence; Tebeje, Bosena; Mukose, Aggrey

2014-06-02

The role of health systems research (HSR) in informing and guiding national programs and policies has been increasingly recognized. Yet, many universities in sub-Saharan African countries have relatively limited capacity to teach HSR. Seven schools of public health (SPHs) in East and Central Africa undertook an HSR institutional capacity assessment, which included a review of current HSR teaching programs. This study determines the extent to which SPHs are engaged in teaching HSR-relevant courses and assessing their capacities to effectively design and implement HSR curricula whose graduates are equipped to address HSR needs while helping to strengthen public health policy. This study used a cross-sectional study design employing both quantitative and qualitative approaches. An organizational profile tool was administered to senior staff across the seven SPHs to assess existing teaching programs. A self-assessment tool included nine questions relevant to teaching capacity for HSR curricula. The analysis triangulates the data, with reflections on the responses from within and across the seven SPHs. Proportions and average of values from the Likert scale are compared to determine strengths and weaknesses, while themes relevant to the objectives are identified and clustered to elicit in-depth interpretation. None of the SPHs offer an HSR-specific degree program; however, all seven offer courses in the Master of Public Health (MPH) degree that are relevant to HSR. The general MPH curricula partially embrace principles of competency-based education. Different strengths in curricula design and staff interest in HSR at each SPH were exhibited but a number of common constraints were identified, including out-of-date curricula, face-to-face delivery approaches, inadequate staff competencies, and limited access to materials. Opportunities to align health system priorities to teaching programs include existing networks. Each SPH has key strengths that can be leveraged to design and implement HSR teaching curricula. We propose networking for standardizing HSR curricula competencies, institutionalizing sharing of teaching resources, creating an HSR eLearning platform to expand access, regularly reviewing HSR teaching content to infuse competency-based approaches, and strengthening staff capacity to deliver such curricula.

Institutional capacity for health systems research in East and Central Africa schools of public health: enhancing capacity to design and implement teaching programs

PubMed Central

2014-01-01

Background The role of health systems research (HSR) in informing and guiding national programs and policies has been increasingly recognized. Yet, many universities in sub-Saharan African countries have relatively limited capacity to teach HSR. Seven schools of public health (SPHs) in East and Central Africa undertook an HSR institutional capacity assessment, which included a review of current HSR teaching programs. This study determines the extent to which SPHs are engaged in teaching HSR-relevant courses and assessing their capacities to effectively design and implement HSR curricula whose graduates are equipped to address HSR needs while helping to strengthen public health policy. Methods This study used a cross-sectional study design employing both quantitative and qualitative approaches. An organizational profile tool was administered to senior staff across the seven SPHs to assess existing teaching programs. A self-assessment tool included nine questions relevant to teaching capacity for HSR curricula. The analysis triangulates the data, with reflections on the responses from within and across the seven SPHs. Proportions and average of values from the Likert scale are compared to determine strengths and weaknesses, while themes relevant to the objectives are identified and clustered to elicit in-depth interpretation. Results None of the SPHs offer an HSR-specific degree program; however, all seven offer courses in the Master of Public Health (MPH) degree that are relevant to HSR. The general MPH curricula partially embrace principles of competency-based education. Different strengths in curricula design and staff interest in HSR at each SPH were exhibited but a number of common constraints were identified, including out-of-date curricula, face-to-face delivery approaches, inadequate staff competencies, and limited access to materials. Opportunities to align health system priorities to teaching programs include existing networks. Conclusions Each SPH has key strengths that can be leveraged to design and implement HSR teaching curricula. We propose networking for standardizing HSR curricula competencies, institutionalizing sharing of teaching resources, creating an HSR eLearning platform to expand access, regularly reviewing HSR teaching content to infuse competency-based approaches, and strengthening staff capacity to deliver such curricula. PMID:24888353

Psychometric properties of the Persian version of the Intensive and Critical Care Nursing Competence Scale version-1 (ICCN-CS-1).

PubMed

Shouryabi, Ali Asghar; Ghahrisarabi, Alireza; Anboohi, Sima Zohari; Nasiri, Malihe; Rassouli, Maryam

2017-11-01

Nursing competence is highly related to patient outcomes and patient safety issues, especially in intensive care units. Competence assessment tools are needed specifically for intensive care nursing. This study was performed to determine psychometric properties of the Intensive and Critical Care Nursing Competence Scale version-1 between Iranian Nurses. The present study was a methodological research in which 289 nurses of Intensive Care Units from nine hospitals in Shahid Beheshti University of Medical Sciences in Tehran were selected between 2015 and 2016. The original version of the scale was translated into Persian and back-translated into English, and the comments of the developer were applied. The validity of the scale was the determined quality (content validity and face validity) and quantity (confirmatory factor analysis). Reliability of the scale was reported by Cronbach's alpha coefficient and Intra class Correlation Coefficient. SPSS-PC (v.21) and LISREL (v.8.5) were used to analyze the data. The intensive and critical care nursing competence scale version-1 is a self-assessment test that consists of 144 items and four domains which are the knowledge base, the skill base, the attitudes and values base and the experience base, which are divided into clinical competence and professional competence. Content and face validity was confirmed by 10 experts and 10 practitioner nurses in the intensive care units. In confirmatory factor analysis, all fitness indexes, except goodness of fit index (0.64), confirmed the four-factor structure of the ICCN-CS-1. The results of the factor analysis, load factor between 0.304 and 0.727 items was estimated; only 4 items out of 144 items, that were loaded were less than 0.3 due to high Cronbach's alpha coefficient (0.984-0.986), all items were preserved, no item was removed and 4 subscales of the original scale were confirmed. The results of this study indicated that the Persian version of "The Intensive and Critical Care Nursing Competence Scale version-1" is a valid and reliable scale for the assessment of competency among Iranian nurses, and it can be used as a reliable scale in nursing management, education and research.

Undergraduate nursing students' perspectives on clinical assessment at transition to practice.

PubMed

Wu, Xi Vivien; Wang, Wenru; Pua, Lay Hoon; Heng, Doreen Gek Noi; Enskär, Karin

2015-01-01

Assessment of clinical competence requires explicitly defined standards meeting the national standards of the nursing profession. This is a complex process because of the diverse nature of nursing practice. To explore the perceptions of final-year undergraduate nursing students regarding clinical assessment at transition to practice. An exploratory qualitative approach was adopted. Twenty-four students participated in three focus group discussions. Thematic analysis was conducted. Five themes emerged: the need for a valid and reliable clinical assessment tool, the need for a flexible style of reflection and specific feedback, the dynamic clinical learning environment, students' efforts in learning and assessment, and the unclear support system for preceptors. Workload, time, resource availability, adequate preparation of preceptors, and the provision of valid and reliable clinical assessment tools were deemed to influence the quality of students' clinical learning and assessment. Nursing leadership in hospitals and educational institutions has a joint responsibility in shaping the clinical learning environment and providing clinical assessments for the students.

Competing values in healthcare: balancing the (un)balanced scorecard.

PubMed

Wicks, Angela M; St Clair, Lynda

2007-01-01

Facing a complex environment driven by two decades of dramatic change, healthcare organizations are adopting new strategic frameworks such as the Balanced Scorecard (BSC) to evaluate performance (Kaplan and Norton 1992). The BSC was not originally developed as a performance management tool, however. Rather, it was designed as a tool to communicate strategy and, as such, provides little guidance when actual outcomes fall short of desired outcomes. In addition, although the BSC is an improvement over exclusively financial measures, it has three conceptual limitations that are especially problematic for evaluating healthcare organizations: (1) it underemphasizes the employee perspective, (2) it is founded on a control-based management philosophy, and (3) it emphasizes making trade-offs. To address these limitations, we propose using the Competing Values Framework (CVF), a theoretically grounded, comprehensive approach to understanding and improving organizational and managerial performance by focusing on four action imperatives: competing, controlling, collaborating, and creating. The CVF pays particular attention to the employee perspective, is consistent with a commitment-based management philosophy, and emphasizes transcending apparent paradoxes to identify win-win solutions. Rather than focusing on customer satisfaction or employee satisfaction, the CVF looks for ways to satisfy customers and employees while still addressing financial constraints and growth opportunities. The CVF also can be used to assess both the culture of the organization and the competencies of individual managers, thereby providing a clear link between strategy and implementation.

Assessing capacity to consent for research in cognitively impaired older patients

PubMed Central

Gilbert, Thomas; Bosquet, Antoine; Thomas-Antérion, Catherine; Bonnefoy, Marc; Le Saux, Olivia

2017-01-01

Background The number of clinical trials including older patients, and particularly patients with cognitive impairment, is increasing. While statutory provisions exist to make sure that the capacity to consent is assessed systematically for each patient, many gray areas remain with regard to how this assessment is made or should be made in the routine practice of clinical research. Objectives The aim of this review was to draw up an inventory of assessment tools evaluating older patients’ capacity to consent specifically applicable to clinical research, which could be used in routine practice. Methods Two authors independently searched PubMed, Cochrane, and Google Scholar data-bases between November 2015 and January 2016. The search was actualized in April 2017. We used keywords (MeSH terms and text words) referring to informed consent, capacity to consent, consent for research, research ethics, cognitive impairment, vulnerable older patients, and assessment tools. Existing reviews were also considered. Results Among the numerous existing tools for assessing capacity to consent, 14 seemed potentially suited for clinical research and six were evaluated in older patients. The MacArthur Competence Assessment Tool for Clinical Research (MacCAT-CR) was the most frequently cited. Conclusion The MacCAT-CR is currently the most used and the best validated questionnaire. However, it appears difficult to use and time-consuming. A more recent tool, the University of California Brief Assessment of Capacity to Consent (UBACC), seems interesting for routine practice because of its simplicity, relevance, and applicability in older patients. PMID:29026293

Exploring the cultural competence of undergraduate nursing students in Saudi Arabia.

PubMed

Halabi, Jehad O; de Beer, Jennifer

2018-03-01

To explore the cultural competence of undergraduate nursing students at a college of nursing, Saudi Arabia. A descriptive exploratory design was used to explore the Saudi undergraduate nursing students' level of cultural competency. The convenience sample included 205 nursing students affiliated with a college of nursing at a health science university in Jeddah, Saudi Arabia. Data was collected using the Inventory for Assessing the Process of Cultural Competence-Revised (IAPCC-R) consisting of 25 items. The tool reported acceptable reliability of Cronbach alpha 0.89. The majority of students were culturally aware and dealt with people from different cultures. One-third preferred to have training on culture over a period of time. Half the students preferred studying a special course related to working with people from different cultures. Cultural desire reported the highest mean while cultural knowledge scored the lowest among the cultural competence subscales despite students being exposed to some cultural knowledge content in their training. Implementing the guidelines for culturally competent care assure covering all aspects of care with consideration of cultural heritage as a main concept. Comparative study of nurses' and students' perception is further recommended. Copyright © 2017 Elsevier Ltd. All rights reserved.

Nursing and conflict communication: avoidance as preferred strategy.

PubMed

Mahon, Margaret M; Nicotera, Anne M

2011-01-01

An exploratory study was conducted to examine nurses' (n = 57) selection of strategies to confront conflict in the workplace. Communication competence is the conceptual framework, defining competent conflict communication as joint problem-solving communication that is both effective and appropriate. Items were drawn from tools assessing nurses' conflict management strategies. Nurses reported a strong preference not to confront conflict directly; nurse managers were less likely to avoid direct communication. Nurses who do choose to confront conflict are more likely to use constructive than destructive strategies. The integration of the social science of health communication into nursing education and practice and other implications are discussed.

Motivational Interviewing in Health Care: Results of a Brief Training in Endocrinology

PubMed Central

Bean, Melanie K.; Biskobing, Diane; Francis, Gary L.; Wickham, Edmond

2012-01-01

Background Despite the importance of lifestyle change in disease management and the growing evidence supporting motivational interviewing (MI) as an effective counseling method to promote behavioral change, to date there are few published reports about MI training in graduate medical education. Objective The study aimed to pilot the feasibility and effectiveness of a brief MI training intervention for endocrinology fellows and other providers. Methods We used a pretest/posttest design to evaluate a brief MI training for 5 endocrinology fellows and 9 other providers. All participants completed subjective assessments of perceived confidence and beliefs about behavioral counseling at pretest and posttest. Objective assessment of MI was conducted using fellows' audiotaped patient encounters, which were coded using a validated tool for adherence to MI before and after the training. Paired t tests examined changes in objective and subjective assessments. Results The training intervention was well received and feasible in the endocrinology setting. At posttest, participants reported increased endorsement of the MI spirit and improved confidence in MI skills. Objective assessment revealed relative improvements in MI skills across several domains. However, most domains, as assessed by a validated tool, did not reach competency level after the training intervention. Conclusions Although more intensive training may be needed to develop MI competence, the results of our pilot study suggest that brief, targeted MI training has short-term efficacy and is well received by endocrinology fellows and other providers. PMID:23997882

[French national survey on neurology resident training].

PubMed

Weiss, N; Guiraud, V; Zuber, M; Touzé, E

2009-01-01

Over the past 20 years, the neurology specialty has changed because of the explosion of information and new treatment modalities that became available, and has consequently become more complex and diversified. The satisfaction of residents concerning their training and the competencies that they acquire has never been thoroughly assessed in France. We conducted a national survey in order to assess (1) the methods for training and validation; (2) the level of knowledge that residents perceived to have acquired in different domains; and (3) their satisfaction towards training and their wishes. One hundred and eight residents replied to the survey. The main sources of training were local teaching (74%), personal work (61%), scientific (57%) and didactic (54%) papers. Residents seemed unable to acquire knowledge on all domains of the curriculum established at a national level, particularly for neurophysiology, neuropsychology, comatose state and sleep disorders, oncology and psychiatry. Even postgraduate year four residents were not fully competent with several technical tools, particularly neurophysiological tests. Fifty eight percent of residents were satisfied with their training, but 16% were not and 26% were half-hearted. Overall the residents were in favor of more standardization in their training, national-level certification, and would be keen on having access to clinics and the use of a follow-up monitoring chart. Our results suggest that it would be useful to better identify the competencies that every neurologist should acquire and to use appropriate tools to reach these objectives.

Development and validation of a questionnaire for evaluation of students' attitudes towards family medicine.

PubMed

Šter, Marija Petek; Švab, Igor; Klemenc-Ketiš, Zalika; Kersnik, Janko

2015-03-01

The development of the EURACT (European Academy of Teachers in General Practice) Educational Agenda helped many family medicine departments in development of clerkship and the aims and objectives of family medicine teaching. Our aims were to develop and validate a tool for assessment of students' attitudes towards family medicine and to evaluate the impact of the clerkship on students' attitudes regarding the competences of family doctor. In the pilot study, experienced family doctors were asked to describe their attitudes towards family medicine by using the Educational Agenda as a template for brainstorming. The statements were paraphrased and developed into a 164-items questionnaire, which was administered to 176 final-year students in academic year 2007/08. The third phase consisted of development of a final tool using statistical analysis, which resulted in the 60-items questionnaire in six domains which was used for the evaluation of students' attitudes. At the beginning of the clerkship, person-centred care and holistic approach scored lower than the other competences. Students' attitudes regarding the competences at the end of 7 weeks clerkship in family medicine were more positive, with exception of the competence regarding primary care management. The students who named family medicine as his or her future career choice, found holistic approach as more important than the students who did not name it as their future career. With the decision tree, which included students' attitudes to the competences of family medicine, we can successfully predict the future career choice in family medicine in 93.5% of the students. This study reports on the first attempt to develop a valid and reliable tool for measuring attitudes towards family medicine based on EURACT Educational Agenda. The questionnaire could be used for evaluating changes of students' attitudes in undergraduate curricula and for prediction of students' preferences regarding their future professional career in family medicine.

Assessing Procedural Competence: Validity Considerations.

PubMed

Pugh, Debra M; Wood, Timothy J; Boulet, John R

2015-10-01

Simulation-based medical education (SBME) offers opportunities for trainees to learn how to perform procedures and to be assessed in a safe environment. However, SBME research studies often lack robust evidence to support the validity of the interpretation of the results obtained from tools used to assess trainees' skills. The purpose of this paper is to describe how a validity framework can be applied when reporting and interpreting the results of a simulation-based assessment of skills related to performing procedures. The authors discuss various sources of validity evidence because they relate to SBME. A case study is presented.

An educational tool for teaching medication history taking to pharmacy students.

PubMed

Sando, Karen R; Elliott, Jennifer; Stanton, Melonie L; Doty, Randell

2013-06-12

To implement and evaluate the use of a situated-learning experience to prepare second-year pharmacy students to conduct medication history interviews in preparation for introductory pharmacy practice experiences (IPPE) at ambulatory clinic sites. Second-year doctor of pharmacy (PharmD) students (n=200) used the Medication Mysteries Infinite Case Tool, a game-like educational tool in which groups of 3 students assumed the roles of pharmacist, patient, and observer and rolled a die and drew cards to determine the drugs, patient personality, medication problems, and other variables that guided a medication history taking session. After the laboratory session, faculty members assessed students' medication history-taking skills. One hundred sixteen (58%) and 78 (39%) of 200 students achieved excellence or competence, respectively, on the final assessment. Two weeks after the assessment, 53 of 200 (26.5%) students completed a survey instrument. The respondents indicated that their self-confidence in conducting medical history taking significantly improved following completion of the learning experience. Using the Medication Mysteries Infinite Case Tool increased students' confidence and skills in conducting medication history taking prior to their clinical IPPE experience.

Stone tool production and utilization by bonobo-chimpanzees (Pan paniscus).

PubMed

Roffman, Itai; Savage-Rumbaugh, Sue; Rubert-Pugh, Elizabeth; Ronen, Avraham; Nevo, Eviatar

2012-09-04

Using direct percussion, language-competent bonobo-chimpanzees Kanzi and Pan-Banisha produced a significantly wider variety of flint tool types than hitherto reported, and used them task-specifically to break wooden logs or to dig underground for food retrieval. For log breaking, small flakes were rotated drill-like or used as scrapers, whereas thick cortical flakes were used as axes or wedges, leaving consistent wear patterns along the glued slits, the weakest areas of the log. For digging underground, a variety of modified stone tools, as well as unmodified flint nodules, were used as shovels. Such tool production and utilization competencies reported here in Pan indicate that present-day Pan exhibits Homo-like technological competencies.

A novel genotoxin-specific qPCR array based on the metabolically competent human HepaRG™ cell line as a rapid and reliable tool for improved in vitro hazard assessment.

PubMed

Ates, Gamze; Mertens, Birgit; Heymans, Anja; Verschaeve, Luc; Milushev, Dimiter; Vanparys, Philippe; Roosens, Nancy H C; De Keersmaecker, Sigrid C J; Rogiers, Vera; Doktorova, Tatyana Y

2018-04-01

Although the value of the regulatory accepted batteries for in vitro genotoxicity testing is recognized, they result in a high number of false positives. This has a major impact on society and industries developing novel compounds for pharmaceutical, chemical, and consumer products, as afflicted compounds have to be (prematurely) abandoned or further tested on animals. Using the metabolically competent human HepaRG ™ cell line and toxicogenomics approaches, we have developed an upgraded, innovative, and proprietary gene classifier. This gene classifier is based on transcriptomic changes induced by 12 genotoxic and 12 non-genotoxic reference compounds tested at sub-cytotoxic concentrations, i.e., IC10 concentrations as determined by the 3-(4,5-dimethylthiazol-2-yl)-2,5-diphenyltetrazolium bromide (MTT) assay. The resulting gene classifier was translated into an easy-to-handle qPCR array that, as shown by pathway analysis, covers several different cellular processes related to genotoxicity. To further assess the predictivity of the tool, a set of 5 known positive and 5 known negative test compounds for genotoxicity was evaluated. In addition, 2 compounds with debatable genotoxicity data were tested to explore how the qPCR array would classify these. With an accuracy of 100%, when equivocal results were considered positive, the results showed that combining HepaRG ™ cells with a genotoxin-specific qPCR array can improve (geno)toxicological hazard assessment. In addition, the developed qPCR array was able to provide additional information on compounds for which so far debatable genotoxicity data are available. The results indicate that the new in vitro tool can improve human safety assessment of chemicals in general by basing predictions on mechanistic toxicogenomics information.

Cultural competence springs up in the desert: the story of the center for cultural competence in health care at Weill Cornell Medical College in Qatar.

PubMed

Elnashar, Maha; Abdelrahim, Huda; Fetters, Michael D

2012-06-01

The authors describe the factors that led Weill Cornell Medical College in Qatar (WCMC-Q) to establish the Center for Cultural Competence in Health Care from the ground up, and they explore challenges and successes in implementing cultural competence training.Qatar's capital, Doha, is an extremely high-density multicultural setting. When WCMC-Q's first class of medical students began their clinical clerkships at the affiliated teaching hospital Hamad Medical Corporation in 2006, the complicated nature of training in a multicultural and multilingual setting became apparent immediately. In response, initiatives to improve students' cultural competence were undertaken. Initiatives included launching a medical interpretation program in 2007; surveying the patients' spoken languages, examining the effect of an orientation program on interpretation requests, and surveying faculty using the Tool for Assessing Cultural Competence Training in 2008; implementing cultural competence training for students and securing research funding in 2009; and expanding awareness to the Qatar community in 2010. These types of initiatives, which are generally highly valued in U.S. and Canadian settings, are also apropos in the Arabian Gulf region.The authors report on their initial efforts, which can serve as a resource for other programs in the Arabian Gulf region.

What makes a child a 'competent' child?

PubMed

van Rooyen, Amanda; Water, Tineke; Rasmussen, Shayne; Diesfeld, Kate

2015-12-04

Competence is a vital component of the informed consent process. The perceived level of a child's competence may influence their degree of participation in health decisions that affect them. It is the responsibility of the health professional to gauge a child's level of competence. Child competence, however, is not a static attribute that is linked to age. Rather, it is dynamic, changing in nature and dependent on a child's previous experiences, personal attributes, network of relationships around them and cultural and environmental context. Consequently, there is no single verified assessment tool to assist in the recognition of competence for New Zealand children. Adding to this complexity are the unclear interpretations of New Zealand health legislation and policy regarding whether or not a child can legally consent or refuse healthcare advice and treatment without the consent of a legal guardian. Under the Care of Children Act 2004 and the Code of Health and Disability Services Consumers' Rights 1996, the Health and Disability Commissioner states "a child may consent themselves [to health treatment] if and when the child achieves sufficient understanding and maturity to understand fully what is proposed". This paper poses the question: What is 'competency' and how is this decided? For the purpose of this article, 'child' pertains to those under the age of 16 years.

[The role of "competent physician" for prevention of accidents at work].

PubMed

Ramistella, E; Bergamaschi, A; Mosconi, G; Rossi, O; Sallese, D

2008-01-01

Using at best the professional and legal tools at his/her disposal, the "competent physician" can have a relevant role in reducing accidents in the workplace. In assessing the worker's suitability to specific tasks, the competent physician checks the presence of pathologies or functional impairments of organs or apparatuses that can be an additional risk for the occurrence of accidents at work. The activity aimed to preventing accidents at work translates also in taking part in the planning and implementation of information activities and workers' training within the company. The various risk factors inherent in the workplace and so-called "human factors" can interact in a negative way to the point of becoming a cause of accidents. The human variables of accident risk at the industrial, occupational and even individual level, may be numerous. In this paper we shall review these important aspects and attempt to clarify the role that can be played by the competent physician in the prevention of accidents at work.

The Desired Concept Maps and Goal Setting for Assessing Professionalism in Medicine.

PubMed

Guraya, Salman Y; Guraya, Shaista S; Mahabbat, Nehal Anam; Fallatah, Khulood Yahya; Al-Ahmadi, Bashaer Ahmad; Alalawi, Hadeel Hadi

2016-05-01

Due to the multi-dimensional characteristics of professionalism, no single assessment modality has shown to reliably assess professionalism. This review aims to describe some of the popular assessment tools that are being used to assess professionalism with a view to formulate a framework of assessment of professionalism in medicine. In December 2015, the online research databases of MEDLINE, the Educational Resources Information Center (ERIC), Elton Bryson Stephens Company (EBSCO), SCOPUS, OVID and PsychINFO were searched for full-text English language articles published during 2000 to 2015. MeSH terms "professionalism" AND "duty" AND "assessment" OR "professionalism behavioural" AND "professionalism-cognitive" were used. The research articles that assessed professionalism across medical fields along with other areas of competencies were included. A final list of 35 articles were selected for this review. Several assessment tools are available for assessing professionalism that includes, but not limited to, mini clinical evaluation exercise, standardised direct observation of procedural skills, professionalism mini-evaluation exercise, multi-source feedback and 360 degree evaluation, and case based discussions. Because professionalism is a complex construct, it is less likely that a single assessment strategy will adequately measure it. Since every single assessment tool has its own weaknesses, triangulation involving multiple tools can compensate the shortcomings associated with any single approach. Assessment of professionalism necessitates a combination of modalities at individual, interpersonal, societal, and institutional levels and should be accompanied by feedback and motivational reflection that will, in turn, lead to behaviour and identity formation. The assessment of professionalism in medicine should meet the criteria of validity, reliability, feasibility and acceptability. Educators are urged to enhance the depth and quality of assessment instruments in the existing medical curricula for ensuring validity and reliability of assessment tools for professionalism.

Effects of mobile and digital support for a structured, competency-based curriculum in neurosurgery residency education.

PubMed

Gonzalez, Nestor R; Dusick, Joshua R; Martin, Neil A

2012-07-01

Changes in neurosurgical practice and graduate medical education impose new challenges for training programs. We present our experience providing neurosurgical residents with digital and mobile educational resources in support of the departmental academic activities. A weekly mandatory conference program for all clinical residents based on the Accreditation Council for Graduate Medical Education competencies, held in protected time, was introduced. Topics were taught through didactic sessions and case discussions. Faculty and residents prepare high-quality presentations, equivalent to peer-review leading papers or case reports. Presentations are videorecorded, stored in a digital library, and broadcasted through our Website and iTunes U. Residents received mobile tablet devices with remote access to the digital library, applications for document/video management, and interactive teaching tools. Residents responded to an anonymous survey, and performances on the Self-Assessment in Neurological Surgery examination before and after the intervention were compared. Ninety-two percent reported increased time used to study outside the hospital and attributed the habit change to the introduction of mobile devices; 67% used the electronic tablets as the primary tool to access the digital library, followed by 17% hospital computers, 8% home computers, and 8% personal laptops. Forty-two percent have submitted operative videos, cases, and documents to the library. One year after introducing the program, results of the Congress of Neurological Surgeons-Self-Assessment in Neurological Surgery examination showed a statistically significant improvement in global scoring and improvement in 16 of the 18 individual areas evaluated, 6 of which reached statistical significance. A structured, competency-based neurosurgical education program supported with digital and mobile resources improved reading habits among residents and performance on the Congress of Neurological Surgeons-Self-Assessment in Neurological Surgery examination.

Assessment of representational competence in kinematics

NASA Astrophysics Data System (ADS)

Klein, P.; Müller, A.; Kuhn, J.

2017-06-01

A two-tier instrument for representational competence in the field of kinematics (KiRC) is presented, designed for a standard (1st year) calculus-based introductory mechanics course. It comprises 11 multiple choice (MC) and 7 multiple true-false (MTF) questions involving multiple representational formats, such as graphs, pictures, and formal (mathematical) expressions (1st tier). Furthermore, students express their answer confidence for selected items, providing additional information (2nd tier). Measurement characteristics of KiRC were assessed in a validation sample (pre- and post-test, N =83 and N =46 , respectively), including usefulness for measuring learning gain. Validity is checked by interviews and by benchmarking KiRC against related measures. Values for item difficulty, discrimination, and consistency are in the desired ranges; in particular, a good reliability was obtained (KR 20 =0.86 ). Confidence intervals were computed and a replication study yielded values within the latter. For practical and research purposes, KiRC as a diagnostic tool goes beyond related extant instruments both for the representational formats (e.g., mathematical expressions) and for the scope of content covered (e.g., choice of coordinate systems). Together with the satisfactory psychometric properties it appears a versatile and reliable tool for assessing students' representational competency in kinematics (and of its potential change). Confidence judgments add further information to the diagnostic potential of the test, in particular for representational misconceptions. Moreover, we present an analytic result for the question—arising from guessing correction or educational considerations—of how the total effect size (Cohen's d ) varies upon combination of two test components with known individual effect sizes, and then discuss the results in the case of KiRC (MC and MTF combination). The introduced method of test combination analysis can be applied to any test comprising two components for the purpose of finding effect size ranges.

An Educational Tool for Teaching Medication History Taking to Pharmacy Students

PubMed Central

Elliott, Jennifer; Stanton, Melonie L.; Doty, Randell

2013-01-01

Objective. To implement and evaluate the use of a situated-learning experience to prepare second-year pharmacy students to conduct medication history interviews in preparation for introductory pharmacy practice experiences (IPPE) at ambulatory clinic sites. Design. Second-year doctor of pharmacy (PharmD) students (n=200) used the Medication Mysteries Infinite Case Tool, a game-like educational tool in which groups of 3 students assumed the roles of pharmacist, patient, and observer and rolled a die and drew cards to determine the drugs, patient personality, medication problems, and other variables that guided a medication history taking session. Assessment. After the laboratory session, faculty members assessed students’ medication history-taking skills. One hundred sixteen (58%) and 78 (39%) of 200 students achieved excellence or competence, respectively, on the final assessment. Two weeks after the assessment, 53 of 200 (26.5%) students completed a survey instrument. The respondents indicated that their self-confidence in conducting medical history taking significantly improved following completion of the learning experience. Conclusion. Using the Medication Mysteries Infinite Case Tool increased students’ confidence and skills in conducting medication history taking prior to their clinical IPPE experience. PMID:23788816

Patient Safety Competence of Nursing Students in Saudi Arabia: A Self-Reported Survey

PubMed Central

Colet, Paolo C.; Cruz, Jonas P.; Cruz, Charlie P.; Al-otaibi, Jazi; Qubeilat, Hikmet; Alquwez, Nahed

2015-01-01

Objective With the growing recognition of the significance of patient safety (PS) in educational institutions and health organizations, it is essential to understand the perspective of nursing students on their own PS competence. This study analyzed the self-reported PS competence of nursing students at a government university in Saudi Arabia. Methodology A cross-sectional self-reported survey of 191 respondents, using the Health Professional Education in Patient Safety Survey (H-PEPSS) was conducted. The survey tool reflected 6 key socio-cultural dimensions assessing competence in classroom and clinical setting. Results Female nursing students reported higher PS competence in both the classroom and clinical settings along the dimensions ‘working in teams’ and ‘communicating effectively’ while males reported higher competence in both settings as to the ‘managing safety risks’ and ‘understanding human and environmental factors’ dimensions. The respondents’ academic level and self–reported PS competence have weak negative correlation in the classroom while a strong negative correlation between the 2 variables existed in the clinical setting. Self-reported PS competence for the dimensions ‘working in teams’, ‘recognize and respond to remove immediate risks of harm’, and ‘culture of safety’ is significantly higher in classroom than in the clinical setting. Conclusion Generally, the Saudi nursing students reported varying levels of competence in the six dimensions of patient safety. Significant gap between the perceived PS competence was observed between learning settings. Educational and training interventions are suggested for implementation to bridge this gap. PMID:26715921

The development of a competency framework for pharmacists providing cancer services.

PubMed

Carrington, Christine; Weir, Janet; Smith, Peter

2011-09-01

Health practitioners should possess relevant, up to date skills and be able to perform within their required scope of practice to ensure that they are competent. Maintaining the competency of health care professionals is a key principle of clinical governance and risk management. The aim of this project was to develop a competency framework for pharmacists providing pharmaceutical care to cancer patients. An initial draft framework was developed based on existing documentation and adapted to the needs of Queensland Health (QH) facilities. Pharmacists in QH and interstate were asked to review the framework for content and applicability. Cancer care pharmacists in QH were invited to evaluate and score the usefulness and relevance of the final framework. The framework consists of competency clusters, which describe core activities within three areas: patient care competencies, knowledge competencies, and advanced level competencies. The characteristics of the levels of practice at foundation, advanced, and consultant are defined. Twelve pharmacists evaluated the framework by self-assessing their own practice. Respondents reported that the framework was very to somewhat reflective of what they usually do and gave overall support for the content and applicability to practice. The framework has been developed using national and international documents and the input of experienced practitioners across Australia. It represents a set of key competencies for the pharmaceutical delivery of cancer care. The next essential step of the competency framework is to implement and integrate the framework into practice and to develop accompanying training tools.

A Shared Focus: Comparing the Australian, Canadian, United Kingdom and United States Pharmacy Learning Outcome Frameworks and the Global Competency Framework.

PubMed

Stupans, Ieva; Atkinson, Jeffrey; Meštrović, Arijana; Nash, Rose; Rouse, Michael J

2016-09-10

This paper presents an analysis of the end of degree expectations, expressed as learning outcomes, for pharmacy graduates from Australia, Canada, United Kingdom and United States. The authors compare the end of degree expectations, through mapping these requirements to the International Pharmaceutical Federation (FIP) Global Competency Framework (GbCF). The anticipated end of degree expectations are similar but also reveal some individual characteristics. Irrespective of degree title, achievement of learning outcomes specified in any one of the four jurisdictions should enable students to become pharmacists who are patient-orientated medicines experts. The mapping provides impetus for cross-border institutional networking to generate a dependable set of assessment tools across national borders developing a common metric for outcome assessment irrespective of different program delivery.

The Desired Concept Maps and Goal Setting for Assessing Professionalism in Medicine

PubMed Central

Guraya, Shaista S.; Mahabbat, Nehal Anam; Fallatah, Khulood Yahya; Al-Ahmadi, Bashaer Ahmad; Alalawi, Hadeel Hadi

2016-01-01

Due to the multi-dimensional characteristics of professionalism, no single assessment modality has shown to reliably assess professionalism. This review aims to describe some of the popular assessment tools that are being used to assess professionalism with a view to formulate a framework of assessment of professionalism in medicine. In December 2015, the online research databases of MEDLINE, the Educational Resources Information Center (ERIC), Elton Bryson Stephens Company (EBSCO), SCOPUS, OVID and PsychINFO were searched for full-text English language articles published during 2000 to 2015. MeSH terms “professionalism” AND “duty” AND “assessment” OR “professionalism behavioural” AND “professionalism–cognitive” were used. The research articles that assessed professionalism across medical fields along with other areas of competencies were included. A final list of 35 articles were selected for this review. Several assessment tools are available for assessing professionalism that includes, but not limited to, mini clinical evaluation exercise, standardised direct observation of procedural skills, professionalism mini-evaluation exercise, multi-source feedback and 360 degree evaluation, and case based discussions. Because professionalism is a complex construct, it is less likely that a single assessment strategy will adequately measure it. Since every single assessment tool has its own weaknesses, triangulation involving multiple tools can compensate the shortcomings associated with any single approach. Assessment of professionalism necessitates a combination of modalities at individual, interpersonal, societal, and institutional levels and should be accompanied by feedback and motivational reflection that will, in turn, lead to behaviour and identity formation. The assessment of professionalism in medicine should meet the criteria of validity, reliability, feasibility and acceptability. Educators are urged to enhance the depth and quality of assessment instruments in the existing medical curricula for ensuring validity and reliability of assessment tools for professionalism. PMID:27437247

Development of a Nurse Preceptor Competency Assessment Tool: A Mixed-Methods Approach

ERIC Educational Resources Information Center

Hoot, Tracy A. T.

2017-01-01

Clinical nurse preceptors are often assigned a nursing student based on their availability and not necessarily on their expertise. The preceptor, as role model and clinical teacher, is instrumental in guiding a nursing student in his or her transition into the role of a new graduate nurse. When a preceptor lacks the knowledge to meet the…

Differentiating Second Language Acquisition from Specific Learning Disability: An Observational Tool Assessing Dual Language Learners' Pragmatic Competence

ERIC Educational Resources Information Center

Farnsworth, Megan

2018-01-01

Overrepresentation of Dual Language Learners (DLLs) in special education remains a problem even after 40 years of inquiry. One factor is that the U.S. federal government has neither clearly explained the definition of Specific Learning Disability (SLD) nor operationally defined it to identify children for special education services. This lack of…

Developing and Testing EVALOE: A Tool for Assessing Spoken Language Teaching and Learning in the Classroom

ERIC Educational Resources Information Center

Gràcia, Marta; Vega, Fàtima; Galván-Bovaira, Maria José

2015-01-01

Broadly speaking, the teaching of spoken language in Spanish schools has not been approached in a systematic way. Changes in school practices are needed in order to allow all children to become competent speakers and to understand and construct oral texts that are appropriate in different contexts and for different audiences both inside and…

The Development of Maritime English Learning Model Using Authentic Assessment Based Bridge Simulator in Merchant Marine Polytechnic, Makassar

ERIC Educational Resources Information Center

Fauzi, Ahmad; Bundu, Patta; Tahmir, Suradi

2016-01-01

Bridge simulator constitutes a very fundamental and vital tool to trigger and ensure that seamen or seafarers possess the standardized competence required. By using the bridge simulator technique, a reality based study can be presented easily and delivered to the students in ongoing basis to their classroom or study place. Afterwards, the validity…

The National Association of School Psychologists' Self-Assessment Tool for School Psychologists: Factor Structure and Relationship to the National Association of School Psychologists' Practice Model

ERIC Educational Resources Information Center

Eklund, Katie; Rossen, Eric; Charvat, Jeff; Meyer, Lauren; Tanner, Nick

2016-01-01

The National Association of School Psychologists' Model for Comprehensive and Integrated School Psychological Services (2010a), often referred to as the National Association of School Psychologists' Practice Model, describes the comprehensive range of professional skills and competencies available from school psychologists across 10 domains. The…

Education for Sustainable Development: Assessment of the Current Situation at the Faculty of Engineering of Notre Dame University--Louaize

ERIC Educational Resources Information Center

Salem, Talal; Harb, Jacques

2012-01-01

There is a growing need to incorporate educational sustainable development (ESD) principles into engineering education. This paper identifies engineering competencies within the Faculty of Engineering at Notre Dame University--Louaize and the means to shift towards sustainability. ESD tools are used to carry the analysis, keeping in mind the…

"An Adjective Is a Word Hanging down from a Noun": Learning to Write and Students with Learning Disabilities

ERIC Educational Resources Information Center

Harris, Karen R.; Graham, Steve

2013-01-01

By the upper elementary grades, writing becomes an essential tool both for learning and for showing what you know. Students who struggle significantly with writing are at a terrible disadvantage. Data from the National Assessment of Educational Progress indicate that only 25% of students can be classified as competent writers; students with…

Ensuring relational competency in critical care: Importance of nursing students' communication skills.

PubMed

Sánchez Expósito, Judit; Leal Costa, César; Díaz Agea, José Luis; Carrillo Izquierdo, María Dolores; Jiménez Rodríguez, Diana

2018-02-01

The aim of this study was to analyse the communication skills of students in interactions with simulated critically-ill patients using a new assessment tool to study the relationships between communication skills, teamwork and clinical skills and to analyse the psychometric properties of the tool. A cross-sectional study was conducted to assess the communications skills of 52 students with critically-ill patients through the use of a new measurement tool to score video recordings of simulated clinical scenarios. The 52 students obtained low scores on their skills in communicating with patients. The reliability of the measuring instrument showed good inter-observer agreement (ICC between 0.71 and 0.90) and the validity yielded a positive correlation (p<0.01). The results provide evidence that nursing students lack skills when communicating with critically ill patients in simulated scenarios. The measuring instrument used is therefore deemed valid and reliable for assessing nursing students through a clinical simulation. Copyright © 2017 Elsevier Ltd. All rights reserved.

Examining validity evidence for a simulation-based assessment tool for basic robotic surgical skills.

PubMed

Havemann, Maria Cecilie; Dalsgaard, Torur; Sørensen, Jette Led; Røssaak, Kristin; Brisling, Steffen; Mosgaard, Berit Jul; Høgdall, Claus; Bjerrum, Flemming

2018-05-14

Increasing focus on patient safety makes it important to ensure surgical competency among surgeons before operating on patients. The objective was to gather validity evidence for a virtual-reality simulator test for robotic surgical skills and evaluate its potential as a training tool. Surgeons with varying experience in robotic surgery were recruited: novices (zero procedures), intermediates (1-50), experienced (> 50). Five experienced surgeons rated five exercises on the da Vinci Skills Simulator. Participants were tested using the five exercises. Participants were invited back 3 times and completed a total of 10 attempts per exercise. The outcome was the average simulator performance score for the 5 exercises. 32 participants from 5 surgical specialties were included. 38 participants completed all 4 sessions. A moderate correlation between the average total score and robotic experience was identified for the first attempt (Spearman r = 0.58; p = 0.0004). A difference in average total score was observed between novices and intermediates [median score 61% (IQR 52-66) vs. 83% (IQR 75-91), adjusted p < 0.0001], as well as novices and experienced [median score 61% (IQR 52-66) vs. 80 (IQR 69-85), adjusted p = 0.002]. All three groups improved their performance between the 1st and 10th attempts (p < 0.00). This study describes validity evidence for a virtual-reality simulator for basic robotic surgical skills, which can be used for assessment of basic competency and as a training tool. However, more validity evidence is needed before it can be used for certification or high-stakes assessment.

The Surgical Learning and Instructional Portfolio (SLIP) as a self-assessment educational tool demonstrating practice-based learning.

PubMed

Webb, Travis P; Aprahamian, Charles; Weigelt, John A; Brasel, Karen J

2006-01-01

Time constraints on the teaching and evaluation of residents continue to alter the way in which medical knowledge must be imparted and assessed. Lifelong learning is a component of the practice-based learning competency. A portfolio is one way to assess practice-based learning, but its use is unfamiliar to most surgical programs. The authors describe the evolution of the Surgical Learning and Instructional Portfolio (SLIP) into a worthwhile educational tool. In March 2001, the authors began a program to encourage residents to develop a case-based portfolio to document their experience and demonstrate acquisition of knowledge in caring for a variety of surgical diseases. The monthly case topic was chosen by the resident and reported using a template: case history, supporting diagnostic studies, differential diagnosis, final diagnosis with ICD-9 coding, management options, treatment used, 3 lessons learned, embellishment of 1 lesson, and 2 articles supporting the experience. Initially, cases were submitted to the program coordinator and reviewed every 6 months with a faculty advisor to provide feedback. After the first 18 months of this program, resident compliance was less than 50%, satisfaction was low, and formal review did not occur. In July 2004, a single faculty member became responsible for evaluating and providing feedback on the monthly SLIPs. The assignments were handled electronically with feedback delivered within the month via e-mail. SLIP quality as measured by resident compliance and satisfaction improved. These SLIPs have matured into a valuable educational tool satisfying multiple ACGME competencies. This portfolio system required direct faculty feedback to become successful.

Analyzing reflective narratives to assess the ethical reasoning of pediatric residents.

PubMed

Moon, Margaret; Taylor, Holly A; McDonald, Erin L; Hughes, Mark T; Beach, Mary Catherine; Carrese, Joseph A

2013-01-01

A limiting factor in ethics education in medical training has been difficulty in assessing competence in ethics. This study was conducted to test the concept that content analysis of pediatric residents' personal reflections about ethics experiences can identify changes in ethical sensitivity and reasoning over time. Analysis of written narratives focused on two of our ethics curriculum's goals: 1) To raise sensitivity to ethical issues in everyday clinical practice and 2) to enhance critical reflection on personal and professional values as they affect patient care. Content analysis of written reflections was guided by a tool developed to identify and assess the level of ethical reasoning in eight domains determined to be important aspects of ethical competence. Based on the assessment of narratives written at two times (12 to 16 months/apart) during their training, residents showed significant progress in two specific domains: use of professional values, and use of personal values. Residents did not show decline in ethical reasoning in any domain. This study demonstrates that content analysis of personal narratives may provide a useful method for assessment of developing ethical sensitivity and reasoning.

Creation of a competency-based professional development program for infection preventionists guided by the APIC Competency Model: steps in the process.

PubMed

Bernard, Heather; Hackbarth, Diana; Olmsted, Russell N; Murphy, Denise

2018-06-07

Infection Preventionists have varying levels of educational preparation. Many have no prior experience in IP. The diversity makes design of professional development programs challenging. Recent surveys suggest that only about half of practicing IPs are board certified. There is an urgent need to employ competent IP's to drive improvement in patient outcomes. This is a project that utilized the APIC Competency Model to create a professional development program characterizing three career stages. Methods included a review of literature on professional development; a survey of IP competence; an assessment of job descriptions and performance evaluations; and a crosswalk of IP competencies. The professional development program includes competency - based IP job descriptions and performance evaluations for each career stage; a professional portfolio; and a toolkit for supervisors. Participants agreed that application of the model resulted in tools which are more closely aligned with current roles for IPs; and increased satisfaction and motivation with the new program. Competent and knowledgeable IP's are crucial to optimizing efficacy of IPC programs. A professional development program has the potential to guide staff orientation, improve satisfaction and retention, improve patient outcomes and promote a positive trajectory in advancing practice. Copyright © 2018 Association for Professionals in Infection Control and Epidemiology, Inc. Published by Elsevier Inc. All rights reserved.

Development and evaluation of a risk communication curriculum for medical students.

PubMed

Han, Paul K J; Joekes, Katherine; Elwyn, Glyn; Mazor, Kathleen M; Thomson, Richard; Sedgwick, Philip; Ibison, Judith; Wong, John B

2014-01-01

To develop, pilot, and evaluate a curriculum for teaching clinical risk communication skills to medical students. A new experience-based curriculum, "Risk Talk," was developed and piloted over a 1-year period among students at Tufts University School of Medicine. An experimental study of 2nd-year students exposed vs. unexposed to the curriculum was conducted to evaluate the curriculum's efficacy. Primary outcome measures were students' objective (observed) and subjective (self-reported) risk communication competence; the latter was assessed using an Observed Structured Clinical Examination (OSCE) employing new measures. Twenty-eight 2nd-year students completed the curriculum, and exhibited significantly greater (p<.001) objective and subjective risk communication competence than a convenience sample of 24 unexposed students. New observational measures of objective competence in risk communication showed promising evidence of reliability and validity. The curriculum was resource-intensive. The new experience-based clinical risk communication curriculum was efficacious, although resource-intensive. More work is needed to develop the feasibility of curriculum delivery, and to improve the measurement of competence in clinical risk communication. Risk communication is an important advanced communication skill, and the Risk Talk curriculum provides a model educational intervention and new assessment tools to guide future efforts to teach and evaluate this skill. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

Advanced Engineering Technology for Measuring Performance.

PubMed

Rutherford, Drew N; D'Angelo, Anne-Lise D; Law, Katherine E; Pugh, Carla M

2015-08-01

The demand for competency-based assessments in surgical training is growing. Use of advanced engineering technology for clinical skills assessment allows for objective measures of hands-on performance. Clinical performance can be assessed in several ways via quantification of an assessee's hand movements (motion tracking), direction of visual attention (eye tracking), levels of stress (physiologic marker measurements), and location and pressure of palpation (force measurements). Innovations in video recording technology and qualitative analysis tools allow for a combination of observer- and technology-based assessments. Overall the goal is to create better assessments of surgical performance with robust validity evidence. Copyright © 2015 Elsevier Inc. All rights reserved.

A validated search assessment tool: assessing practice-based learning and improvement in a residency program.

PubMed

Rana, Gurpreet K; Bradley, Doreen R; Hamstra, Stanley J; Ross, Paula T; Schumacher, Robert E; Frohna, John G; Haftel, Hilary M; Lypson, Monica L

2011-01-01

The objective of this study was to validate an assessment instrument for MEDLINE search strategies at an academic medical center. Two approaches were used to investigate if the search assessment tool could capture performance differences in search strategy construction. First, data from an evaluation of MEDLINE searches from a pediatric resident's longitudinal assessment were investigated. Second, a cross-section of search strategies from residents in one incoming class was compared with strategies of residents graduating a year later. MEDLINE search strategies formulated by faculty who had been identified as having search expertise were used as a gold standard comparison. Participants were presented with a clinical scenario and asked to identify the search question and conduct a MEDLINE search. Two librarians rated the blinded search strategies. Search strategy scores were significantly higher for residents who received training than the comparison group with no training. There was no significant difference in search strategy scores between senior residents who received training and faculty experts. The results provide evidence for the validity of the instrument to evaluate MEDLINE search strategies. This assessment tool can measure improvements in information-seeking skills and provide data to fulfill Accreditation Council for Graduate Medical Education competencies.

The model for Fundamentals of Endovascular Surgery (FEVS) successfully defines the competent endovascular surgeon.

PubMed

Duran, Cassidy; Estrada, Sean; O'Malley, Marcia; Sheahan, Malachi G; Shames, Murray L; Lee, Jason T; Bismuth, Jean

2015-12-01

Fundamental skills testing is now required for certification in general surgery. No model for assessing fundamental endovascular skills exists. Our objective was to develop a model that tests the fundamental endovascular skills and differentiates competent from noncompetent performance. The Fundamentals of Endovascular Surgery model was developed in silicon and virtual-reality versions. Twenty individuals (with a range of experience) performed four tasks on each model in three separate sessions. Tasks on the silicon model were performed under fluoroscopic guidance, and electromagnetic tracking captured motion metrics for catheter tip position. Image processing captured tool tip position and motion on the virtual model. Performance was evaluated using a global rating scale, blinded video assessment of error metrics, and catheter tip movement and position. Motion analysis was based on derivations of speed and position that define proficiency of movement (spectral arc length, duration of submovement, and number of submovements). Performance was significantly different between competent and noncompetent interventionalists for the three performance measures of motion metrics, error metrics, and global rating scale. The mean error metric score was 6.83 for noncompetent individuals and 2.51 for the competent group (P < .0001). Median global rating scores were 2.25 for the noncompetent group and 4.75 for the competent users (P < .0001). The Fundamentals of Endovascular Surgery model successfully differentiates competent and noncompetent performance of fundamental endovascular skills based on a series of objective performance measures. This model could serve as a platform for skills testing for all trainees. Copyright © 2015 Society for Vascular Surgery. Published by Elsevier Inc. All rights reserved.

Development of a tool to support holistic generic assessment of clinical procedure skills.

PubMed

McKinley, Robert K; Strand, Janice; Gray, Tracey; Schuwirth, Lambert; Alun-Jones, Tom; Miller, Helen

2008-06-01

The challenges of maintaining comprehensive banks of valid checklists make context-specific checklists for assessment of clinical procedural skills problematic. This paper reports the development of a tool which supports generic holistic assessment of clinical procedural skills. We carried out a literature review, focus groups and non-participant observation of assessments with interview of participants, participant evaluation of a pilot objective structured clinical examination (OSCE), a national modified Delphi study with prior definitions of consensus and an OSCE. Participants were volunteers from a large acute teaching trust, a teaching primary care trust and a national sample of National Health Service staff. Results In total, 86 students, trainees and staff took part in the focus groups, observation of assessments and pilot OSCE, 252 in the Delphi study and 46 candidates and 50 assessors in the final OSCE. We developed a prototype tool with 5 broad categories amongst which were distributed 38 component competencies. There was > 70% agreement (our prior definition of consensus) at the first round of the Delphi study for inclusion of all categories and themes and no consensus for inclusion of additional categories or themes. Generalisability was 0.76. An OSCE based on the instrument has a predicted reliability of 0.79 with 12 stations and 1 assessor per station or 10 stations and 2 assessors per station. This clinical procedural skills assessment tool enables reliable assessment and has content and face validity for the assessment of clinical procedural skills. We have designated it the Leicester Clinical Procedure Assessment Tool (LCAT).

Experimental Methods to Evaluate Science Utility Relative to the Decadal Survey

NASA Technical Reports Server (NTRS)

Widergren, Cynthia

2012-01-01

The driving factor for competed missions is the science that it plans on performing once it has reached its target body. These science goals are derived from the science recommended by the most current Decadal Survey. This work focuses on science goals in previous Venus mission proposals with respect to the 2013 Decadal Survey. By looking at how the goals compare to the survey and how much confidence NASA has in the mission's ability to accomplish these goals, a method was created to assess the science return utility of each mission. This method can be used as a tool for future Venus mission formulation and serves as a starting point for future development of create science utility assessment tools.

Beyond JCAHO: using competency models to change healthcare organizations. Part 2: Developing competence assessment systems.

PubMed

Decker, P J; Strader, M K; Wise, R J

1997-01-01

In 1996, JCAHO required hospitals to assess, prove, track, and improve the competence of all employees. This article is the second part of a review of the concept of competency assessment and the implications of meeting and exceeding the JCAHO standards. Part 1 (in the previous issue of Hospital Topics) provided the theory of competence assessment, the current situation in JCAHO surveys, and an overview of the problems inherent in competency assessment. This part puts competence assessment in the context of quality improvement and provides the details of developing competence assessment systems.

Clinical nurse leaders' and academics' perspectives in clinical assessment of final-year nursing students: A qualitative study.

PubMed

Wu, Xi Vivien; Enskär, Karin; Pua, Lay Hoon; Heng, Doreen Gek Noi; Wang, Wenru

2017-09-01

The nature of nursing practice is diverse; therefore, clinical assessment is a complex process. This study explores the perceptions of clinical nurse leaders and academics on clinical assessment for undergraduate nursing education during transition to practice. An explorative qualitative approach was applied. Eight nurse managers, six clinical nurse educators, and eight academics from two tertiary hospitals and a university in Singapore participated in four focus group discussions. Thematic analysis was conducted. Four overriding themes were revealed: the need for a valid and reliable clinical assessment tool, preceptors' competency in clinical assessment, challenges encountered by the students in clinical assessment, and the need for close academic and clinical collaboration to support preceptors and students. Closer academic-clinical partnership is recommended to review the clinical education curriculum. Clinical and educational institutions need to work closely to design a learning program to enhance preceptors' competence in clinical pedagogy and assessment. Furthermore, a stress management program could build students' resiliency in coping with unfamiliar clinical environments. Ongoing support needs to be provided for both preceptors and students to enrich the preceptorship and learning experiences. © 2017 John Wiley & Sons Australia, Ltd.

Assessing physiotherapists' communication skills for promoting patient autonomy for self-management: reliability and validity of the communication evaluation in rehabilitation tool.

PubMed

Murray, Aileen; Hall, Amanda; Williams, Geoffrey C; McDonough, Suzanne M; Ntoumanis, Nikos; Taylor, Ian; Jackson, Ben; Copsey, Bethan; Hurley, Deirdre A; Matthews, James

2018-02-27

To assess the inter-rater reliability and concurrent validity of the Communication Evaluation in Rehabilitation Tool, which aims to externally assess physiotherapists competency in using Self-Determination Theory-based communication strategies in practice. Audio recordings of initial consultations between 24 physiotherapists and 24 patients with chronic low back pain in four hospitals in Ireland were obtained as part of a larger randomised controlled trial. Three raters, all of whom had Ph.Ds in psychology and expertise in motivation and physical activity, independently listened to the 24 audio recordings and completed the 18-item Communication Evaluation in Rehabilitation Tool. Inter-rater reliability between all three raters was assessed using intraclass correlation coefficients. Concurrent validity was assessed using Pearson's r correlations with a reference standard, the Health Care Climate Questionnaire. The total score for the Communication Evaluation in Rehabilitation Tool is an average of all 18 items. Total scores demonstrated good inter-rater reliability (Intraclass Correlation Coefficient (ICC) = 0.8) and concurrent validity with the Health Care Climate Questionnaire total score (range: r = 0.7-0.88). Item-level scores of the Communication Evaluation in Rehabilitation Tool identified five items that need improvement. Results provide preliminary evidence to support future use and testing of the Communication Evaluation in Rehabilitation Tool. Implications for Rehabilitation Promoting patient autonomy is a learned skill and while interventions exist to train clinicians in these skills there are no tools to assess how well clinicians use these skills when interacting with a patient. The lack of robust assessment has severe implications regarding both the fidelity of clinician training packages and resulting outcomes for promoting patient autonomy. This study has developed a novel measurement tool Communication Evaluation in Rehabilitation Tool and a comprehensive user manual to assess how well health care providers use autonomy-supportive communication strategies in real world-clinical settings. This tool has demonstrated good inter-rater reliability and concurrent validity in its initial testing phase. The Communication Evaluation in Rehabilitation Tool can be used in future studies to assess autonomy-supportive communication and undergo further measurement property testing as per our recommendations.

Assessing nursing clinical skills competence through objective structured clinical examination (OSCE) for open distance learning students in Open University Malaysia.

PubMed

Oranye, Nelson Ositadimma; Ahmad, Che'an; Ahmad, Nora; Bakar, Rosnida Abu

2012-06-01

The objective structured clinical skills examination (OSCE) has over the years emerged as a method of evaluating clinical skills in most medical and allied professions. Although its validity and objectivity has evoked so much debate in the literature, little has been written about its application in non-traditional education systems such as in distance learning. This study examined clinical skills competence among practising nursing students who were enrolled in a distance learning programme. The study examined the effect of work and years of nursing practice on nurses' clinical skills competence. This study used observational design whereby nursing students' clinical skills were observed and scored in five OSCE stations. Two instruments were used for the data collection - A self-administered questionnaire on the students' bio-demographic data, and a check list on the clinical skills which the examiners rated on a four point scale. The findings revealed that 14% of the nurses had level four competence, which indicated that they could perform the tasks correctly and complete. However, 12% failed the OSCE, even though they had more than 10 years experience in nursing and post basic qualifications. Inter-rater reliability was 0.92 for the five examiners. Factor analysis indicated that five participant factors accounted for 74.1% of the variations in clinical skills performance. An OSCE is a necessary assessment tool that should be continuously applied in nursing education, regardless of the mode of the education program, the student's years of experience or his/her clinical placement. This study validates the need for OSCE in both the design of tertiary nursing degree programs and the assessment of nurses' clinical competency level.

Resident-Specific Morbidity Reduced Following ACS NSQIP Data-Driven Quality Program.

PubMed

Turrentine, Florence E; Hanks, John B; Tracci, Megan C; Jones, R Scott; Schirmer, Bruce D; Smith, Philip W

2018-04-16

The Accreditation Council for Graduate Medical Education Milestone Project for general surgery provided a more robust method for developing and tracking residents' competence. This framework enhanced systematic and progressive development of residents' competencies in surgical quality improvement. A 22-month interactive, educational program based on resident-specific surgical outcomes data culminated in a quality improvement project for postgraduate year 4 surgery residents. Self- assessment, quality knowledge test, and resident-specific American College of Surgeons National Surgical Quality Improvement Program Quality In-Training Initiative morbidity were compared before and after the intervention. Quality in-training initiative morbidity decreased from 25% (82/325) to 18% (93/517), p = 0.015 despite residents performing more complex cases. All participants achieved level 4 competency (4/4) within the general surgery milestones improvement of care, practice-based learning and improvement competency. Institutional American College of Surgeons National Surgical Quality Improvement Program general surgery morbidity improved from the ninth to the sixth decile. Quality assessment and improvement self-assessment postintervention scores (M = 23.80, SD = 4.97) were not significantly higher than preintervention scores (M = 19.20, SD = 5.26), p = 0.061. Quality Improvement Knowledge Application Tool postintervention test scores (M = 17.4, SD = 4.88), were not significantly higher than pretest scores (M = 13.2, SD = 1.92), p = 0.12. Sharing validated resident-specific clinical data with participants was associated with improved surgical outcomes. Participating fourth year surgical residents achieved the highest score, a level 4, in the practice based learning and improvement competency of the improvement of care practice domain and observed significantly reduced surgical morbidity for cases in which they participated. Copyright © 2018. Published by Elsevier Inc.

Does emotional intelligence predict breaking bad news skills in pediatric interns? A pilot study

PubMed Central

Reed, Suzanne; Kassis, Karyn; Nagel, Rollin; Verbeck, Nicole; Mahan, John D.; Shell, Richard

2015-01-01

Background While both patients and physicians identify communication of bad news as an area of great challenge, the factors underlying this often complex task remain largely unknown. Emotional intelligence (EI) has been positively correlated with good general communication skills and successful leadership, but there is no literature relating EI to the delivery of bad news. Purpose Our objectives were to determine: 1) performance of first-year pediatric residents in the delivery of bad news in a standardized patient (SP) setting; and 2) the role of EI in these assessments. Our hypothesis was that pediatric trainees with higher EI would demonstrate more advanced skills in this communication task. Methods Forty first- year residents participated. Skill in bad news delivery was assessed via SP encounters using a previously published assessment tool (GRIEV_ING Death Notification Protocol). Residents completed the Emotional and Social Competency Inventory (ESCI) as a measure of EI. Results Residents scored poorly on bad news delivery skills but scored well on EI. Intraclass correlation coefficients indicated moderate to substantial inter-rater reliability among raters using the delivering bad news assessment tool. However, no correlation was found between bad news delivery performance and EI. Conclusions We concluded that first-year pediatric residents have inadequate skills in the delivery of bad news. In addition, our data suggest that higher EI alone is not sufficient to effectively deliver death news and more robust skill training is necessary for residents to gain competence and acquire mastery in this important communication domain. PMID:26286897

Does emotional intelligence predict breaking bad news skills in pediatric interns? A pilot study.

PubMed

Reed, Suzanne; Kassis, Karyn; Nagel, Rollin; Verbeck, Nicole; Mahan, John D; Shell, Richard

2015-01-01

While both patients and physicians identify communication of bad news as an area of great challenge, the factors underlying this often complex task remain largely unknown. Emotional intelligence (EI) has been positively correlated with good general communication skills and successful leadership, but there is no literature relating EI to the delivery of bad news. Our objectives were to determine: 1) performance of first-year pediatric residents in the delivery of bad news in a standardized patient (SP) setting; and 2) the role of EI in these assessments. Our hypothesis was that pediatric trainees with higher EI would demonstrate more advanced skills in this communication task. Forty first-year residents participated. Skill in bad news delivery was assessed via SP encounters using a previously published assessment tool (GRIEV_ING Death Notification Protocol). Residents completed the Emotional and Social Competency Inventory (ESCI) as a measure of EI. Residents scored poorly on bad news delivery skills but scored well on EI. Intraclass correlation coefficients indicated moderate to substantial inter-rater reliability among raters using the delivering bad news assessment tool. However, no correlation was found between bad news delivery performance and EI. We concluded that first-year pediatric residents have inadequate skills in the delivery of bad news. In addition, our data suggest that higher EI alone is not sufficient to effectively deliver death news and more robust skill training is necessary for residents to gain competence and acquire mastery in this important communication domain.

Does emotional intelligence predict breaking bad news skills in pediatric interns? A pilot study.

PubMed

Reed, Suzanne; Kassis, Karyn; Nagel, Rollin; Verbeck, Nicole; Mahan, John D; Shell, Richard

2015-01-01

Background While both patients and physicians identify communication of bad news as an area of great challenge, the factors underlying this often complex task remain largely unknown. Emotional intelligence (EI) has been positively correlated with good general communication skills and successful leadership, but there is no literature relating EI to the delivery of bad news. Purpose Our objectives were to determine: 1) performance of first-year pediatric residents in the delivery of bad news in a standardized patient (SP) setting; and 2) the role of EI in these assessments. Our hypothesis was that pediatric trainees with higher EI would demonstrate more advanced skills in this communication task. Methods Forty first- year residents participated. Skill in bad news delivery was assessed via SP encounters using a previously published assessment tool (GRIEV_ING Death Notification Protocol). Residents completed the Emotional and Social Competency Inventory (ESCI) as a measure of EI. Results Residents scored poorly on bad news delivery skills but scored well on EI. Intraclass correlation coefficients indicated moderate to substantial inter-rater reliability among raters using the delivering bad news assessment tool. However, no correlation was found between bad news delivery performance and EI. Conclusions We concluded that first-year pediatric residents have inadequate skills in the delivery of bad news. In addition, our data suggest that higher EI alone is not sufficient to effectively deliver death news and more robust skill training is necessary for residents to gain competence and acquire mastery in this important communication domain.

A short questionnaire to assess pediatric resident’s competencies: the validation process

PubMed Central

2013-01-01

Background In order to help assess resident performance during training, the Residency Affair Committee of the Pediatric Residency Program of the University of Padua (Italy) administered a Resident Assessment Questionnaire (ReAQ), which both residents and faculty were asked to complete. The aim of this article is to present the ReAQ and its validation. Methods The ReAQ consists of 20 items that assess the six core competencies identified by the Accreditation Council of Graduate Medical Education (ACGME). A many-facet Rasch measurement analysis was used for validating the ReAQ. Results Between July 2011 and June 2012, 211 evaluations were collected from residents and faculty. Two items were removed because their functioning changed with the gender of respondents. The step calibrations were ordered. The self evaluations (residents rating themselves) positively correlated with the hetero evaluations (faculty rating residents; Spearman’s ρ = 0.75, p < 0.001). Unfortunately, the observed agreement among faculty was smaller than expected (Exp = 47.1%; Obs = 41%), which indicates that no enough training to faculty for using the tool was provided. Conclusions In its final form, the ReAQ provides a valid unidimensional measure of core competences in pediatric residents. It produces reliable measures, distinguishes among groups of residents according to different levels of performance, and provides a resident evaluation that holds an analogous meaning for residents and faculty. PMID:23830041

The Evolution of the American Board of Ophthalmology Written Qualifying Examination.

PubMed

Wilson, David J; Tasman, William S; Skuta, Gregory L; Sheth, Bhavna P

2016-09-01

Since the inception of board certification in ophthalmology in 1916, a written assessment of candidates' knowledge base has been an integral part of the certification process. Although the committee structure and technique for writing examination questions has evolved over the past 100 years, the written qualifying examination remains an essential tool for assessing the competency of physicians entering the workforce. To develop a fair and valid examination, the American Board of Ophthalmology builds examination questions using evidence-based, peer-reviewed literature and adheres to accepted psychometric assessment standards. Copyright © 2016 American Academy of Ophthalmology. All rights reserved.

Design, implementation, and demographic differences of HEAL: a self-report health care leadership instrument

PubMed Central

Murphy, Kelly R; McManigle, John E; Wildman-Tobriner, Benjamin M; Little Jones, Amy; Dekker, Travis J; Little, Barrett A; Doty, Joseph P; Taylor, Dean C

2016-01-01

The medical community has recognized the importance of leadership skills among its members. While numerous leadership assessment tools exist at present, few are specifically tailored to the unique health care environment. The study team designed a 24-item survey (Healthcare Evaluation & Assessment of Leadership [HEAL]) to measure leadership competency based on the core competencies and core principles of the Duke Healthcare Leadership Model. A novel digital platform was created for use on handheld devices to facilitate its distribution and completion. This pilot phase involved 126 health care professionals self-assessing their leadership abilities. The study aimed to determine both the content validity of the survey and the feasibility of its implementation and use. The digital platform for survey implementation was easy to complete, and there were no technical problems with survey use or data collection. With regard to reliability, initial survey results revealed that each core leadership tenet met or exceeded the reliability cutoff of 0.7. In self-assessment of leadership, women scored themselves higher than men in questions related to patient centeredness (P=0.016). When stratified by age, younger providers rated themselves lower with regard to emotional intelligence and integrity. There were no differences in self-assessment when stratified by medical specialty. While only a pilot study, initial data suggest that HEAL is a reliable and easy-to-administer survey for health care leadership assessment. Differences in responses by sex and age with respect to patient centeredness, integrity, and emotional intelligence raise questions about how providers view themselves amid complex medical teams. As the survey is refined and further administered, HEAL will be used not only as a self-assessment tool but also in “360” evaluation formats. PMID:29355186

Design, implementation, and demographic differences of HEAL: a self-report health care leadership instrument.

PubMed

Murphy, Kelly R; McManigle, John E; Wildman-Tobriner, Benjamin M; Little Jones, Amy; Dekker, Travis J; Little, Barrett A; Doty, Joseph P; Taylor, Dean C

2016-01-01

The medical community has recognized the importance of leadership skills among its members. While numerous leadership assessment tools exist at present, few are specifically tailored to the unique health care environment. The study team designed a 24-item survey (Healthcare Evaluation & Assessment of Leadership [HEAL]) to measure leadership competency based on the core competencies and core principles of the Duke Healthcare Leadership Model. A novel digital platform was created for use on handheld devices to facilitate its distribution and completion. This pilot phase involved 126 health care professionals self-assessing their leadership abilities. The study aimed to determine both the content validity of the survey and the feasibility of its implementation and use. The digital platform for survey implementation was easy to complete, and there were no technical problems with survey use or data collection. With regard to reliability, initial survey results revealed that each core leadership tenet met or exceeded the reliability cutoff of 0.7. In self-assessment of leadership, women scored themselves higher than men in questions related to patient centeredness ( P =0.016). When stratified by age, younger providers rated themselves lower with regard to emotional intelligence and integrity. There were no differences in self-assessment when stratified by medical specialty. While only a pilot study, initial data suggest that HEAL is a reliable and easy-to-administer survey for health care leadership assessment. Differences in responses by sex and age with respect to patient centeredness, integrity, and emotional intelligence raise questions about how providers view themselves amid complex medical teams. As the survey is refined and further administered, HEAL will be used not only as a self-assessment tool but also in "360" evaluation formats.

A Key to Evaluation: The Transition Competence Battery for Deaf Adolescents and Young Adults

ERIC Educational Resources Information Center

Davis, Cheryl; Leonard, Annette

2007-01-01

Guiding deaf and hard of hearing students in making important decisions about their adult careers can be a difficult task. Twenty years after its inception, the TCB, as adapted for deaf and hard of hearing students, may help. The TCB is a unique assessment tool designed to measure the transition skills of deaf high school adolescents and young…

The Certification of Teachers' Language Competence in Quebec in French and English: Two Different Perspectives?

ERIC Educational Resources Information Center

Laurier, Michel; Baker, Beverly

2015-01-01

Language tools are reflections of the academic traditions of assessment developers, as well as the perceptions of language quality as held by a society. The certification of Quebec's teachers offers an interesting case in point: the Québec Government's "ministère de l'Éducation, du Loisir et du Sport" (MELS) requires that all students in…

Enriching Arizona's Knowledge Economy: Creating the Research Connections, Attention, and Talent Arizona Needs to Compete. Proposition 301 Investments at Arizona State University, FY 2002-FY 2005

ERIC Educational Resources Information Center

Heffernon, Rick

2006-01-01

This report presents results tracked by the CAT Measures, a 21st century assessment tool for enabling policymakers to monitor "en route" performance of their public investments in science and technology research. Developed by Morrison Institute for Public Policy at Arizona State University, the CAT Measures analyze growth supporting…

An Evaluation of Curriculum-Based Measurement of Mathematics Word Problem--Solving Measures for Monitoring Third-Grade Students' Mathematics Competence

ERIC Educational Resources Information Center

Leh, Jayne M.; Jitendra, Asha K.; Caskie, Grace I. L.; Griffin, Cynthia C.

2007-01-01

The purpose of this study was to examine the tenability of a curriculum-based mathematical word problem-solving (WPS) measure as a progress-monitoring tool to index students' rate of growth or slope of achievement over time. Participants consisted of 58 third-grade students, who were assessed repeatedly over 16 school weeks. Students were measured…

Information Literacy and Office Tool Competencies: A Benchmark Study

ERIC Educational Resources Information Center

Heinrichs, John H.; Lim, Jeen-Su

2010-01-01

Present information science literature recognizes the importance of information technology to achieve information literacy. The authors report the results of a benchmarking student survey regarding perceived functional skills and competencies in word-processing and presentation tools. They used analysis of variance and regression analysis to…

Simulation for Teaching Orthopaedic Residents in a Competency-based Curriculum: Do the Benefits Justify the Increased Costs?

PubMed

Nousiainen, Markku T; McQueen, Sydney A; Ferguson, Peter; Alman, Benjamin; Kraemer, William; Safir, Oleg; Reznick, Richard; Sonnadara, Ranil

2016-04-01

Although simulation-based training is becoming widespread in surgical education and research supports its use, one major limitation is cost. Until now, little has been published on the costs of simulation in residency training. At the University of Toronto, a novel competency-based curriculum in orthopaedic surgery has been implemented for training selected residents, which makes extensive use of simulation. Despite the benefits of this intensive approach to simulation, there is a need to consider its financial implications and demands on faculty time. This study presents a cost and faculty work-hours analysis of implementing simulation as a teaching and evaluation tool in the University of Toronto's novel competency-based curriculum program compared with the historic costs of using simulation in the residency training program. All invoices for simulation training were reviewed to determine the financial costs before and after implementation of the competency-based curriculum. Invoice items included costs for cadavers, artificial models, skills laboratory labor, associated materials, and standardized patients. Costs related to the surgical skills laboratory rental fees and orthopaedic implants were waived as a result of special arrangements with the skills laboratory and implant vendors. Although faculty time was not reimbursed, faculty hours dedicated to simulation were also evaluated. The academic year of 2008 to 2009 was chosen to represent an academic year that preceded the introduction of the competency-based curriculum. During this year, 12 residents used simulation for teaching. The academic year of 2010 to 2011 was chosen to represent an academic year when the competency-based curriculum training program was functioning parallel but separate from the regular stream of training. In this year, six residents used simulation for teaching and assessment. The academic year of 2012 to 2013 was chosen to represent an academic year when simulation was used equally among the competency-based curriculum and regular stream residents for teaching (60 residents) and among 14 competency-based curriculum residents and 21 regular stream residents for assessment. The total costs of using simulation to teach and assess all residents in the competency-based curriculum and regular stream programs (academic year 2012-2013) (CDN 155,750, USD 158,050) were approximately 15 times higher than the cost of using simulation to teach residents before the implementation of the competency-based curriculum (academic year 2008-2009) (CDN 10,090, USD 11,140). The number of hours spent teaching and assessing trainees increased from 96 to 317 hours during this period, representing a threefold increase. Although the financial costs and time demands on faculty in running the simulation program in the new competency-based curriculum at the University of Toronto have been substantial, augmented learner and trainer satisfaction has been accompanied by direct evidence of improved and more efficient learning outcomes. The higher costs and demands on faculty time associated with implementing simulation for teaching and assessment must be considered when it is used to enhance surgical training.

Risk-based approach in valuation of workplace injury rate for transportation and construction industry

NASA Astrophysics Data System (ADS)

Pykhtin, Kirill; Simankina, Tatiana; Sharmanov, Vladimir; Kopytova, Anna

2017-10-01

The danger of injuries and accidents in various industries such as transportation and construction urges the government to control the occupational health and safety more strictly. However, in order to do so with the minimal costs modern risk management tools, have to be implemented. Risk-based approach is an essential tool for competent risk- assessment and used in a great variety of other countries, demonstrating great results in providing of safe working environment. The article describes the problems that the implementation of the method faces in Russia and suggests certain ways to resolve them.

Demonstration of the Health Literacy Universal Precautions Toolkit

PubMed Central

Mabachi, Natabhona M.; Cifuentes, Maribel; Barnard, Juliana; Brega, Angela G.; Albright, Karen; Weiss, Barry D.; Brach, Cindy; West, David

2016-01-01

The Agency for Healthcare Research and Quality Health Literacy Universal Precautions Toolkit was developed to help primary care practices assess and make changes to improve communication with and support for patients. Twelve diverse primary care practices implemented assigned tools over a 6-month period. Qualitative results revealed challenges practices experienced during implementation, including competing demands, bureaucratic hurdles, technological challenges, limited quality improvement experience, and limited leadership support. Practices used the Toolkit flexibly and recognized the efficiencies of implementing tools in tandem and in coordination with other quality improvement initiatives. Practices recommended reducing Toolkit density and making specific refinements. PMID:27232681

Demonstration of the Health Literacy Universal Precautions Toolkit: Lessons for Quality Improvement.

PubMed

Mabachi, Natabhona M; Cifuentes, Maribel; Barnard, Juliana; Brega, Angela G; Albright, Karen; Weiss, Barry D; Brach, Cindy; West, David

2016-01-01

The Agency for Healthcare Research and Quality Health Literacy Universal Precautions Toolkit was developed to help primary care practices assess and make changes to improve communication with and support for patients. Twelve diverse primary care practices implemented assigned tools over a 6-month period. Qualitative results revealed challenges practices experienced during implementation, including competing demands, bureaucratic hurdles, technological challenges, limited quality improvement experience, and limited leadership support. Practices used the Toolkit flexibly and recognized the efficiencies of implementing tools in tandem and in coordination with other quality improvement initiatives. Practices recommended reducing Toolkit density and making specific refinements.

Validity evidence for an OSCE to assess competency in systems-based practice and practice-based learning and improvement: a preliminary investigation.

PubMed

Varkey, Prathibha; Natt, Neena; Lesnick, Timothy; Downing, Steven; Yudkowsky, Rachel

2008-08-01

To determine the psychometric properties and validity of an OSCE to assess the competencies of Practice-Based Learning and Improvement (PBLI) and Systems-Based Practice (SBP) in graduate medical education. An eight-station OSCE was piloted at the end of a three-week Quality Improvement elective for nine preventive medicine and endocrinology fellows at Mayo Clinic. The stations assessed performance in quality measurement, root cause analysis, evidence-based medicine, insurance systems, team collaboration, prescription errors, Nolan's model, and negotiation. Fellows' performance in each of the stations was assessed by three faculty experts using checklists and a five-point global competency scale. A modified Angoff procedure was used to set standards. Evidence for the OSCE's validity, feasibility, and acceptability was gathered. Evidence for content and response process validity was judged as excellent by institutional content experts. Interrater reliability of scores ranged from 0.85 to 1 for most stations. Interstation correlation coefficients ranged from -0.62 to 0.99, reflecting case specificity. Implementation cost was approximately $255 per fellow. All faculty members agreed that the OSCE was realistic and capable of providing accurate assessments. The OSCE provides an opportunity to systematically sample the different subdomains of Quality Improvement. Furthermore, the OSCE provides an opportunity for the demonstration of skills rather than the testing of knowledge alone, thus making it a potentially powerful assessment tool for SBP and PBLI. The study OSCE was well suited to assess SBP and PBLI. The evidence gathered through this study lays the foundation for future validation work.

Active learning through a debate series in a first-year pharmacy self-care course.

PubMed

Lampkin, Stacie J; Collins, Christine; Danison, Ryan; Lewis, Michelle

2015-03-25

To evaluate the usefulness of formal debates in the pharmacy classroom as a way to learn course material and as a tool for developing competency in essential skills including critical thinking, communication, public speaking, research methods, and teamwork. Debates were incorporated into a self-care course, where students were assigned different debate topics focused on controversial issues. Quantitative analysis was completed to assess debate style learning, knowledge about the subjects presented, and the impact on necessary skills. Quizzes given before and after debates showed up to a 36% improvement in grades and up to a 31% change in opinions on the topic. Students assessed themselves as more competent in the skill sets at the completion of the debate series. Incorporation of debates into didactic style courses offers students an opportunity to improve upon skills that will help them succeed as pharmacists.

Competence for older people nursing in care and nursing homes: An integrative review.

PubMed

Kiljunen, Outi; Välimäki, Tarja; Kankkunen, Päivi; Partanen, Pirjo

2017-09-01

People living in care and nursing homes are vulnerable individuals with complex needs; therefore, a wide array of nursing competence is needed to ensure their well-being. When developing the quality of care in these units, it is essential to know what type of competence is required for older people nursing. The aim of this integrative review was to identify the competence needed for older people nursing in licensed practical nurses' and registered nurses' work in care and nursing homes. Integrative literature review. We performed an integrative review using Whittemore and Knafl's method. The CINAHL, MEDLINE, PsycINFO, SocINDEX and Scopus databases were searched for studies published from 2006 to April 2016. We assessed the quality of the studies using Joanna Briggs Institute critical appraisal tools and analysed the data by applying qualitative content analysis. Ten articles were included in the review. Most of the studies focused on registered nurses' work. We identified five competence areas that are needed for older people nursing in registered nurses' work in care and nursing homes: attitudinal and ethical, interactional, evidence-based care, pedagogical, and leadership and development competence. Empirical evidence of competence requirements related to licensed practical nurses' work in these facilities was scarce. The competence required for registered nurses and licensed practical nurses should be clearly identified to support competence management in the care and nursing home context. Well-educated nursing staff are needed in care and nursing homes to provide high-quality care because comprehensive and advanced nurse competence is required to meet the needs of older people. © 2016 John Wiley & Sons Ltd.

Going DEEP: guidelines for building simulation-based team assessments.

PubMed

Grand, James A; Pearce, Marina; Rench, Tara A; Chao, Georgia T; Fernandez, Rosemarie; Kozlowski, Steve W J

2013-05-01

Whether for team training, research or evaluation, making effective use of simulation-based technologies requires robust, reliable and accurate assessment tools. Extant literature on simulation-based assessment practices has primarily focused on scenario and instructional design; however, relatively little direct guidance has been provided regarding the challenging decisions and fundamental principles related to assessment development and implementation. The objective of this manuscript is to introduce a generalisable assessment framework supplemented by specific guidance on how to construct and ensure valid and reliable simulation-based team assessment tools. The recommendations reflect best practices in assessment and are designed to empower healthcare educators, professionals and researchers with the knowledge to design and employ valid and reliable simulation-based team assessments. Information and actionable recommendations associated with creating assessments of team processes (non-technical 'teamwork' activities) and performance (demonstration of technical proficiency) are presented which provide direct guidance on how to Distinguish the underlying competencies one aims to assess, Elaborate the measures used to capture team member behaviours during simulation activities, Establish the content validity of these measures and Proceduralise the measurement tools in a way that is systematically aligned with the goals of the simulation activity while maintaining methodological rigour (DEEP). The DEEP framework targets fundamental principles and critical activities that are important for effective assessment, and should benefit healthcare educators, professionals and researchers seeking to design or enhance any simulation-based assessment effort.

Individualized education and competency development of Croatian community pharmacists using the general level framework.

PubMed

Meštrović, Arijana; Staničić, Zivka; Hadžiabdić, Maja Ortner; Mucalo, Iva; Bates, Ian; Duggan, Catherine; Carter, Sarah; Bruno, Andreia; Košiček, Miljenko

2012-03-12

To measure Croatian community pharmacists' progress in competency development using the General Level Framework (GLF) as an educational tool in a longitudinal study. Patient care competencies of 100 community pharmacists were evaluated twice, in 2009 and in 2010 in a prospective cohort study. During this 12-month period, tailored educational programs based on the GLF were organized and conducted, new services and standard operating procedures were implemented, and documentation of contributions to patient care in the pharmacist's portfolio became mandatory. Pharmacists' development of all GLF patient care competencies was significant with the greatest improvements seen in the following competencies: patient consultation, monitoring drug therapy, medicine information and patient education, and evaluation of outcomes. This study, which retested the effectiveness of an evidence-based competency framework, confirmed that GLF is a valid educational tool for pharmacist development.

Teaching nurses teamwork: Integrative review of competency-based team training in nursing education.

PubMed

Barton, Glenn; Bruce, Anne; Schreiber, Rita

2017-12-20

Widespread demands for high reliability healthcare teamwork have given rise to many educational initiatives aimed at building team competence. Most effort has focused on interprofessional team training however; Registered Nursing teams comprise the largest human resource delivering direct patient care in hospitals. Nurses also influence many other health team outcomes, yet little is known about the team training curricula they receive, and furthermore what specific factors help translate teamwork competency to nursing practice. The aim of this review is to critically analyse empirical published work reporting on teamwork education interventions in nursing, and identify key educational considerations enabling teamwork competency in this group. CINAHL, Web of Science, Academic Search Complete, and ERIC databases were searched and detailed inclusion-exclusion criteria applied. Studies (n = 19) were selected and evaluated using established qualitative-quantitative appraisal tools and a systematic constant comparative approach. Nursing teamwork knowledge is rooted in High Reliability Teams theory and Crew or Crisis Resource Management sources. Constructivist pedagogy is used to teach, practice, and refine teamwork competency. Nursing teamwork assessment is complex; involving integrated yet individualized determinations of knowledge, skills, and attitudes. Future initiatives need consider frontline leadership, supportive followership and skilled communication emphasis. Collective stakeholder support is required to translate teamwork competency into nursing practice. Copyright © 2017 Elsevier Ltd. All rights reserved.

[Psychosocial Risk Evaluation in the Workplace: Expert-based Development of a Checklist for Occupational Physicians].

PubMed

Weigl, M; Müller, A; Angerer, P; Petru, R

2016-03-01

The implementation of psychosocial risk assessment at the workplace often fails in practice. One reason is the lack of competence of those who are in charge of the process. We present a checklist for the effective implementation of psychosocial risk assessment at workplace. This tool shall support occupational physicians in the preparation, planning and implementation of a psychosocial risks assessment process. Based on a stepwise development and validation process, specific steps and factors for the successful implementation were identified qualitatively with 15 occupational physicians and experts. This was conducted in a 2-stage Delphi study. In the following, the identified steps and factors were transferred into a checklist. Subsequently, the checklist was evaluated in a focus group of occupational physicians (user evaluation). Thereafter, the contents were subjected to an expert evaluation. Our checklist for the effective implementation of the process of psychosocial risk management in the workplace aims to strengthen the competence of occupational physicians, especially in the implementation of risk assessments in small and medium-sized enterprises (SMEs). © Georg Thieme Verlag KG Stuttgart · New York.

The relevance of the alliance for CME competencies for planning, organizing, and sustaining an interorganizational educational collaborative.

PubMed

Balmer, Jann T; Bellande, Bruce J; Addleton, Robert L; Havens, Carol S

2011-01-01

The heightened demand for accountability, access, and quality performance from health care professionals has resulted in linkages between continuing education (CE), performance improvement (PI), and outcomes. CE health professionals must also expand their skills and abilities to design, implement, and measure CE activities consistent with these new expectations. In addition to administrative and meeting-planning activities, new competencies associated with educational consultation and performance coaching are needed. This article utilizes the Alliance competencies as the framework for discussion of the competencies of CE professionals and applies it to the unique setting of a collaborative. The CS2day initiative serves as an example of the application of these competencies in this environment. The framework of the Alliance competencies can serve as a guide and a tool for self-assessment, work design, and professional development at individual, organization, and systems levels. Continual reassessment of the Alliance competencies for CE in the health professions will be critical to the continued effectiveness of CE that is linked to performance improvement and outcomes for the CE professional and the health care professionals we serve. A collaborative can provide one option for meeting these new expectations for professional development for CE professionals and the creation of effective educational initiatives. Copyright © 2011 The Alliance for Continuing Medical Education, the Society for Academic Continuing Medical Education, and the Council on CME, Association for Hospital Medical Education.

Development and psychometric testing of the nursing student mentors' competence instrument (MCI): A cross-sectional study.

PubMed

Tuomikoski, Anna-Maria; Ruotsalainen, Heidi; Mikkonen, Kristina; Miettunen, Jouko; Kääriäinen, Maria

2018-06-05

Mentors require competence at a diverse array of skills to mentor students during clinical practice. According to the latest evidence, competence at mentoring includes: knowledge, skills and attributes of individual students' learning objectives, core elements of nursing, learning processes, a reciprocal and trustful relationship, feedback, evaluation, cooperation with stakeholders, and the mentor's personal qualities. The purpose of the study was to test psychometric properties of a mentor's competence instrument developed to self-evaluate mentors' competence at mentoring nursing students in clinical practice. A cross-sectional, descriptive, explorative study design was used. Data were collected from mentors at five university hospitals in Finland in 2016. A total of 576 mentors participated in this study. The instrument was developed through systematic review, experts' evaluations, and pilot versions of the instrument tested in previous studies. The construct validity and reliability of the instrument were tested using exploratory factor analysis (EFA) with promax rotation and Cronbach's alpha. A 10-factor model showed that the instrument has acceptable construct validity. Cronbach's alpha values for the subscales observed ranged from 0.76 to 0.90. The instrument exhibited acceptable psychometric properties, thereby proving itself a valuable tool for evaluating mentors' competence at mentoring students. Further assessments of its reliability, validity and generality for measuring mentor's competence for mentoring students in different contexts and cultures are recommended. Copyright © 2018 Elsevier Ltd. All rights reserved.

The development of a cultural standardized patient examination for a general surgery residency program.

PubMed

Chun, Maria B J; Young, Keane G M; Honda, Andrea F; Belcher, Gary F; Maskarinec, Gregory G

2012-01-01

Cultural competency and cross-cultural care issues in surgery resident education are areas of recognized need. The Accreditation Council for Graduate Medical Education (ACGME) has developed 6 core competencies addressing training to provide high quality care. Of these, cultural training is addressed under 3: patient care, professionalism, and interpersonal and communication skills. Our study sought to develop a measurable tool-a cultural standardized patient (SP) examination-that integrates cross-cultural care issues within the core competencies. All first year surgery residents (PGY-1) were required to participate in the videotaped cultural SP examination as part of the general surgery residency curriculum. Two measures were utilized to assess resident performance. On the same day, we administered a Cross-Cultural Care Survey. The SP examination was assessed by trained surgery teaching faculty using a written checklist that was developed to evaluate residents on all 6 ACGME competencies. Of the 26 eligible participants over 2 years, we were able to analyze the pre- and post-test results for 24 residents. The post-test score of the "attitude toward cross-cultural care" subscale of the Cross-Cultural Care Survey was significantly lower than the pre-test score (p = 0.012; Wilcoxon signed-ranks test). There were significant differences by ethnicity on all 3 subscales of the Cross-Cultural Care Survey (attitude = p < 0.05, knowledge = p < 0.01, skills = p < 0.05) on the pre-test. However, only the knowledge subscale scores remained significantly different between ethnicities on the post-test (p < 0.01). After additional assessment, evaluation, and refinement, our goal is to incorporate cross-cultural health care training as a permanent part of our curriculum. Our hope is that efforts to provide training in cross-cultural healthcare leads to high quality care and positive outcomes for the patient. This will not only enhance our training program, but may also become a useful tool for other surgery residency programs. Copyright © 2012 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Tool & Die Technician.

ERIC Educational Resources Information Center

Ohio State Univ., Columbus. Center on Education and Training for Employment.

This document contains 23 units to consider for use in a tech prep competency profile for the occupation of tool and die technician. All the units listed will not necessarily apply to every situation or tech prep consortium, nor will all the competencies within each unit be appropriate. Several units appear within each specific occupation and…

Psychometric properties of the Persian version of the Intensive and Critical Care Nursing Competence Scale version-1 (ICCN-CS-1)

PubMed Central

Shouryabi, Ali Asghar; Ghahrisarabi, Alireza; Anboohi, Sima Zohari; Nasiri, Malihe; Rassouli, Maryam

2017-01-01

Background Nursing competence is highly related to patient outcomes and patient safety issues, especially in intensive care units. Competence assessment tools are needed specifically for intensive care nursing. Objective This study was performed to determine psychometric properties of the Intensive and Critical Care Nursing Competence Scale version-1 between Iranian Nurses. Methods The present study was a methodological research in which 289 nurses of Intensive Care Units from nine hospitals in Shahid Beheshti University of Medical Sciences in Tehran were selected between 2015 and 2016. The original version of the scale was translated into Persian and back-translated into English, and the comments of the developer were applied. The validity of the scale was the determined quality (content validity and face validity) and quantity (confirmatory factor analysis). Reliability of the scale was reported by Cronbach’s alpha coefficient and Intra class Correlation Coefficient. SPSS-PC (v.21) and LISREL (v.8.5) were used to analyze the data. Results The intensive and critical care nursing competence scale version-1 is a self-assessment test that consists of 144 items and four domains which are the knowledge base, the skill base, the attitudes and values base and the experience base, which are divided into clinical competence and professional competence. Content and face validity was confirmed by 10 experts and 10 practitioner nurses in the intensive care units. In confirmatory factor analysis, all fitness indexes, except goodness of fit index (0.64), confirmed the four-factor structure of the ICCN-CS-1. The results of the factor analysis, load factor between 0.304 and 0.727 items was estimated; only 4 items out of 144 items, that were loaded were less than 0.3 due to high Cronbach’s alpha coefficient (0.984–0.986), all items were preserved, no item was removed and 4 subscales of the original scale were confirmed. Conclusion The results of this study indicated that the Persian version of “The Intensive and Critical Care Nursing Competence Scale version-1” is a valid and reliable scale for the assessment of competency among Iranian nurses, and it can be used as a reliable scale in nursing management, education and research. PMID:29403620

The effects of a visualization-centered curriculum on conceptual understanding and representational competence in high school biology

NASA Astrophysics Data System (ADS)

Wilder, Anna

The purpose of this study was to investigate the effects of a visualization-centered curriculum, Hemoglobin: A Case of Double Identity, on conceptual understanding and representational competence in high school biology. Sixty-nine students enrolled in three sections of freshman biology taught by the same teacher participated in this study. Online Chemscape Chime computer-based molecular visualizations were incorporated into the 10-week curriculum to introduce students to fundamental structure and function relationships. Measures used in this study included a Hemoglobin Structure and Function Test, Mental Imagery Questionnaire, Exam Difficulty Survey, the Student Assessment of Learning Gains, the Group Assessment of Logical Thinking, the Attitude Toward Science in School Assessment, audiotapes of student interviews, students' artifacts, weekly unit activity surveys, informal researcher observations and a teacher's weekly questionnaire. The Hemoglobin Structure and Function Test, consisting of Parts A and B, was administered as a pre and posttest. Part A used exclusively verbal test items to measure conceptual understanding, while Part B used visual-verbal test items to measure conceptual understanding and representational competence. Results of the Hemoglobin Structure and Function pre and posttest revealed statistically significant gains in conceptual understanding and representational competence, suggesting the visualization-centered curriculum implemented in this study was effective in supporting positive learning outcomes. The large positive correlation between posttest results on Part A, comprised of all-verbal test items, and Part B, using visual-verbal test items, suggests this curriculum supported students' mutual development of conceptual understanding and representational competence. Evidence based on student interviews, Student Assessment of Learning Gains ratings and weekly activity surveys indicated positive attitudes toward the use of Chemscape Chime software and the computer-based molecular visualization activities as learning tools. Evidence from these same sources also indicated that students felt computer-based molecular visualization activities in conjunction with other classroom activities supported their learning. Implications for instructional design are discussed.

Development and validation of a nursing professionalism evaluation model in a career ladder system.

PubMed

Kim, Yeon Hee; Jung, Young Sun; Min, Ja; Song, Eun Young; Ok, Jung Hui; Lim, Changwon; Kim, Kyunghee; Kim, Ji-Su

2017-01-01

The clinical ladder system categorizes the degree of nursing professionalism and rewards and is an important human resource tool for managing nursing. We developed a model to evaluate nursing professionalism, which determines the clinical ladder system levels, and verified its validity. Data were collected using a clinical competence tool developed in this study, and existing methods such as the nursing professionalism evaluation tool, peer reviews, and face-to-face interviews to evaluate promotions and verify the presented content in a medical institution. Reliability and convergent and discriminant validity of the clinical competence evaluation tool were verified using SmartPLS software. The validity of the model for evaluating overall nursing professionalism was also analyzed. Clinical competence was determined by five dimensions of nursing practice: scientific, technical, ethical, aesthetic, and existential. The structural model explained 66% of the variance. Clinical competence scales, peer reviews, and face-to-face interviews directly determined nursing professionalism levels. The evaluation system can be used for evaluating nurses' professionalism in actual medical institutions from a nursing practice perspective. A conceptual framework for establishing a human resources management system for nurses and a tool for evaluating nursing professionalism at medical institutions is provided.

Intercultural Teaching Competence: A Multi-Disciplinary Model for Instructor Reflection

ERIC Educational Resources Information Center

Dimitrov, Nanda; Haque, Aisha

2016-01-01

This article presents a model for Intercultural Teaching Competence (ITC) that instructors may use as a tool for reflection as they prepare to facilitate learning across cultures. Building on previous research on intercultural competence, culturally relevant teaching, intercultural trainer competencies, and student-centred approaches to teaching,…

A core competency-based objective structured clinical examination (OSCE) can predict future resident performance.

PubMed

Wallenstein, Joshua; Heron, Sheryl; Santen, Sally; Shayne, Philip; Ander, Douglas

2010-10-01

This study evaluated the ability of an objective structured clinical examination (OSCE) administered in the first month of residency to predict future resident performance in the Accreditation Council for Graduate Medical Education (ACGME) core competencies. Eighteen Postgraduate Year 1 (PGY-1) residents completed a five-station OSCE in the first month of postgraduate training. Performance was graded in each of the ACGME core competencies. At the end of 18 months of training, faculty evaluations of resident performance in the emergency department (ED) were used to calculate a cumulative clinical evaluation score for each core competency. The correlations between OSCE scores and clinical evaluation scores at 18 months were assessed on an overall level and in each core competency. There was a statistically significant correlation between overall OSCE scores and overall clinical evaluation scores (R = 0.48, p < 0.05) and in the individual competencies of patient care (R = 0.49, p < 0.05), medical knowledge (R = 0.59, p < 0.05), and practice-based learning (R = 0.49, p < 0.05). No correlation was noted in the systems-based practice, interpersonal and communication skills, or professionalism competencies. An early-residency OSCE has the ability to predict future postgraduate performance on a global level and in specific core competencies. Used appropriately, such information can be a valuable tool for program directors in monitoring residents' progress and providing more tailored guidance. © 2010 by the Society for Academic Emergency Medicine.

The effectiveness of gynaecology teaching associates in teaching pelvic examination to medical students: a randomised controlled trial.

PubMed

Janjua, Aisha; Smith, P; Chu, J; Raut, N; Malick, S; Gallos, I; Singh, R; Irani, S; Gupta, J K; Parle, J; Clark, T J

2017-03-01

To assess whether teaching female pelvic examinations using gynaecological teaching associates (GTAs); women who are trained to give instruction and feedback on gynaecological examination technique, improves the competence, confidence and communication skills of medical students compared to conventional teaching. Randomised controlled trial. Ten University of Birmingham (UoB) affiliated teaching hospitals in the UK. 492 final year medical students. GTA teaching of gynaecological examination compared with conventional pelvic manikin based teaching at the start of a five week clinical placement in obstetrics and gynaecology (O&G). Student's perception of their confidence was measured on a 10cm visual analogue scale (VAS). Domains of competence were measured by a senior clinical examiner using a standardised assessment tool which utilised 10cm VAS and by a GTA using a four point Likert scale. Assessors were blinded to the allocated teaching intervention. 407/492 (83%) students completed both the intervention and outcome assessment. Self-reported confidence was higher in students taught by GTAs compared with those taught on manikins (median score GTA 6.3; vs. conventional 5.8; p=0.03). Competence was also higher in those taught by GTAs when assessed by an examiner (median global score GTA 7.1 vs. conventional 6.0; p<0.001) and by a GTA (p<0.001). GTA teaching of female pelvic examination at the start of undergraduate medical student O&G clinical placements improves their confidence and competence compared with conventional pelvic manikin based teaching. GTAs should be introduced into undergraduate medical curricula to teach pelvic examination. Crown Copyright © 2016. Published by Elsevier B.V. All rights reserved.

Assessing computer skills in Tanzanian medical students: an elective experience

PubMed Central

Samuel, Miriam; Coombes, John C; Miranda, J Jaime; Melvin, Rob; Young, Eoin JW; Azarmina, Pejman

2004-01-01

Background One estimate suggests that by 2010 more than 30% of a physician's time will be spent using information technology tools. The aim of this study is to assess the information and communication technologies (ICT) skills of medical students in Tanzania. We also report a pilot intervention of peer mentoring training in ICT by medical students from the UK tutoring students in Tanzania. Methods Design: Cross sectional study and pilot intervention study. Participants: Fourth year medical students (n = 92) attending Muhimbili University College of Health Sciences, Dar es Salaam, Tanzania. Main outcome measures: Self-reported assessment of competence on ICT-related topics and ability to perform specific ICT tasks. Further information related to frequency of computer use (hours per week), years of computer use, reasons for use and access to computers. Skills at specific tasks were reassessed for 12 students following 4 to 6 hours of peer mentoring training. Results The highest levels of competence in generic ICT areas were for email, Internet and file management. For other skills such as word processing most respondents reported low levels of competence. The abilities to perform specific ICT skills were low – less than 60% of the participants were able to perform the core specific skills assessed. A period of approximately 5 hours of peer mentoring training produced an approximate doubling of competence scores for these skills. Conclusion Our study has found a low level of ability to use ICT facilities among medical students in a leading university in sub-Saharan Africa. A pilot scheme utilising UK elective students to tutor basic skills showed potential. Attention is required to develop interventions that can improve ICT skills, as well as computer access, in order to bridge the digital divide. PMID:15306029

Assessing computer skills in Tanzanian medical students: an elective experience.

PubMed

Samuel, Miriam; Coombes, John C; Miranda, J Jaime; Melvin, Rob; Young, Eoin J W; Azarmina, Pejman

2004-08-12

One estimate suggests that by 2010 more than 30% of a physician's time will be spent using information technology tools. The aim of this study is to assess the information and communication technologies (ICT) skills of medical students in Tanzania. We also report a pilot intervention of peer mentoring training in ICT by medical students from the UK tutoring students in Tanzania. Cross sectional study and pilot intervention study. Fourth year medical students (n = 92) attending Muhimbili University College of Health Sciences, Dar es Salaam, Tanzania. Self-reported assessment of competence on ICT-related topics and ability to perform specific ICT tasks. Further information related to frequency of computer use (hours per week), years of computer use, reasons for use and access to computers. Skills at specific tasks were reassessed for 12 students following 4 to 6 hours of peer mentoring training. The highest levels of competence in generic ICT areas were for email, Internet and file management. For other skills such as word processing most respondents reported low levels of competence. The abilities to perform specific ICT skills were low - less than 60% of the participants were able to perform the core specific skills assessed. A period of approximately 5 hours of peer mentoring training produced an approximate doubling of competence scores for these skills. Our study has found a low level of ability to use ICT facilities among medical students in a leading university in sub-Saharan Africa. A pilot scheme utilising UK elective students to tutor basic skills showed potential. Attention is required to develop interventions that can improve ICT skills, as well as computer access, in order to bridge the digital divide.

Validity Evidence for the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT).

PubMed

Breimer, Gerben E; Haji, Faizal A; Cinalli, Giuseppe; Hoving, Eelco W; Drake, James M

2017-02-01

Growing demand for transparent and standardized methods for evaluating surgical competence prompted the construction of the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). To provide validity evidence of the NEVAT by reporting on the tool's internal structure and its relationship with surgical expertise during simulation-based training. The NEVAT was used to assess performance of trainees and faculty at an international neuroendoscopy workshop. All participants performed an endoscopic third ventriculostomy (ETV) on a synthetic simulator. Participants were simultaneously scored by 2 raters using the NEVAT procedural checklist and global rating scale (GRS). Evidence of internal structure was collected by calculating interrater reliability and internal consistency of raters' scores. Evidence of relationships with other variables was collected by comparing the ETV performance of experts, experienced trainees, and novices using Jonckheere's test (evidence of construct validity). Thirteen experts, 11 experienced trainees, and 10 novices participated. The interrater reliability by the intraclass correlation coefficient for the checklist and GRS was 0.82 and 0.94, respectively. Internal consistency (Cronbach's α) for the checklist and the GRS was 0.74 and 0.97, respectively. Median scores with interquartile range on the checklist and GRS for novices, experienced trainees, and experts were 0.69 (0.58-0.86), 0.85 (0.63-0.89), and 0.85 (0.81-0.91) and 3.1 (2.5-3.8), 3.7 (2.2-4.3) and 4.6 (4.4-4.9), respectively. Jonckheere's test showed that the median checklist and GRS score increased with performer expertise ( P = .04 and .002, respectively). This study provides validity evidence for the NEVAT to support its use as a standardized method of evaluating neuroendoscopic competence during simulation-based training. Copyright © 2016 by the Congress of Neurological Surgeons

Management of an affiliated Physics Residency Program using a commercial software tool.

PubMed

Zacarias, Albert S; Mills, Michael D

2010-06-01

A review of commercially available allied health educational management software tools was performed to evaluate their capacity to manage program data associated with a CAMPEP-accredited Therapy Physics Residency Program. Features of these software tools include: a) didactic course reporting and organization, b) competency reporting by topic, category and didactic course, c) student time management and accounting, and d) student patient case reporting by topic, category and course. The software package includes features for recording school administrative information; setting up lists of courses, faculty, clinical sites, categories, competencies, and time logs; and the inclusion of standardized external documents. There are provisions for developing evaluation and survey instruments. The mentors and program may be evaluated by residents, and residents may be evaluated by faculty members using this feature. Competency documentation includes the time spent on the problem or with the patient, time spent with the mentor, date of the competency, and approval by the mentor and program director. Course documentation includes course and lecture title, lecturer, topic information, date of lecture and approval by the Program Director. These software tools have the facility to include multiple clinical sites, with local subadministrators having the ability to approve competencies and attendance at clinical conferences. In total, these software tools have the capability of managing all components of a CAMPEP-accredited residency program. The application database lends the software to the support of multiple affiliated clinical sites within a single residency program. Such tools are a critical and necessary component if the medical physics profession is to meet the projected needs for qualified medical physicists in future years.

The back squat: A proposed assessment of functional deficits and technical factors that limit performance

PubMed Central

Myer, Gregory D.; Kushner, Adam M.; Brent, Jensen L.; Schoenfeld, Brad J.; Hugentobler, Jason; Lloyd, Rhodri S.; Vermeil, Al; Chu, Donald A.; Harbin, Jason; McGill, Stuart M.

2014-01-01

Fundamental movement competency is essential for participation in physical activity and for mitigating the risk of injury, which are both key elements of health throughout life. The squat movement pattern is arguably one of the most primal and critical fundamental movements necessary to improve sport performance, to reduce injury risk and to support lifelong physical activity. Based on current evidence, this first (1 of 2) report deconstructs the technical performance of the back squat as a foundation training exercise and presents a novel dynamic screening tool that incorporates identification techniques for functional deficits that limit squat performance and injury resilience. The follow-up report will outline targeted corrective methodology for each of the functional deficits presented in the assessment tool. PMID:25506270

Behavioral and Psychological Symptoms Impact Clinical Competence in Alzheimer’s Disease

PubMed Central

Bertrand, Elodie; van Duinkerken, Eelco; Landeira-Fernandez, J.; Dourado, Marcia C. N.; Santos, Raquel L.; Laks, Jerson; Mograbi, Daniel C.

2017-01-01

Decision-making is considered a fundamental aspect of personal autonomy and can be affected in psychiatric and neurologic diseases. It has been shown that cognitive deficits in dementia impact negatively on decision-making. Moreover, studies highlighted impaired clinical competence in neuropsychiatric disorders, such as schizophrenia and bipolar disorder. In this context, the current study explored the relationship between behavioral and psychological symptoms of dementia (BPSD) and clinical competence, especially the capacity to consent to treatment, in Alzheimer’s disease (AD). Seventy-one patients with mild to moderate AD participated, completing assessments for capacity to consent to treatment, general cognition and neuropsychiatric disturbances. For each neuropsychiatric symptom, patients with and without the particular disturbance were compared on the different subscales of the MacArthur Competence Tool for Treatment (MacCAT-T; Understanding, Appreciation, Reasoning and Expression). The results showed that patients presenting delusions, as well as apathetic patients, had a lower ability to express a clear treatment choice compared to patients without these symptoms. By contrast, patients with dysphoria/depression had higher scores on this variable. Additionally, AD patients with euphoria had more difficulties discussing consequences of treatment alternatives compared to patients without this disturbance. None of the differences were confounded by global cognition. There were no between-group differences in clinical decision-making for patients with hallucinations, agitation/aggression, anxiety, irritability, disinhibition and aberrant motor behavior. These findings highlight the importance of taking BPSD into account when assessing decision-making capacity, especially clinical competence, in AD. Furthermore, reducing BPSD may lead to better clinical competence in patients with AD, as well as to improvements in patients and caregivers’ quality of life. PMID:28670272

Calibrating perceived understanding and competency in probability concepts: A diagnosis of learning difficulties based on Rasch probabilistic model

NASA Astrophysics Data System (ADS)

Mahmud, Zamalia; Porter, Anne; Salikin, Masniyati; Ghani, Nor Azura Md

2015-12-01

Students' understanding of probability concepts have been investigated from various different perspectives. Competency on the other hand is often measured separately in the form of test structure. This study was set out to show that perceived understanding and competency can be calibrated and assessed together using Rasch measurement tools. Forty-four students from the STAT131 Understanding Uncertainty and Variation course at the University of Wollongong, NSW have volunteered to participate in the study. Rasch measurement which is based on a probabilistic model is used to calibrate the responses from two survey instruments and investigate the interactions between them. Data were captured from the e-learning platform Moodle where students provided their responses through an online quiz. The study shows that majority of the students perceived little understanding about conditional and independent events prior to learning about it but tend to demonstrate a slightly higher competency level afterward. Based on the Rasch map, there is indication of some increase in learning and knowledge about some probability concepts at the end of the two weeks lessons on probability concepts.

In-training assessment: qualitative study of effects on supervision and feedback in an undergraduate clinical rotation.

PubMed

Daelmans, H E M; Overmeer, R M; van der Hem-Stokroos, H H; Scherpbier, A J J A; Stehouwer, C D A; van der Vleuten, C P M

2006-01-01

Supervision and feedback are essential factors that contribute to the learning environment in the context of workplace learning and their frequency and quality can be improved. Assessment is a powerful tool with which to influence students' learning and supervisors' teaching and thus the learning environment. To investigate an in-training assessment (ITA) programme in action and to explore its effects on supervision and feedback. A qualitative study using individual, semistructured interviews. Eight students and 17 assessors (9 members of staff and 8 residents) in the internal medicine undergraduate clerkship at Vrije Universiteit Medical Centre, Amsterdam, the Netherlands. The ITA programme in action differed from the intended programme. Assessors provided hardly any follow-up on supervision and feedback given during assessments. Although students wanted more supervision and feedback, they rarely asked for it. Students and assessors failed to integrate the whole range of competencies included in the ITA programme into their respective learning and supervision and feedback. When giving feedback, assessors rarely gave borderline or fail judgements. If an ITA programme in action is to be congruent with the intended programme, the implementation of the programme must be monitored. It is also necessary to provide full information about the programme and to ensure this information is given repeatedly. Introducing an ITA programme that includes the assessment of several competencies does not automatically lead to more attention being paid to these competencies in terms of supervision and feedback. Measures that facilitate change in the learning environment seem to be a prerequisite for enabling the assessment programme to steer the learning environment.

Development of a competency mapping tool for undergraduate professional degree programmes, using mechanical engineering as a case study

NASA Astrophysics Data System (ADS)

Holmes, David W.; Sheehan, Madoc; Birks, Melanie; Smithson, John

2018-01-01

Mapping the curriculum of a professional degree to the associated competency standard ensures graduates have the competence to perform as professionals. Existing approaches to competence mapping vary greatly in depth, complexity, and effectiveness, and a standardised approach remains elusive. This paper describes a new mapping software tool that streamlines and standardises the competency mapping process. The available analytics facilitate ongoing programme review, management, and accreditation. The complete mapping and analysis of an Australian mechanical engineering degree programme is described as a case study. Each subject is mapped by evaluating the amount and depth of competence development present. Combining subject results then enables highly detailed programme level analysis. The mapping process is designed to be administratively light, with aspects of professional development embedded in the software. The effective competence mapping described in this paper enables quantification of learning within a professional degree programme, and provides a mechanism for holistic programme improvement.

Building Culturally & Linguistically Competent Services to Support Young Children, Their Families, and School Readiness

ERIC Educational Resources Information Center

Hepburn, Kathy Seitzinger

2004-01-01

The purpose of this tool kit is to provide guidance, tools, and resources that will assist communities in building culturally and linguistically competent services, supports, programs, and practices related to young children, their families. By offering services in culturally and linguistically meaningful ways, communities can engage all families…

Evaluation Tools in the European Higher Education Area (EHEA): An Assessment for Evaluating the Competences of the Final Year Project in the Social Sciences

ERIC Educational Resources Information Center

Mateo, Joan; Escofet, Anna; Martinez-Olmo, Francesc; Ventura, Javier; Vlachopoulos, Dimitrios

2012-01-01

The guidelines of the European Higher Education Area (EHEA) imply the rethinking of many of the current evaluation systems, since the new pedagogical models now focus on the learning acquired through the students' personal work and on the establishment of the ideal conditions for them to achieve the learning outcomes of the proposed educational…

Profile of an excellent nurse manager: identifying and developing health care team leaders.

PubMed

Kallas, Kathryn D

2014-01-01

The purpose of this research was to identify the profile of an excellent nurse manager who can lead effective health care teams. Leadership attributes and competencies that characterize an excellent nurse manager and tools to identify them are lacking in the literature but are required to efficiently and effectively address the growing shortage of registered nurses (RNs) in health care team leadership roles and the critical linkage of these roles to patient outcomes. A profile of an excellent nurse manager was developed on the basis of the responses of nurse managers across the United States who had been identified as excellent or competent by chief nurse executive assessment or/and the Nurse Manager Ability, Leadership, and Support of Nurses staff survey to the Kouzes and Posner Leadership Practices Inventory: Self Instrument. Statistically significant distinctions exist between nurse managers who are excellent and those who are competent as assessed by the Five Practices of Exemplary Leadership, which together comprise the profile of an excellent nurse manager. The Kouzes and Posner Leadership Practices Inventory: Self Instrument can be used to identify, recruit, and develop RNs in the nurse manager role as excellent leaders of effective health care teams.

Assessing capacity to consent to treatment with cholinesterase inhibitors in dementia using a specific and standardized version of the MacArthur Competence Assessment Tool (MacCAT-T).

PubMed

Mueller, Tanja; Haberstroh, Julia; Knebel, Maren; Oswald, Frank; Kaspar, Roman; Kemper, Christoph J; Halder-Sinn, Petra; Schroeder, Johannes; Pantel, Johannes

2017-02-01

The use of assessment tools has been shown to improve the inter-rater reliability of capacity assessments. However, instrument-based capacity assessments of people with dementia face challenges. In dementia research, measuring capacity with instruments like the MacArthur Competence Assessment Tool for Treatment (MacCAT-T) mostly employ hypothetical treatment vignettes that can overwhelm the abstraction capabilities of people with dementia and are thus not always suitable for this target group. The primary aim of this study was to provide a standardized real informed consent paradigm that enables the dementia-specific properties of capacity to consent to treatment in people with dementia to be identified in a real informed consent process that is both externally valid and ethically justifiable. The sample consisted of 53 people with mild to moderate dementia and a group of 133 people without cognitive impairment. Rather than using a hypothetical treatment vignette, we used a standardized version of the MacCAT-T to assess capacity to consent to treatment with cholinesterase inhibitors in people with dementia. Inter-rater reliability, item statistics, and psychometric properties were also investigated. Intraclass correlations (ICCs) (0.951-0.990) indicated high inter-rater reliability of the standardized real informed consent paradigm. In the dementia group, performance on different items of the MacCAT-T varied. Most people with dementia were able to express a treatment choice, and were aware of the need to take a tablet. Further information on the course of the disorder and the benefits and risks of the treatment were less understood, as was comparative reasoning regarding treatment alternatives. The standardized real informed consent paradigm enabled us to detect dementia-specific characteristics of patients' capacity to consent to treatment with cholinesterase inhibitors. In order to determine suitable enhanced consent procedures for this treatment, we recommend the consideration of MacCAT-T results on an item level. People with dementia seem to understand only basic information. Our data indicate that one useful strategy to enhance capacity to consent is to reduce attention and memory demands as far as possible.

The use of CellaVision competency software for external quality assessment and continuing professional development.

PubMed

Horiuchi, Yuki; Tabe, Yoko; Idei, Mayumi; Bengtsson, Hans-Inge; Ishii, Kiyoshi; Horii, Takashi; Miyake, Kazunori; Satoh, Naotake; Miida, Takashi; Ohsaka, Akimichi

2011-07-01

Quality assessment of blood cell morphological testing, such as white blood cell (WBC) differential and its interpretation, is one of the most important and difficult assignments in haematology laboratories. A monthly survey was performed to assess the possible role of the proficiency testing program produced by CellaVision competency software (CCS) in external quality assessment (EQA) of the clinical laboratories of affiliated university hospitals and the effective utilisation of this program in continuing professional development (CPD). Four monthly proficiency surveys were conducted in collaboration with four clinical laboratories affiliated with the teaching hospitals of Juntendo University of Medicine in Japan. EQA results by the CCS proficiency testing program revealed a difference of performance levels of WBC differential and morphological interpretation and a discrepancy in the WBC differential criteria among laboratories. With regard to the utilisation of this proficiency program as a tool for CPD, this program successfully improved the performance of the low-scoring laboratories and less experienced individuals. The CCS proficiency testing program was useful for the quality assessment of laboratory performance, for education, and for the storage and distribution of cell images to be utilised for further standardisation and education.

Identification of nursing assessment models/tools validated in clinical practice for use with diverse ethno-cultural groups: an integrative review of the literature

PubMed Central

2011-01-01

Background High income nations are currently exhibiting increasing ethno-cultural diversity which may present challenges for nursing practice. We performed an integrative review of literature published in North America and Europe between 1990 and 2007, to map the state of knowledge and to identify nursing assessment tools/models which are have an associated research or empirical perspective in relation to ethno-cultural dimensions of nursing care. Methods Data was retrieved from a wide variety of sources, including key electronic bibliographic databases covering research in biomedical fields, nursing and allied health, and culture, e.g. CINAHL, MEDline, PUBmed, Cochrane library, PsycINFO, Web of Science, and HAPI. We used the Critical Appraisal Skills Programme tools for quality assessment. We applied Torraco's definition and method of an integrative review that aims to create new knowledge and perspectives on a given phenomena. To add methodological rigor with respect to the search strategy and other key review components we also used the principles established by the Centre for Reviews and Dissemination. Results Thirteen thousand and thirteen articles were retrieved, from which 53 full papers were assessed for inclusion. Eight papers met the inclusion criteria, describing research on a total of eight ethno-cultural assessment tools/models. The tools/models are described and synthesized. Conclusions While many ethno-cultural assessment tools exist to guide nursing practice, few are informed by research perspectives. An increased focus on the efficiency and effectiveness of health services, patient safety, and risk management, means that provision of culturally responsive and competent health services will inevitably become paramount. PMID:21812960

Training and Assessing Interprofessional Virtual Teams Using a Web-Based Case System.

PubMed

Dow, Alan W; Boling, Peter A; Lockeman, Kelly S; Mazmanian, Paul E; Feldman, Moshe; DiazGranados, Deborah; Browning, Joel; Coe, Antoinette; Selby-Penczak, Rachel; Hobgood, Sarah; Abbey, Linda; Parsons, Pamela; Delafuente, Jeffrey; Taylor, Suzanne F

2016-01-01

Today, clinical care is often provided by interprofessional virtual teams-groups of practitioners who work asynchronously and use technology to communicate. Members of such teams must be competent in interprofessional practice and the use of information technology, two targets for health professions education reform. The authors created a Web-based case system to teach and assess these competencies in health professions students. They created a four-module, six-week geriatric learning experience using a Web-based case system. Health professions students were divided into interprofessional virtual teams. Team members received profession-specific information, entered a summary of this information into the case system's electronic health record, answered knowledge questions about the case individually, then collaborated asynchronously to answer the same questions as a team. Individual and team knowledge scores and case activity measures--number of logins, message board posts/replies, views of message board posts--were tracked. During academic year 2012-2013, 80 teams composed of 522 students from medicine, nursing, pharmacy, and social work participated. Knowledge scores varied by profession and within professions. Team scores were higher than individual scores (P < .001). Students and teams with higher knowledge scores had higher case activity measures. Team score was most highly correlated with number of message board posts/replies and was not correlated with number of views of message board posts. This Web-based case system provided a novel approach to teach and assess the competencies needed for virtual teams. This approach may be a valuable new tool for measuring competency in interprofessional practice.

Criterion-Referenced Test Items for Auto Body.

ERIC Educational Resources Information Center

Tannehill, Dana, Ed.

This test item bank on auto body repair contains criterion-referenced test questions based upon competencies found in the Missouri Auto Body Competency Profile. Some test items are keyed for multiple competencies. The tests cover the following 26 competency areas in the auto body curriculum: auto body careers; measuring and mixing; tools and…

Competency-Based Evaluation in Higher Education--Design and Use of Competence Rubrics by University Educators

ERIC Educational Resources Information Center

Velasco-Martínez, Leticia-Concepción; Tójar-Hurtado, Juan-Carlos

2018-01-01

Competency-based learning requires making changes in the higher education model in response to current socio-educational demands. Rubrics are an innovative educational tool for competence evaluation, for both students and educators. Ever since arriving at the university systems, the application of rubrics in evaluation programs has grown…

Patient simulation: a literary synthesis of assessment tools in anesthesiology.

PubMed

Edler, Alice A; Fanning, Ruth G; Chen, Michael I; Claure, Rebecca; Almazan, Dondee; Struyk, Brain; Seiden, Samuel C

2009-12-20

High-fidelity patient simulation (HFPS) has been hypothesized as a modality for assessing competency of knowledge and skill in patient simulation, but uniform methods for HFPS performance assessment (PA) have not yet been completely achieved. Anesthesiology as a field founded the HFPS discipline and also leads in its PA. This project reviews the types, quality, and designated purpose of HFPS PA tools in anesthesiology. We used the systematic review method and systematically reviewed anesthesiology literature referenced in PubMed to assess the quality and reliability of available PA tools in HFPS. Of 412 articles identified, 50 met our inclusion criteria. Seventy seven percent of studies have been published since 2000; more recent studies demonstrated higher quality. Investigators reported a variety of test construction and validation methods. The most commonly reported test construction methods included "modified Delphi Techniques" for item selection, reliability measurement using inter-rater agreement, and intra-class correlations between test items or subtests. Modern test theory, in particular generalizability theory, was used in nine (18%) of studies. Test score validity has been addressed in multiple investigations and shown a significant improvement in reporting accuracy. However the assessment of predicative has been low across the majority of studies. Usability and practicality of testing occasions and tools was only anecdotally reported. To more completely comply with the gold standards for PA design, both shared experience of experts and recognition of test construction standards, including reliability and validity measurements, instrument piloting, rater training, and explicit identification of the purpose and proposed use of the assessment tool, are required.

Assessment and self-assessment of the pharmacists' competencies using the global competency framework (GbCF) in Serbia.

PubMed

Stojkov, Svetlana; Tadić, Ivana; Crnjanski, Tatjana; Krajnović, Dušanka

2016-09-01

Pharmacists' competence represents a dynamic framework of knowledge, skills and abilities to carry out tasks, and it reflects on improving the quality of life and on patients’ health. One of the documents for the Evaluation and Competency Development of Pharmacists is the Global Competency Framework (GbCF). The aim of this study was to implement the GBCF document into Serbian pharmacies, to perform assessment and self assessment of the competencies. The assessment and self-assessment of pharmacists’ competencies were performed during the period 2012−13 year in eight community pharmacy chains, in seven cities in Serbia. For assessment and self-assessment of pharmacists competencies the GbCF model was applied, which was adjusted to pharmaceutical practice and legislation in Serbia. External assessment was conducted by teams of pharmacists using the structured observation of the work of pharmacists during regular working hours. Evaluated pharmacists filled out the questionnaire about demographic indicators about the pharmacist and the pharmacy where they work. A total of 123 pharmacists were evaluated. Pharmacists’ Professional Competency Cluster (KK1) had the lowest score (average value 2.98), while the cluster Management and Organizational Competency (KK2) had the highest score (average value 3.15). The competence Recognition of the Diagnosis and Patient Counseling (K8), which belonged to the cluster KK1, had the lowest score (average value for assessment and self-assessment were 2.09, and 2.34, respectively) among the all evaluated competencies. GbCF might be considered as an instrument for the competencies' evaluation/selfevaluation and their improvement, accordingly.

Exploring the acquisition of entry-to-practice competencies by second-degree nursing students during a preceptorship experience.

PubMed

Sedgwick, Monique; Kellett, Peter; Kalischuck, Ruth Grant

2014-03-01

Nursing programs across Canada have begun to implement at an unprecedented rate second-degree nursing programs in response to consumer demands and a nursing shortage. While these types of programs are enjoying considerable popularity among prospective students and employers, it is imperative that nursing programs assess their graduates' ability to meet Registered Nursing entry-to-practice competencies (ETCs). This study sought to determine if second-degree undergraduate nursing students achieved the entry-to-practice competencies established by the provincial regulatory body for registered nurses of Alberta, Canada. The study took place in southern Alberta, Canada as the first cohort of second-degree undergraduate nursing students were completing the final practice course for the program. In this exploratory study, quantitative and qualitative data generation approaches were used. Quantitative data were collected using the nursing program's standardized Clinical Evaluation Tool which is mapped to the 119 ETCs established by the regulatory body. Qualitative data were generated by conducting focus group interviews with students, faculty advisors, and preceptors. A convenience sample consisting of both male and female students (n=14) submitted their mid-term and final clinical evaluations for inclusion in the dataset. Thirteen preceptors submitted mid-term and final clinical evaluations. Three students, three faculty advisors, and two preceptors participated in focus group interviews. At mid-term, statistically significant differences were noted on 31% of the indicators within the clinical evaluation tool between students and preceptors with preceptors consistently ranking students higher than the students' ratings of their performance. Student and preceptor ratings of students' clinical performance were more consistent on the final evaluation. However, where there were differences, preceptors rated students higher than student ratings. Qualitative data analysis suggests that the concept of competence is complex and multifaceted and understood differently by students, preceptors, and advisors. The findings of this study suggest that there is ambiguity among second-degree students, preceptors and faculty advisors surrounding the concept of competence. In order to develop an understanding of competence, nursing program administrators must encourage faculty advisors, preceptors and students to engage in a discussion at the outset of the preceptored practice experience in regard to what is meant by competence within various practice setting. Further, we suggest nursing programs in collaboration with their clinical partners and re-examine their practice evaluation tools to determine the degree to which they are sensitive to the clinical practice context. Copyright © 2013 Elsevier Ltd. All rights reserved.

Tools to assess psychological trauma & its correlates in child sexual abuse: A review & current needs in Asia.

PubMed

Satapathy, Sujata; Choudhary, Vandana; Sagar, Rajesh

2017-02-01

Absence of visible physical symptoms and limited capacity to express trauma directly, pose significant challenges in assessment of its exact nature of trauma and its correlates in child sexual abuse. There are numerous assessment tools however, deciding upon the appropriateness is often challenging in Asian socio-cultural and health care set up. A review would provide a ready reference to the practioner regarding the exact clinically utility of the tools and also would guide them in the direction of culture specific modifications. Computerized databases namely Medline, PsycINFO, Health and Psychosocial Instruments, and Social Sciences Citation Index were used. 52 scales were obtained and analysed in terms of scale characteristics, reference to theory and DSM, and cultural competency. Despite of a wide variety of methods, and newer instruments, many of the traditionally used techniques of child's internal thinking and emotional assessment appear outdated while reviewing the recent theories of CSA related psychological trauma. An integrated format, incroporating child-parent-clinicain rating, with multiple domain speciafic items and verbal and non-verbal tasks, is the current need in the Asian region. Copyright © 2016 Elsevier B.V. All rights reserved.

Tools to investigate how interprofessional education activities link to competencies.

PubMed

West, Courtney; Veronin, Michael; Landry, Karen; Kurz, Terri; Watzak, Bree; Quiram, Barbara; Graham, Lori

2015-01-01

Integrating interprofessional education (IPE) activities and curricular components in health professions education has been emphasized recently by the inclusion of accreditation standards across disciplines. The Interprofessional Education Collaborative (IPEC) established IPE competencies in 2009, but evaluating how activities link to competencies has not been investigated in depth. The purpose of this project is to investigate how well two IPE activities align with IPEC competencies. To evaluate how our IPE activities met IPEC competencies, we developed a checklist and an observation instrument. A brief description of each is included as well as the outcomes. We analyzed Disaster Day, a simulation exercise that includes participants from Nursing, Medicine, and Pharmacy, and Interprofessional Healthcare Ethics (IPHCE), a course that introduced medical, nursing, and pharmacy students to ethical issues using didactic sessions and case discussions. While both activities appeared to facilitate the development of IPE competencies, Disaster Day aligned more with IPEC competencies than the IPHCE course and appears to be a more comprehensive way of addressing IPEC competencies. However, offering one IPE activity or curricular element is not sufficient. Having several IPE options available, utilizing the tools we developed to map the IPE curriculum and evaluating competency coverage is recommended.

A teaching skills assessment tool inspired by the Calgary-Cambridge model and the patient-centered approach.

PubMed

Sommer, Johanna; Lanier, Cédric; Perron, Noelle Junod; Nendaz, Mathieu; Clavet, Diane; Audétat, Marie-Claude

2016-04-01

The aim of this study was to develop a descriptive tool for peer review of clinical teaching skills. Two analogies framed our research: (1) between the patient-centered and the learner-centered approach; (2) between the structures of clinical encounters (Calgary-Cambridge communication model) and teaching sessions. During the course of one year, each step of the action research was carried out in collaboration with twelve clinical teachers from an outpatient general internal medicine clinic and with three experts in medical education. The content validation consisted of a literature review, expert opinion and the participatory research process. Interrater reliability was evaluated by three clinical teachers coding thirty audiotaped standardized learner-teacher interactions. This tool contains sixteen items covering the process and content of clinical supervisions. Descriptors define the expected teaching behaviors for three levels of competence. Interrater reliability was significant for eleven items (Kendall's coefficient p<0.05). This peer assessment tool has high reliability and can be used to facilitate the acquisition of teaching skills. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

The Individualized Classroom Assessment Scoring System (inCLASS): Preliminary Reliability and Validity of a System for Observing Preschoolers’ Competence in Classroom Interactions

PubMed Central

Downer, Jason T.; Booren, Leslie M.; Lima, Olivia K.; Luckner, Amy E.; Pianta, Robert C.

2012-01-01

This paper introduces the Individualized Classroom Assessment Scoring System (inCLASS), an observation tool that targets children’s interactions in preschool classrooms with teachers, peers, and tasks. In particular, initial evidence is reported of the extent to which the inCLASS meets the following psychometric criteria: inter-rater reliability, normal distributions and adequate range, construct validity, and criterion-related validity. These initial findings suggest that the inCLASS has the potential to provide an authentic, contextualized assessment of young children’s classroom behaviors. Future directions for research with the inCLASS are discussed. PMID:23175598