Enabling performance skills: Assessment in engineering education

NASA Astrophysics Data System (ADS)

Ferrone, Jenny Kristina

Current reform in engineering education is part of a national trend emphasizing student learning as well as accountability in instruction. Assessing student performance to demonstrate accountability has become a necessity in academia. In newly adopted criterion proposed by the Accreditation Board for Engineering and Technology (ABET), undergraduates are expected to demonstrate proficiency in outcomes considered essential for graduating engineers. The case study was designed as a formative evaluation of freshman engineering students to assess the perceived effectiveness of performance skills in a design laboratory environment. The mixed methodology used both quantitative and qualitative approaches to assess students' performance skills and congruency among the respondents, based on individual, team, and faculty perceptions of team effectiveness in three ABET areas: Communications Skills. Design Skills, and Teamwork. The findings of the research were used to address future use of the assessment tool and process. The results of the study found statistically significant differences in perceptions of Teamwork Skills (p < .05). When groups composed of students and professors were compared, professors were less likely to perceive student's teaming skills as effective. The study indicated the need to: (1) improve non-technical performance skills, such as teamwork, among freshman engineering students; (2) incorporate feedback into the learning process; (3) strengthen the assessment process with a follow-up plan that specifically targets performance skill deficiencies, and (4) integrate the assessment instrument and practice with ongoing curriculum development. The findings generated by this study provides engineering departments engaged in assessment activity, opportunity to reflect, refine, and develop their programs as it continues. It also extends research on ABET competencies of engineering students in an under-investigated topic of factors correlated with team

Urban College Graduates: Their Investments in and Returns for Strong Quantitative Skills, Social Capital Skills, and Soft Skills

ERIC Educational Resources Information Center

Haynes, Marie Ellen

2010-01-01

This case study examined strong quantitative skills, social capital skills, and soft skills of urban college graduates using data from the Multi-City Study of Urban Inequality Household Survey. The urban college graduates lived in Atlanta, Boston, or Los Angeles and had bachelor's, master's, PhD, and professional degrees. Among the three skills…

National Assessment's Consumer Skills Assessments.

ERIC Educational Resources Information Center

Education Commission of the States, Denver, CO. National Assessment of Educational Progress.

The National Assessment of Educational Progress (NAEP) encountered problems when developing a comprehensive assessment of consumer skills of 17-year-olds in 1978. When NAEP conducted a nationwide assessment of basic life skills of 17-year-olds in 1977, there was considerable interest in the consumer skills portion. Therefore, it was decided to…

Factors Influencing Students' Perceptions of Their Quantitative Skills

ERIC Educational Resources Information Center

Matthews, Kelly E.; Hodgson, Yvonne; Varsavsky, Cristina

2013-01-01

There is international agreement that quantitative skills (QS) are an essential graduate competence in science. QS refer to the application of mathematical and statistical thinking and reasoning in science. This study reports on the use of the Science Students Skills Inventory to capture final year science students' perceptions of their QS across…

Nursing students' evaluation of a new feedback and reflection tool for use in high-fidelity simulation - Formative assessment of clinical skills. A descriptive quantitative research design.

PubMed

Solheim, Elisabeth; Plathe, Hilde Syvertsen; Eide, Hilde

2017-11-01

Clinical skills training is an important part of nurses' education programmes. Clinical skills are complex. A common understanding of what characterizes clinical skills and learning outcomes needs to be established. The aim of the study was to develop and evaluate a new reflection and feedback tool for formative assessment. The study has a descriptive quantitative design. 129 students participated who were at the end of the first year of a Bachelor degree in nursing. After highfidelity simulation, data were collected using a questionnaire with 19 closed-ended and 2 open-ended questions. The tool stimulated peer assessment, and enabled students to be more thorough in what to assess as an observer in clinical skills. The tool provided a structure for selfassessment and made visible items that are important to be aware of in clinical skills. This article adds to simulation literature and provides a tool that is useful in enhancing peer learning, which is essential for nurses in practice. The tool has potential for enabling students to learn about reflection and developing skills for guiding others in practice after they have graduated. Copyright © 2017 Elsevier Ltd. All rights reserved.

Two Thinking Skills Assessment Approaches: "Assessment of Pupils' Thinking Skills" and "Individual Thinking Skills Assessments"

ERIC Educational Resources Information Center

Burke, Lynsey A.; Williams, Joanne M.

2012-01-01

This paper is linked to a previous paper outlining an evaluation of a thinking skills intervention (Burke & Williams, 2008). Following extensive requests for the assessment tools used in the intervention, this short paper presents the development and potential uses of two thinking skills assessment tools. The aim of the paper is simply to make…

Assessing Leadership and Problem-Solving Skills and Their Impacts in the Community.

ERIC Educational Resources Information Center

Rohs, F. Richard; Langone, Christine A.

1993-01-01

A pretest-posttest control group design was used to assess the leadership and problem-solving skills of 281 participants and 110 controls in a statewide community leadership development program. Quantitative and qualitative data demonstrate that the program has been a catalyst to influence leadership and problem-solving skills for community…

Quantitative methods in assessment of neurologic function.

PubMed

Potvin, A R; Tourtellotte, W W; Syndulko, K; Potvin, J

1981-01-01

Traditionally, neurologists have emphasized qualitative techniques for assessing results of clinical trials. However, in recent years qualitative evaluations have been increasingly augmented by quantitative tests for measuring neurologic functions pertaining to mental state, strength, steadiness, reactions, speed, coordination, sensation, fatigue, gait, station, and simulated activities of daily living. Quantitative tests have long been used by psychologists for evaluating asymptomatic function, assessing human information processing, and predicting proficiency in skilled tasks; however, their methodology has never been directly assessed for validity in a clinical environment. In this report, relevant contributions from the literature on asymptomatic human performance and that on clinical quantitative neurologic function are reviewed and assessed. While emphasis is focused on tests appropriate for evaluating clinical neurologic trials, evaluations of tests for reproducibility, reliability, validity, and examiner training procedures, and for effects of motivation, learning, handedness, age, and sex are also reported and interpreted. Examples of statistical strategies for data analysis, scoring systems, data reduction methods, and data display concepts are presented. Although investigative work still remains to be done, it appears that carefully selected and evaluated tests of sensory and motor function should be an essential factor for evaluating clinical trials in an objective manner.

Information Literacy Follow-Through: Enhancing Preservice Teachers' Information Evaluation Skills through Formative Assessment

ERIC Educational Resources Information Center

Seely, Sara Robertson; Fry, Sara Winstead; Ruppel, Margie

2011-01-01

An investigation into preservice teachers' information evaluation skills at a large university suggests that formative assessment can improve student achievement. Preservice teachers were asked to apply information evaluation skills in the areas of currency, relevancy, authority, accuracy, and purpose. The study used quantitative methods to assess…

The Influence of Undergraduate Science Curriculum Reform on Students' Perceptions of Their Quantitative Skills

ERIC Educational Resources Information Center

Matthews, Kelly E.; Adams, Peter; Goos, Merrilyn

2015-01-01

In this study, the Science Student Skills Inventory was used to gain understanding of student perceptions about their quantitative skills and compare perceptions of cohorts graduating before and after the implementation of a new science curriculum intent on developing quantitative skills. The study involved 600 responses from final-year…

Engaging Business Students in Quantitative Skills Development

ERIC Educational Resources Information Center

Cronin, Anthony; Carroll, Paula

2015-01-01

In this paper the complex problems of developing quantitative and analytical skills in undergraduate first year, first semester business students are addressed. An action research project, detailing how first year business students perceive the relevance of data analysis and inferential statistics in light of the economic downturn and the…

Laparoscopic training using a quantitative assessment and instructional system.

PubMed

Yamaguchi, T; Nakamura, R

2018-04-28

Laparoscopic surgery requires complex surgical skills; hence, surgeons require regular training to improve their surgical techniques. The quantitative assessment of a surgeon's skills and the provision of feedback are important processes for conducting effective training. The aim of this study was to develop an inexpensive training system that provides automatic technique evaluation and feedback. We detected the instrument using image processing of commercial web camera images and calculated the motion analysis parameters (MAPs) of the instrument to quantify performance features. Upon receiving the results, we developed a method of evaluating the surgeon's skill level. The feedback system was developed using MAPs-based radar charts and scores for determining the skill level. These methods were evaluated using the videos of 38 surgeons performing a suturing task. There were significant differences in MAPs among surgeons; therefore, MAPs can be effectively used to quantify a surgeon's performance features. The results of skill evaluation and feedback differed greatly between skilled and unskilled surgeons, and it was possible to indicate points of improvement for the procedure performed in this study. Furthermore, the results obtained for certain novice surgeons were similar to those obtained for skilled surgeons. This system can be used to assess the skill level of surgeons, independent of the years of experience, and provide an understanding of the individual's current surgical skill level effectively. We conclude that our system is useful as an inexpensive laparoscopic training system that might aid in skill improvement.

Improving quantitative skills in introductory geoscience courses at a four-year public institution using online math modules

NASA Astrophysics Data System (ADS)

Gordon, E. S.

2011-12-01

Fitchburg State University has a diverse student population comprised largely of students traditionally underrepresented in higher education, including first-generation, low-income, and/or students with disabilities. Approximately half of our incoming students require developmental math coursework, but often enroll in science classes prior to completing those courses. Since our introductory geoscience courses (Oceanography, Meteorology, Geology, Earth Systems Science) do not have prerequisites, many students who take them lack basic math skills, but are taking these courses alongside science majors. In order to provide supplemental math instruction without sacrificing time for content, "The Math You Need, When You Need It (TMYN), a set of online math tutorials placed in a geoscience context, will be implemented in three of our introductory courses (Oceanography, Meteorology, and Earth Systems Science) during Fall, 2011. Students will complete 5-6 modules asynchronously, the topics of which include graphing skills, calculating rates, unit conversions, and rearranging equations. Assessment of quantitative skills will be tracked with students' pre- and post-test results, as well as individual module quiz scores. In addition, student assessment results from Oceanography will be compared to student data from Academic Year 2010-11, during which quantitative skills were evaluated with pre- and post-test questions, but students did not receive online supplemental instruction.

The Relationship between Quantitative and Qualitative Measures of Writing Skills.

ERIC Educational Resources Information Center

Howerton, Mary Lou P.; And Others

The relationships of quantitative measures of writing skills to overall writing quality as measured by the E.T.S. Composition Evaluation Scale (CES) were examined. Quantitative measures included indices of language productivity, vocabulary diversity, spelling, and syntactic maturity. Power of specific indices to account for variation in overall…

Assessing locomotor skills development in childhood using wearable inertial sensor devices: the running paradigm.

PubMed

Masci, Ilaria; Vannozzi, Giuseppe; Bergamini, Elena; Pesce, Caterina; Getchell, Nancy; Cappozzo, Aurelio

2013-04-01

Objective quantitative evaluation of motor skill development is of increasing importance to carefully drive physical exercise programs in childhood. Running is a fundamental motor skill humans adopt to accomplish locomotion, which is linked to physical activity levels, although the assessment is traditionally carried out using qualitative evaluation tests. The present study aimed at investigating the feasibility of using inertial sensors to quantify developmental differences in the running pattern of young children. Qualitative and quantitative assessment tools were adopted to identify a skill-sensitive set of biomechanical parameters for running and to further our understanding of the factors that determine progression to skilled running performance. Running performances of 54 children between the ages of 2 and 12 years were submitted to both qualitative and quantitative analysis, the former using sequences of developmental level, the latter estimating temporal and kinematic parameters from inertial sensor measurements. Discriminant analysis with running developmental level as dependent variable allowed to identify a set of temporal and kinematic parameters, within those obtained with the sensor, that best classified children into the qualitative developmental levels (accuracy higher than 67%). Multivariate analysis of variance with the quantitative parameters as dependent variables allowed to identify whether and which specific parameters or parameter subsets were differentially sensitive to specific transitions between contiguous developmental levels. The findings showed that different sets of temporal and kinematic parameters are able to tap all steps of the transitional process in running skill described through qualitative observation and can be prospectively used for applied diagnostic and sport training purposes. Copyright © 2012 Elsevier B.V. All rights reserved.

Skill Assessment in the Interpretation of 3D Fracture Patterns from Radiographs

PubMed Central

Rojas-Murillo, Salvador; Hanley, Jessica M; Kreiter, Clarence D; Karam, Matthew D; Anderson, Donald D

2016-01-01

Abstract Background Interpreting two-dimensional radiographs to ascertain the three-dimensional (3D) position and orientation of fracture planes and bone fragments is an important component of orthopedic diagnosis and clinical management. This skill, however, has not been thoroughly explored and measured. Our primary research question is to determine if 3D radiographic image interpretation can be reliably assessed, and whether this assessment varies by level of training. A test designed to measure this skill among orthopedic surgeons would provide a quantitative benchmark for skill assessment and training research. Methods Two tests consisting of a series of online exercises were developed to measure this skill. Each exercise displayed a pair of musculoskeletal radiographs. Participants selected one of three CT slices of the same or similar fracture patterns that best matched the radiographs. In experiment 1, 10 orthopedic residents and staff responded to nine questions. In experiment 2, 52 residents from both orthopedics and radiology responded to 12 questions. Results Experiment 1 yielded a Cronbach alpha of 0.47. Performance correlated with experience; r(8) = 0.87, p<0.01, suggesting that the test could be both valid and reliable with a slight increase in test length. In experiment 2, after removing three non-discriminating items, the Cronbach coefficient alpha was 0.28 and performance correlated with experience; r(50) = 0.25, p<0.10. Conclusions Although evidence for reliability and validity was more compelling with the first experiment, the analyses suggest motivation and test duration are important determinants of test efficacy. The interpretation of radiographs to discern 3D information is a promising and a relatively unexplored area for surgical skill education and assessment. The online test was useful and reliable. Further test development is likely to increase test effectiveness. Clinical Relevance Accurately interpreting radiographic images is an

Gender, Math Confidence, and Grit: Relationships with Quantitative Skills and Performance in an Undergraduate Biology Course

ERIC Educational Resources Information Center

Flanagan, K. M.; Einarson, J.

2017-01-01

In a world filled with big data, mathematical models, and statistics, the development of strong quantitative skills is becoming increasingly critical for modern biologists. Teachers in this field must understand how students acquire quantitative skills and explore barriers experienced by students when developing these skills. In this study, we…

MathPatch - Raising Retention and Performance in an Intro-geoscience Class by Raising Students' Quantitative Skills

NASA Astrophysics Data System (ADS)

Baer, E. M.; Whittington, C.; Burn, H.

2008-12-01

The geological sciences are fundamentally quantitative. However, the diversity of students' mathematical preparation and skills makes the successful use of quantitative concepts difficult in introductory level classes. At Highline Community College, we have implemented a one-credit co-requisite course to give students supplemental instruction for quantitative skills used in the course. The course, formally titled "Quantitative Geology," nicknamed "MathPatch," runs parallel to our introductory Physical Geology course. MathPatch teaches the quantitative skills required for the geology class right before they are needed. Thus, students learn only the skills they need and are given opportunities to apply them immediately. Topics include complex-graph reading, unit conversions, large numbers, scientific notation, scale and measurement, estimation, powers of 10, and other fundamental mathematical concepts used in basic geological concepts. Use of this course over the past 8 years has successfully accomplished the goals of increasing students' quantitative skills, success and retention. Students master the quantitative skills to a greater extent than before the course was implemented, and less time is spent covering basic quantitative skills in the classroom. Because the course supports the use of quantitative skills, the large number of faculty that teach Geology 101 are more comfortable in using quantitative analysis, and indeed see it as an expectation of the course at Highline. Also significant, retention in the geology course has increased substantially, from 75% to 85%. Although successful, challenges persist with requiring MathPatch as a supplementary course. One, we have seen enrollments decrease in Geology 101, which may be the result of adding this co-requisite. Students resist mandatory enrollment in the course, although they are not good at evaluating their own need for the course. The logistics utilizing MathPatch in an evening class with fewer and longer

Scientists and Mathematicians Collaborating to Build Quantitative Skills in Undergraduate Science

ERIC Educational Resources Information Center

Rylands, Leanne; Simbag, Vilma; Matthews, Kelly E.; Coady, Carmel; Belward, Shaun

2013-01-01

There is general agreement in Australia and beyond that quantitative skills (QS) in science, the ability to use mathematics and statistics in context, are important for science. QS in the life sciences are becoming ever more important as these sciences become more quantitative. Consequently, undergraduates studying the life sciences require better…

Integrating Quantitative Skills in Introductory Ecology: Investigations of Wild Bird Feeding Preferences

ERIC Educational Resources Information Center

Small, Christine J.; Newtoff, Kiersten N.

2013-01-01

Undergraduate biology education is undergoing dramatic changes, emphasizing student training in the "tools and practices" of science, particularly quantitative and problem-solving skills. We redesigned a freshman ecology lab to emphasize the importance of scientific inquiry and quantitative reasoning in biology. This multi-week investigation uses…

Creating Research-Rich Learning Experiences and Quantitative Skills in a 1st Year Earth Systems Course

NASA Astrophysics Data System (ADS)

King, P. L.; Eggins, S.; Jones, S.

2014-12-01

We are creating a 1st year Earth Systems course at the Australian National University that is built around research-rich learning experiences and quantitative skills. The course has top students including ≤20% indigenous/foreign students; nonetheless, students' backgrounds in math and science vary considerably posing challenges for learning. We are addressing this issue and aiming to improve knowledge retention and deep learning by changing our teaching approach. In 2013-2014, we modified the weekly course structure to a 1hr lecture; a 2hr workshop with hands-on activities; a 2hr lab; an assessment piece covering all face-to-face activities; and a 1hr tutorial. Our new approach was aimed at: 1) building student confidence with data analysis and quantitative skills through increasingly difficult tasks in science, math, physics, chemistry, climate science and biology; 2) creating effective learning groups using name tags and a classroom with 8-person tiered tables; 3) requiring students to apply new knowledge to new situations in group activities, two 1-day field trips and assessment items; 4) using pre-lab and pre-workshop exercises to promote prior engagement with key concepts; 5) adding open-ended experiments to foster structured 'scientific play' or enquiry and creativity; and 6) aligning the assessment with the learning outcomes and ensuring that it contains authentic and challenging southern hemisphere problems. Students were asked to design their own ocean current experiment in the lab and we were astounded by their ingenuity: they simulated the ocean currents off Antarctica; varied water density to verify an equation; and examined the effect of wind and seafloor topography on currents. To evaluate changes in student learning, we conducted surveys in 2013 and 2014. In 2014, we found higher levels of student engagement with the course: >~80% attendance rates and >~70% satisfaction (20% neutral). The 2014 cohort felt that they were more competent in writing

Employability Skills Assessment Tool Development

ERIC Educational Resources Information Center

Rasul, Mohamad Sattar; Rauf, Rose Amnah Abd; Mansor, Azlin Norhaini; Puvanasvaran, A. P.

2012-01-01

Research nationally and internationally found that technical graduates are lacking in employability skills. As employability skills are crucial in outcome-based education, the main goal of this research is to develop an Employability Skill Assessment Tool to help students and lecturers produce competent graduates in employability skills needed by…

Factors influencing students' perceptions of their quantitative skills

NASA Astrophysics Data System (ADS)

Matthews, Kelly E.; Hodgson, Yvonne; Varsavsky, Cristina

2013-09-01

There is international agreement that quantitative skills (QS) are an essential graduate competence in science. QS refer to the application of mathematical and statistical thinking and reasoning in science. This study reports on the use of the Science Students Skills Inventory to capture final year science students' perceptions of their QS across multiple indicators, at two Australian research-intensive universities. Statistical analysis reveals several variables predicting higher levels of self-rated competence in QS: students' grade point average, students' perceptions of inclusion of QS in the science degree programme, their confidence in QS, and their belief that QS will be useful in the future. The findings are discussed in terms of implications for designing science curricula more effectively to build students' QS throughout science degree programmes. Suggestions for further research are offered.

Movement skill assessment of typically developing preschool children: a review of seven movement skill assessment tools.

PubMed

Cools, Wouter; Martelaer, Kristine De; Samaey, Christiane; Andries, Caroline

2009-06-01

The importance of movement is often overlooked because it is such a natural part of human life. It is, however, crucial for a child's physical, cognitive and social development. In addition, experiences support learning and development of fundamental movement skills. The foundations of those skills are laid in early childhood and essential to encourage a physically active lifestyle. Fundamental movement skill performance can be examined with several assessment tools. The choice of a test will depend on the context in which the assessment is planned. This article compares seven assessment tools which are often referred to in European or international context. It discusses the tools' usefulness for the assessment of movement skill development in general population samples. After a brief description of each assessment tool the article focuses on contents, reliability, validity and normative data. A conclusion outline of strengths and weaknesses of all reviewed assessment tools focusing on their use in educational research settings is provided and stresses the importance of regular data collection of fundamental movement skill development among preschool children. Key pointsThis review discusses seven movement skill assessment tool's test content, reliability, validity and normative samples.The seven assessment tools all showed to be of great value. Strengths and weaknesses indicate that test choice will depend on specific purpose of test use.Further data collection should also include larger data samples of able bodied preschool children.Admitting PE specialists in assessment of fundamental movement skill performance among preschool children is recommended.The assessment tool's normative data samples would benefit from frequent movement skill performance follow-up of today's children. MOT 4-6: Motoriktest fur vier- bis sechsjährige Kinder, M-ABC: Movement Assessment Battery for Children, PDMS: Peabody Development Scales, KTK: Körper-Koordinationtest für Kinder

Two Reading Assessments for Youth in Alternative Basic Skills and Livelihood Skills Training Programs

ERIC Educational Resources Information Center

Comings, John P.; Strucker, John; Bell, Brenda

2017-01-01

This article describes two assessment tools that have been used to assess the reading skills of youth participating in alternative basic skills and livelihood skills training programs. The Rapid Assessment of Reading Skills (RARS) was developed to identify potential participants who needed to improve their reading skills before beginning training…

Assessing Hourly Precipitation Forecast Skill with the Fractions Skill Score

NASA Astrophysics Data System (ADS)

Zhao, Bin; Zhang, Bo

2018-02-01

Statistical methods for category (yes/no) forecasts, such as the Threat Score, are typically used in the verification of precipitation forecasts. However, these standard methods are affected by the so-called "double-penalty" problem caused by slight displacements in either space or time with respect to the observations. Spatial techniques have recently been developed to help solve this problem. The fractions skill score (FSS), a neighborhood spatial verification method, directly compares the fractional coverage of events in windows surrounding the observations and forecasts. We applied the FSS to hourly precipitation verification by taking hourly forecast products from the GRAPES (Global/Regional Assimilation Prediction System) regional model and quantitative precipitation estimation products from the National Meteorological Information Center of China during July and August 2016, and investigated the difference between these results and those obtained with the traditional category score. We found that the model spin-up period affected the assessment of stability. Systematic errors had an insignificant role in the fraction Brier score and could be ignored. The dispersion of observations followed a diurnal cycle and the standard deviation of the forecast had a similar pattern to the reference maximum of the fraction Brier score. The coefficient of the forecasts and the observations is similar to the FSS; that is, the FSS may be a useful index that can be used to indicate correlation. Compared with the traditional skill score, the FSS has obvious advantages in distinguishing differences in precipitation time series, especially in the assessment of heavy rainfall.

The Quantitative Reasoning for College Science (QuaRCS) Assessment: Emerging Themes from 5 Years of Data

NASA Astrophysics Data System (ADS)

Follette, Katherine; Dokter, Erin; Buxner, Sanlyn

2018-01-01

The Quantitative Reasoning for College Science (QuaRCS) Assessment is a validated assessment instrument that was designed to measure changes in students' quantitative reasoning skills, attitudes toward mathematics, and ability to accurately assess their own quantitative abilities. It has been administered to more than 5,000 students at a variety of institutions at the start and end of a semester of general education college science instruction. I will begin by briefly summarizing our published work surrounding validation of the instrument and identification of underlying attitudinal factors (composite variables identified via factor analysis) that predict 50% of the variation in students' scores on the assessment. I will then discuss more recent unpublished work, including: (1) Development and validation of an abbreviated version of the assessment (The QuaRCS Light), which results in marked improvements in students' ability to maintain a high effort level throughout the assessment and has broad implications for quantitative reasoning assessments in general, and (2) Our efforts to revise the attitudinal portion of the assessment to better assess math anxiety level, another key factor in student performance on numerical assessments.

Movement Skill Assessment of Typically Developing Preschool Children: A Review of Seven Movement Skill Assessment Tools

PubMed Central

Cools, Wouter; Martelaer, Kristine De; Samaey, Christiane; Andries, Caroline

2009-01-01

The importance of movement is often overlooked because it is such a natural part of human life. It is, however, crucial for a child’s physical, cognitive and social development. In addition, experiences support learning and development of fundamental movement skills. The foundations of those skills are laid in early childhood and essential to encourage a physically active lifestyle. Fundamental movement skill performance can be examined with several assessment tools. The choice of a test will depend on the context in which the assessment is planned. This article compares seven assessment tools which are often referred to in European or international context. It discusses the tools’ usefulness for the assessment of movement skill development in general population samples. After a brief description of each assessment tool the article focuses on contents, reliability, validity and normative data. A conclusion outline of strengths and weaknesses of all reviewed assessment tools focusing on their use in educational research settings is provided and stresses the importance of regular data collection of fundamental movement skill development among preschool children. Key pointsThis review discusses seven movement skill assessment tool’s test content, reliability, validity and normative samples.The seven assessment tools all showed to be of great value. Strengths and weaknesses indicate that test choice will depend on specific purpose of test use.Further data collection should also include larger data samples of able bodied preschool children.Admitting PE specialists in assessment of fundamental movement skill performance among preschool children is recommended.The assessment tool’s normative data samples would benefit from frequent movement skill performance follow-up of today’s children. Abbreviations MOT 4-6: Motoriktest fur vier- bis sechsjährige Kinder, M-ABC: Movement Assessment Battery for Children, PDMS: Peabody Development Scales, KTK: K�

Crowd-sourced assessment of technical skills: an opportunity for improvement in the assessment of laparoscopic surgical skills.

PubMed

Deal, Shanley B; Lendvay, Thomas S; Haque, Mohamad I; Brand, Timothy; Comstock, Bryan; Warren, Justin; Alseidi, Adnan

2016-02-01

Objective, unbiased assessment of surgical skills remains a challenge in surgical education. We sought to evaluate the feasibility and reliability of Crowd-Sourced Assessment of Technical Skills. Seven volunteer general surgery interns were given time for training and then testing, on laparoscopic peg transfer, precision cutting, and intracorporeal knot-tying. Six faculty experts (FEs) and 203 Amazon.com Mechanical Turk crowd workers (CWs) evaluated 21 deidentified video clips using the Global Objective Assessment of Laparoscopic Skills validated rating instrument. Within 19 hours and 15 minutes we received 662 eligible ratings from 203 CWs and 126 ratings from 6 FEs over 10 days. FE video ratings were of borderline internal consistency (Krippendorff's alpha = .55). FE ratings were highly correlated with CW ratings (Pearson's correlation coefficient = .78, P < .001). We propose the use of Crowd-Sourced Assessment of Technical Skills as a reliable, basic tool to standardize the evaluation of technical skills in general surgery. Copyright © 2016 Elsevier Inc. All rights reserved.

Ecosystem Model Skill Assessment. Yes We Can!

PubMed Central

Olsen, Erik; Fay, Gavin; Gaichas, Sarah; Gamble, Robert; Lucey, Sean; Link, Jason S.

2016-01-01

Need to Assess the Skill of Ecosystem Models Accelerated changes to global ecosystems call for holistic and integrated analyses of past, present and future states under various pressures to adequately understand current and projected future system states. Ecosystem models can inform management of human activities in a complex and changing environment, but are these models reliable? Ensuring that models are reliable for addressing management questions requires evaluating their skill in representing real-world processes and dynamics. Skill has been evaluated for just a limited set of some biophysical models. A range of skill assessment methods have been reviewed but skill assessment of full marine ecosystem models has not yet been attempted. Northeast US Atlantis Marine Ecosystem Model We assessed the skill of the Northeast U.S. (NEUS) Atlantis marine ecosystem model by comparing 10-year model forecasts with observed data. Model forecast performance was compared to that obtained from a 40-year hindcast. Multiple metrics (average absolute error, root mean squared error, modeling efficiency, and Spearman rank correlation), and a suite of time-series (species biomass, fisheries landings, and ecosystem indicators) were used to adequately measure model skill. Overall, the NEUS model performed above average and thus better than expected for the key species that had been the focus of the model tuning. Model forecast skill was comparable to the hindcast skill, showing that model performance does not degenerate in a 10-year forecast mode, an important characteristic for an end-to-end ecosystem model to be useful for strategic management purposes. Skill Assessment Is Both Possible and Advisable We identify best-practice approaches for end-to-end ecosystem model skill assessment that would improve both operational use of other ecosystem models and future model development. We show that it is possible to not only assess the skill of a complicated marine ecosystem model, but that

Teaching and assessing procedural skills: a qualitative study

PubMed Central

2013-01-01

Background Graduating Internal Medicine residents must possess sufficient skills to perform a variety of medical procedures. Little is known about resident experiences of acquiring procedural skills proficiency, of practicing these techniques, or of being assessed on their proficiency. The purpose of this study was to qualitatively investigate resident 1) experiences of the acquisition of procedural skills and 2) perceptions of procedural skills assessment methods available to them. Methods Focus groups were conducted in the weeks following an assessment of procedural skills incorporated into an objective structured clinical examination (OSCE). Using fundamental qualitative description, emergent themes were identified and analyzed. Results Residents perceived procedural skills assessment on the OSCE as a useful formative tool for direct observation and immediate feedback. This positive reaction was regularly expressed in conjunction with a frustration with available assessment systems. Participants reported that proficiency was acquired through resident directed learning with no formal mechanism to ensure acquisition or maintenance of skills. Conclusions The acquisition and assessment of procedural skills in Internal Medicine programs should move toward a more structured system of teaching, deliberate practice and objective assessment. We propose that directed, self-guided learning might meet these needs. PMID:23672617

Teaching and assessing procedural skills: a qualitative study.

PubMed

Touchie, Claire; Humphrey-Murto, Susan; Varpio, Lara

2013-05-14

Graduating Internal Medicine residents must possess sufficient skills to perform a variety of medical procedures. Little is known about resident experiences of acquiring procedural skills proficiency, of practicing these techniques, or of being assessed on their proficiency. The purpose of this study was to qualitatively investigate resident 1) experiences of the acquisition of procedural skills and 2) perceptions of procedural skills assessment methods available to them. Focus groups were conducted in the weeks following an assessment of procedural skills incorporated into an objective structured clinical examination (OSCE). Using fundamental qualitative description, emergent themes were identified and analyzed. Residents perceived procedural skills assessment on the OSCE as a useful formative tool for direct observation and immediate feedback. This positive reaction was regularly expressed in conjunction with a frustration with available assessment systems. Participants reported that proficiency was acquired through resident directed learning with no formal mechanism to ensure acquisition or maintenance of skills. The acquisition and assessment of procedural skills in Internal Medicine programs should move toward a more structured system of teaching, deliberate practice and objective assessment. We propose that directed, self-guided learning might meet these needs.

Assessment of Surgical Skills and Competency.

PubMed

Bhatti, Nasir I

2017-10-01

Evaluation of surgical skills and competency are important aspects of the medical education process. Measurable and reproducible methods of assessment with objective feedback are essential components of surgical training. Objective Structured Assessment of Technical Skills (OSATS) is widely used across the medical specialties and otolaryngology-specific tools have been developed and validated for sinus and mastoid surgery. Although assessment of surgical skills can be time-consuming and requires human and financial resources, new evaluation methods and emerging technology may alleviate these barriers while also improving data collection practices. Copyright © 2017 Elsevier Inc. All rights reserved.

An Alumni Assessment of MIS Related Job Skill Importance and Skill Gaps

ERIC Educational Resources Information Center

Wilkerson, Jerod W.

2012-01-01

This paper presents the results of a job skill survey of Management Information Systems (MIS) alumni from a Northeastern U.S. university. The study assesses job skill importance and skill gaps associated with 104 technical and non-technical skill items. Survey items were grouped into 6 categories based on prior research. Skill importance and skill…

Skill assessment of Korea operational oceanographic system (KOOS)

NASA Astrophysics Data System (ADS)

Kim, J.; Park, K.

2016-02-01

For the ocean forecast system in Korea, the Korea operational oceanographic system (KOOS) has been developed and pre-operated since 2009 by the Korea institute of ocean science and technology (KIOST) funded by the Korean government. KOOS provides real time information and forecasts for marine environmental conditions in order to support all kinds of activities in the sea. Furthermore, more significant purpose of the KOOS information is to response and support to maritime problems and accidents such as oil spill, red-tide, shipwreck, extraordinary wave, coastal inundation and so on. Accordingly, it is essential to evaluate prediction accuracy and efforts to improve accuracy. The forecast accuracy should meet or exceed target benchmarks before its products are approved for release to the public.In this paper, we conduct error quantification of the forecasts using skill assessment technique for judgement of the KOOS performance. Skill assessment statistics includes the measures of errors and correlations such as root-mean-square-error (RMSE), mean bias (MB), correlation coefficient (R), and index of agreement (IOA) and the frequency with which errors lie within specified limits termed the central frequency (CF).The KOOS provides 72-hour daily forecast data such as air pressure, wind, water elevation, currents, wave, water temperature, and salinity produced by meteorological and hydrodynamic numerical models of WRF, ROMS, MOM5, WAM, WW3, and MOHID. The skill assessment has been performed through comparison of model results with in-situ observation data (Figure 1) for the period from 1 July, 2010 to 31 March, 2015 in Table 1 and model errors have been quantified with skill scores and CF determined by acceptable criteria depending on predicted variables (Table 2). Moreover, we conducted quantitative evaluation of spatio-temporal pattern correlation between numerical models and observation data such as sea surface temperature (SST) and sea surface current produced by

Examining the Role of Numeracy in College STEM Courses: Results from the Quantitative Reasoning for College Science (QuaRCS) Assessment Instrument

NASA Astrophysics Data System (ADS)

Follette, Katherine B.; McCarthy, Donald W.; Dokter, Erin F.; Buxner, Sanlyn; Prather, Edward E.

2016-01-01

Is quantitative literacy a prerequisite for science literacy? Can students become discerning voters, savvy consumers and educated citizens without it? Should college science courses for nonmajors be focused on "science appreciation", or should they engage students in the messy quantitative realities of modern science? We will present results from the recently developed and validated Quantitative Reasoning for College Science (QuaRCS) Assessment, which probes both quantitative reasoning skills and attitudes toward mathematics. Based on data from nearly two thousand students enrolled in nineteen general education science courses, we show that students in these courses did not demonstrate significant skill or attitude improvements over the course of a single semester, but find encouraging evidence for longer term trends.

Teacher Assessment of Practical Skills in A-Level Chemistry

ERIC Educational Resources Information Center

Wood, R.; Ferguson, Carolyn M.

1975-01-01

Discusses a two-year assessment undertaken to evaluate the Nuffield A-Level chemistry course. Secondary teachers selected chemistry experiments for assessment purposes and assessed their students in manipulative skills, observational skills, interpretation skills, creative skills, and attitudes. (MLH)

Resources and Approaches for Teaching Quantitative and Computational Skills in the Geosciences and Allied Fields

NASA Astrophysics Data System (ADS)

Orr, C. H.; Mcfadden, R. R.; Manduca, C. A.; Kempler, L. A.

2016-12-01

Teaching with data, simulations, and models in the geosciences can increase many facets of student success in the classroom, and in the workforce. Teaching undergraduates about programming and improving students' quantitative and computational skills expands their perception of Geoscience beyond field-based studies. Processing data and developing quantitative models are critically important for Geoscience students. Students need to be able to perform calculations, analyze data, create numerical models and visualizations, and more deeply understand complex systems—all essential aspects of modern science. These skills require students to have comfort and skill with languages and tools such as MATLAB. To achieve comfort and skill, computational and quantitative thinking must build over a 4-year degree program across courses and disciplines. However, in courses focused on Geoscience content it can be challenging to get students comfortable with using computational methods to answers Geoscience questions. To help bridge this gap, we have partnered with MathWorks to develop two workshops focused on collecting and developing strategies and resources to help faculty teach students to incorporate data, simulations, and models into the curriculum at the course and program levels. We brought together faculty members from the sciences, including Geoscience and allied fields, who teach computation and quantitative thinking skills using MATLAB to build a resource collection for teaching. These materials, and the outcomes of the workshops are freely available on our website. The workshop outcomes include a collection of teaching activities, essays, and course descriptions that can help faculty incorporate computational skills at the course or program level. The teaching activities include in-class assignments, problem sets, labs, projects, and toolboxes. These activities range from programming assignments to creating and using models. The outcomes also include workshop

Peer Assessment of Elementary Science Teaching Skills

ERIC Educational Resources Information Center

Kilic, Gulsen Bagci; Cakan, Mehtap

2007-01-01

In this study, peer assessment was applied in assessing elementary science teaching skills. Preservice teachers taught a science topic as a team to their peers in an elementary science methods course. The peers participating in the science lesson assessed teacher-groups' elementary science teaching skills on an assessment form provided by the…

Ecosystem Model Skill Assessment. Yes We Can!

PubMed

Olsen, Erik; Fay, Gavin; Gaichas, Sarah; Gamble, Robert; Lucey, Sean; Link, Jason S

2016-01-01

Accelerated changes to global ecosystems call for holistic and integrated analyses of past, present and future states under various pressures to adequately understand current and projected future system states. Ecosystem models can inform management of human activities in a complex and changing environment, but are these models reliable? Ensuring that models are reliable for addressing management questions requires evaluating their skill in representing real-world processes and dynamics. Skill has been evaluated for just a limited set of some biophysical models. A range of skill assessment methods have been reviewed but skill assessment of full marine ecosystem models has not yet been attempted. We assessed the skill of the Northeast U.S. (NEUS) Atlantis marine ecosystem model by comparing 10-year model forecasts with observed data. Model forecast performance was compared to that obtained from a 40-year hindcast. Multiple metrics (average absolute error, root mean squared error, modeling efficiency, and Spearman rank correlation), and a suite of time-series (species biomass, fisheries landings, and ecosystem indicators) were used to adequately measure model skill. Overall, the NEUS model performed above average and thus better than expected for the key species that had been the focus of the model tuning. Model forecast skill was comparable to the hindcast skill, showing that model performance does not degenerate in a 10-year forecast mode, an important characteristic for an end-to-end ecosystem model to be useful for strategic management purposes. We identify best-practice approaches for end-to-end ecosystem model skill assessment that would improve both operational use of other ecosystem models and future model development. We show that it is possible to not only assess the skill of a complicated marine ecosystem model, but that it is necessary do so to instill confidence in model results and encourage their use for strategic management. Our methods are applicable

Clinical Skills Assessment in the Twenty-First Century.

PubMed

Elder, Andrew

2018-05-01

Clinical skills remain fundamental to the practice of medicine and form a core component of the professional identity of the physician. However, evidence exists to suggest that the practice of some clinical skills is declining, particularly in the United States. A decline in practice of any skill can lead to a decline in its teaching and assessment, with further decline in practice as a result. Consequently, assessment not only drives learning of clinical skills, but their practice. This article summarizes contemporary approaches to clinical skills assessment that, if more widely adopted, could support the maintenance and reinvigoration of bedside clinical skills. Crown Copyright © 2017. Published by Elsevier Inc. All rights reserved.

Quantitative Literacy across the Curriculum: Integrating Skills from English Composition, Mathematics, and the Substantive Disciplines

ERIC Educational Resources Information Center

Miller, Jane E.

2010-01-01

Quantitative literacy is an important proficiency that pertains to "word problems" from science, history, and other fields. Unfortunately, teaching how to solve such problems often is relegated to math courses alone. This article examines how quantitative literacy also involves concepts and skills from English composition and the substantive…

Assessing Pupils' Skills in Experimentation

ERIC Educational Resources Information Center

Hammann, Marcus; Phan, Thi Thanh Hoi; Ehmer, Maike; Grimm, Tobias

2008-01-01

This study is concerned with different forms of assessment of pupils' skills in experimentation. The findings of three studies are reported. Study 1 investigates whether it is possible to develop reliable multiple-choice tests for the skills of forming hypotheses, designing experiments and analysing experimental data. Study 2 compares scores from…

Does Computer-Based Motor Skill Assessment Training Transfer to Live Assessing?

ERIC Educational Resources Information Center

Kelly, Luke E.; Taliaferro, Andrea; Krause, Jennifer

2012-01-01

Developing competency in motor skill assessment has been identified as a critical need in physical educator preparation. We conducted this study to evaluate (a) the effectiveness of a web-based instructional program--Motor Skill Assessment Program (MSAP)--for developing assessment competency, and specifically (b) whether competency developed by…

The Quantitative Methods Boot Camp: Teaching Quantitative Thinking and Computing Skills to Graduate Students in the Life Sciences

PubMed Central

Stefan, Melanie I.; Gutlerner, Johanna L.; Born, Richard T.; Springer, Michael

2015-01-01

The past decade has seen a rapid increase in the ability of biologists to collect large amounts of data. It is therefore vital that research biologists acquire the necessary skills during their training to visualize, analyze, and interpret such data. To begin to meet this need, we have developed a “boot camp” in quantitative methods for biology graduate students at Harvard Medical School. The goal of this short, intensive course is to enable students to use computational tools to visualize and analyze data, to strengthen their computational thinking skills, and to simulate and thus extend their intuition about the behavior of complex biological systems. The boot camp teaches basic programming using biological examples from statistics, image processing, and data analysis. This integrative approach to teaching programming and quantitative reasoning motivates students’ engagement by demonstrating the relevance of these skills to their work in life science laboratories. Students also have the opportunity to analyze their own data or explore a topic of interest in more detail. The class is taught with a mixture of short lectures, Socratic discussion, and in-class exercises. Students spend approximately 40% of their class time working through both short and long problems. A high instructor-to-student ratio allows students to get assistance or additional challenges when needed, thus enhancing the experience for students at all levels of mastery. Data collected from end-of-course surveys from the last five offerings of the course (between 2012 and 2014) show that students report high learning gains and feel that the course prepares them for solving quantitative and computational problems they will encounter in their research. We outline our course here which, together with the course materials freely available online under a Creative Commons License, should help to facilitate similar efforts by others. PMID:25880064

The quantitative methods boot camp: teaching quantitative thinking and computing skills to graduate students in the life sciences.

PubMed

Stefan, Melanie I; Gutlerner, Johanna L; Born, Richard T; Springer, Michael

2015-04-01

The past decade has seen a rapid increase in the ability of biologists to collect large amounts of data. It is therefore vital that research biologists acquire the necessary skills during their training to visualize, analyze, and interpret such data. To begin to meet this need, we have developed a "boot camp" in quantitative methods for biology graduate students at Harvard Medical School. The goal of this short, intensive course is to enable students to use computational tools to visualize and analyze data, to strengthen their computational thinking skills, and to simulate and thus extend their intuition about the behavior of complex biological systems. The boot camp teaches basic programming using biological examples from statistics, image processing, and data analysis. This integrative approach to teaching programming and quantitative reasoning motivates students' engagement by demonstrating the relevance of these skills to their work in life science laboratories. Students also have the opportunity to analyze their own data or explore a topic of interest in more detail. The class is taught with a mixture of short lectures, Socratic discussion, and in-class exercises. Students spend approximately 40% of their class time working through both short and long problems. A high instructor-to-student ratio allows students to get assistance or additional challenges when needed, thus enhancing the experience for students at all levels of mastery. Data collected from end-of-course surveys from the last five offerings of the course (between 2012 and 2014) show that students report high learning gains and feel that the course prepares them for solving quantitative and computational problems they will encounter in their research. We outline our course here which, together with the course materials freely available online under a Creative Commons License, should help to facilitate similar efforts by others.

On the Edge of Mathematics and Biology Integration: Improving Quantitative Skills in Undergraduate Biology Education

ERIC Educational Resources Information Center

Feser, Jason; Vasaly, Helen; Herrera, Jose

2013-01-01

In this paper, the authors describe how two institutions are helping their undergraduate biology students build quantitative competencies. Incorporation of quantitative skills and reasoning in biology are framed through a discussion of two cases that both concern introductory biology courses, but differ in the complexity of the mathematics and the…

"The Math You Need" When Faculty Need It: Enhancing Quantitative Skills at a Broad Spectrum of Higher Education Institutions

NASA Astrophysics Data System (ADS)

Baer, E. M.; Wenner, J. M.

2014-12-01

Implementation of "The Math You Need, When You Need It" (TMYN) modules at a wide variety of institutions suggests a broad need for faculty support in helping students develop quantitative skills necessary in introductory geoscience courses. Designed to support students in applying geoscience relevant quantitative skills, TMYN modules are web-based, self-paced and commonly assigned outside of class. They include topics such as calculating slope, rearranging equations, and unit conversions and provide several applications of the mathematical technique to geoscience problems. Each instructor chooses modules that are applicable to the content in his/her individual course and students typically work through the module immediately before the module topic is applied in lab or class. Instructors assigned TMYN modules in their courses at more than 40 diverse institutions, including four-year colleges and universities (4YCs) that vary from non-selective to highly selective and open-door two-year colleges (2YCs). Analysis of module topics assigned, frequency of module use, and institutional characteristics reveals similarities and differences among faculty perception of required quantitative skills and incoming student ability at variably selective institutions. Results indicate that institutional type and selectivity are not correlated with module topic; that is, faculty apply similar quantitative skills in all introductory geoscience courses. For example, nearly every instructor assigned the unit conversions module, whereas very few required the trigonometry module. However, differences in number of assigned modules and faculty expectations are observed between 2YCs and 4YCs (no matter the selectivity). Two-year college faculty typically assign a higher number of modules per course and faculty at 4YCs more often combine portions of multiple modules or cover multiple mathematical concepts in a single assignment. These observations suggest that quantitative skills required

The Relationship between Student's Quantitative Skills, Application of Math, Science Courses, and Science Marks at Single-Sex Independent High Schools

ERIC Educational Resources Information Center

Cambridge, David

2012-01-01

For independent secondary schools who offer rigorous curriculum to attract students, integration of quantitative skills in the science courses has become an important definition of rigor. However, there is little research examining students' quantitative skills in relation to high school science performance within the single-sex independent school…

Adult Survival Skills Assessment.

ERIC Educational Resources Information Center

Walsko, Gregory M.

The purpose of this instrument is to supplement data from the Adult Basic Learning Examination in assessing the functional level of adults in daily situations. It may also be used as a teaching tool for adults requesting tutoring in specific concepts and skills presented in the instrument. This instrument is an informal assessment instrument and…

Assessment of technical and nontechnical skills in surgical residents.

PubMed

Ponton-Carss, Alicia; Kortbeek, John B; Ma, Irene W Y

2016-11-01

Surgical competence encompasses both technical and nontechnical skills. This study seeks to evaluate the validity evidence for a comprehensive surgical skills examination and to examine the relationship between technical and nontechnical skills. Six examination stations assessing both technical and nontechnical skills, conducted yearly for surgical trainees (n = 120) between 2010 and 2014 are included. The assessment tools demonstrated acceptable internal consistency. Interstation reliability for technical skills was low (alpha = .39). Interstation reliability for the nontechnical skills was lower (alpha range -.05 to .31). Nontechnical skills domains were strongly correlated, ranging from r = .65, P < .001 to .86, P < .001. The associations between nontechnical and technical skills were inconsistent, ranging from poor (r = -.06; P = .54) to moderate (r = .45; P < .001). Multiple samplings of integrated technical and nontechnical skills are necessary to assess overall surgical competency. Copyright © 2016 Elsevier Inc. All rights reserved.

Fostering the Development of Quantitative Life Skills through Introductory Astronomy: Can it be Done?

NASA Astrophysics Data System (ADS)

Follette, Katherine B.; McCarthy, D. W.

2012-01-01

We present preliminary results from a student survey designed to test whether the all-important life skill of numeracy/quantitative literacy can be fostered and improved upon in college students through the vehicle of non-major introductory courses in Astronomy. Many instructors of introductory science courses for non-majors would state that a major goal of our classes is to teach our students to distinguish between science and pseudoscience, truth and fiction, in their everyday lives. It is difficult to believe that such a skill can truly be mastered without a fair amount of mathematical sophistication in the form of arithmetic, statistical and graph reading skills that many American college students unfortunately lack when they enter our classrooms. In teaching what is frequently their "terminal science course in life” can we instill in our students the numerical skills that they need to be savvy consumers, educated citizens and discerning interpreters of the ever-present polls, studies and surveys in which our society is awash? In what may well be their final opportunity to see applied mathematics in the classroom, can we impress upon them the importance of mathematical sophistication in interpreting the statistics that they are bombarded with by the media? Our study is in its second semester, and is designed to investigate to what extent it is possible to improve important quantitative skills in college students through a single semester introductory Astronomy course.

Assessing Student Status and Progress in Science Reasoning and Quantitative Literacy at a Very Large Undergraduate Institution

NASA Astrophysics Data System (ADS)

Donahue, Megan; Kaplan, J.; Ebert-May, D.; Ording, G.; Melfi, V.; Gilliland, D.; Sikorski, A.; Johnson, N.

2009-01-01

The typical large liberal-arts, tier-one research university requires all of its graduates to achieve some minimal standards of quantitative literacy and scientific reasoning skills. But how do we know what we are doing, as instructors and as a university, is working the way we think it should? At Michigan State University, a cross-disciplinary team of scientists, statisticians, and teacher education experts have begun a large-scale investigation about student mastery of quantitative and scientific skills, beginning with an assessment of 3,000 freshmen before they start their university careers. We will describe the process we used for developing and testing an instrument, for expanding faculty involvement and input on high-level goals. For this limited presentation, we will limit the discussion mainly to the scientific reasoning perspective, but we will briefly mention some intriguing observations regarding quantitative literacy as well. This project represents the beginning of long-term, longitudinal tracking of the progress of students at our institution. We will discuss preliminary results our 2008 assessment of incoming freshman at Michigan State, and where we plan to go from here. We acknowledge local support from the Quality Fund from the Office of the Provost at MSU. We also acknowledge the Center for Assessment at James Madison University and the NSF for their support at the very beginning of our work.

Non-technical skills assessment in surgery.

PubMed

Sharma, Bharat; Mishra, Amit; Aggarwal, Rajesh; Grantcharov, Teodor P

2011-09-01

Adverse events in surgery have highlighted the importance of non-technical skills, such as communication, decision-making, teamwork, situational awareness and leadership, to effective organizational performance. These skills carry particular importance to surgical oncology, as members of a multidisciplinary team must work cohesively to formulate effective patient care plans. Several non-technical skills evaluation tools have been developed for use in surgery, without adequate comparison and consensus on which should be standard for training. Eleven articles describing the use of three non-technical evaluation tools related to surgery: NOTSS (Non Technical Skills for Surgeons), NOTECHS (Non Technical Skills) and OTAS (Observational Teamwork Assessment for Surgery) were analyzed with respect to scale formulation, validity, reliability and feasibility. Furthermore, their use in training thus far and the future of non-technical rating scales in surgical curricula was discussed. Future work should focus on incorporating these assessment tools into training and into a real operating room setting to provide formative evaluations for surgical residents. Copyright © 2010 Elsevier Ltd. All rights reserved.

A model for communication skills assessment across the undergraduate curriculum.

PubMed

Rider, Elizabeth A; Hinrichs, Margaret M; Lown, Beth A

2006-08-01

Physicians' interpersonal and communication skills have a significant impact on patient care and correlate with improved healthcare outcomes. Some studies suggest, however, that communication skills decline during the four years of medical school. Regulatory and other medical organizations, recognizing the importance of interpersonal and communication skills in the practice of medicine, now require competence in communication skills. Two challenges exist: to select a framework of interpersonal and communication skills to teach across undergraduate medical education, and to develop and implement a uniform model for the assessment of these skills. The authors describe a process and model for developing and institutionalizing the assessment of communication skills across the undergraduate curriculum. Consensus was built regarding communication skill competencies by working with course leaders and examination directors, a uniform framework of competencies was selected to both teach and assess communication skills, and the framework was implemented across the Harvard Medical School undergraduate curriculum. The authors adapted an assessment framework based on the Bayer-Fetzer Kalamazoo Consensus Statement adapted a patient and added and satisfaction tool to bring patients' perspectives into the assessment of the learners. The core communication competencies and evaluation instruments were implemented in school-wide courses and assessment exercises including the first-year Patient-Doctor I Clinical Assessment, second-year Objective Structured Clinical Exam (OSCE), third-year Patient-Doctor III Clinical Assessment, fourth-year Comprehensive Clinical Practice Examination and the Core Medicine Clerkships. Faculty were offered workshops and interactive web-based teaching to become familiar with the framework, and students used the framework with repeated opportunities for faculty feedback on these skills. A model is offered for educational leaders and others who are involved

Assessing students' communication skills: validation of a global rating.

PubMed

Scheffer, Simone; Muehlinghaus, Isabel; Froehmel, Annette; Ortwein, Heiderose

2008-12-01

Communication skills training is an accepted part of undergraduate medical programs nowadays. In addition to learning experiences its importance should be emphasised by performance-based assessment. As detailed checklists have been shown to be not well suited for the assessment of communication skills for different reasons, this study aimed to validate a global rating scale. A Canadian instrument was translated to German and adapted to assess students' communication skills during an end-of-semester-OSCE. Subjects were second and third year medical students at the reformed track of the Charité-Universitaetsmedizin Berlin. Different groups of raters were trained to assess students' communication skills using the global rating scale. Validity testing included concurrent validity and construct validity: Judgements of different groups of raters were compared to expert ratings as a defined gold standard. Furthermore, the amount of agreement between scores obtained with this global rating scale and a different instrument for assessing communication skills was determined. Results show that communication skills can be validly assessed by trained non-expert raters as well as standardised patients using this instrument.

Neurosurgical Skills Assessment: Measuring Technical Proficiency in Neurosurgery Residents Through Intraoperative Video Evaluations.

PubMed

Sarkiss, Christopher A; Philemond, Steven; Lee, James; Sobotka, Stanislaw; Holloway, Terrell D; Moore, Maximillian M; Costa, Anthony B; Gordon, Errol L; Bederson, Joshua B

2016-05-01

Although technical skills are fundamental in neurosurgery, there is little agreement on how to describe, measure, or compare skills among surgeons. The primary goal of this study was to develop a quantitative grading scale for technical surgical performance that distinguishes operator skill when graded by domain experts (residents, attendings, and nonsurgeons). Scores provided by raters should be highly reliable with respect to scores from other observers. Neurosurgery residents were fitted with a head-mounted video camera while performing craniotomies under attending supervision. Seven videos, 1 from each postgraduate year (PGY) level (1-7), were anonymized and scored by 16 attendings, 8 residents, and 7 nonsurgeons using a grading scale. Seven skills were graded: incision, efficiency of instrument use, cauterization, tissue handling, drilling/craniotomy, confidence, and training level. A strong correlation was found between skills score and PGY year (P < 0.001, analysis of variance). Junior residents (PGY 1-3) had significantly lower scores than did senior residents (PGY 4-7, P < 0.001, t test). Significant variation among junior residents was observed, and senior residents' scores were not significantly different from one another. Interrater reliability, measured against other observers, was high (r = 0.581 ± 0.245, Spearman), as was assessment of resident training level (r = 0.583 ± 0.278, Spearman). Both variables were strongly correlated (r = 0.90, Pearson). Attendings, residents, and nonsurgeons did not score differently (P = 0.46, analysis of variance). Technical skills of neurosurgery residents recorded during craniotomy can be measured with high interrater reliability. Surgeons and nonsurgeons alike readily distinguish different skill levels. This type of assessment could be used to coach residents, to track performance over time, and potentially to compare skill levels. Developing an objective tool to evaluate surgical performance would be useful

Objective assessment of gynecologic laparoscopic skills using the LapSimGyn virtual reality simulator.

PubMed

Larsen, C R; Grantcharov, T; Aggarwal, R; Tully, A; Sørensen, J L; Dalsgaard, T; Ottesen, B

2006-09-01

Safe realistic training and unbiased quantitative assessment of technical skills are required for laparoscopy. Virtual reality (VR) simulators may be useful tools for training and assessing basic and advanced surgical skills and procedures. This study aimed to investigate the construct validity of the LapSimGyn VR simulator, and to determine the learning curves of gynecologists with different levels of experience. For this study, 32 gynecologic trainees and consultants (juniors or seniors) were allocated into three groups: novices (0 advanced laparoscopic procedures), intermediate level (>20 and <60 procedures), and experts (>100 procedures). All performed 10 sets of simulations consisting of three basic skill tasks and an ectopic pregnancy program. The simulations were carried out on 3 days within a maximum period of 2 weeks. Assessment of skills was based on time, economy of movement, and error parameters measured by the simulator. The data showed that expert gynecologists performed significantly and consistently better than intermediate and novice gynecologists. The learning curves differed significantly between the groups, showing that experts start at a higher level and more rapidly reach the plateau of their learning curve than do intermediate and novice groups of surgeons. The LapSimGyn VR simulator package demonstrates construct validity on both the basic skills module and the procedural gynecologic module for ectopic pregnancy. Learning curves can be obtained, but to reach the maximum performance for the more complex tasks, 10 repetitions do not seem sufficient at the given task level and settings. LapSimGyn also seems to be flexible and widely accepted by the users.

Using Quantitative Literacy to Enhance Critical Thinking Skills in Undergraduate Nursing Students.

PubMed

Asknes, Edna

2017-04-01

Critical thinking and quantitative literacy (QL) are similarly grounded: both focus on analyzing and evaluating evidence, identifying implications and consequences, drawing inferences, and communicating information. This teaching strategy was based on those commonalities and was designed so that undergraduate nursing students would enhance their critical thinking skills as they used their QL skills. QL skills are most effective when taught, learned, and used to solve significant, pertinent problems. Using the principles of learner-centered, team-based learning, QL was integrated into the curriculum of the Maternal-Newborn Nursing course at an urban community college with a diverse student population. Students were engaged and demonstrated enhanced and ongoing development of their critical thinking and problem-solving skills. They also reported a better understanding of data interpretation and use. The positive outcome of this project revealed further opportunities for incorporating QL into nursing curricula and highlighted the need for research on the use of QL in nursing education. [J Nurs Educ. 2017;56(4):240-242.]. Copyright 2017, SLACK Incorporated.

The Quantitative Reasoning for College Science (QuaRCS) Assessment in non-Astro 101 Courses

NASA Astrophysics Data System (ADS)

Kirkman, Thomas W.; Jensen, Ellen

2016-06-01

The innumeracy of American students and adults is a much lamented educational problem. The quantitative reasoning skills of college students may be particularly addressed and improved in "general education" science courses like Astro 101. Demonstrating improvement requires a standardized instrument. Among the non-proprietary instruments the Quantitative Literacy and Reasoning Assessment[1] (QRLA) and the Quantitative Reasoning for College Science (QuaRCS) Assessment[2] stand out.Follette et al. developed the QuaRCS in the context of Astro 101 at University of Arizona. We report on QuaRCS results in different contexts: pre-med physics and pre-nursing microbiology at a liberal arts college. We report on the mismatch between students' contemporaneous report of a question's difficulty and the actual probability of success. We report correlations between QuaRCS and other assessments of overall student performance in the class. We report differences in attitude towards mathematics in these two different but health-related student populations .[1] QLRA, Gaze et al., 2014, DOI: http://dx.doi.org/10.5038/1936-4660.7.2.4[2] QuaRCS, Follette, et al., 2015, DOI: http://dx.doi.org/10.5038/1936-4660.8.2.2

Brief Report: Suitability of the Social Skills Performance Assessment (SSPA) for the Assessment of Social Skills in Adults with Autism Spectrum Disorders

ERIC Educational Resources Information Center

Verhoeven, E. W. M.; Smeekens, I.; Didden, R.

2013-01-01

The present study aims at examining whether the "Social Skills Performance Assessment" (SSPA; Patterson et al. in "Schizophr Res" 48(2-3):351-360, 2001) is a suitable performance-based measure to assess social skills in adults with autism spectrum disorders (ASD). For this purpose, social skills of individuals with ASD and…

A Qualitative Analysis of Narrative Preclerkship Assessment Data to Evaluate Teamwork Skills.

PubMed

Dolan, Brigid M; O'Brien, Celia Laird; Cameron, Kenzie A; Green, Marianne M

2018-04-16

Construct: Students entering the health professions require competency in teamwork. Although many teamwork curricula and assessments exist, studies have not demonstrated robust longitudinal assessment of preclerkship students' teamwork skills and attitudes. Assessment portfolios may serve to fill this gap, but it is unknown how narrative comments within portfolios describe student teamwork behaviors. We performed a qualitative analysis of narrative data in 15 assessment portfolios. Student portfolios were randomly selected from 3 groups stratified by quantitative ratings of teamwork performance gathered from small-group and clinical preceptor assessment forms. Narrative data included peer and faculty feedback from these same forms. Data were coded for teamwork-related behaviors using a constant comparative approach combined with an identification of the valence of the coded statements as either "positive observation" or "suggestion for improvement." Eight codes related to teamwork emerged: attitude and demeanor, information facilitation, leadership, preparation and dependability, professionalism, team orientation, values team member contributions, and nonspecific teamwork comments. The frequency of codes and valence varied across the 3 performance groups, with students in the low-performing group receiving more suggestions for improvement across all teamwork codes. Narrative data from assessment portfolios included specific descriptions of teamwork behavior, with important contributions provided by both faculty and peers. A variety of teamwork domains were represented. Such feedback as collected in an assessment portfolio can be used for longitudinal assessment of preclerkship student teamwork skills and attitudes.

Comparing new BSN RN self skills assessment to actual skills demonstration.

PubMed

Adair, Jean; Hughes, Lin; Davis, Sue; Wolcott-Breci, Mary

2014-01-01

The purpose of the study was to compare the self-skills assessment with the skill competence during an actual skills demonstration of newly hired bachelor of science in nursing (BSN) registered nurse graduates. This retrospective study included 32 randomly selected BSN registered nurse graduates from January 2010 to December 31, 2010. The participants were already hired into a midwest health system. Because this was a retrospective study, no demographic data were collected, and no consent from participants was needed. This study included a clinical skills check list where the participants rated themselves on specific skills utilizing a Likert scale ranging from 1 (no knowledge) to 4 (able to perform independently). The same clinical check list was utilized by an expert registered nurse when the skill was demonstrated. This study compared the difference between the subject's self-rating of skills and the clinical demonstration of the skills. We used t tests in the analysis to demonstrate the differences between the participant's self-rating of skills and the expert evaluation of the clinical demonstration of the skills. The data were inserted into the Statistical Package for the Social Sciences 19 software program to assist in the analysis process. The study demonstrated 17 significant differences in the skills ratings between the participant and competency demonstration of new BSN graduates. These significant results (2 tailed) ranged from .000 to .048.The 17 out of 46 specific skills where differences were noted included the following: staple removal, nasal pharyngeal suctioning, urinary catheter specimen collection, site care dressing change, urinary catheter irrigation, Juzo application and measurement, 5-lead telemetry, oral airway insertion, hemovac/Jackson Pratt, oral pharyngeal suctioning, urinary catheter insertion, dry suction chest drainage, bed to cart/slider board, urinary catheter removal, antiembolism stockings, measurement and application, removal of

Guide to an Assessment of Consumer Skills.

ERIC Educational Resources Information Center

Education Commission of the States, Denver, CO.

This guide is intended to assist those interested in developing and/or assessing consumer skills. It is an accompanyment to a separate collection of survey items (mostly in a multiple choice format) designed to assess seventeen-year-olds' consumer skills. It is suggested that the items can be used as part of an item pool, as an instructional tool,…

The Effectiveness of the Geritalk Communication Skills Course: A Real-Time Assessment of Skill Acquisition and Deliberate Practice

PubMed Central

Gelfman, Laura P.; Lindenberger, Elizabeth; Fernandez, Helen; Goldberg, Gabrielle R.; Lim, Betty B.; Litrivis, Evgenia; O’Neill, Lynn; Smith, Cardinale B.; Kelley, Amy S.

2014-01-01

Background Communication skills are critical in Geriatrics and Palliative Medicine because these patients confront complex clinical scenarios. We evaluated effectiveness of the Geritalk communication skills course by comparing pre- and post-course real-time assessment of participants leading family meetings. We also evaluated the participants’ sustained skills practice. Measures We compare participants’ skill acquisition before and after Geritalk using a direct observation Family Meeting Communication Assessment Tool, and assessed their deliberate practice at follow-up. Intervention First-year Geriatrics or Palliative Medicine fellows at Mount Sinai Medical Center and the James J. Peters Bronx VA Medical Center participated in Geritalk. Outcomes Pre- and post-course family meeting assessments were compared. An average net gain of 6.8 skills represented a greater than 20% improvement in use of applicable skills. At two-month follow-up, most participants reported deliberate practice of fundamental and advanced skills. Conclusions This intensive training and family meeting assessment offers evidence-based communication skills training. PMID:24681183

The effectiveness of the Geritalk communication skills course: a real-time assessment of skill acquisition and deliberate practice.

PubMed

Gelfman, Laura P; Lindenberger, Elizabeth; Fernandez, Helen; Goldberg, Gabrielle R; Lim, Betty B; Litrivis, Evgenia; O'Neill, Lynn; Smith, Cardinale B; Kelley, Amy S

2014-10-01

Communication skills are critical in Geriatrics and Palliative Medicine because these patients confront complex clinical scenarios. We evaluated the effectiveness of the Geritalk communication skills course by comparing pre- and post-course real-time assessment of the participants leading family meetings. We also evaluated the participants' sustained skills practice. We compare the participants' skill acquisition before and after Geritalk using a direct observation Family Meeting Communication Assessment Tool and assess their deliberate practice at follow-up. First-year Geriatrics or Palliative Medicine fellows at Mount Sinai Medical Center and the James J. Peters Bronx VA Medical Center participated in Geritalk. Pre- and post-course family meeting assessments were compared. An average net gain of 6.8 skills represented a greater than 20% improvement in use of applicable skills. At two month follow-up, most participants reported deliberate practice of fundamental and advanced skills. This intensive training and family meeting assessment offers evidence-based communication skills training. Published by Elsevier Inc.

Teamwork Skills Assessment for Cooperative Learning

ERIC Educational Resources Information Center

Strom, Paris S.; Strom, Robert D.

2011-01-01

Teamwork skills are required at work, but teacher efforts in many countries to track achievement within this context have been hindered by lack of assessment tools and input from students. The Teamwork Skills Inventory relies on peer and self-evaluation to establish accountability, identify competencies, and detect learning needs. Twenty-five…

The development and validation of measures to assess cooking skills and food skills.

PubMed

Lavelle, Fiona; McGowan, Laura; Hollywood, Lynsey; Surgenor, Dawn; McCloat, Amanda; Mooney, Elaine; Caraher, Martin; Raats, Monique; Dean, Moira

2017-09-02

With the increase use of convenience food and eating outside the home environment being linked to the obesity epidemic, the need to assess and monitor individuals cooking and food skills is key to help intervene where necessary to promote the usage of these skills. Therefore, this research aimed to develop and validate a measure for cooking skills and one for food skills, that are clearly described, relatable, user-friendly, suitable for different types of studies, and applicable across all sociodemographic levels. Two measures were developed in light of the literature and expert opinion and piloted for clarity and ease of use. Following this, four studies were undertaken across different cohorts (including a sample of students, both 'Food preparation novices' and 'Experienced food preparers', and a nationally representative sample) to assess temporal stability, psychometrics, internal consistency reliability and construct validity of both measures. Analysis included T-tests, Pearson's correlations, factor analysis, and Cronbach's alphas, with a significance level of 0.05. Both measures were found to have a significant level of temporal stability (P < 0.001). Factor analysis revealed three factors with eigenvalues over 1, with two items in a third factor outside the two suggested measures. The internal consistency reliability for the cooking skills confidence measure ranged from 0.78 to 0.93 across all cohorts. The food skills confidence measure's Cronbach's alpha's ranged from 0.85 to 0.94. The two measures also showed a high discriminate validity as there were significant differences (P < 0.05 for cooking skills confidence and P < 0.01 for food skills confidence) between Food preparation novices' and 'Experienced food preparers.' The cooking skills confidence measure and the food skills confidence measure have been shown to have a very satisfactory reliability, validity and are consistent over time. Their user-friendly applicability make both measures

Teaching Fundamental Skills in Microsoft Excel to First-Year Students in Quantitative Analysis

ERIC Educational Resources Information Center

Rubin, Samuel J.; Abrams, Binyomin

2015-01-01

Despite their technological savvy, most students entering university lack the necessary computer skills to succeed in a quantitative analysis course, in which they are often expected to input, analyze, and plot results of experiments without any previous formal education in Microsoft Excel or similar programs. This lack of formal education results…

Clinical skills assessment of procedural and advanced communication skills: performance expectations of residency program directors.

PubMed

Langenau, Erik E; Zhang, Xiuyuan; Roberts, William L; DeChamplain, Andre F; Boulet, John R

2012-01-01

High stakes medical licensing programs are planning to augment and adapt current examinations to be relevant for a two-decision point model for licensure: entry into supervised practice and entry into unsupervised practice. Therefore, identifying which skills should be assessed at each decision point is critical for informing examination development, and gathering input from residency program directors is important. Using data from previously developed surveys and expert panels, a web-delivered survey was distributed to 3,443 residency program directors. For each of the 28 procedural and 18 advanced communication skills, program directors were asked which clinical skills should be assessed, by whom, when, and how. Descriptive statistics were collected, and Intraclass Correlations (ICC) were conducted to determine consistency across different specialties. Among 347 respondents, program directors reported that all advanced communication and some procedural tasks are important to assess. The following procedures were considered 'important' or 'extremely important' to assess: sterile technique (93.8%), advanced cardiovascular life support (ACLS) (91.1%), basic life support (BLS) (90.0%), interpretation of electrocardiogram (89.4%) and blood gas (88.7%). Program directors reported that most clinical skills should be assessed at the end of the first year of residency (or later) and not before graduation from medical school. A minority were considered important to assess prior to the start of residency training: demonstration of respectfulness (64%), sterile technique (67.2%), BLS (68.9%), ACLS (65.9%) and phlebotomy (63.5%). Results from this study support that assessing procedural skills such as cardiac resuscitation, sterile technique, and phlebotomy would be amenable to assessment at the end of medical school, but most procedural and advanced communications skills would be amenable to assessment at the end of the first year of residency training or later. Gathering

Quantitative assessment of developmental levels in overarm throwing using wearable inertial sensing technology.

PubMed

Grimpampi, Eleni; Masci, Ilaria; Pesce, Caterina; Vannozzi, Giuseppe

2016-09-01

Motor proficiency in childhood has been recently recognised as a public health determinant, having a potential impact on the physical activity level and possible sedentary behaviour of the child later in life. Among fundamental motor skills, ballistic skills assessment based on in-field quantitative observations is progressively needed in the motor development community. The aim of this study was to propose an in-field quantitative approach to identify different developmental levels in overarm throwing. Fifty-eight children aged 5-10 years performed an overarm throwing task while wearing three inertial sensors located at the wrist, trunk and pelvis level and were then categorised using a developmental sequence of overarm throwing. A set of biomechanical parameters were defined and analysed using multivariate statistics to evaluate whether they can be used as developmental indicators. Trunk and pelvis angular velocities and time durations before the ball release showed increasing/decreasing trends with increasing developmental level. Significant differences between developmental level pairs were observed for selected biomechanical parameters. The results support the suitability and feasibility of objective developmental measures in ecological learning contexts, suggesting their potential supportiveness to motor learning experiences in educational and youth sports training settings.

Assessing Business Student Thinking Skills

ERIC Educational Resources Information Center

Smith, Gerald F.

2014-01-01

The development of student thinking skills is a major goal of business education. As with other such goals, student outcomes assessment must be undertaken to measure goal achievement. Thinking is difficult to teach; it is also difficult to assess. The purpose of this article is to improve management educators' understanding of student thinking…

Assessing Motor Skill Competency in Elementary School Students: A Three-Year Study.

PubMed

Chen, Weiyun; Mason, Steve; Hypnar, Andrew; Bennett, Austin

2016-03-01

This study was to examine how well fourth- and fifth-grade students demonstrated motor skill competency assessed with selected PE Metrics assessment rubrics (2009). Fourth- and fifth-grade students (n = 1,346-1,926) were assessed on their performance of three manipulative skills using the PE Metrics Assessment Rubrics during the pre-intervention year, the post-intervention year 1, and the post-intervention year 3. Descriptive statistics, independent t-test, ANOVA, and follow-up comparisons were conducted for data analysis. The results indicated that the post-intervention year 2 cohort performed significantly more competent than the pre-intervention cohort and the post-intervention year 1 cohort on the three manipulative skill assessments. The post-intervention year 1 cohort significantly outperformed the pre-intervention cohort on the soccer dribbling, passing, and receiving and the striking skill assessments, but not on the throwing skill assessment. Although the boys in the three cohorts performed significantly better than the girls on all three skills, the girls showed substantial improvement on the overhand throwing and the soccer skills from baseline to the post-intervention year 1 and the post-intervention year 2. However, the girls, in particular, need to improve striking skill. The CTACH PE was conducive to improving fourth- and fifth-grade students' motor skill competency in the three manipulative skills. This study suggest that PE Metrics assessment rubrics are feasible tools for PE teachers to assess levels of students' demonstration of motor skill competency during a regular PE lesson. Key pointsCATCH PE is an empirically-evidenced quality PE curricular that is conducive to improving students' manipulative skill competency.Boys significantly outperformed than girls in all three manipulative skills.Girls need to improve motor skill competency in striking skill. PE Metrics are feasible assessment rubrics that can be easily used by trained physical

Assessing Motor Skill Competency in Elementary School Students: A Three-Year Study

PubMed Central

Chen, Weiyun; Mason, Steve; Hypnar, Andrew; Bennett, Austin

2016-01-01

This study was to examine how well fourth- and fifth-grade students demonstrated motor skill competency assessed with selected PE Metrics assessment rubrics (2009). Fourth- and fifth-grade students (n = 1,346-1,926) were assessed on their performance of three manipulative skills using the PE Metrics Assessment Rubrics during the pre-intervention year, the post-intervention year 1, and the post-intervention year 3. Descriptive statistics, independent t-test, ANOVA, and follow-up comparisons were conducted for data analysis. The results indicated that the post-intervention year 2 cohort performed significantly more competent than the pre-intervention cohort and the post-intervention year 1 cohort on the three manipulative skill assessments. The post-intervention year 1 cohort significantly outperformed the pre-intervention cohort on the soccer dribbling, passing, and receiving and the striking skill assessments, but not on the throwing skill assessment. Although the boys in the three cohorts performed significantly better than the girls on all three skills, the girls showed substantial improvement on the overhand throwing and the soccer skills from baseline to the post-intervention year 1 and the post-intervention year 2. However, the girls, in particular, need to improve striking skill. The CTACH PE was conducive to improving fourth- and fifth-grade students’ motor skill competency in the three manipulative skills. This study suggest that PE Metrics assessment rubrics are feasible tools for PE teachers to assess levels of students’ demonstration of motor skill competency during a regular PE lesson. Key points CATCH PE is an empirically-evidenced quality PE curricular that is conducive to improving students’ manipulative skill competency. Boys significantly outperformed than girls in all three manipulative skills. Girls need to improve motor skill competency in striking skill. PE Metrics are feasible assessment rubrics that can be easily used by trained

A Perkins Challenge: Assessing Technical Skills in CTE

ERIC Educational Resources Information Center

Stone, James R., III

2009-01-01

Federal law requires state to develop performance measures and data-collection systems for secondary and postsecondary technical-skill attainment. This poses many challenges, such as defining a technical skills, measurement and when to assess students. In this article, the author outlines various assessment models and looks at the challenges…

The Quantitative Reasoning for College Science (QuaRCS) Assessment in non-Astro 101 Courses II

NASA Astrophysics Data System (ADS)

Kirkman, Thomas W.; Jensen, Ellen

2017-06-01

The Quantitative Reasoning for College Science (QuaRCS) Assessment[1] aims to measure the pre-algebra mathematical skills that are often part of "general education" science courses like Astro 101. In four majors STEM classes, we report comparisons between QuaRCS metrics, ACT math, GPAO, and the course grade. In three of four classes QuaRCS QR score and ACT math were statistically significantly correlated (with r˜.6), however in the fourth course —a senior-level microbiology course— there was no statistically significantly correlation (in fact, r<0). In all courses —even in courses with seemingly little quantitative content— course grade was statistically significantly correlated to GPAO and QR. A QuaRCS metric aiming to report the students belief in the importance of math in science was seen to grow with the course level. Pre/post QuaRCS testing in Physics courses showed fractional sigma gains in QR, self-estimated math fluency and math importance, but not all of those increases were statistically significant. Using a QuaRCS map relating the questions to skill areas, we found graph reading, percentages, and proportional reasoning to be the most misunderstood skills in all four courses.[1] QuaRCS, Follette, et al.,2015, DOI: http://dx.doi.org/10.5038/1936-4660.8.2.2

The use of advanced physical assessment skills by cardiac nurses.

PubMed

Edmunds, Linda; Ward, Susan; Barnes, Rhian

To establish what advanced physical assessment skills are being used by cardiac nurses after they undertook a clinical patient assessment module; and to explore the factors that influence their use of these skills. A longitudinal descriptive approach using convenience sampling was employed. Qualitative data was obtained from individual interviews, non-participant observation within the participants' clinical environment and self-reported activity logs. Five key themes emerged: use of advanced physical assessment skills varied; use and development of skills was linked to personal characteristics; use influenced by perceptions of role boundaries, permission and cooperation; use influenced by participants' perception of nursing and the development of their own nursing practice; and use influenced by the physical environment and the human support within it. Cardiac nurses selectively use physical assessment skills, predominately related to the cardiorespiratory systems. Organisational structure, professional relationships and the professionalism of the individual nurse appear to play a significant part in the use of physical assessment skills. Although the findings from this qualitative study cannot be generalized, they concur with findings from recent research into physical assessment skills used by a variety of UK nurses. The implications identified are: first, for those who provide the education, in terms of what should be taught and facilitated; and second, for organizations, in ensuring staff have assessment skills relevant to their role and that systems are in place to enable the development of a supportive and progressive culture that embraces modernization congruent with healthcare policy.

A Formal Investigation of Human Spatial Control Skills: Mathematical Formalization, Skill Development, and Skill Assessment

NASA Astrophysics Data System (ADS)

Li, Bin

Spatial control behaviors account for a large proportion of human everyday activities from normal daily tasks, such as reaching for objects, to specialized tasks, such as driving, surgery, or operating equipment. These behaviors involve intensive interactions within internal processes (i.e. cognitive, perceptual, and motor control) and with the physical world. This dissertation builds on a concept of interaction pattern and a hierarchical functional model. Interaction pattern represents a type of behavior synergy that humans coordinates cognitive, perceptual, and motor control processes. It contributes to the construction of the hierarchical functional model that delineates humans spatial control behaviors as the coordination of three functional subsystems: planning, guidance, and tracking/pursuit. This dissertation formalizes and validates these two theories and extends them for the investigation of human spatial control skills encompassing development and assessment. Specifically, this dissertation first presents an overview of studies in human spatial control skills encompassing definition, characteristic, development, and assessment, to provide theoretical evidence for the concept of interaction pattern and the hierarchical functional model. The following, the human experiments for collecting motion and gaze data and techniques to register and classify gaze data, are described. This dissertation then elaborates and mathematically formalizes the hierarchical functional model and the concept of interaction pattern. These theories then enables the construction of a succinct simulation model that can reproduce a variety of human performance with a minimal set of hypotheses. This validates the hierarchical functional model as a normative framework for interpreting human spatial control behaviors. The dissertation then investigates human skill development and captures the emergence of interaction pattern. The final part of the dissertation applies the hierarchical

Assessment of dental students’ communication skills with patients

PubMed Central

MEMARPOUR, MAHTAB; BAZRAFKAN, LEILA; ZAREI, ZAHRA

2016-01-01

Introduction Establishment of effective communication between the clinician and patient is essential in order to increase the effectiveness of treatment. These skills have been less investigated among dental students. This study aimed to evaluate communication skills of dental students in Shiraz with patients through direct observation, patients' perspectives and students' self-assessments. Methods This cross-sectional study enrolled the fifth and sixth year dental students and one of each student’s patients who was chosen using simple random sampling method. We used a checklist for data collection. Students’ communication skills were assessed at three steps of the student-patient interview – at the beginning of the interview, during the interview, and at the end of the interview. The checklist was completed by three groups: 1) an observer, 2) the patient and 3) the student, as self-assessment. The validity of the checklist was confirmed by clinical professors and the reliability was determined by Cronbach's alpha test. Data were analyzed using descriptive statistics and Student’s t test. A repeated measure MANOVA was used to compare the mean communication skills in the researcher, patients, and students at each step of the patient interviews. Results There were 110 students (mean age: 22.3±8.4 years) and 110 patients (mean age: 32±8.8 years) who completed the checklists. Overall, the communication skills of dental students were rated as good according to the patients. However, the observer and student participants rated the skills at the moderate level. We observed significant differences between communication skills in all three groups and in the three steps of the patient interviews (p<0.001). According to patients' beliefs and students' self assessments, there were no differences between male and female students in communication skills in the three steps of the patient interviews (all p>0.05). However from the observer’s viewpoint, female students

Clinical skills assessment of procedural and advanced communication skills: performance expectations of residency program directors

PubMed Central

Langenau, Erik E.; Zhang, Xiuyuan; Roberts, William L.; DeChamplain, Andre F.; Boulet, John R.

2012-01-01

Background High stakes medical licensing programs are planning to augment and adapt current examinations to be relevant for a two-decision point model for licensure: entry into supervised practice and entry into unsupervised practice. Therefore, identifying which skills should be assessed at each decision point is critical for informing examination development, and gathering input from residency program directors is important. Methods Using data from previously developed surveys and expert panels, a web-delivered survey was distributed to 3,443 residency program directors. For each of the 28 procedural and 18 advanced communication skills, program directors were asked which clinical skills should be assessed, by whom, when, and how. Descriptive statistics were collected, and Intraclass Correlations (ICC) were conducted to determine consistency across different specialties. Results Among 347 respondents, program directors reported that all advanced communication and some procedural tasks are important to assess. The following procedures were considered ‘important’ or ‘extremely important’ to assess: sterile technique (93.8%), advanced cardiovascular life support (ACLS) (91.1%), basic life support (BLS) (90.0%), interpretation of electrocardiogram (89.4%) and blood gas (88.7%). Program directors reported that most clinical skills should be assessed at the end of the first year of residency (or later) and not before graduation from medical school. A minority were considered important to assess prior to the start of residency training: demonstration of respectfulness (64%), sterile technique (67.2%), BLS (68.9%), ACLS (65.9%) and phlebotomy (63.5%). Discussion Results from this study support that assessing procedural skills such as cardiac resuscitation, sterile technique, and phlebotomy would be amenable to assessment at the end of medical school, but most procedural and advanced communications skills would be amenable to assessment at the end of the first

A Procedural Skills OSCE: Assessing Technical and Non-Technical Skills of Internal Medicine Residents

ERIC Educational Resources Information Center

Pugh, Debra; Hamstra, Stanley J.; Wood, Timothy J.; Humphrey-Murto, Susan; Touchie, Claire; Yudkowsky, Rachel; Bordage, Georges

2015-01-01

Internists are required to perform a number of procedures that require mastery of technical and non-technical skills, however, formal assessment of these skills is often lacking. The purpose of this study was to develop, implement, and gather validity evidence for a procedural skills objective structured clinical examination (PS-OSCE) for internal…

Communicating With Patients: Skills Assessment in US Colleges of Pharmacy

PubMed Central

2006-01-01

Objective To describe current practices in assessing patient communication skills in US colleges and schools of pharmacy. Methods Syllabi and behavioral assessment forms were solicited and key faculty members were interviewed. Forms were analyzed to determine skills most commonly assessed in communication with simulated or role-playing patients. Results Fifty schools submitted behavioral assessment forms for patient communication skills. Individuals from 47 schools were interviewed. Colleges were found to vary in the way communication skills were assessed. Assessment forms focused more on dispensing a new prescription than monitoring ongoing therapy. Providing information was emphasized more than promoting adherence. Common faculty concerns were lack of continuity and congruence of assessment across the curriculum. Conclusions A common understanding of the standards and procedures for determining competence is needed. Experience and assessment activities should be sequenced throughout a program to build competence. PMID:17136187

Learning Style Differences in Adult Students' Perceptions of the Effectiveness of Classroom Techniques for Teaching Quantitative Skills

ERIC Educational Resources Information Center

Deever, Walter Thomas

2012-01-01

More than half of adults in the USA have quantitative literacy ratings at or below a basic level. This lack of literacy often becomes a barrier to employability. To overcome this barrier, adults are returning to college to improve their quantitative skills and complete an undergraduate education, often through an accelerated degree program. A…

A quantitative framework for assessing ecological resilience

EPA Science Inventory

Quantitative approaches to measure and assess resilience are needed to bridge gaps between science, policy, and management. In this paper, we suggest a quantitative framework for assessing ecological resilience. Ecological resilience as an emergent ecosystem phenomenon can be de...

A Study of Assessments Designed for Student Success

ERIC Educational Resources Information Center

Delepine, Sidney G., III

2012-01-01

The purpose of this quantitative study is to compare a new assessment tool, the SkillsUSA Connect Assessment with the NOCTI assessment to determine which test results in more students achieving success. A quantitative study, designed to compare test scores of students taking the NOCTI assessment and new assessments from SkillsUSA, called the…

Quantitative assessment of paretic limb dexterity and interlimb coordination during bilateral arm rehabilitation training.

PubMed

Xu, Chang; Li, Siyi; Wang, Kui; Hou, Zengguang; Yu, Ningbo

2017-07-01

In neuro-rehabilitation after stroke, the conventional constrained induced movement therapy (CIMT) has been well-accepted. Existing bilateral trainings are mostly on mirrored symmetrical motion. However, complementary bilateral movements are dominantly involved in activities of daily living (ADLs), and functional bilateral therapies may bring better skill transfer from trainings to daily life. Neurophysiological evidence is also growing. In this work, we firstly introduce our bilateral arm training system realized with a haptic interface and a motion sensor, as well as the tasks that have been designed to train both the manipulation function of the paretic arm and coordination of bilateral upper limbs. Then, we propose quantitative measures for functional assessment of complementary bilateral training performance, including kinematic behavior indices, smoothness, submovement and bimanual coordination. After that, we describe the experiments with healthy subjects and the results with respect to these quantitative measures. Feasibility and sensitivity of the proposed indices were evaluated through comparison of unilateral and bilateral training outcomes. The proposed bilateral training system and tasks, as well as the quantitative measures, have been demonstrated effective for training and assessment of unilateral and bilateral arm functions.

Teaching and assessing procedural skills using simulation: metrics and methodology.

PubMed

Lammers, Richard L; Davenport, Moira; Korley, Frederick; Griswold-Theodorson, Sharon; Fitch, Michael T; Narang, Aneesh T; Evans, Leigh V; Gross, Amy; Rodriguez, Elliot; Dodge, Kelly L; Hamann, Cara J; Robey, Walter C

2008-11-01

Simulation allows educators to develop learner-focused training and outcomes-based assessments. However, the effectiveness and validity of simulation-based training in emergency medicine (EM) requires further investigation. Teaching and testing technical skills require methods and assessment instruments that are somewhat different than those used for cognitive or team skills. Drawing from work published by other medical disciplines as well as educational, behavioral, and human factors research, the authors developed six research themes: measurement of procedural skills; development of performance standards; assessment and validation of training methods, simulator models, and assessment tools; optimization of training methods; transfer of skills learned on simulator models to patients; and prevention of skill decay over time. The article reviews relevant and established educational research methodologies and identifies gaps in our knowledge of how physicians learn procedures. The authors present questions requiring further research that, once answered, will advance understanding of simulation-based procedural training and assessment in EM.

Accuracy of quantitative visual soil assessment

NASA Astrophysics Data System (ADS)

van Leeuwen, Maricke; Heuvelink, Gerard; Stoorvogel, Jetse; Wallinga, Jakob; de Boer, Imke; van Dam, Jos; van Essen, Everhard; Moolenaar, Simon; Verhoeven, Frank; Stoof, Cathelijne

2016-04-01

Visual soil assessment (VSA) is a method to assess soil quality visually, when standing in the field. VSA is increasingly used by farmers, farm organisations and companies, because it is rapid and cost-effective, and because looking at soil provides understanding about soil functioning. Often VSA is regarded as subjective, so there is a need to verify VSA. Also, many VSAs have not been fine-tuned for contrasting soil types. This could lead to wrong interpretation of soil quality and soil functioning when contrasting sites are compared to each other. We wanted to assess accuracy of VSA, while taking into account soil type. The first objective was to test whether quantitative visual field observations, which form the basis in many VSAs, could be validated with standardized field or laboratory measurements. The second objective was to assess whether quantitative visual field observations are reproducible, when used by observers with contrasting backgrounds. For the validation study, we made quantitative visual observations at 26 cattle farms. Farms were located at sand, clay and peat soils in the North Friesian Woodlands, the Netherlands. Quantitative visual observations evaluated were grass cover, number of biopores, number of roots, soil colour, soil structure, number of earthworms, number of gley mottles and soil compaction. Linear regression analysis showed that four out of eight quantitative visual observations could be well validated with standardized field or laboratory measurements. The following quantitative visual observations correlated well with standardized field or laboratory measurements: grass cover with classified images of surface cover; number of roots with root dry weight; amount of large structure elements with mean weight diameter; and soil colour with soil organic matter content. Correlation coefficients were greater than 0.3, from which half of the correlations were significant. For the reproducibility study, a group of 9 soil scientists and 7

Surgical Crisis Management Skills Training and Assessment

PubMed Central

Moorthy, Krishna; Munz, Yaron; Forrest, Damien; Pandey, Vikas; Undre, Shabnam; Vincent, Charles; Darzi, Ara

2006-01-01

Background: Intraoperative surgical crisis management is learned in an unstructured manner. In aviation, simulation training allows aircrews to coordinate and standardize recovery strategies. Our aim was to develop a surgical crisis simulation and evaluate its feasibility, realism, and validity of the measures used to assess performance. Methods: Surgical trainees were exposed to a bleeding crisis in a simulated operating theater. Assessment of performance consisted of a trainee’s technical ability to control the bleeding and of their team/human factors skills. This assessment was performed in a blinded manner by 2 surgeons and one human factors expert. Other measures consisted of time measures such as time to diagnose the bleeding (TD), inform team members (TT), achieve control (TC), and close the laceration (TL). Blood loss was used as a surrogate outcome measures. Results: There were considerable variations within both senior (n = 10) and junior (n = 10) trainees for technical and team skills. However, while the senior trainees scored higher than the juniors for technical skills (P = 0.001), there were no differences in human factors skills. There were also significant differences between the 2 groups for TD (P = 0.01), TC (P = 0.001), and TL (0.001). The blood loss was higher in the junior group. Conclusions: We have described the development of a novel simulated setting for the training of crisis management skills and the variability in performance both in between and within the 2 groups. PMID:16794399

Towards quantitative assessment of calciphylaxis

NASA Astrophysics Data System (ADS)

Deserno, Thomas M.; Sárándi, István.; Jose, Abin; Haak, Daniel; Jonas, Stephan; Specht, Paula; Brandenburg, Vincent

2014-03-01

Calciphylaxis is a rare disease that has devastating conditions associated with high morbidity and mortality. Calciphylaxis is characterized by systemic medial calcification of the arteries yielding necrotic skin ulcerations. In this paper, we aim at supporting the installation of multi-center registries for calciphylaxis, which includes a photographic documentation of skin necrosis. However, photographs acquired in different centers under different conditions using different equipment and photographers cannot be compared quantitatively. For normalization, we use a simple color pad that is placed into the field of view, segmented from the image, and its color fields are analyzed. In total, 24 colors are printed on that scale. A least-squares approach is used to determine the affine color transform. Furthermore, the card allows scale normalization. We provide a case study for qualitative assessment. In addition, the method is evaluated quantitatively using 10 images of two sets of different captures of the same necrosis. The variability of quantitative measurements based on free hand photography is assessed regarding geometric and color distortions before and after our simple calibration procedure. Using automated image processing, the standard deviation of measurements is significantly reduced. The coefficients of variations yield 5-20% and 2-10% for geometry and color, respectively. Hence, quantitative assessment of calciphylaxis becomes practicable and will impact a better understanding of this rare but fatal disease.

Assessing Motor Skills in Multiply Handicapped Children.

ERIC Educational Resources Information Center

DuBose, Rebecca F.

Examined are the effects of motor skill development and impairment on the infant's and young child's overall functioning, and suggested are guidelines for assessing motor skills in multiply handicapped children. It is explained that motor delays and deficits limit a child's learning during critical developmental periods. Examples of delayed motor…

A comparison of students' self-assessments with faculty evaluations of their communication skills.

PubMed

Lundquist, Lisa M; Shogbon, Angela O; Momary, Kathryn M; Rogers, Hannah K

2013-05-13

To compare students' self-assessment of their communication skills with faculty members' formal evaluation of their skills in a therapeutics course. Over a 3-year period, faculty members evaluated second-year pharmacy students' communication skills as part of a requirement in a therapeutics course. Immediately following an individual oral assessment and again following a group oral assessment, students self-assessed their communication skills using the same rubric the faculty members had used. Students' self-assessments were then compared with faculty members' evaluation of students' communication skills. Four hundred one (97.3%) students consented to participate in this study. Faculty evaluation scores of students for both the individual and group oral assessments were significantly higher than students' self-assessment scores. Students' self-assessment scores of their communication skills increased from the individual to the group oral assessment. Students' self-assessments of communication skills were consistently lower than faculty members' evaluations. Greater use of oral assessments throughout the pharmacy curriculum may help to improve students' confidence in and self-assessment of their communication skills.

Quantitative CMMI Assessment for Offshoring through the Analysis of Project Management Repositories

NASA Astrophysics Data System (ADS)

Sunetnanta, Thanwadee; Nobprapai, Ni-On; Gotel, Olly

The nature of distributed teams and the existence of multiple sites in offshore software development projects pose a challenging setting for software process improvement. Often, the improvement and appraisal of software processes is achieved through a turnkey solution where best practices are imposed or transferred from a company’s headquarters to its offshore units. In so doing, successful project health checks and monitoring for quality on software processes requires strong project management skills, well-built onshore-offshore coordination, and often needs regular onsite visits by software process improvement consultants from the headquarters’ team. This paper focuses on software process improvement as guided by the Capability Maturity Model Integration (CMMI) and proposes a model to evaluate the status of such improvement efforts in the context of distributed multi-site projects without some of this overhead. The paper discusses the application of quantitative CMMI assessment through the collection and analysis of project data gathered directly from project repositories to facilitate CMMI implementation and reduce the cost of such implementation for offshore-outsourced software development projects. We exemplify this approach to quantitative CMMI assessment through the analysis of project management data and discuss the future directions of this work in progress.

Quantitative Assessment of Motor and Sensory/Motor Acquisition in Handicapped and Nonhandicapped Infants and Young Children. Volume III: Replication of the Procedures.

ERIC Educational Resources Information Center

Guess, Doug; And Others

Ten replication studies based on quantitative procedures developed to measure motor and sensory/motor skill acquisition among handicapped and nonhandicapped infants and children are presented. Each study follows the original assessment procedures, and emphasizes the stability of interobserver reliability across time, consistency in the response…

Data Nuggets: Bringing Real Data into the Classroom to Unearth Students' Quantitative & Inquiry Skills

ERIC Educational Resources Information Center

Schultheis, Elizabeth H.; Kjelvik, Melissa K.

2015-01-01

Current educational reform calls for increased integration between science and mathematics to overcome the shortcomings in students' quantitative skills. Data Nuggets (free online resource, http://datanuggets.org) are worksheets that bring data into the classroom, repeatedly guiding students through the scientific method and making claims…

A qualitative study of advanced nurse practitioners' use of physical assessment skills in the community: shifting skills across professional boundaries.

PubMed

Raleigh, Mary; Allan, Helen

2017-07-01

To explore multiple perspectives on the use of physical assessment skills by advanced nurse practitioners in the UK. Physical assessment skills practices are embedded in advanced nursing practice roles in the UK. There is little evidence on how these skills are used by advanced nurse practitioners in the community. Case study. A qualitative interpretative single-embedded case study of 22 participants from South of England. A framework method analysed interview data collected by the researcher between March-August 2013. Participants included nurses, doctors, nurse educators and managers. Physical assessment skills education at universities is part of a policy shift to develop a flexible workforce in the UK. Shared physical assessment practices are less to do with role substitution and more about preparing practitioners with skills that are fit for purpose. Competence, capability and performance with physical assessment skills are an expectation of advanced nursing practice. These skills are used successfully by community advanced nurse practitioners to deliver a wide range of services in response to changing patient need. The introduction of physical assessment skills education to undergraduate professional preparation would create a firm foundation to develop these skills in postgraduate education. Physical assessment education prepares nurses with the clinical competencies to carry out healthcare reforms in the UK. Shared sets of clinical assessment competencies between disciplines have better outcomes for patients. Levels of assessment competence can depend on the professional attributes of individual practitioners. Unsupportive learning cultures can hinder professional development of advanced nursing practice. © 2016 John Wiley & Sons Ltd.

How Effective Are Self- and Peer Assessment of Oral Presentation Skills Compared with Teachers' Assessments?

ERIC Educational Resources Information Center

De Grez, Luc; Valcke, Martin; Roozen, Irene

2012-01-01

Assessment of oral presentation skills is an underexplored area. The study described here focuses on the agreement between professional assessment and self- and peer assessment of oral presentation skills and explores student perceptions about peer assessment. The study has the merit of paying attention to the inter-rater reliability of the…

Helpers' Self-Assessment Biases Before and after Helping Skills Training.

PubMed

Jaeken, Marine; Zech, Emmanuelle; Brison, Céline; Verhofstadt, Lesley L; Van Broeck, Nady; Mikolajczak, Moïra

2017-01-01

Several studies have shown that therapists are generally biased concerning their performed helping skills, as compared to judges' ratings. As clients' ratings of therapists' performance are better predictors of psychotherapy effectiveness than judges' ratings, this study examined the validity and effectiveness of a helping skills training program at reducing novice helpers' self-enhancement biases concerning their helping skills, in comparison to their clients' ratings. Helping skills were assessed by three objective measures (a knowledge multiple choice test, a video test and a role play), as well as by a self- and peer-reported questionnaire. In addition, some performed helping skills' correlates (relationship quality, session quality, and helpers' therapeutic attitudes) were assessed both by helpers and their simulated helpees. Seventy-two sophomores in psychology participated to this study, 37 being assigned to a 12-h helping skills training program, and 35 to a control group. Helpers were expected to assess the aforementioned performed helping skills and correlates as being better than their helpees' assessments at pretest, thus revealing a self-enhancement bias. At posttest, we expected that trained helpers would objectively exhibit better helping skills than untrained helpers while beginning to underestimate their performance, thus indexing a self-diminishment bias. In contrast, we hypothesized that untrained helpers would continue to overestimate their performance. Our hypotheses were only partly confirmed but results reflected a skilled-unaware pattern among trainees. Trained helpers went either from a pretest overestimation to a posttest equivalence (performed helping skills and performed therapeutic attitudes), or from a pretest equivalence to a posttest underestimation (performed session quality and performed therapeutic relationship), as compared to helpees' ratings. Results showed that trained helpers improved on all helping skills objective measures

Helpers' Self-Assessment Biases Before and after Helping Skills Training

PubMed Central

Jaeken, Marine; Zech, Emmanuelle; Brison, Céline; Verhofstadt, Lesley L.; Van Broeck, Nady; Mikolajczak, Moïra

2017-01-01

Several studies have shown that therapists are generally biased concerning their performed helping skills, as compared to judges' ratings. As clients' ratings of therapists' performance are better predictors of psychotherapy effectiveness than judges' ratings, this study examined the validity and effectiveness of a helping skills training program at reducing novice helpers' self-enhancement biases concerning their helping skills, in comparison to their clients' ratings. Helping skills were assessed by three objective measures (a knowledge multiple choice test, a video test and a role play), as well as by a self- and peer-reported questionnaire. In addition, some performed helping skills' correlates (relationship quality, session quality, and helpers' therapeutic attitudes) were assessed both by helpers and their simulated helpees. Seventy-two sophomores in psychology participated to this study, 37 being assigned to a 12-h helping skills training program, and 35 to a control group. Helpers were expected to assess the aforementioned performed helping skills and correlates as being better than their helpees' assessments at pretest, thus revealing a self-enhancement bias. At posttest, we expected that trained helpers would objectively exhibit better helping skills than untrained helpers while beginning to underestimate their performance, thus indexing a self-diminishment bias. In contrast, we hypothesized that untrained helpers would continue to overestimate their performance. Our hypotheses were only partly confirmed but results reflected a skilled-unaware pattern among trainees. Trained helpers went either from a pretest overestimation to a posttest equivalence (performed helping skills and performed therapeutic attitudes), or from a pretest equivalence to a posttest underestimation (performed session quality and performed therapeutic relationship), as compared to helpees' ratings. Results showed that trained helpers improved on all helping skills objective measures

Assessing Elementary Students' Writing Skills. Publication No. 78.74.

ERIC Educational Resources Information Center

Friedman, Myron; Fowler, Elaine

An instrument was developed for use in the evaluation of a pilot program to improve the writing skills of elementary school students in the Austin (Texas) Independent School District. Called the "Assessment of Writing Skills" (AWS), the instrument assesses writing maturity, productivity, and writing mechanics by collecting a holistic evaluation…

Assessing Students' Communication Skills: Validation of a Global Rating

ERIC Educational Resources Information Center

Scheffer, Simone; Muehlinghaus, Isabel; Froehmel, Annette; Ortwein, Heiderose

2008-01-01

Communication skills training is an accepted part of undergraduate medical programs nowadays. In addition to learning experiences its importance should be emphasised by performance-based assessment. As detailed checklists have been shown to be not well suited for the assessment of communication skills for different reasons, this study aimed to…

Automated surgical skill assessment in RMIS training.

PubMed

Zia, Aneeq; Essa, Irfan

2018-05-01

Manual feedback in basic robot-assisted minimally invasive surgery (RMIS) training can consume a significant amount of time from expert surgeons' schedule and is prone to subjectivity. In this paper, we explore the usage of different holistic features for automated skill assessment using only robot kinematic data and propose a weighted feature fusion technique for improving score prediction performance. Moreover, we also propose a method for generating 'task highlights' which can give surgeons a more directed feedback regarding which segments had the most effect on the final skill score. We perform our experiments on the publicly available JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS) and evaluate four different types of holistic features from robot kinematic data-sequential motion texture (SMT), discrete Fourier transform (DFT), discrete cosine transform (DCT) and approximate entropy (ApEn). The features are then used for skill classification and exact skill score prediction. Along with using these features individually, we also evaluate the performance using our proposed weighted combination technique. The task highlights are produced using DCT features. Our results demonstrate that these holistic features outperform all previous Hidden Markov Model (HMM)-based state-of-the-art methods for skill classification on the JIGSAWS dataset. Also, our proposed feature fusion strategy significantly improves performance for skill score predictions achieving up to 0.61 average spearman correlation coefficient. Moreover, we provide an analysis on how the proposed task highlights can relate to different surgical gestures within a task. Holistic features capturing global information from robot kinematic data can successfully be used for evaluating surgeon skill in basic surgical tasks on the da Vinci robot. Using the framework presented can potentially allow for real-time score feedback in RMIS training and help surgical trainees have more focused training.

Rubric Authoring Tool Supporting Cognitive Skills Assessment across an Institution

ERIC Educational Resources Information Center

Simper, Natalie

2018-01-01

This paper explores a method to support instructors in assessing cognitive skills in their course, designed to enable aggregation of data across an institution. A rubric authoring tool, "BASICS" (Building Assessment Scaffolds for Intellectual Cognitive Skills) was built as part of the Queen's University Learning Outcomes Assessment (LOA)…

Medical and psychology students' self-assessed communication skills: A pilot study.

PubMed

Tiuraniemi, Juhani; Läärä, Riitta; Kyrö, Tuuli; Lindeman, Sari

2011-05-01

The purpose of this study was to describe how psychology and medical students assess their own competency and skills before and after training, in which role-play was used to teach interpersonal and communication skills. Interpersonal and communication skills were assessed with a semi-structured questionnaire before and after the training. The students of both medicine and psychology estimated their skill levels to be higher after the course. The psychology students estimated their skills for communication, motivating interviewing, empathy and reflection, and change orientation to be better at the end of the course. Medical students estimated their communication skills, motivating interviewing skills, and change orientation skills to be better at the end of the course. Even a short period of training in interpersonal and communication skills can positively affect the self-assessed skills of the medical students. In the future, it would be worthwhile to pay attention to reflective teaching practices in the training of both medical and psychology students. The cognitive and emotional components of these practices help students to develop their own communication skills. Copyright © 2010 Elsevier Ireland Ltd. All rights reserved.

Crowd-sourced assessment of surgical skills in cricothyrotomy procedure.

PubMed

Aghdasi, Nava; Bly, Randall; White, Lee W; Hannaford, Blake; Moe, Kris; Lendvay, Thomas S

2015-06-15

Objective assessment of surgical skills is resource intensive and requires valuable time of expert surgeons. The goal of this study was to assess the ability of a large group of laypersons using a crowd-sourcing tool to grade a surgical procedure (cricothyrotomy) performed on a simulator. The grading included an assessment of the entire procedure by completing an objective assessment of technical skills survey. Two groups of graders were recruited as follows: (1) Amazon Mechanical Turk users and (2) three expert surgeons from University of Washington Department of Otolaryngology. Graders were presented with a video of participants performing the procedure on the simulator and were asked to grade the video using the objective assessment of technical skills questions. Mechanical Turk users were paid $0.50 for each completed survey. It took 10 h to obtain all responses from 30 Mechanical Turk users for 26 training participants (26 videos/tasks), whereas it took 60 d for three expert surgeons to complete the same 26 tasks. The assessment of surgical performance by a group (n = 30) of laypersons matched the assessment by a group (n = 3) of expert surgeons with a good level of agreement determined by Cronbach alpha coefficient = 0.83. We found crowd sourcing was an efficient, accurate, and inexpensive method for skills assessment with a good level of agreement to experts' grading. Copyright © 2015 Elsevier Inc. All rights reserved.

A Systems Approach to Scaffold Communication Skills Development

PubMed Central

Er, Nelson L.

2008-01-01

Objectives To implement a communication skills development (CSD) system and evaluate its effectiveness in a clinical communications course. Design Students conducted baseline interviews and wrote SOAP notes, and based on faculty, patient, self- and peer assessments, set goals for improvement of their communication skills. Students participated in various activities to scaffold their learning, several of which took place in a web-based environment to enhance access and function for both students and faculty members. Quantitative and qualitative analyses were performed. Assessment Students' communication skills improved as evidenced by assessment scores. Student and faculty comments offered additional evidence of the effectiveness of standardized patient interviews, learning strategies, and assessment methods. Conclusion The CSD system effectively integrated various types of learning activities and feedback processes. The use of scaffolding strategies appeared to enhance the development of students' communication skills. PMID:18483601

Implementing online quantitative support modules in an intermediate-level course

NASA Astrophysics Data System (ADS)

Daly, J.

2011-12-01

While instructors typically anticipate that students in introductory geology courses enter a class with a wide range of quantitative ability, we often overlook the fact that this may also be true in upper-level courses. Some students are drawn to the subject and experience success in early courses with an emphasis on descriptive geology, then experience frustration and disappointment in mid- and upper-level courses that are more quantitative. To bolster student confidence in quantitative skills and enhance their performance in an upper-level course, I implemented several modules from The Math You Need (TMYN) online resource with a 200-level geomorphology class. Student facility with basic quantitative skills (rearranging equations, manipulating units, and graphing) was assessed with an online pre- and post-test. During the semester, modules were assigned to complement existing course activities (for example, the module on manipulating units was assigned prior to a lab on measurement of channel area and water velocity, then calculation of discharge). The implementation was designed to be a concise review of relevant skills for students with higher confidence in their quantitative abilities, and to provide a self-paced opportunity for students with less quantitative facility to build skills. This course already includes a strong emphasis on quantitative data collection, analysis, and presentation; in the past, student performance in the course has been strongly influenced by their individual quantitative ability. I anticipate that giving students the opportunity to improve mastery of fundamental quantitative skills will improve their performance on higher-stakes assignments and exams, and will enhance their sense of accomplishment in the course.

Validation of the self-assessment of communication skills and professionalism for nurses.

PubMed

Pereira, Thaís Josgrilberg; Puggina, Ana Claudia

2017-01-01

to translate, adapt cross-culturally and validate into Brazilian Portuguese the following instrument: "Self-assessment of communication skills and professionalism in residents" for the nursing professional, and to determine if personal characteristics and performance of the nurse interfere in the self-assessment about professionalism and interpersonal communication. quantitative study. the sample consisted of 110 nurses with mean age of 32 years old (± 7.3), most of them were women (n = 80; 72.7%). The internal consistency of the scale "Autoavaliação sobre profissionalismo e comunicação interpessoal entre enfermeiro e paciente" presented moderate and satisfactory reliability (α=0,712). Factorial analysis identified four factors: Interpersonal Skills, Exchange of Information, Honesty in the Relationship and Professionalism. the instrument is valid and reliable in Portuguese and for Brazilian culture. Interpersonal Skills changed with gender and marital status. Ability to exchange information was influenced by gender and working sector. Self-assessment of professionalism changed with marital status. traduzir, adaptar culturalmente e validar para o português o instrumento Self-assessment of communication skills and professionalism in residents para o profissional enfermeiro e avaliar se características pessoais e de atuação do enfermeiro interferem na autoavaliação sobre o profissionalismo e comunicação interpessoal. estudo metodológico quantitativo. A amostra foi de 110 enfermeiros com média de idade de 32 anos (±7,3) e a maioria mulheres (n=80; 72,7%). A consistência interna da escala "Autoavaliação sobre profissionalismo e comunicação interpessoal entre enfermeiro e paciente" apresentou confiabilidade moderada e satisfatória (α=0,712). A análise fatorial identificou quatro fatores: Habilidade Interpessoal, Troca de Informação, Sinceridade na Relação e Profissionalismo. o instrumento é válido e confiável na língua portuguesa e para a

Development of a Biological Science Quantitative Reasoning Exam (BioSQuaRE)

ERIC Educational Resources Information Center

Stanhope, Liz; Ziegler, Laura; Haque, Tabassum; Le, Laura; Vinces, Marcelo; Davis, Gregory K.; Zieffler, Andrew; Brodfuehrer, Peter; Preest, Marion; Belitsky, Jason M.; Umbanhowar, Charles, Jr.; Overvoorde, Paul J.

2017-01-01

Multiple reports highlight the increasingly quantitative nature of biological research and the need to innovate means to ensure that students acquire quantitative skills. We present a tool to support such innovation. The Biological Science Quantitative Reasoning Exam (BioSQuaRE) is an assessment instrument designed to measure the quantitative…

The role of multimedia in surgical skills training and assessment.

PubMed

Shariff, Umar; Seretis, Charalampos; Lee, Doreen; Balasubramanian, Saba P

2016-06-01

Multimedia is an educational resource that can be used to supplement surgical skills training. The aim of this review was to determine the role of multimedia in surgical training and assessment by performing a systematic review of the literature. A systematic review for published articles was conducted on the following databases: PubMed/MEDLINE (1992 to November 2014), SCOPUS (1992 to November 2014) and EMBASE (1992 to November 2014). For each study the educational content, study design, surgical skill assessed and outcomes were recorded. A standard data extraction form was created to ensure systematic retrieval of relevant information. 21 studies were included; 14 randomized controlled trials (RCTs) and 7 non-randomized controlled trials (Non-RCTs). Technical skills were assessed in 7 RCTs and 3 non-RCTs; cognitive skills were assessed in 9 RCTs and 4 non-RCTs. In controlled studies, multimedia was associated with significant improvement in technical skills (4 studies; 4 RCTs) and cognitive skills (7 studies; 6 RCTs). In two studies multimedia was inferior in comparison to conventional teaching. Evaluation of multimedia (9 studies) demonstrated strongly favourable results. This review suggests that multimedia effectively facilitates both technical and cognitive skills acquisition and is well accepted as an educational resource. Copyright © 2015 Royal College of Surgeons of Edinburgh (Scottish charity number SC005317) and Royal College of Surgeons in Ireland. Published by Elsevier Ltd. All rights reserved.

Video and accelerometer-based motion analysis for automated surgical skills assessment.

PubMed

Zia, Aneeq; Sharma, Yachna; Bettadapura, Vinay; Sarin, Eric L; Essa, Irfan

2018-03-01

Basic surgical skills of suturing and knot tying are an essential part of medical training. Having an automated system for surgical skills assessment could help save experts time and improve training efficiency. There have been some recent attempts at automated surgical skills assessment using either video analysis or acceleration data. In this paper, we present a novel approach for automated assessment of OSATS-like surgical skills and provide an analysis of different features on multi-modal data (video and accelerometer data). We conduct a large study for basic surgical skill assessment on a dataset that contained video and accelerometer data for suturing and knot-tying tasks. We introduce "entropy-based" features-approximate entropy and cross-approximate entropy, which quantify the amount of predictability and regularity of fluctuations in time series data. The proposed features are compared to existing methods of Sequential Motion Texture, Discrete Cosine Transform and Discrete Fourier Transform, for surgical skills assessment. We report average performance of different features across all applicable OSATS-like criteria for suturing and knot-tying tasks. Our analysis shows that the proposed entropy-based features outperform previous state-of-the-art methods using video data, achieving average classification accuracies of 95.1 and 92.2% for suturing and knot tying, respectively. For accelerometer data, our method performs better for suturing achieving 86.8% average accuracy. We also show that fusion of video and acceleration features can improve overall performance for skill assessment. Automated surgical skills assessment can be achieved with high accuracy using the proposed entropy features. Such a system can significantly improve the efficiency of surgical training in medical schools and teaching hospitals.

Development of a Biological Science Quantitative Reasoning Exam (BioSQuaRE)

PubMed Central

Stanhope, Liz; Ziegler, Laura; Haque, Tabassum; Le, Laura; Vinces, Marcelo; Davis, Gregory K.; Zieffler, Andrew; Brodfuehrer, Peter; Preest, Marion; M. Belitsky, Jason; Umbanhowar, Charles; Overvoorde, Paul J.

2017-01-01

Multiple reports highlight the increasingly quantitative nature of biological research and the need to innovate means to ensure that students acquire quantitative skills. We present a tool to support such innovation. The Biological Science Quantitative Reasoning Exam (BioSQuaRE) is an assessment instrument designed to measure the quantitative skills of undergraduate students within a biological context. The instrument was developed by an interdisciplinary team of educators and aligns with skills included in national reports such as BIO2010, Scientific Foundations for Future Physicians, and Vision and Change. Undergraduate biology educators also confirmed the importance of items included in the instrument. The current version of the BioSQuaRE was developed through an iterative process using data from students at 12 postsecondary institutions. A psychometric analysis of these data provides multiple lines of evidence for the validity of inferences made using the instrument. Our results suggest that the BioSQuaRE will prove useful to faculty and departments interested in helping students acquire the quantitative competencies they need to successfully pursue biology, and useful to biology students by communicating the importance of quantitative skills. We invite educators to use the BioSQuaRE at their own institutions. PMID:29196427

Assessing Students' Technical Skill Attainment

ERIC Educational Resources Information Center

Jorgensen, Haley

2010-01-01

The Wisconsin Technical College System (WTCS) is working to comply with the Carl D. Perkins Career and Technical Education Improvement Act of 2006 (Perkins) to ensure that its graduates have mastered the technical skills needed by business and industry. The legislation requires that each state identify and approve program assessment strategies…

Ensuring the Criminological Skills of the Next Generation: A Case Study on the Importance of Enhanced Quantitative Method Teaching Provision

ERIC Educational Resources Information Center

Chamberlain, John Martyn

2017-01-01

Against the backdrop of contemporary debates surrounding the public role of criminology, this article argues that a key barrier to ensuring that the next generation of criminologists is equipped with the skills necessary to engage in critical forms of citizenship, is the quantitative "skills gap" that undergraduate students possess as a…

Soft Skills Assessment: Theory Development and the Research Agenda

ERIC Educational Resources Information Center

Gibb, Stephen

2014-01-01

Soft skills have become a subject of increasing interest in lifelong learning. Soft skills development is intended to enable and enhance personal development, participation in learning and success in employment. The assessment of soft skill is therefore widely practised, but there is little in the way of research or evidence on how well this…

The development of aerobic and skill assessment in soccer.

PubMed

O'Reilly, John; Wong, Stephen H S

2012-12-01

Methods of assessing soccer players' performance have developed significantly in recent times. The fitness profiles and skill levels of a prospective elite soccer player is a valuable resource for coaches in the process of identifying talent. Traditional means to measure aerobic fitness have centred on the 'aerobic capacity' or '&OV0312;O(2max)' test (also known as the maximal oxygen consumption test) but, over time, this has been shown not to be a sensitive measure for specific aspects of soccer in a match situation. Therefore, numerous soccer-specific simulations have been designed to re-create exercise patterns similar to those experienced during a match. Some of these studies have yet to be validated, while others have been shown to result in a similar physiological load to that encountered during regular match play. Further developments have led to specifically designed intermittent sprint tests, which are used as a sensitive tool to accurately measure the fluctuations in players' ability both between and within soccer seasons. Testing procedures have also been developed that incorporate elements of both skill and physical ability. Soccer-specific field tests have been designed, incorporating skill and dynamic movements, and this opens up the possibility of teams testing the aerobic capacity of their elite players using soccer-specific movements. Valid studies assessing soccer-specific skills in an ecologically sound environment have been quite rare until recently. Some test protocols have been deemed largely irrelevant to soccer match play, while others have had limited impact on scientific literature. More recently, skill tests have been developed and shown to be valid and reliable methods of assessing soccer skill performance. Many new skill tests continue to be developed, and some have been shown to be highly reliable, but further study of these relatively novel concepts is required before a more solid recommendation can be made. Overall, while significant

Identifying criteria for the assessment of pharmacy students' communication skills with patients.

PubMed

Mackellar, Adele; Ashcroft, Darren M; Bell, Dawn; James, Delyth Higman; Marriott, John

2007-06-15

To identify criteria by which patients can assess the communication skills of pharmacy students. Potential assessment criteria were generated from 2 main sources: a literature review and a focus group discussion. A modified two-round Delphi survey was subsequently conducted with 35 professionals who were actively involved in teaching and assessing communication skills of pharmacy students to determine the importance and reliability of each criterion. Consensus ratings identified 7 criteria that were important measures of pharmacy students' communication skills and could be reliably assessed by patients. A modified two-round Delphi consultation survey successfully identified criteria that can be used by patients to assess the communication skills of pharmacy undergraduates. Future work will examine the feasibility of using patients as assessors of communication skills of pharmacy students, preregistration pharmacists, and qualified pharmacists.

Evaluating Attitudes, Skill, and Performance in a Learning-Enhanced Quantitative Methods Course: A Structural Modeling Approach.

ERIC Educational Resources Information Center

Harlow, Lisa L.; Burkholder, Gary J.; Morrow, Jennifer A.

2002-01-01

Used a structural modeling approach to evaluate relations among attitudes, initial skills, and performance in a Quantitative Methods course that involved students in active learning. Results largely confirmed hypotheses offering support for educational reform efforts that propose actively involving students in the learning process, especially in…

The Postoperative Pain Assessment Skills pilot trial.

PubMed

McGillion, Michael; Dubrowski, Adam; Stremler, Robyn; Watt-Watson, Judy; Campbell, Fiona; McCartney, Colin; Victor, Charles; Wiseman, Jeffrey; Snell, Linda; Costello, Judy; Robb, Anja; Nelson, Sioban; Stinson, Jennifer; Hunter, Judith; Dao, Thuan; Promislow, Sara; McNaughton, Nancy; White, Scott; Shobbrook, Cindy; Jeffs, Lianne; Mauch, Kianda; Leegaard, Marit; Beattie, W Scott; Schreiber, Martin; Silver, Ivan

2011-01-01

BACKGROUND⁄ Pain-related misbeliefs among health care professionals (HCPs) are common and contribute to ineffective postoperative pain assessment. While standardized patients (SPs) have been effectively used to improve HCPs' assessment skills, not all centres have SP programs. The present equivalence randomized controlled pilot trial examined the efficacy of an alternative simulation method - deteriorating patient-based simulation (DPS) - versus SPs for improving HCPs' pain knowledge and assessment skills. Seventy-two HCPs were randomly assigned to a 3 h SP or DPS simulation intervention. Measures were recorded at baseline, immediate postintervention and two months postintervention. The primary outcome was HCPs' pain assessment performance as measured by the postoperative Pain Assessment Skills Tool (PAST). Secondary outcomes included HCPs knowledge of pain-related misbeliefs, and perceived satisfaction and quality of the simulation. These outcomes were measured by the Pain Beliefs Scale (PBS), the Satisfaction with Simulated Learning Scale (SSLS) and the Simulation Design Scale (SDS), respectively. Student's t tests were used to test for overall group differences in postintervention PAST, SSLS and SDS scores. One-way analysis of covariance tested for overall group differences in PBS scores. DPS and SP groups did not differ on post-test PAST, SSLS or SDS scores. Knowledge of pain-related misbeliefs was also similar between groups. These pilot data suggest that DPS is an effective simulation alternative for HCPs' education on postoperative pain assessment, with improvements in performance and knowledge comparable with SP-based simulation. An equivalence trial to examine the effectiveness of deteriorating patient-based simulation versus standardized patients is warranted.

Attitudes to proposed assessment of pharmacy skills in Korean pharmacist licensure examination.

PubMed

Kim, Joo Hee; Lee, Ju-Yeun; Lee, Young Sook; Yong, Chul-Soon; Han, Nayoung; Gwak, Hye Sun; Oh, Jungmi; Lee, Byung Koo; Lee, Sukhyang

2017-01-01

The survey aimed to obtain opinions about a proposed implementation of pharmacy skills assessment in Korean pharmacist licensure examination (KPLE). A 16-question survey was distributed electronically to 2,738 people including 570 pharmacy professors of 35 pharmacy schools, 550 preceptors from 865 practice sites and 1,618 students who graduated in 2015. The survey solicited responses concerning the adequacy of the current KPLE in assessing pharmacy knowledge/skills/attitudes, deficiencies of pharmacy skills testing in assessing the professional competencies necessary for pharmacists, plans for pharmacy skills tests in the current KPLE, and subject areas of pharmacy practice. A total of 466 surveys were returned. The current exam is not adequate for assessing skills and attitudes according to 42%-48% of respondents. Sixty percent felt that skills test is necessary to assess qualifications and professional competencies. Almost two-thirds of participants stated that testing should be implemented within 5 years. More than 60% agreed that candidates should be graduates and that written and skills test scores can be combined for pass-fail decisions. About 70% of respondents felt that the test should be less than 2 hours in duration. Over half of the respondents thought that the assessor should be a pharmacy faculty member with at least 5 years of clinical experience. Up to 70% stated that activities related to patient care were appropriate and practical for the scope of skills test. Pharmacy skills assessment was supported by the majority of respondents.

Enhancing Student Communication Skills Through Arabic Language Competency and Simulated Patient Assessments.

PubMed

Hasan, Sanah; Tarazi, Hamadeh M Khier; Halim Hilal, Dana Abdel

2017-05-01

Objective. To assess student communication and patient management skill with introduction of Arabic and use of simulated patient assessments to a communication and counseling course. Design. Five, 3-hour tutorials (clinical skill laboratory) were added to the course covering: listening and empathic responding, non-verbal communications, interviewing skills, assertiveness, counseling in special situations: conflict, anger, worry or rushed situations, and professional decision making. Arabic content was introduced to the course to enhance Arabic communications and competence among students. Simulated patient assessment was used to evaluate student skills. Students' feedback about course changes was evaluated. Assessment. The course now covers a wider content and Arabic language. Students' scores were similar in the assessment and other assessments within the course and between Arabic and English groups. Students favorably rated the changes in the course and provided constructive feedback on content usefulness and adequacy. Conclusion. Expanding the course to include Arabic language and content and simulated patient assessments enhanced student communication skills.

Enhancing Student Communication Skills Through Arabic Language Competency and Simulated Patient Assessments

PubMed Central

Tarazi, Hamadeh (M. Khier); Halim Hilal, Dana Abdel

2017-01-01

Objective. To assess student communication and patient management skill with introduction of Arabic and use of simulated patient assessments to a communication and counseling course. Design. Five, 3-hour tutorials (clinical skill laboratory) were added to the course covering: listening and empathic responding, non-verbal communications, interviewing skills, assertiveness, counseling in special situations: conflict, anger, worry or rushed situations, and professional decision making. Arabic content was introduced to the course to enhance Arabic communications and competence among students. Simulated patient assessment was used to evaluate student skills. Students’ feedback about course changes was evaluated. Assessment. The course now covers a wider content and Arabic language. Students’ scores were similar in the assessment and other assessments within the course and between Arabic and English groups. Students favorably rated the changes in the course and provided constructive feedback on content usefulness and adequacy. Conclusion. Expanding the course to include Arabic language and content and simulated patient assessments enhanced student communication skills. PMID:28630517

Development of performance assessment instrument based contextual learning for measuring students laboratory skills

NASA Astrophysics Data System (ADS)

Susilaningsih, E.; Khotimah, K.; Nurhayati, S.

2018-04-01

The assessment of laboratory skill in general hasn’t specific guideline in assessment, while the individual assessment of students during a performance and skill in performing laboratory is still not been observed and measured properly. Alternative assessment that can be used to measure student laboratory skill is use performance assessment. The purpose of this study was to determine whether the performance assessment instrument that the result of research can be used to assess basic skills student laboratory. This research was conducted by the Research and Development. The result of the data analysis performance assessment instruments developed feasible to implement and validation result 62.5 with very good categories for observation sheets laboratory skills and all of the components with the very good category. The procedure is the preliminary stages of research and development stages. Preliminary stages are divided in two, namely the field studies and literature studies. The development stages are divided into several parts, namely 1) development of the type instrument, 2) validation by an expert, 3) a limited scale trial, 4) large-scale trials and 5) implementation of the product. The instrument included in the category of effective because 26 from 29 students have very high laboratory skill and high laboratory skill. The research of performance assessment instrument is standard and can be used to assess basic skill student laboratory.

Developing Employability Skills: Peer Assessment in Higher Education

ERIC Educational Resources Information Center

Cassidy, Simon

2006-01-01

Purpose: Reports examining graduate employment issues suggest that employers are concerned by the lack of employability skills exhibited by entry-level job applications. It is also suggested that employers consider it the responsibility of educational institutions to develop such skills. The current study seeks to identify peer assessment as a…

Defining and Assessing Team Skills of Business and Accountancy Students

ERIC Educational Resources Information Center

Alghalith, Nabil; Blum, Michael; Medlock, Amanda; Weber, Sandy

2004-01-01

The objectives of the project are (1) to define the skills necessary for students to work effectively with others to achieve common goals, and (2) to develop an assessment instrument to measure student progress toward achieving these skills. The defined skill set will form a basis for common expectations related to team skills that will be shared…

Accuracy and reliability of peer assessment of athletic training psychomotor laboratory skills.

PubMed

Marty, Melissa C; Henning, Jolene M; Willse, John T

2010-01-01

Peer assessment is defined as students judging the level or quality of a fellow student's understanding. No researchers have yet demonstrated the accuracy or reliability of peer assessment in athletic training education. To determine the accuracy and reliability of peer assessment of athletic training students' psychomotor skills. Cross-sectional study. Entry-level master's athletic training education program. First-year (n  =  5) and second-year (n  =  8) students. Participants evaluated 10 videos of a peer performing 3 psychomotor skills (middle deltoid manual muscle test, Faber test, and Slocum drawer test) on 2 separate occasions using a valid assessment tool. Accuracy of each peer-assessment score was examined through percentage correct scores. We used a generalizability study to determine how reliable athletic training students were in assessing a peer performing the aforementioned skills. Decision studies using generalizability theory demonstrated how the peer-assessment scores were affected by the number of participants and number of occasions. Participants had a high percentage of correct scores: 96.84% for the middle deltoid manual muscle test, 94.83% for the Faber test, and 97.13% for the Slocum drawer test. They were not able to reliably assess a peer performing any of the psychomotor skills on only 1 occasion. However, the φ increased (exceeding the 0.70 minimal standard) when 2 participants assessed the skill on 3 occasions (φ  =  0.79) for the Faber test, with 1 participant on 2 occasions (φ  =  0.76) for the Slocum drawer test, and with 3 participants on 2 occasions for the middle deltoid manual muscle test (φ  =  0.72). Although students did not detect all errors, they assessed their peers with an average of 96% accuracy. Having only 1 student assess a peer performing certain psychomotor skills was less reliable than having more than 1 student assess those skills on more than 1 occasion. Peer assessment of psychomotor skills

Quantitative Microbial Risk Assessment Tutorial - Primer

EPA Science Inventory

This document provides a Quantitative Microbial Risk Assessment (QMRA) primer that organizes QMRA tutorials. The tutorials describe functionality of a QMRA infrastructure, guide the user through software use and assessment options, provide step-by-step instructions for implementi...

Non-technical skills assessment for prelicensure nursing students: An integrative review.

PubMed

Pires, Sara; Monteiro, Sara; Pereira, Anabela; Chaló, Daniela; Melo, Elsa; Rodrigues, Alexandre

2017-11-01

In nursing, non-technical skills are recognized as playing an important role to increase patient safety and successful clinical outcomes (Pearson and McLafferty, 2011). Non-technical skills are cognitive and social resource skills that complement technical skills and contribute to safe and efficient task performance (Flin et al., 2008). In order to effectively provide non-technical skills training, it is essential to have an instrument to measure these skills. An online search was conducted. Articles were selected if they referred to and/or described instruments assessing non-technical skills for nurses and/or prelicensure nursing students in educational, clinical and/or simulated settings with validation evidence (inclusion criteria). Of the 53 articles located, 26 met the inclusion criteria. Those referred to and/or described 16 instruments with validation evidence developed to assess non-technical skills in multidisciplinary teams including nurses. Although articles have shown 16 valid and reliable instruments, to our knowledge, no instrument has been published or developed and validated for the assessment of non-technical skills of only nurses in general, relevant for use in high-fidelity simulation-based training for prelicensure nursing students. Therefore, there is a need for the development of such an instrument. Copyright © 2017 Elsevier Ltd. All rights reserved.

Attitudes to proposed assessment of pharmacy skills in Korean pharmacist licensure examination

PubMed Central

2017-01-01

Purpose The survey aimed to obtain opinions about a proposed implementation of pharmacy skills assessment in Korean pharmacist licensure examination (KPLE). Methods A 16-question survey was distributed electronically to 2,738 people including 570 pharmacy professors of 35 pharmacy schools, 550 preceptors from 865 practice sites and 1,618 students who graduated in 2015. The survey solicited responses concerning the adequacy of the current KPLE in assessing pharmacy knowledge/skills/attitudes, deficiencies of pharmacy skills testing in assessing the professional competencies necessary for pharmacists, plans for pharmacy skills tests in the current KPLE, and subject areas of pharmacy practice. Results A total of 466 surveys were returned. The current exam is not adequate for assessing skills and attitudes according to 42%–48% of respondents. Sixty percent felt that skills test is necessary to assess qualifications and professional competencies. Almost two-thirds of participants stated that testing should be implemented within 5 years. More than 60% agreed that candidates should be graduates and that written and skills test scores can be combined for pass-fail decisions. About 70% of respondents felt that the test should be less than 2 hours in duration. Over half of the respondents thought that the assessor should be a pharmacy faculty member with at least 5 years of clinical experience. Up to 70% stated that activities related to patient care were appropriate and practical for the scope of skills test. Conclusion Pharmacy skills assessment was supported by the majority of respondents. PMID:28352065

Competency Assessment in Senior Emergency Medicine Residents for Core Ultrasound Skills.

PubMed

Schmidt, Jessica N; Kendall, John; Smalley, Courtney

2015-11-01

Quality resident education in point-of-care ultrasound (POC US) is becoming increasingly important in emergency medicine (EM); however, the best methods to evaluate competency in graduating residents has not been established. We sought to design and implement a rigorous assessment of image acquisition and interpretation in POC US in a cohort of graduating residents at our institution. We evaluated nine senior residents in both image acquisition and image interpretation for five core US skills (focused assessment with sonography for trauma (FAST), aorta, echocardiogram (ECHO), pelvic, central line placement). Image acquisition, using an observed clinical skills exam (OSCE) directed assessment with a standardized patient model. Image interpretation was measured with a multiple-choice exam including normal and pathologic images. Residents performed well on image acquisition for core skills with an average score of 85.7% for core skills and 74% including advanced skills (ovaries, advanced ECHO, advanced aorta). Residents scored well but slightly lower on image interpretation with an average score of 76%. Senior residents performed well on core POC US skills as evaluated with a rigorous assessment tool. This tool may be developed further for other EM programs to use for graduating resident evaluation.

How to Assess Creative Thinking Skill in Making Products of Liquid Pressure?

NASA Astrophysics Data System (ADS)

Chasanah, L.; Kaniawati, I.; Hernani, H.

2017-09-01

The primary skills that must be possessed in the 21st century curriculum are learning and innovation skills. One of the learning strategies that can train students to innovate and improve creative thinking skills is by applying Science, Technology, Engineering and Mathematics (STEM) in learning. Based on an interview to one of the science teachers that learning that aims to train learning and innovation skills has not been applied to learning in the classroom because there is not enough time, learning materials and assessment instruments used. This study aims to determine the results of the implementation of performance assessment of creative thinking skills on STEM-based learning in junior high school for the material of liquid pressure. This research uses descriptive method. Participants in this study were junior high school students 8th in Kudus area. The research instrument consists of observation sheet, performance assessment and documentation. The result showed that creative thinking skills performance assessment can assess student’s creativity in making products of STEM-based learning for junior high school.

Advanced physical assessment skills: implementation of a module.

PubMed

Aldridge-Bent, Sharon

2011-02-01

This article aims to explore and examine advanced physical assessment skills and the role of the district nurse. It will particularly highlight district nurses' perceptions of how they may implement skills learnt on a new module introduced into the Community Health Care Nursing degree at a university in London. Physical assessment skills have traditionally been viewed as part of a doctor's role; however, with the advancement of nursing roles, it is argued that it has become a key nursing skill. As Government policy continues to expect health professionals to keep patients in the community who have complex health and social care needs, the role of the district nurse presents as 'best placed' to take on this challenge (Department of Health (DH), 2005a; 2005b). Evaluation of the district nurses' perceptions of their practice is shared here, highlighting some of the challenges that they face. The article will address the complexity of developing a curriculum in response to the DH initiatives and the importance of listening to students on courses.

Communication skills assessment: the perceptions of medical students at the University of Nottingham.

PubMed

Rees, Charlotte; Sheard, Charlotte; McPherson, Amy

2002-09-01

Despite the wealth of literature surrounding communication curricula within medical education, there is a lack of in-depth research into medical students' perceptions of communication skills assessment. This study aims to address this gap in the research literature. Five focus group discussions were conducted with 32 students, with representatives from each of the 5 years of the medical degree course at Nottingham University. Audiotapes of the discussions were transcribed in full and the transcripts were theme analysed independently by 2 analysts. Two assessment-related themes emerged from the analysis: namely, students' perceptions of formative assessment and students' perceptions of summative assessment. While students seemed to value formative methods of assessing their communication skills, they did not appear to value summative methods like objective structured clinical examinations (OSCEs). Students had mixed views about who should assess their oral communication skills. Some students preferred self-assessment while others preferred peer assessment. Although students appeared to value medical educators assessing their communication skills, other students preferred feedback from patients. Although summative methods like OSCEs were criticized widely, students suggested that examinations were essential to motivate students' learning of communication skills. This study begins to illustrate medical students' perceptions of communication skills assessment. However, further research using large-scale surveys is required to validate these findings. Medical educators should provide students with feedback on their communication skills wherever possible. This feedback should ideally come from a combination of different assessors. Over-assessment in other subject areas should be minimized to prevent students being discouraged from learning communication skills.

Skill and independence weighting for multi-model assessments

DOE PAGES

Sanderson, Benjamin M.; Wehner, Michael; Knutti, Reto

2017-06-28

We present a weighting strategy for use with the CMIP5 multi-model archive in the fourth National Climate Assessment, which considers both skill in the climatological performance of models over North America as well as the inter-dependency of models arising from common parameterizations or tuning practices. The method exploits information relating to the climatological mean state of a number of projection-relevant variables as well as metrics representing long-term statistics of weather extremes. The weights, once computed can be used to simply compute weighted means and significance information from an ensemble containing multiple initial condition members from potentially co-dependent models of varyingmore » skill. Two parameters in the algorithm determine the degree to which model climatological skill and model uniqueness are rewarded; these parameters are explored and final values are defended for the assessment. The influence of model weighting on projected temperature and precipitation changes is found to be moderate, partly due to a compensating effect between model skill and uniqueness. However, more aggressive skill weighting and weighting by targeted metrics is found to have a more significant effect on inferred ensemble confidence in future patterns of change for a given projection.« less

Does a Rater's Professional Background Influence Communication Skills Assessment?

PubMed

Artemiou, Elpida; Hecker, Kent G; Adams, Cindy L; Coe, Jason B

2015-01-01

There is increasing pressure in veterinary education to teach and assess communication skills, with the Objective Structured Clinical Examination (OSCE) being the most common assessment method. Previous research reveals that raters are a large source of variance in OSCEs. This study focused on examining the effect of raters' professional background as a source of variance when assessing students' communication skills. Twenty-three raters were categorized according to their professional background: clinical sciences (n=11), basic sciences (n=4), clinical communication (n=5), or hospital administrator/clinical skills technicians (n=3). Raters from each professional background were assigned to the same station and assessed the same students during two four-station OSCEs. Students were in year 2 of their pre-clinical program. Repeated-measures ANOVA results showed that OSCE scores awarded by the rater groups differed significantly: (F(matched_station_1) [2,91]=6.97, p=.002), (F(matched_station_2) [3,90]=13.95, p=.001), (F(matched_station_3) [3,90]=8.76, p=.001), and ((Fmatched_station_4) [2,91]=30.60, p=.001). A significant time effect between the two OSCEs was calculated for matched stations 1, 2, and 4, indicating improved student performances. Raters with a clinical communication skills background assigned scores that were significantly lower compared to the other rater groups. Analysis of written feedback provided by the clinical sciences raters showed that they were influenced by the students' clinical knowledge of the case and that they did not rely solely on the communication checklist items. This study shows that it is important to consider rater background both in recruitment and training programs for communication skills' assessment.

Skill Assessment for Coupled Biological/Physical Models of Marine Systems.

PubMed

Stow, Craig A; Jolliff, Jason; McGillicuddy, Dennis J; Doney, Scott C; Allen, J Icarus; Friedrichs, Marjorie A M; Rose, Kenneth A; Wallhead, Philip

2009-02-20

Coupled biological/physical models of marine systems serve many purposes including the synthesis of information, hypothesis generation, and as a tool for numerical experimentation. However, marine system models are increasingly used for prediction to support high-stakes decision-making. In such applications it is imperative that a rigorous model skill assessment is conducted so that the model's capabilities are tested and understood. Herein, we review several metrics and approaches useful to evaluate model skill. The definition of skill and the determination of the skill level necessary for a given application is context specific and no single metric is likely to reveal all aspects of model skill. Thus, we recommend the use of several metrics, in concert, to provide a more thorough appraisal. The routine application and presentation of rigorous skill assessment metrics will also serve the broader interests of the modeling community, ultimately resulting in improved forecasting abilities as well as helping us recognize our limitations.

Assessing Arthroscopic Skills Using Wireless Elbow-Worn Motion Sensors.

PubMed

Kirby, Georgina S J; Guyver, Paul; Strickland, Louise; Alvand, Abtin; Yang, Guang-Zhong; Hargrove, Caroline; Lo, Benny P L; Rees, Jonathan L

2015-07-01

Assessment of surgical skill is a critical component of surgical training. Approaches to assessment remain predominantly subjective, although more objective measures such as Global Rating Scales are in use. This study aimed to validate the use of elbow-worn, wireless, miniaturized motion sensors to assess the technical skill of trainees performing arthroscopic procedures in a simulated environment. Thirty participants were divided into three groups on the basis of their surgical experience: novices (n = 15), intermediates (n = 10), and experts (n = 5). All participants performed three standardized tasks on an arthroscopic virtual reality simulator while wearing wireless wrist and elbow motion sensors. Video output was recorded and a validated Global Rating Scale was used to assess performance; dexterity metrics were recorded from the simulator. Finally, live motion data were recorded via Bluetooth from the wireless wrist and elbow motion sensors and custom algorithms produced an arthroscopic performance score. Construct validity was demonstrated for all tasks, with Global Rating Scale scores and virtual reality output metrics showing significant differences between novices, intermediates, and experts (p < 0.001). The correlation of the virtual reality path length to the number of hand movements calculated from the wireless sensors was very high (p < 0.001). A comparison of the arthroscopic performance score levels with virtual reality output metrics also showed highly significant differences (p < 0.01). Comparisons of the arthroscopic performance score levels with the Global Rating Scale scores showed strong and highly significant correlations (p < 0.001) for both sensor locations, but those of the elbow-worn sensors were stronger and more significant (p < 0.001) than those of the wrist-worn sensors. A new wireless assessment of surgical performance system for objective assessment of surgical skills has proven valid for assessing arthroscopic skills. The elbow

Structured assessment of microsurgery skills in the clinical setting.

PubMed

Chan, WoanYi; Niranjan, Niri; Ramakrishnan, Venkat

2010-08-01

Microsurgery is an essential component in plastic surgery training. Competence has become an important issue in current surgical practice and training. The complexity of microsurgery requires detailed assessment and feedback on skills components. This article proposes a method of Structured Assessment of Microsurgery Skills (SAMS) in a clinical setting. Three types of assessment (i.e., modified Global Rating Score, errors list and summative rating) were incorporated to develop the SAMS method. Clinical anastomoses were recorded on videos using a digital microscope system and were rated by three consultants independently and in a blinded fashion. Fifteen clinical cases of microvascular anastomoses performed by trainees and a consultant microsurgeon were assessed using SAMS. The consultant had consistently the highest scores. Construct validity was also demonstrated by improvement of SAMS scores of microsurgery trainees. The overall inter-rater reliability was strong (alpha=0.78). The SAMS method provides both formative and summative assessment of microsurgery skills. It is demonstrated to be a valid, reliable and feasible assessment tool of operating room performance to provide systematic and comprehensive feedback as part of the learning cycle. Copyright 2009 British Association of Plastic, Reconstructive and Aesthetic Surgeons. Published by Elsevier Ltd. All rights reserved.

Training self-assessment and task-selection skills to foster self-regulated learning: Do trained skills transfer across domains?

PubMed

Raaijmakers, Steven F; Baars, Martine; Paas, Fred; van Merriënboer, Jeroen J G; van Gog, Tamara

2018-01-01

Students' ability to accurately self-assess their performance and select a suitable subsequent learning task in response is imperative for effective self-regulated learning. Video modeling examples have proven effective for training self-assessment and task-selection skills, and-importantly-such training fostered self-regulated learning outcomes. It is unclear, however, whether trained skills would transfer across domains. We investigated whether skills acquired from training with either a specific, algorithmic task-selection rule or a more general heuristic task-selection rule in biology would transfer to self-regulated learning in math. A manipulation check performed after the training confirmed that both algorithmic and heuristic training improved task-selection skills on the biology problems compared with the control condition. However, we found no evidence that students subsequently applied the acquired skills during self-regulated learning in math. Future research should investigate how to support transfer of task-selection skills across domains.

Technical skills assessment toolbox: a review using the unitary framework of validity.

PubMed

Ghaderi, Iman; Manji, Farouq; Park, Yoon Soo; Juul, Dorthea; Ott, Michael; Harris, Ilene; Farrell, Timothy M

2015-02-01

The purpose of this study was to create a technical skills assessment toolbox for 35 basic and advanced skills/procedures that comprise the American College of Surgeons (ACS)/Association of Program Directors in Surgery (APDS) surgical skills curriculum and to provide a critical appraisal of the included tools, using contemporary framework of validity. Competency-based training has become the predominant model in surgical education and assessment of performance is an essential component. Assessment methods must produce valid results to accurately determine the level of competency. A search was performed, using PubMed and Google Scholar, to identify tools that have been developed for assessment of the targeted technical skills. A total of 23 assessment tools for the 35 ACS/APDS skills modules were identified. Some tools, such as Operative Performance Rating System (OSATS) and Objective Structured Assessment of Technical Skill (OPRS), have been tested for more than 1 procedure. Therefore, 30 modules had at least 1 assessment tool, with some common surgical procedures being addressed by several tools. Five modules had none. Only 3 studies used Messick's framework to design their validity studies. The remaining studies used an outdated framework on the basis of "types of validity." When analyzed using the contemporary framework, few of these studies demonstrated validity for content, internal structure, and relationship to other variables. This study provides an assessment toolbox for common surgical skills/procedures. Our review shows that few authors have used the contemporary unitary concept of validity for development of their assessment tools. As we progress toward competency-based training, future studies should provide evidence for various sources of validity using the contemporary framework.

Educational Assessment Knowledge and Skills for Teachers

ERIC Educational Resources Information Center

Brookhart, Susan M.

2011-01-01

The 1990 Standards for Teacher Competence in Educational Assessment of Students (AFT, NCME, & NEA, 1990) made a documentable contribution to the field. However, the Standards have become a bit dated, most notably in two ways: (1) the Standards do not consider current conceptions of formative assessment knowledge and skills, and (2) the Standards…

Key Skills Portfolio Assessment: Good Practice Guide

ERIC Educational Resources Information Center

Rolfe, Martin; Wilson, Lorraine

2007-01-01

This guide offers advice and guidance to key skills managers, coordinators and other practitioners who are responsible for ensuring that the standards of portfolio assessment in their center are consistent across the center and with national standards. It describes and offers advice and guidance on the stages of assessment and internal moderation…

Assessment Training Effects on Student Assessment Skills and Task Performance in a Technology-Facilitated Peer Assessment

ERIC Educational Resources Information Center

Liu, Xiongyi; Li, Lan

2014-01-01

This study examines the impact of an assessment training module on student assessment skills and task performance in a technology-facilitated peer assessment. Seventy-eight undergraduate students participated in the study. The participants completed an assessment training exercise, prior to engaging in peer-assessment activities. During the…

Interpersonal Skills in MBA Admissions: How Are They Conceptualized and Assessed?

ERIC Educational Resources Information Center

Beenen, Gerard; Pichler, Shaun; Davoudpour, Shahin

2018-01-01

Employers and students concur that soft skills or interpersonal skills are critical to managerial success, yet we know little about how MBA program admissions professionals conceptualize and assess these skills in the context of global management education. Such practices have key implications for interpersonal skills curriculum and training in…

Crowdsourcing Assessment of Surgeon Dissection of Renal Artery and Vein During Robotic Partial Nephrectomy: A Novel Approach for Quantitative Assessment of Surgical Performance.

PubMed

Powers, Mary K; Boonjindasup, Aaron; Pinsky, Michael; Dorsey, Philip; Maddox, Michael; Su, Li-Ming; Gettman, Matthew; Sundaram, Chandru P; Castle, Erik P; Lee, Jason Y; Lee, Benjamin R

2016-04-01

We sought to describe a methodology of crowdsourcing for obtaining quantitative performance ratings of surgeons performing renal artery and vein dissection of robotic partial nephrectomy (RPN). We sought to compare assessment of technical performance obtained from the crowdsourcers with that of surgical content experts (CE). Our hypothesis is that the crowd can score performances of renal hilar dissection comparably to surgical CE using the Global Evaluative Assessment of Robotic Skills (GEARS). A group of resident and attending robotic surgeons submitted a total of 14 video clips of RPN during hilar dissection. These videos were rated by both crowd and CE for technical skills performance using GEARS. A minimum of 3 CE and 30 Amazon Mechanical Turk crowdworkers evaluated each video with the GEARS scale. Within 13 days, we received ratings of all videos from all CE, and within 11.5 hours, we received 548 GEARS ratings from crowdworkers. Even though CE were exposed to a training module, internal consistency across videos of CE GEARS ratings remained low (ICC = 0.38). Despite this, we found that crowdworker GEARS ratings of videos were highly correlated with CE ratings at both the video level (R = 0.82, p < 0.001) and surgeon level (R = 0.84, p < 0.001). Similarly, crowdworker ratings of the renal artery dissection were highly correlated with expert assessments (R = 0.83, p < 0.001) for the unique surgery-specific assessment question. We conclude that crowdsourced assessment of qualitative performance ratings may be an alternative and/or adjunct to surgical experts' ratings and would provide a rapid scalable solution to triage technical skills.

Energy-Based Metrics for Arthroscopic Skills Assessment.

PubMed

Poursartip, Behnaz; LeBel, Marie-Eve; McCracken, Laura C; Escoto, Abelardo; Patel, Rajni V; Naish, Michael D; Trejos, Ana Luisa

2017-08-05

Minimally invasive skills assessment methods are essential in developing efficient surgical simulators and implementing consistent skills evaluation. Although numerous methods have been investigated in the literature, there is still a need to further improve the accuracy of surgical skills assessment. Energy expenditure can be an indication of motor skills proficiency. The goals of this study are to develop objective metrics based on energy expenditure, normalize these metrics, and investigate classifying trainees using these metrics. To this end, different forms of energy consisting of mechanical energy and work were considered and their values were divided by the related value of an ideal performance to develop normalized metrics. These metrics were used as inputs for various machine learning algorithms including support vector machines (SVM) and neural networks (NNs) for classification. The accuracy of the combination of the normalized energy-based metrics with these classifiers was evaluated through a leave-one-subject-out cross-validation. The proposed method was validated using 26 subjects at two experience levels (novices and experts) in three arthroscopic tasks. The results showed that there are statistically significant differences between novices and experts for almost all of the normalized energy-based metrics. The accuracy of classification using SVM and NN methods was between 70% and 95% for the various tasks. The results show that the normalized energy-based metrics and their combination with SVM and NN classifiers are capable of providing accurate classification of trainees. The assessment method proposed in this study can enhance surgical training by providing appropriate feedback to trainees about their level of expertise and can be used in the evaluation of proficiency.

The objective assessment of experts' and novices' suturing skills using an image analysis program.

PubMed

Frischknecht, Adam C; Kasten, Steven J; Hamstra, Stanley J; Perkins, Noel C; Gillespie, R Brent; Armstrong, Thomas J; Minter, Rebecca M

2013-02-01

To objectively assess suturing performance using an image analysis program and to provide validity evidence for this assessment method by comparing experts' and novices' performance. In 2009, the authors used an image analysis program to extract objective variables from digital images of suturing end products obtained during a previous study involving third-year medical students (novices) and surgical faculty and residents (experts). Variables included number of stitches, stitch length, total bite size, travel, stitch orientation, total bite-size-to-travel ratio, and symmetry across the incision ratio. The authors compared all variables between groups to detect significant differences and two variables (total bite-size-to-travel ratio and symmetry across the incision ratio) to ideal values. Five experts and 15 novices participated. Experts' and novices' performances differed significantly (P < .05) with large effect sizes attributable to experience (Cohen d > 0.8) for total bite size (P = .009, d = 1.5), travel (P = .045, d = 1.1), total bite-size-to-travel ratio (P < .0001, d = 2.6), stitch orientation (P = .014,d = 1.4), and symmetry across the incision ratio (P = .022, d = 1.3). The authors found that a simple computer algorithm can extract variables from digital images of a running suture and rapidly provide quantitative summative assessment feedback. The significant differences found between groups confirm that this system can discriminate between skill levels. This image analysis program represents a viable training tool for objectively assessing trainees' suturing, a foundational skill for many medical specialties.

The Differential Effects of Task Complexity on Domain-Specific and Peer Assessment Skills

ERIC Educational Resources Information Center

van Zundert, Marjo J.; Sluijsmans, Dominique M. A.; Konings, Karen D.; van Merrienboer, Jeroen J. G.

2012-01-01

In this study the relationship between domain-specific skills and peer assessment skills as a function of task complexity is investigated. We hypothesised that peer assessment skills were superposed on domain-specific skills and will therefore suffer more when higher cognitive load is induced by increased task complexity. In a mixed factorial…

Training and Assessment of Hysteroscopic Skills: A Systematic Review.

PubMed

Savran, Mona Meral; Sørensen, Stine Maya Dreier; Konge, Lars; Tolsgaard, Martin G; Bjerrum, Flemming

2016-01-01

The aim of this systematic review was to identify studies on hysteroscopic training and assessment. PubMed, Excerpta Medica, the Cochrane Library, and Web of Science were searched in January 2015. Manual screening of references and citation tracking were also performed. Studies on hysteroscopic educational interventions were selected without restrictions on study design, populations, language, or publication year. A qualitative data synthesis including the setting, study participants, training model, training characteristics, hysteroscopic skills, assessment parameters, and study outcomes was performed by 2 authors working independently. Effect sizes were calculated when possible. Overall, 2 raters independently evaluated sources of validity evidence supporting the outcomes of the hysteroscopy assessment tools. A total of 25 studies on hysteroscopy training were identified, of which 23 were performed in simulated settings. Overall, 10 studies used virtual-reality simulators and reported effect sizes for technical skills ranging from 0.31 to 2.65; 12 used inanimate models and reported effect sizes for technical skills ranging from 0.35 to 3.19. One study involved live animal models; 2 studies were performed in clinical settings. The validity evidence supporting the assessment tools used was low. Consensus between the 2 raters on the reported validity evidence was high (94%). This systematic review demonstrated large variations in the effect of different tools for hysteroscopy training. The validity evidence supporting the assessment of hysteroscopic skills was limited. Copyright © 2016 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Training self‐assessment and task‐selection skills to foster self‐regulated learning: Do trained skills transfer across domains?

PubMed Central

Baars, Martine; Paas, Fred; van Merriënboer, Jeroen J. G.; van Gog, Tamara

2018-01-01

Summary Students' ability to accurately self‐assess their performance and select a suitable subsequent learning task in response is imperative for effective self‐regulated learning. Video modeling examples have proven effective for training self‐assessment and task‐selection skills, and—importantly—such training fostered self‐regulated learning outcomes. It is unclear, however, whether trained skills would transfer across domains. We investigated whether skills acquired from training with either a specific, algorithmic task‐selection rule or a more general heuristic task‐selection rule in biology would transfer to self‐regulated learning in math. A manipulation check performed after the training confirmed that both algorithmic and heuristic training improved task‐selection skills on the biology problems compared with the control condition. However, we found no evidence that students subsequently applied the acquired skills during self‐regulated learning in math. Future research should investigate how to support transfer of task‐selection skills across domains. PMID:29610547

Exploring Predictability of Instructor Ratings Using a Quantitative Tool for Evaluating Soft Skills among MBA Students

ERIC Educational Resources Information Center

Brill, Robert T.; Gilfoil, David M.; Doll, Kristen

2014-01-01

Academic researchers have often touted the growing importance of "soft skills" for modern day business leaders, especially leadership and communication skills. Despite this growing interest and attention, relatively little work has been done to develop and validate tools to assess soft skills. Forty graduate students from nine MBA…

Teaching and Assessing Clinical Reasoning Skills.

PubMed

Modi, Jyoti Nath; Anshu; Gupta, Piyush; Singh, Tejinder

2015-09-01

Clinical reasoning is a core competency expected to be acquired by all clinicians. It is the ability to integrate and apply different types of knowledge, weigh evidence critically and reflect upon the process used to arrive at a diagnosis. Problems with clinical reasoning often occur because of inadequate knowledge, flaws in data gathering and improper approach to information processing. Some of the educational strategies which can be used to encourage acquisition of clinical reasoning skills are: exposure to a wide variety of clinical cases, activation of previous knowledge, development of illness scripts, sharing expert strategies to arrive at a diagnosis, forcing students to prioritize differential diagnoses; and encouraging reflection, metacognition, deliberate practice and availability of formative feedback. Assessment of clinical reasoning abilities should be done throughout the training course in diverse settings. Use of scenario based multiple choice questions, key feature test and script concordance test are some ways of theoretically assessing clinical reasoning ability. In the clinical setting, these skills can be tested in most forms of workplace based assessment. We recommend that clinical reasoning must be taught at all levels of medical training as it improves clinician performance and reduces cognitive errors.

Nurses' Evaluation of Their Use and Mastery in Health Assessment Skills: Selected Iran's Hospitals

PubMed Central

Adib-Hajbaghery, Mohsen; Safa, Azade

2013-01-01

Background: Health assessment skills are of the most important skills which nurses require. The more precise assessment, the better results would be obtained and the quality of patient care would be improved. However, in Iran, few studies have investigated nurses’ assessment skills. Objectives: This study was aimed to assessnurses' evaluation of the learned skills of health assessment and their use. Materials and Methods: This cross-sectional study was conducted on 200 nurses in Isfahan province hospitals. Data was collected by a questionnaire including demographic data and 120 health assessment skills. Nurses scored their frequency of using and proficiency in skills. Statistical analysis was conducted by ANOVA, Tukey test and independent sample T-tests. Results: The highest level of using and proficiency in skills was related to taking history. Nurses received 87.25% of score in this field. The lowest level of application was in assessment of the urogenital system so that nurses received 16.37% of score in this area. Also the lowest proficiency was in assessment of the nervous system and nurses received 34.58% of score in this area. Conclusions: The level of nurses' proficiency in the health assessment skills was not satisfactory. Modifying the curriculum and cooperating of nurse managers and nursing schools can help to improve the situation. PMID:25414875

Nurses' Evaluation of Their Use and Mastery in Health Assessment Skills: Selected Iran's Hospitals.

PubMed

Adib-Hajbaghery, Mohsen; Safa, Azade

2013-09-01

Health assessment skills are of the most important skills which nurses require. The more precise assessment, the better results would be obtained and the quality of patient care would be improved. However, in Iran, few studies have investigated nurses' assessment skills. This study was aimed to assessnurses' evaluation of the learned skills of health assessment and their use. This cross-sectional study was conducted on 200 nurses in Isfahan province hospitals. Data was collected by a questionnaire including demographic data and 120 health assessment skills. Nurses scored their frequency of using and proficiency in skills. Statistical analysis was conducted by ANOVA, Tukey test and independent sample T-tests. The highest level of using and proficiency in skills was related to taking history. Nurses received 87.25% of score in this field. The lowest level of application was in assessment of the urogenital system so that nurses received 16.37% of score in this area. Also the lowest proficiency was in assessment of the nervous system and nurses received 34.58% of score in this area. The level of nurses' proficiency in the health assessment skills was not satisfactory. Modifying the curriculum and cooperating of nurse managers and nursing schools can help to improve the situation.

Objective Assessment of Bimanual Laparoscopic Surgical Skills via Functional Near Infrared Spectroscopy (fNIRS)

NASA Astrophysics Data System (ADS)

Nemani, Arun

Surgical simulators are effective methods for training and assessing surgical technical skills, particularly those that are bimanual. These simulators are now ubiquitous in surgical training and assessment programs for residents. Simulators are used in programs such as the Fundamentals of Laparoscopic Surgery (FLS) and Fundamentals of Endoscopic Surgery (FES), which are pre-requisites for Board certification in general surgery. Although these surgical simulators have been validated for clinical use, they have significant limitations, such as subjectivity in assessment metrics, poor correlation of transfer from simulation to clinically relevant environments, poor correlation of task performance scores to learning motor skill levels, and ultimately inconsistent reliability of these assessment methods as an indicator of positive patient outcomes. These limitations present an opportunity for more objective and analytical approaches to assess surgical motor skills. To address these surgical skill assessment limitations, we present functional near-infrared spectroscopic (fNIRS), a non-invasive brain imaging method, to objectively differentiate and classify subjects with varying degrees of laparoscopic surgical motor skill levels based on measurements of functional activation changes. In this work, we show that fNIRS based metrics can objectively differentiate and classify surgical motor skill levels with significantly more accuracy than established metrics. Using classification approaches such as multivariate linear discriminant analysis, we show evidence that fNIRS metrics reduce the misclassification error, defined as the probability that a trained subject is misclassified as an untrained subject and vice versa, from 53-61% to 4.2-4.4% compared to conventional metrics for surgical skill assessment. This evidence also translates to surgical skill transfer metrics, where such metrics assess surgical motor skill transfer from simulation to clinically relevant environments

Using the Academic Skills Inventory to Assess the Biology Major

ERIC Educational Resources Information Center

Seifert, Kyle; Hurney, Carol A.; Wigtil, Clifton J.; Sundre, Donna L.

2009-01-01

The Academic Skills Inventory (Kruger and Zechmeister, 2001) was developed at Loyola University of Chicago and originally designed for use with psychology majors. It was later extended for use in a variety of academic programs. The Academic Skills Inventory (ASI) assesses student self-reports of behaviors in 10 skill areas: (1) written and oral…

Assessing computer skills in Tanzanian medical students: an elective experience.

PubMed

Samuel, Miriam; Coombes, John C; Miranda, J Jaime; Melvin, Rob; Young, Eoin J W; Azarmina, Pejman

2004-08-12

One estimate suggests that by 2010 more than 30% of a physician's time will be spent using information technology tools. The aim of this study is to assess the information and communication technologies (ICT) skills of medical students in Tanzania. We also report a pilot intervention of peer mentoring training in ICT by medical students from the UK tutoring students in Tanzania. Cross sectional study and pilot intervention study. Fourth year medical students (n = 92) attending Muhimbili University College of Health Sciences, Dar es Salaam, Tanzania. Self-reported assessment of competence on ICT-related topics and ability to perform specific ICT tasks. Further information related to frequency of computer use (hours per week), years of computer use, reasons for use and access to computers. Skills at specific tasks were reassessed for 12 students following 4 to 6 hours of peer mentoring training. The highest levels of competence in generic ICT areas were for email, Internet and file management. For other skills such as word processing most respondents reported low levels of competence. The abilities to perform specific ICT skills were low - less than 60% of the participants were able to perform the core specific skills assessed. A period of approximately 5 hours of peer mentoring training produced an approximate doubling of competence scores for these skills. Our study has found a low level of ability to use ICT facilities among medical students in a leading university in sub-Saharan Africa. A pilot scheme utilising UK elective students to tutor basic skills showed potential. Attention is required to develop interventions that can improve ICT skills, as well as computer access, in order to bridge the digital divide.

Teaching and Assessing Communication Skills in Medical Undergraduate Training.

PubMed

Modi, Jyoti Nath; Anshu, -; Chhatwal, Jugesh; Gupta, Piyush; Singh, Tejinder

2016-06-08

Good communication skills are essential for an optimal doctor-patient relationship, and also contribute to improved health outcomes. Although the need for training in communication skills is stated as a requirement in the 1997 Graduate Medical Education Regulations of the Medical Council of India, formal training in these skills has been fragmentary and non-uniform in most Indian curricula. The Vision 2015 document of the Medical Council of India reaffirms the need to include training in communication skills in the MBBS curriculum. Training in communication skills needs approaches which are different from that of teaching other clinical subjects. It is also a challenge to ensure that students not only imbibe the nuances of communication and interpersonal skills, but adhere to them throughout their careers. This article addresses the possible ways of standardizing teaching and assessment of communication skills and integrating them into the existing curriculum.

Attention to gender in communication skills assessment instruments in medical education: a review.

PubMed

Dielissen, Patrick; Bottema, Ben; Verdonk, Petra; Lagro-Janssen, Toine

2011-03-01

Gender is increasingly regarded as an important factor in doctor-patient communication education. This review aims to assess if and how gender is addressed by current assessment instruments for communication skills in medical education. In 2009 at Radboud University Nijmegen Medical Centre, an online search was conducted in the bibliographic databases PubMed, PsycINFO and ERIC for references about communication skills assessment instruments designed to be completed by trained faculty staff and used in medical education. The search strategy used the following search terms: 'consultation skills'; 'doctor-patient communication'; 'physician-patient relations'; 'medical education'; 'instruments'; 'measurement', and 'assessment'. Papers published between January 1999 and June 2009 were included. The assessment instruments identified were analysed for gender-specific content. The search yielded 21 communication skills assessment instruments. Only two of the 17 checklists obtained explicitly considered gender as a communication-related issue. Only six of 21 manuals considered gender in any way and none gave specific details to explain which aspects of communication behaviour should be assessed with regard to gender. Very few communication assessment instruments in medical education focus on gender. Nevertheless, interest exists in using gender in communication skills assessment. The criteria for and purpose of assessing gender in communication skills in medical education are yet to be clarified. © Blackwell Publishing Ltd 2011.

Reliability and Validity of the Research Methods Skills Assessment

ERIC Educational Resources Information Center

Smith, Tamarah; Smith, Samantha

2018-01-01

The Research Methods Skills Assessment (RMSA) was created to measure psychology majors' statistics knowledge and skills. The American Psychological Association's Guidelines for the Undergraduate Major in Psychology (APA, 2007, 2013) served as a framework for development. Results from a Rasch analysis with data from n = 330 undergraduates showed…

Validation of a global assessment of arthroscopic skills in a cadaveric knee model.

PubMed

Slade Shantz, Jesse A; Leiter, Jeff R; Collins, John B; MacDonald, Peter B

2013-01-01

The purpose of this study was to determine whether a global assessment of arthroscopic skills was valid for blinded assessment of cadaveric diagnostic knee arthroscopy. A global skills assessment for arthroscopy was created using a published theory of the development of expertise. Faculty surgeons, fellows, and residents were consented and enrolled in this institutional review board-approved validation study. All participants were oriented to the equipment and procedures for diagnostic arthroscopy of the knee. After reviewing the anatomic structures to be visualized, participants were allowed 10 minutes to complete a diagnostic arthroscopy of the knee. The hands and arthroscopic view were recorded during this attempt. Resident participants completed a second filmed diagnostic arthroscopy 1 week after the initial attempt. Five blinded reviewers watched the synchronized videos and assessed arthroscopic skills with a procedure-specific checklist and the newly developed global skills assessment. The agreement between reviewers was determined by intraclass correlation coefficient. Internal consistency was determined with Cronbach's α. Test-retest reliability was measured by correlating repeated arthroscopies by residents. The ability of the global assessment to discriminate skill levels was determined with between-group Mann-Whitney U tests. The agreement between global assessment scores was strong (I.C.C. = 0.80, 95% C.I. 0.68-0.92). The internal consistency of evaluations was excellent (Cronbach's α = 0.97), and the test-retest reliability was strong (r = 0.52). The global assessment score was shown to be able to discriminate between skill levels by an analysis of variance indicating the difference in means among the various levels of training (P < .0001). The Objective Assessment of Arthroscopic Skills is a useful adjunct to arthroscopic educators and learners and could be used for in-training evaluations. The Objective Assessment of Arthroscopic Skills is an

Assessing Thinking Skills in Astro 101: Do We Make an Impact?

NASA Astrophysics Data System (ADS)

Bruning, D.

2005-12-01

Most instructors agree that a major goal of "Astronomy 101" is to develop thinking skills in our students (Partridge and Greenstein, AER 2, 46, 2003). Much educational research in astronomy has initially concentrated on "best practices" for improving student learning (development of "think-pair-share", lecture tutorials, peer tutoring, etc.). Little has been done to date to assess our efforts to improve student thinking skills and students' desire to think more deeply about the cognitively rich ideas offered in the typical astronomy class. This study surveys several astronomy and physics courses to determine whether general analytical thinking skills increase because of the science course and whether students' attitudes toward cognition improve. Cacioppo, Petty and Kao's "Need for Cognition" scale is used for the latter assessment (J. Personality Assessment 48, 306, 1984). A shortened version of Whimbey and Lochhead's ASI skills instrument is used to assess analytical skills ("Problem Solving and Comprehension," 1986). Preliminary results suggest that students need for cognition does not change in general, although there may be a correlation between increasing need for cognition and improvement in grades through the semester. There is a suggestion that need for cognition is slightly predictive of course performance, but a greater correlation exists between the post-course survey and grades. Gains in general analytical skills have been seen in initial surveys, but correlations with course performance appear elusive.

Formative assessment of GP trainees' clinical skills.

PubMed

Wiener-Ogilvie, Sharon; Begg, Drummond

2012-03-01

Clinical skill assessment (CSA) has been an integral part of the Royal College of General Practitioners' membership examination (MRCGP) since 2008. It is an expensive, high-stakes examination with first time pass rates ranging from 76.4 to 81.3. In this paper we describe the South East Scotland Deanery, NHS Education Scotland, pilot of a formative clinical skills assessment (fCSA) using the principles of formative assessment and OSCE. The purpose of the study was to assess the acceptability of the fCSA and to examine whether trainees, identified during the fCSA as 'at risk of failing the MRCGP CSA exam', are more likely to fail the MRCGP CSA exam later on in the year. Trainees were assessed in four clinical skills stations under exam conditions. After each station they were given verbal feedback and subsequently both trainee and their trainer received written feedback. We assessed the value of the exercise through written feedback from trainees and trainers. Each trainee's performance in fCSA was triangulated with trainer assessment to identify 'flagged trainees'. We compared flagged and non-flagged trainees' performance in MRCGP CSA. Both trainees and trainers highly rated the fCSA. Overall 97% of non-flagged trainees have passed the RCGP CSA exam by May of that year in comparison to 80% of flagged trainees who have passed the RCGP CSA (P = 0.005). Trainers and trainees rated the fCSA as excellent and useful. We were able to demonstrate that the fCSA can be used to identify those trainees likely to fail the RCGP CSA. Contrary to reservations about the potential to demoralise trainees, the fCSA was viewed as a useful and a positive experience by both trainees and trainers. In addition, we suggest that feedback from fCSA was useful in triggering appropriate educational interventions. Early intervention with trainees who are predicted to fail the CSA has the potential to reduce deaneries overall fail rate. Preventing one trainee failure could save over £30 000.

Development of assessment instruments to measure critical thinking skills

NASA Astrophysics Data System (ADS)

Sumarni, W.; Supardi, K. I.; Widiarti, N.

2018-04-01

Assessment instruments that is commonly used in the school generally have not been orientated on critical thinking skills. The purpose of this research is to develop assessment instruments to measure critical thinking skills, to test validity, reliability, and practicality. This type of research is Research and Development. There are two stages on the preface step, which are field study and literacy study. On the development steps, there some parts, which are 1) instrument construction, 2) expert validity, 3) limited scale tryout and 4) narrow scale try-out. The developed assessment instrument are analysis essay and problem solving. Instruments were declared valid, reliable and practical.

The development and validation of a golf swing and putt skill assessment for children.

PubMed

Barnett, Lisa M; Hardy, Louise L; Brian, Ali S; Robertson, Sam

2015-03-01

The aim was to describe development of a process-oriented instrument designed to assess the golf swing and putt stroke, and to assess the instrument's discriminative validity in terms of age and reliability (intra-rater and re-test). A Delphi consultation (with golf industry professionals and researchers in movement skill assessment) was used to develop an assessment for each skill based on existing skill assessment protocols. Each skill had six components to be marked as present/absent. Individual scores were based on the number of performance components successfully demonstrated over two trials for each skill (potential score range 0 to 24). Children (n = 43) aged 6-10 years (M = 7.8 years, SD = 1.3) were assessed in both skills live in the field by one rater at Time 1(T1). A subset of children (n = 28) had consent for assessments to be videoed. Six weeks later 19 children were reassessed, five days apart (T2, T3). An ANOVA assessed discriminative validity i.e. whether skill competence at T1 differed by age (6 years, 7/8 years and 9/10 years). Intraclass correlations (ICC) assessed intra-rater reliability between the live and video assessment at T1 and test-retest reliability (between T2 and T3). Paired t-tests assessed any systematic differences between live and video assessments (T1) and between T2 and T3. Older children were more skilled (F (2, 40) = 11.18, p < 0.001). The live assessment reflected the video assessment (ICC = 0.79, 95% CI 0.59, 0.90) and scores did not differ between live and video assessments. Test retest reliability was acceptable (ICC = 0.60, 95% CI 0.23, 0.82), although the mean score was slightly higher at retest. This instrument could be used reliably by golf coaches and physical education teachers as part of systematic early player assessment and feedback. Key pointsGolf is becoming an increasingly popular sport among young children, however there is no standard protocol available to assess and identify skill deficits, mastery level

Integrating quantitative thinking into an introductory biology course improves students' mathematical reasoning in biological contexts.

PubMed

Hester, Susan; Buxner, Sanlyn; Elfring, Lisa; Nagy, Lisa

2014-01-01

Recent calls for improving undergraduate biology education have emphasized the importance of students learning to apply quantitative skills to biological problems. Motivated by students' apparent inability to transfer their existing quantitative skills to biological contexts, we designed and taught an introductory molecular and cell biology course in which we integrated application of prerequisite mathematical skills with biology content and reasoning throughout all aspects of the course. In this paper, we describe the principles of our course design and present illustrative examples of course materials integrating mathematics and biology. We also designed an outcome assessment made up of items testing students' understanding of biology concepts and their ability to apply mathematical skills in biological contexts and administered it as a pre/postcourse test to students in the experimental section and other sections of the same course. Precourse results confirmed students' inability to spontaneously transfer their prerequisite mathematics skills to biological problems. Pre/postcourse outcome assessment comparisons showed that, compared with students in other sections, students in the experimental section made greater gains on integrated math/biology items. They also made comparable gains on biology items, indicating that integrating quantitative skills into an introductory biology course does not have a deleterious effect on students' biology learning.

Integrating Quantitative Thinking into an Introductory Biology Course Improves Students’ Mathematical Reasoning in Biological Contexts

PubMed Central

Hester, Susan; Buxner, Sanlyn; Elfring, Lisa; Nagy, Lisa

2014-01-01

Recent calls for improving undergraduate biology education have emphasized the importance of students learning to apply quantitative skills to biological problems. Motivated by students’ apparent inability to transfer their existing quantitative skills to biological contexts, we designed and taught an introductory molecular and cell biology course in which we integrated application of prerequisite mathematical skills with biology content and reasoning throughout all aspects of the course. In this paper, we describe the principles of our course design and present illustrative examples of course materials integrating mathematics and biology. We also designed an outcome assessment made up of items testing students’ understanding of biology concepts and their ability to apply mathematical skills in biological contexts and administered it as a pre/postcourse test to students in the experimental section and other sections of the same course. Precourse results confirmed students’ inability to spontaneously transfer their prerequisite mathematics skills to biological problems. Pre/postcourse outcome assessment comparisons showed that, compared with students in other sections, students in the experimental section made greater gains on integrated math/biology items. They also made comparable gains on biology items, indicating that integrating quantitative skills into an introductory biology course does not have a deleterious effect on students’ biology learning. PMID:24591504

Are general surgeons able to accurately self-assess their level of technical skills?

PubMed

Rizan, C; Ansell, J; Tilston, T W; Warren, N; Torkington, J

2015-11-01

Self-assessment is a way of improving technical capabilities without the need for trainer feedback. It can identify areas for improvement and promote professional medical development. The aim of this review was to identify whether self-assessment is an accurate form of technical skills appraisal in general surgery. The PubMed, MEDLINE(®), Embase(™) and Cochrane databases were searched for studies assessing the reliability of self-assessment of technical skills in general surgery. For each study, we recorded the skills assessed and the evaluation methods used. Common endpoints between studies were compared to provide recommendations based on the levels of evidence. Twelve studies met the inclusion criteria from 22,292 initial papers. There was no level 1 evidence published. All papers compared the correlation between self-appraisal versus an expert score but differed in the technical skills assessment and the evaluation tools used. The accuracy of self-assessment improved with increasing experience (level 2 recommendation), age (level 3 recommendation) and the use of video playback (level 3 recommendation). Accuracy was reduced by stressful learning environments (level 2 recommendation), lack of familiarity with assessment tools (level 3 recommendation) and in advanced surgical procedures (level 3 recommendation). Evidence exists to support the reliability of self-assessment of technical skills in general surgery. Several variables have been shown to affect the accuracy of self-assessment of technical skills. Future work should focus on evaluating the reliability of self-assessment during live operating procedures.

African American Preschoolers' Emotion Explanations Can Provide Evidence of Their Pragmatic Skills

ERIC Educational Resources Information Center

Curenton, Stephanie M.

2015-01-01

This study provides qualitative and quantitative evidence of how an emotion explanation task can reflect African American preschoolers' pragmatic skills. We used an emotion explanation task to assess pragmatic skills among 19 children (aged 3-5 years) related to (1) engaging in conversational turn-taking, (2) answering "Wh-" questions,…

The assessment of mindfulness skills: the "what" and the "how".

PubMed

Iani, Luca; Lauriola, Marco; Cafaro, Valentina

2017-10-06

The five facets mindfulness questionnaire-short form (FFMQ-SF) is a new, brief measure for the assessment of mindfulness skills in clinical and nonclinical samples. The construct validity of the FFMQ-SF has not been previously assessed in community samples. The present study investigated the factor structure of the Italian version of the FFMQ-SF. Structured equation modeling was used to test the fit of three alternative models in a sample of highly educated adults (n = 211). A hierarchical model with a single second-order factor loaded by observing, describing, and acting with awareness (i.e. the mindfulness "what" skills) performed slightly better than both a five-factor model with correlated factors and a hierarchical model with a general second-order factor. The FFMQ-SF scores were significantly higher than those reported in both Dutch depressed patients and Australian undergraduate students for all facets (but nonreactivity for the Australian sample). Data support the multifaceted nature of mindfulness skills. Because of its brevity and simplicity of use, the FFMQ-SF is a promising questionnaire in longitudinal and clinical research. This questionnaire can serve as a guideline to help clinicians assess and monitor mindfulness skills acquisition, strengthening, and generalization, and prioritize mindfulness skills that need immediate attention.

Quantitative risk assessment system (QRAS)

NASA Technical Reports Server (NTRS)

Tan, Zhibin (Inventor); Mosleh, Ali (Inventor); Weinstock, Robert M (Inventor); Smidts, Carol S (Inventor); Chang, Yung-Hsien (Inventor); Groen, Francisco J (Inventor); Swaminathan, Sankaran (Inventor)

2001-01-01

A quantitative risk assessment system (QRAS) builds a risk model of a system for which risk of failure is being assessed, then analyzes the risk of the system corresponding to the risk model. The QRAS performs sensitivity analysis of the risk model by altering fundamental components and quantifications built into the risk model, then re-analyzes the risk of the system using the modifications. More particularly, the risk model is built by building a hierarchy, creating a mission timeline, quantifying failure modes, and building/editing event sequence diagrams. Multiplicities, dependencies, and redundancies of the system are included in the risk model. For analysis runs, a fixed baseline is first constructed and stored. This baseline contains the lowest level scenarios, preserved in event tree structure. The analysis runs, at any level of the hierarchy and below, access this baseline for risk quantitative computation as well as ranking of particular risks. A standalone Tool Box capability exists, allowing the user to store application programs within QRAS.

Assessing computer skills in Tanzanian medical students: an elective experience

PubMed Central

Samuel, Miriam; Coombes, John C; Miranda, J Jaime; Melvin, Rob; Young, Eoin JW; Azarmina, Pejman

2004-01-01

Background One estimate suggests that by 2010 more than 30% of a physician's time will be spent using information technology tools. The aim of this study is to assess the information and communication technologies (ICT) skills of medical students in Tanzania. We also report a pilot intervention of peer mentoring training in ICT by medical students from the UK tutoring students in Tanzania. Methods Design: Cross sectional study and pilot intervention study. Participants: Fourth year medical students (n = 92) attending Muhimbili University College of Health Sciences, Dar es Salaam, Tanzania. Main outcome measures: Self-reported assessment of competence on ICT-related topics and ability to perform specific ICT tasks. Further information related to frequency of computer use (hours per week), years of computer use, reasons for use and access to computers. Skills at specific tasks were reassessed for 12 students following 4 to 6 hours of peer mentoring training. Results The highest levels of competence in generic ICT areas were for email, Internet and file management. For other skills such as word processing most respondents reported low levels of competence. The abilities to perform specific ICT skills were low – less than 60% of the participants were able to perform the core specific skills assessed. A period of approximately 5 hours of peer mentoring training produced an approximate doubling of competence scores for these skills. Conclusion Our study has found a low level of ability to use ICT facilities among medical students in a leading university in sub-Saharan Africa. A pilot scheme utilising UK elective students to tutor basic skills showed potential. Attention is required to develop interventions that can improve ICT skills, as well as computer access, in order to bridge the digital divide. PMID:15306029

Assessment of and standardization for quantitative nondestructive test

NASA Technical Reports Server (NTRS)

Neuschaefer, R. W.; Beal, J. B.

1972-01-01

Present capabilities and limitations of nondestructive testing (NDT) as applied to aerospace structures during design, development, production, and operational phases are assessed. It will help determine what useful structural quantitative and qualitative data may be provided from raw materials to vehicle refurbishment. This assessment considers metal alloys systems and bonded composites presently applied in active NASA programs or strong contenders for future use. Quantitative and qualitative data has been summarized from recent literature, and in-house information, and presented along with a description of those structures or standards where the information was obtained. Examples, in tabular form, of NDT technique capabilities and limitations have been provided. NDT techniques discussed and assessed were radiography, ultrasonics, penetrants, thermal, acoustic, and electromagnetic. Quantitative data is sparse; therefore, obtaining statistically reliable flaw detection data must be strongly emphasized. The new requirements for reusable space vehicles have resulted in highly efficient design concepts operating in severe environments. This increases the need for quantitative NDT evaluation of selected structural components, the end item structure, and during refurbishment operations.

Counselling on breastfeeding: assessing knowledge and skills.

PubMed Central

Rea, M. F.; Venancio, S. I.; Martines, J. C.; Savage, F.

1999-01-01

Reported are the results of a randomized controlled trial to assess the effectiveness of the WHO/UNICEF 40-hour course "Breastfeeding counselling: a training course". The course was conducted in a maternity hospital which provides care to a low-income population in a metropolitan area in São Paulo, Brazil. Health workers from 60 health units were randomly assigned to be either participants (20) or controls (40), and their breastfeeding knowledge and skills were assessed before and immediately after the course, as well as 3 months later. Immediately after the course the participants' knowledge of breastfeeding had increased significantly compared to controls. Both their clinical and counselling skills also improved significantly. When assessed 3 months later, the scores remained high with only a small decrease. The implementation of the course was also evaluated. The methods used were participatory observation, key interviews and focus group discussion. In the 33 sessions of the course, the average score was 8.43 out of 10. Scores were highest for content and methodology of the theory sessions, and lowest for "use of time", "clinical management of lactation", and "discussion of clinical practice". "Breastfeeding counselling: a training course" therefore effectively increases health workers' knowledge and their clinical and counselling skills for the support of breastfeeding. The course can be conducted adequately using the material and methodology proposed, but could be more satisfactory if the time allocated to exercises and clinical practice sessions were increased. PMID:10427934

Investigating Secondary School Students' Unmediated Peer Assessment Skills

ERIC Educational Resources Information Center

Tsivitanidou, Olia E.; Zacharia, Zacharias C.; Hovardas, Tasos

2011-01-01

The purpose of this study was to investigate secondary school students' unmediated peer assessment skills. Specifically, 36 seventh graders, without receiving any kind of support, were anonymously assigned to reciprocally assess their peers' science web-portfolios. Additionally, students' attitudes towards and intentions about the use of…

Developing Enterprise Skills through Peer-Assessed Pitch Presentations

ERIC Educational Resources Information Center

Faherty, Anna

2015-01-01

Purpose: The purpose of this paper is to investigate the impact of using summative peer assessment to develop enterprise skills within higher education. Design/methodology/approach: An empirical investigation analysing students own perceptions of the peer assessment process to evaluate its impact. Findings: Participating students indicate that…

A contextual approach to social skills assessment in the peer group: who is the best judge?

PubMed

Kwon, Kyongboon; Kim, Elizabeth Moorman; Sheridan, Susan M

2012-09-01

Using a contextual approach to social skills assessment in the peer group, this study examined the criterion-related validity of contextually relevant social skills and the incremental validity of peers and teachers as judges of children's social skills. Study participants included 342 (180 male and 162 female) students and their classroom teachers (N = 22) from rural communities. As expected, contextually relevant social skills were significantly related to a variety of social status indicators (i.e., likability, peer- and teacher-assessed popularity, reciprocated friendships, clique centrality) and positive school functioning (i.e., school liking and academic competence). Peer-assessed social skills, not teacher-assessed social skills, demonstrated consistent incremental validity in predicting various indicators of social status outcomes; peer- and teacher-assessed social skills alike showed incremental validity in predicting positive school functioning. The relation between contextually relevant social skills and study outcomes did not vary by child gender. Findings are discussed in terms of the significance of peers in the assessment of children's social skills in the peer group as well as the usefulness of a contextual approach to social skills assessment.

Using the Entrustable Professional Activities Framework in the Assessment of Procedural Skills.

PubMed

Pugh, Debra; Cavalcanti, Rodrigo B; Halman, Samantha; Ma, Irene W Y; Mylopoulos, Maria; Shanks, David; Stroud, Lynfa

2017-04-01

The entrustable professional activity (EPA) framework has been identified as a useful approach to assessment in competency-based education. To apply an EPA framework for assessment, essential skills necessary for entrustment to occur must first be identified. Using an EPA framework, our study sought to (1) define the essential skills required for entrustment for 7 bedside procedures expected of graduates of Canadian internal medicine (IM) residency programs, and (2) develop rubrics for the assessment of these procedural skills. An initial list of essential skills was defined for each procedural EPA by focus groups of experts at 4 academic centers using the nominal group technique. These lists were subsequently vetted by representatives from all Canadian IM training programs through a web-based survey. Consensus (more than 80% agreement) about inclusion of each item was sought using a modified Delphi exercise. Qualitative survey data were analyzed using a framework approach to inform final assessment rubrics for each procedure. Initial lists of essential skills for procedural EPAs ranged from 10 to 24 items. A total of 111 experts completed the national survey. After 2 iterations, consensus was reached on all items. Following qualitative analysis, final rubrics were created, which included 6 to 10 items per procedure. These EPA-based assessment rubrics represent a national consensus by Canadian IM clinician educators. They provide a practical guide for the assessment of procedural skills in a competency-based education model, and a robust foundation for future research on their implementation and evaluation.

Integrated Environmental Modeling: Quantitative Microbial Risk Assessment

EPA Science Inventory

The presentation discusses the need for microbial assessments and presents a road map associated with quantitative microbial risk assessments, through an integrated environmental modeling approach. A brief introduction and the strengths of the current knowledge are illustrated. W...

Basic Skills Assessment Program. Technical Report.

ERIC Educational Resources Information Center

Los Angeles County Superintendent of Schools, CA.

This is the second technical report on the Basic Skills Assessment Program for the Department of Defense Dependents Schools (DoDDS). The report includes a summary of 1979-80 testing activities organized into five chapters. The first chapter provides an overview of report contents and a brief history of the testing program. The second chapter…

Impact of tailored feedback in assessment of communication skills for medical students.

PubMed

Uhm, Seilin; Lee, Gui H; Jin, Jeong K; Bak, Yong I; Jeoung, Yeon O; Kim, Chan W

2015-01-01

Finding out the effective ways of teaching and assessing communication skills remain a challenging part of medication education. This study aims at exploring the usefulness and effectiveness of having additional feedback using qualitative analysis in assessment of communication skills in undergraduate medical training. We also determined the possibilities of using qualitative analysis in developing tailored strategies for improvement in communication skills training. This study was carried out on medical students (n=87) undergoing their final year clinical performance examination on communication skills using standardized patient by video-recording and transcribing their performances. Video-recordings of 26 students were randomly selected for qualitative analysis, and additional feedback was provided. We assessed the level of acceptance of communication skills scores between the study and nonstudy group and within the study group, before and after receiving feedback based on qualitative analysis. There was a statistically significant increase in the level of acceptance of feedback after delivering additional feedback using qualitative analysis, where the percentage of agreement with feedback increased from 15.4 to 80.8% (p<0.001). Incorporating feedback based on qualitative analysis for communication skills assessment gives essential information for medical students to learn and self-reflect, which could potentially lead to improved communication skills. As evident from our study, feedback becomes more meaningful and effective with additional feedback using qualitative analysis.

Impact of tailored feedback in assessment of communication skills for medical students

PubMed Central

Uhm, Seilin; Lee, Gui H.; Jin, Jeong K.; Bak, Yong I.; Jeoung, Yeon O.; Kim, Chan W.

2015-01-01

Background Finding out the effective ways of teaching and assessing communication skills remain a challenging part of medication education. This study aims at exploring the usefulness and effectiveness of having additional feedback using qualitative analysis in assessment of communication skills in undergraduate medical training. We also determined the possibilities of using qualitative analysis in developing tailored strategies for improvement in communication skills training. Methods This study was carried out on medical students (n=87) undergoing their final year clinical performance examination on communication skills using standardized patient by video-recording and transcribing their performances. Video-recordings of 26 students were randomly selected for qualitative analysis, and additional feedback was provided. We assessed the level of acceptance of communication skills scores between the study and nonstudy group and within the study group, before and after receiving feedback based on qualitative analysis. Results There was a statistically significant increase in the level of acceptance of feedback after delivering additional feedback using qualitative analysis, where the percentage of agreement with feedback increased from 15.4 to 80.8% (p<0.001). Conclusions Incorporating feedback based on qualitative analysis for communication skills assessment gives essential information for medical students to learn and self-reflect, which could potentially lead to improved communication skills. As evident from our study, feedback becomes more meaningful and effective with additional feedback using qualitative analysis. PMID:26154864

Instructor and Dental Student Perceptions of Clinical Communication Skills via Structured Assessments.

PubMed

McKenzie, Carly T

2016-05-01

The aim of this study was to use structured assessments to assess dental students' clinical communication skills exhibited during patient appointments. Fourth-year dental students (n=55) at the University of Alabama at Birmingham evaluated their own interpersonal skills in a clinical setting utilizing the Four Habits Coding Scheme. An instructor also assessed student-patient clinical communication. These assessments were used to identify perceived strengths and weaknesses in students' clinical communication. Both instructor assessments and student self-assessments pinpointed the following clinical communication skills as effective the most often: patient greeting, avoidance of jargon, and non-verbal behavior. There was also relative agreement between instructor assessments and student self-assessments regarding clinical communication skills that were rated as not effective most frequently: ensuring patient comprehension, identification of patient feelings, and exploration of barriers to treatment. These resulted pointed to strengths and weaknesses in the portion of the curriculum designed to prepare students for effective provider-patient communication. These results may suggest a need for the school's current behavioral science curriculum to better address discussion of potential treatment barriers and patient feelings as well as techniques to ensure patient comprehension.

Assessment of DoD Job Skill Enhancement Programs.

ERIC Educational Resources Information Center

Fletcher, J. D.; And Others

In response to Congressional direction, an assessment was undertaken of programs developed by the Department of Defense (DoD) that can be made available to civilian organizations to provide immediate support and assistance to upgrade skills for better civilian employment opportunities. The assessment focuses on interactive courseware programs and…

Assessment of Inquiry Skills in the SAILS Project

ERIC Educational Resources Information Center

Harrison, Chris

2014-01-01

Inquiry provides both the impetus and experience that helps students acquire problem solving and lifelong learning skills. Teachers on the Strategies for Assessment of Inquiry Learning in Science Project (SAILS) strengthened their inquiry pedagogy, through focusing on seeking assessment evidence for formative action. This paper reports on both the…

The Development and Validation of a Golf Swing and Putt Skill Assessment for Children

PubMed Central

Barnett, Lisa M.; Hardy, Louise L.; Brian, Ali S.; Robertson, Sam

2015-01-01

The aim was to describe development of a process-oriented instrument designed to assess the golf swing and putt stroke, and to assess the instrument’s discriminative validity in terms of age and reliability (intra-rater and re-test). A Delphi consultation (with golf industry professionals and researchers in movement skill assessment) was used to develop an assessment for each skill based on existing skill assessment protocols. Each skill had six components to be marked as present/absent. Individual scores were based on the number of performance components successfully demonstrated over two trials for each skill (potential score range 0 to 24). Children (n = 43) aged 6-10 years (M = 7.8 years, SD = 1.3) were assessed in both skills live in the field by one rater at Time 1(T1). A subset of children (n = 28) had consent for assessments to be videoed. Six weeks later 19 children were reassessed, five days apart (T2, T3). An ANOVA assessed discriminative validity i.e. whether skill competence at T1 differed by age (6 years, 7/8 years and 9/10 years). Intraclass correlations (ICC) assessed intra-rater reliability between the live and video assessment at T1 and test-retest reliability (between T2 and T3). Paired t-tests assessed any systematic differences between live and video assessments (T1) and between T2 and T3. Older children were more skilled (F (2, 40) = 11.18, p < 0.001). The live assessment reflected the video assessment (ICC = 0.79, 95% CI 0.59, 0.90) and scores did not differ between live and video assessments. Test retest reliability was acceptable (ICC = 0.60, 95% CI 0.23, 0.82), although the mean score was slightly higher at retest. This instrument could be used reliably by golf coaches and physical education teachers as part of systematic early player assessment and feedback. Key points Golf is becoming an increasingly popular sport among young children, however there is no standard protocol available to assess and identify skill deficits, mastery level

Using the Dynamic Model to Identify Stages of Teacher Skills in Assessment

ERIC Educational Resources Information Center

Christoforidou, Margarita; Xirafidou, Elisavet

2014-01-01

The article presents the results of two cross-sectional studies that investigate teachers' skills in using various techniques of assessment in mathematics by taking into account the four phases of assessment. The five dimensions of the dynamic model are also taken into account in proposing a framework for measuring teacher skills in assessment.…

[Use of physical assessment skills and education needs of advanced practice nurses and nurse specialists].

PubMed

Shin, Hyunsook; Kim, Bog-Ja; Kang, Hee Sun

2009-10-01

The study was done to investigate physical assessment skills used by, and educational needs of, advanced practice nurses (APNs) and nurse specialists in Korea. A total of 123 APNs and nurse specialists working in five major hospitals in Seoul were surveyed from July 15 to August 20, 2007. Fourteen skills out of 126 items were reported as being performed on a regular basis by participants. The majority of these skills involved general observation. Forty-six skills were rarely used. Some participants showed a lack of confidence in certain assessment skills, such as in doing a rectal or pelvic exam, and the use of some assessment equipment. Over 90% of participants required in-depth education on health assessment provided by specialists or nursing professional organizations. More educational opportunities in physical assessment should be provided including education programs based on the nurses' skill levels and needs. This effort will help to increase confidence of APNs and nurse specialists in physical assessment skills, ultimately resulting in better nursing outcomes.

Assessing Change in High School Student Information Literacy Using the Tool for Real-Time Assessment of Information Literacy Skills

ERIC Educational Resources Information Center

Kovalik, Cindy L.; Yutzey, Susan D.; Piazza, Laura M.

2012-01-01

Change in high school student information literacy (IL) knowledge and skills, from freshman year to senior year in high school was the focus of this quasi-experimental research project. Researchers used a free information literacy skills assessment tool entitled TRAILS (Tool for Real-time Assessment of Information Literacy Skills) to measure…

A procedural skills OSCE: assessing technical and non-technical skills of internal medicine residents.

PubMed

Pugh, Debra; Hamstra, Stanley J; Wood, Timothy J; Humphrey-Murto, Susan; Touchie, Claire; Yudkowsky, Rachel; Bordage, Georges

2015-03-01

Internists are required to perform a number of procedures that require mastery of technical and non-technical skills, however, formal assessment of these skills is often lacking. The purpose of this study was to develop, implement, and gather validity evidence for a procedural skills objective structured clinical examination (PS-OSCE) for internal medicine (IM) residents to assess their technical and non-technical skills when performing procedures. Thirty-five first to third-year IM residents participated in a 5-station PS-OSCE, which combined partial task models, standardized patients, and allied health professionals. Formal blueprinting was performed and content experts were used to develop the cases and rating instruments. Examiners underwent a frame-of-reference training session to prepare them for their rater role. Scores were compared by levels of training, experience, and to evaluation data from a non-procedural OSCE (IM-OSCE). Reliability was calculated using Generalizability analyses. Reliabilities for the technical and non-technical scores were 0.68 and 0.76, respectively. Third-year residents scored significantly higher than first-year residents on the technical (73.5 vs. 62.2%) and non-technical (83.2 vs. 75.1%) components of the PS-OSCE (p < 0.05). Residents who had performed the procedures more frequently scored higher on three of the five stations (p < 0.05). There was a moderate disattenuated correlation (r = 0.77) between the IM-OSCE and the technical component of the PS-OSCE scores. The PS-OSCE is a feasible method for assessing multiple competencies related to performing procedures and this study provides validity evidence to support its use as an in-training examination.

Towards an Integrated Model for Developing Sustainable Assessment Skills

ERIC Educational Resources Information Center

Fastre, Greet M. J.; van der Klink, Marcel R.; Sluijsmans, Dominique; van Merrienboer, Jeroen J. G.

2013-01-01

One of the goals of current education is to ensure that graduates can act as independent lifelong learners. Graduates need to be able to assess their own learning and interpret assessment results. The central question in this article is how to acquire sustainable assessment skills, enabling students to assess their performance and learning…

Nursing assessment of older people who are in hospital: exploring registered nurses' understanding of their assessment skills.

PubMed

Penney, Wendy; Poulter, Nola; Cole, Clare; Wellard, Sally

2016-01-01

Nurses worldwide are expected to take a leading role in caring for older people. Considerable literature dedicated to the range and application of assessment skills used by nurses vary. There is limited knowledge of registered nurses' (RNs) views of their assessment of older adults. The aim of this project was to explore RNs current perceptions of nursing assessment, and the core skills they identified as necessary. A qualitative descriptive design study was conducted in three inpatient units in one regional hospital in Victoria. Date were collected through participant observation of RNs (n = 13) followed by 1:1 semi-structured interviews. Data were analysed thematically. This research has illuminated that an ill-defined repertoire of skills was used by RNs when assessing older persons. Skills identified appeared to be based on years of personal-professional experience. Differences were noted between the descriptions nurses gave and what was observed during interactions with older persons.

Assessment of HIV/AIDS and Life Skills Delivery in Primary Schools in Tanzania

ERIC Educational Resources Information Center

Rushahu, Bernadetha Gabriel

2015-01-01

This study was conducted to assess the effectiveness of HIV/AIDS and the Life Skills Education delivery in primary schools in Tanzania. Specifically the study investigated pupils' views about the effect of HIV/AIDS and Life Skills teaching in primary schools in Tanzania; assessed pupils' knowledge related to HIV/AIDS and Life Skills education, and…

Physiologic Basis for Understanding Quantitative Dehydration Assessment

DTIC Science & Technology

2012-01-01

Perspective Physiologic basis for understanding quantitative dehydration assessment1–4 Samuel N Cheuvront, Robert W Kenefick, Nisha Charkoudian, and...Michael N Sawka ABSTRACT Dehydration (body water deficit) is a physiologic state that can have profound implications for human health and performance...review the physiologic basis for understanding quantitative dehydration as- sessment. We highlight how phenomenologic interpretations of de- hydration

Concrete Steps for Assessing the "Soft Skills" in an MBA Program

ERIC Educational Resources Information Center

Ingols, Cynthia; Shapiro, Mary

2014-01-01

In 2006, our School of Management began the serious path of assessing both the "hard skills" (such as accounting, finance, and strategy) and the "soft skills" (such as leadership, team work, and ethics) of our MBA Program. The data generated from examining the "soft skills" that we want students to learn within our…

Developing Rubrics to Assess Complex (Generic) Skills in the Classroom: How to Distinguish Skills' Mastery Levels?

ERIC Educational Resources Information Center

Rusman, Ellen; Dirkx, Kim

2017-01-01

Many schools use analytic rubrics to (formatively) assess complex, generic or transversal (21st century) skills, such as collaborating and presenting. In rubrics, performance indicators on different levels of mastering a skill (e.g., novice, practiced, advanced, talented) are described. However, the dimensions used to describe the different…

Spatio-temporal pattern clustering for skill assessment of the Korea Operational Oceanographic System

NASA Astrophysics Data System (ADS)

Kim, J.; Park, K.

2016-12-01

In order to evaluate the performance of operational forecast models in the Korea operational oceanographic system (KOOS) which has been developed by Korea Institute of Ocean Science and Technology (KIOST), a skill assessment (SA) tool has developed and provided multiple skill metrics including not only correlation and error skills by comparing predictions and observation but also pattern clustering with numerical models, satellite, and observation. The KOOS has produced 72 hours forecast information on atmospheric and hydrodynamic forecast variables of wind, pressure, current, tide, wave, temperature, and salinity at every 12 hours per day produced by operating numerical models such as WRF, ROMS, MOM5, WW-III, and SWAN and the SA has conducted to evaluate the forecasts. We have been operationally operated several kinds of numerical models such as WRF, ROMS, MOM5, MOHID, WW-III. Quantitative assessment of operational ocean forecast model is very important to provide accurate ocean forecast information not only to general public but also to support ocean-related problems. In this work, we propose a method of pattern clustering using machine learning method and GIS-based spatial analytics to evaluate spatial distribution of numerical models and spatial observation data such as satellite and HF radar. For the clustering, we use 10 or 15 years-long reanalysis data which was computed by the KOOS, ECMWF, and HYCOM to make best matching clusters which are classified physical meaning with time variation and then we compare it with forecast data. Moreover, for evaluating current, we develop extraction method of dominant flow and apply it to hydrodynamic models and HF radar's sea surface current data. By applying pattern clustering method, it allows more accurate and effective assessment of ocean forecast models' performance by comparing not only specific observation positions which are determined by observation stations but also spatio-temporal distribution of whole model

Assessment of Teacher Perceived Skill in Classroom Assessment Practices Using IRT Models

ERIC Educational Resources Information Center

Koloi-Keaikitse, Setlhomo

2017-01-01

The purpose of this study was to assess teacher perceived skill in classroom assessment practices. Data were collected from a sample of (N = 691) teachers selected from government primary, junior secondary, and senior secondary schools in Botswana. Item response theory models were used to identify teacher response on items that measured their…

Quantitative Measurement of Critical Thinking Skills in Novice and Experienced Physical Therapists

ERIC Educational Resources Information Center

Mulhall, Michele L.

2011-01-01

Critical thinking skills (CTS) have been emphasized in educational curricula and professional development of physical therapists. Studies assessing the measurement and development of CTS in healthcare professionals have primarily focused on students enrolled in professional phases of allied health educational programs. Despite the breadth of…

The IDEA Assessment Tool: Assessing the Reporting, Diagnostic Reasoning, and Decision-Making Skills Demonstrated in Medical Students' Hospital Admission Notes.

PubMed

Baker, Elizabeth A; Ledford, Cynthia H; Fogg, Louis; Way, David P; Park, Yoon Soo

2015-01-01

Construct: Clinical skills are used in the care of patients, including reporting, diagnostic reasoning, and decision-making skills. Written comprehensive new patient admission notes (H&Ps) are a ubiquitous part of student education but are underutilized in the assessment of clinical skills. The interpretive summary, differential diagnosis, explanation of reasoning, and alternatives (IDEA) assessment tool was developed to assess students' clinical skills using written comprehensive new patient admission notes. The validity evidence for assessment of clinical skills using clinical documentation following authentic patient encounters has not been well documented. Diagnostic justification tools and postencounter notes are described in the literature (1,2) but are based on standardized patient encounters. To our knowledge, the IDEA assessment tool is the first published tool that uses medical students' H&Ps to rate students' clinical skills. The IDEA assessment tool is a 15-item instrument that asks evaluators to rate students' reporting, diagnostic reasoning, and decision-making skills based on medical students' new patient admission notes. This study presents validity evidence in support of the IDEA assessment tool using Messick's unified framework, including content (theoretical framework), response process (interrater reliability), internal structure (factor analysis and internal-consistency reliability), and relationship to other variables. Validity evidence is based on results from four studies conducted between 2010 and 2013. First, the factor analysis (2010, n = 216) yielded a three-factor solution, measuring patient story, IDEA, and completeness, with reliabilities of .79, .88, and .79, respectively. Second, an initial interrater reliability study (2010) involving two raters demonstrated fair to moderate consensus (κ = .21-.56, ρ =.42-.79). Third, a second interrater reliability study (2011) with 22 trained raters also demonstrated fair to moderate agreement

High-fidelity, low-cost, automated method to assess laparoscopic skills objectively.

PubMed

Gray, Richard J; Kahol, Kanav; Islam, Gazi; Smith, Marshall; Chapital, Alyssa; Ferrara, John

2012-01-01

We sought to define the extent to which a motion analysis-based assessment system constructed with simple equipment could measure technical skill objectively and quantitatively. An "off-the-shelf" digital video system was used to capture the hand and instrument movement of surgical trainees (beginner level = PGY-1, intermediate level = PGY-3, and advanced level = PGY-5/fellows) while they performed a peg transfer exercise. The video data were passed through a custom computer vision algorithm that analyzed incoming pixels to measure movement smoothness objectively. The beginner-level group had the poorest performance, whereas those in the advanced group generated the highest scores. Intermediate-level trainees scored significantly (p < 0.04) better than beginner trainees. Advanced-level trainees scored significantly better than intermediate-level trainees and beginner-level trainees (p < 0.04 and p < 0.03, respectively). A computer vision-based analysis of surgical movements provides an objective basis for technical expertise-level analysis with construct validity. The technology to capture the data is simple, low cost, and readily available, and it obviates the need for expert human assessment in this setting. Copyright © 2012 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Cognitive skills assessment during robot-assisted surgery: separating the wheat from the chaff.

PubMed

Guru, Khurshid A; Esfahani, Ehsan T; Raza, Syed J; Bhat, Rohit; Wang, Katy; Hammond, Yana; Wilding, Gregory; Peabody, James O; Chowriappa, Ashirwad J

2015-01-01

To investigate the utility of cognitive assessment during robot-assisted surgery (RAS) to define skills in terms of cognitive engagement, mental workload, and mental state; while objectively differentiating between novice and expert surgeons. In all, 10 surgeons with varying operative experience were assigned to beginner (BG), combined competent and proficient (CPG), and expert (EG) groups based on the Dreyfus model. The participants performed tasks for basic, intermediate and advanced skills on the da Vinci Surgical System. Participant performance was assessed using both tool-based and cognitive metrics. Tool-based metrics showed significant differences between the BG vs CPG and the BG vs EG, in basic skills. While performing intermediate skills, there were significant differences only on the instrument-to-instrument collisions between the BG vs CPG (2.0 vs 0.2, P = 0.028), and the BG vs EG (2.0 vs 0.1, P = 0.018). There were no significant differences between the CPG and EG for both basic and intermediate skills. However, using cognitive metrics, there were significant differences between all groups for the basic and intermediate skills. In advanced skills, there were no significant differences between the CPG and the EG except time (1116 vs 599.6 s), using tool-based metrics. However, cognitive metrics revealed significant differences between both groups. Cognitive assessment of surgeons may aid in defining levels of expertise performing complex surgical tasks once competence is achieved. Cognitive assessment may be used as an adjunct to the traditional methods for skill assessment during RAS. © 2014 The Authors. BJU International © 2014 BJU International.

Assessing Gross Motor Skills of Kosovar Preschool Children

ERIC Educational Resources Information Center

Shala, Merita

2009-01-01

In the light of the new developments in preschool education in Kosovo, this study attempts to carry out an assessment of the development of gross motor skills of preschool children attending institutional education. The emphasis is on creating a set of tests to measure the motor attainments of these children by conducting assessments of the…

Assessment Work: Examining the Prevalence and Nature of Assessment Competencies and Skills in Student Affairs Job Postings

ERIC Educational Resources Information Center

Hoffman, John L.; Bresciani, Marilee J.

2010-01-01

This mixed method study explored the assessment-related skills and job duties that student affairs administrators expect from new employees as reflected in 1,759 job openings posted in 2008, of which seven job postings were specialist positions in outcomes-based assessment. The skills and duties required of these seven positions were primarily…

QUANTITATIVE PROCEDURES FOR NEUROTOXICOLOGY RISK ASSESSMENT

EPA Science Inventory

In this project, previously published information on biologically based dose-response model for brain development was used to quantitatively evaluate critical neurodevelopmental processes, and to assess potential chemical impacts on early brain development. This model has been ex...

Quantitative Evaluation of Performance in Interventional Neuroradiology: An Integrated Curriculum Featuring Theoretical and Practical Challenges.

PubMed

Ernst, Marielle; Kriston, Levente; Romero, Javier M; Frölich, Andreas M; Jansen, Olav; Fiehler, Jens; Buhk, Jan-Hendrik

2016-01-01

We sought to develop a standardized curriculum capable of assessing key competencies in Interventional Neuroradiology by the use of models and simulators in an objective, quantitative, and efficient way. In this evaluation we analyzed the associations between the practical experience, theoretical knowledge, and the skills lab performance of interventionalists. We evaluated the endovascular skills of 26 participants of the Advanced Course in Endovascular Interventional Neuroradiology of the European Society of Neuroradiology with a set of three tasks (aneurysm coiling and thrombectomy in a virtual simulator and placement of an intra-aneurysmal flow disruptor in a flow model). Practical experience was assessed by a survey. Participants completed a written and oral examination to evaluate theoretical knowledge. Bivariate and multivariate analyses were performed. In multivariate analysis knowledge of materials and techniques in Interventional Neuroradiology was moderately associated with skills in aneurysm coiling and thrombectomy. Experience in mechanical thrombectomy was moderately associated with thrombectomy skills, while age was negatively associated with thrombectomy skills. We found no significant association between age, sex, or work experience and skills in aneurysm coiling. Our study gives an example of how an integrated curriculum for reasonable and cost-effective assessment of key competences of an interventional neuroradiologist could look. In addition to traditional assessment of theoretical knowledge practical skills are measured by the use of endovascular simulators yielding objective, quantitative, and constructive data for the evaluation of the current performance status of participants as well as the evolution of their technical competency over time.

Accelerated skills preparation and assessment for senior medical students entering surgical internship.

PubMed

Brunt, L Michael; Halpin, Valerie J; Klingensmith, Mary E; Tiemann, Debra; Matthews, Brent D; Spitler, Jennifer A; Pierce, Richard A

2008-05-01

Skills training plays an increasing role in residency training. Few medical schools have skills courses for senior students entering surgical residency. A skills course for 4(th)-year medical students matched in a surgical specialty was conducted in 2006 and 2007 during 7 weekly 3-hour sessions. Topics included suturing, knot tying, procedural skills (eg, chest tube insertion), laparoscopic skills, use of energy devices, and on-call management problems. Materials for outside practice were provided. Pre- and postcourse assessment of suturing skills was performed; laparoscopic skills were assessed postcourse using the Society of American Gastrointestinal and Endoscopic Surgeons' Fundamentals of Laparoscopic Surgery program. Students' perceived preparedness for internship was assessed by survey (1 to 5 Likert scale). Data are mean +/- SD and statistical analyses were performed. Thirty-one 4(th)-year students were enrolled. Pre- versus postcourse surveys of 45 domains related to acute patient management and technical and procedural skills indicated an improved perception of preparedness for internship overall (mean pre versus post) for 28 questions (p < 0.05). Students rated course relevance as "highly useful" (4.8 +/- 0.5) and their ability to complete skills as "markedly improved" (4.5 +/- 0.6). Suturing and knot-tying skills showed substantial time improvement pre- versus postcourse for 4 of 5 tasks: simple interrupted suturing (283 +/- 73 versus 243 +/- 52 seconds), subcuticular suturing (385 +/- 132 versus 274 +/- 80 seconds), 1-handed knot tying (73 +/- 33 versus 58 +/- 22 seconds), and tying in a restricted space (54 +/- 18 versus 44 +/- 16 seconds) (p < 0.02). Only 2-handed knot tying did not change substantially (65 +/- 24 versus 59 +/- 24 seconds). Of 13 students who took the Fundamentals of Laparoscopic Surgery skills test, 5 passed all 5 components and 3 passed 4 of 5 components. Skills instruction for senior students entering surgical internship results in

Retrospectively Assessed Early Motor and Current Pragmatic Language Skills in Autistic and Neurotypical Children.

PubMed

Stevenson, Jennifer L; Lindley, Caitlin E; Murlo, Nicole

2017-08-01

Autistic individuals often struggle developmentally, even in areas that are not explicit diagnostic criteria, such as motor skills. This study explored the relation between early motor skills, assessed retrospectively, and current pragmatic language skills. Caregivers of neurotypical and autistic children, matched on gender and age, completed assessments of their child's early motor development and current language abilities. Early motor skills were correlated with later pragmatic language skills, and autistic children exhibited fewer motor skills than neurotypical children. In fact, motor skills were a better predictor of an autism spectrum diagnosis than were scores on a measure of current pragmatic language. These results highlight the important role of motor skills in autism spectrum disorders.

Teacher Compliance and Accuracy in State Assessment of Student Motor Skill Performance

ERIC Educational Resources Information Center

Hall, Tina J.; Hicklin, Lori K.; French, Karen E.

2015-01-01

Purpose: The purpose of this study was to investigate teacher compliance with state mandated assessment protocols and teacher accuracy in assessing student motor skill performance. Method: Middle school teachers (N = 116) submitted eighth grade student motor skill performance data from 318 physical education classes to a trained monitoring…

Assessing Applied Skills

ERIC Educational Resources Information Center

DiMartino, Joe; Castaneda, Andrea

2007-01-01

A recent employer survey sponsored by the Partnership for 21st Century Skills found that the skills new job entrants most need for success in the workplace--oral and written communication, time management, critical thinking, problem solving, personal accountability, and the ability to work effectively with others--are the areas in which recent…

Objective assessment of laparoscopic skills using a virtual reality stimulator.

PubMed

Eriksen, J R; Grantcharov, T

2005-09-01

Virtual reality simulation has a great potential as a training and assessment tool of laparoscopic skills. The study was carried out to investigate whether the LapSim system (Surgical Science Ltd., Gothenburg, Sweden) was able to differentiate between subjects with different laparoscopic experience and thus to demonstrate its construct validity. Subjects 24 were divided into two groups: experienced (performed > 100 laparoscopic procedures, n = 10) and beginners (performed <10 laparoscopic procedures, n = 14). Assessment of laparoscopic skills was based on parameters measured by the computer system. Experienced surgeons performed consistently better than the residents. Significant differences in the parameters time and economy of motion existed between the two groups in seven of seven tasks. Regarding error parameters, differences existed in most but not all tasks. LapSim was able to differentiate between subjects with different laparoscopic experience. This indicates that the system measures skills relevant for laparoscopic surgery and can be used in training programs as a valid assessment tool.

Science Journals in the Garden: Developing the Skill of Observation in Elementary Age Students

NASA Astrophysics Data System (ADS)

Kelly, Karinsa Michelle

The ability to make and record scientific observations is critical in order for students to engage in successful inquiry, and provides a sturdy foundation for children to develop higher order cognitive processes. Nevertheless, observation is taken for granted in the elementary classroom. This study explores how linking school garden experience with the use of science journals can support this skill. Students participated in a month-long unit in which they practiced their observation skills in the garden and recorded those observations in a science journal. Students' observational skills were assessed using pre- and post-assessments, student journals, and student interviews using three criteria: Accuracy, Detail, and Quantitative Data. Statistically significant improvements were found in the categories of Detail and Quantitative Data. Scores did improve in the category of Accuracy, but it was not found to be a statistically significant improvement.

Assessment of motor and process skills: assessing client work performance in Belgium.

PubMed

Vandamme, Dirk

2010-01-01

The aim of this study is to establish whether the Assessment of Motor and Process Skills (AMPS) is an appropriate tool to evaluate the quality of work performance by comparing clients' results on the AMPS with the quality of the skills that they demonstrate on the shop floor. A convenience sample of chronically unemployed (vocationally disabled) participants (N=139) with no formal training who were seeking unskilled work through Jobcentrum West-Vlaanderen (West Flanders Job Centre, Belgium) was used. Results demonstrated that in 75.2% of cases the prediction of employment outcome was correct; it is suggested that an AMPS motor score < 2.5 and a process score < 1.2 is insufficient for regular employment, while a motor score > 3.1 and process score > 1.5 indicates that regular employment is a realistic goal. The quality of the motor skills measured by the AMPS and measured on the shop floor are comparable, but little similarity was found in the measurement of process skills.

Multi-informant assessment of transition-related skills and skill importance in adolescents with autism spectrum disorder.

PubMed

Hume, Kara; Dykstra Steinbrenner, Jessica; Sideris, John; Smith, Leann; Kucharczyk, Suzanne; Szidon, Kate

2018-01-01

Adolescents with autism spectrum disorder have limited participation in the transition planning process, despite the link between active participation and an improvement in postsecondary education and employment outcomes. The Secondary School Success Checklist was designed to support transition planning for adolescents with autism spectrum disorder by incorporating their own assessments of strengths, skill deficits, and prioritization for instruction along with those of their parents and teachers across multiple skill domains. Findings from more than 500 adolescents with autism spectrum disorder across the United States indicate discrepancies between adolescent, teacher, and parent ratings of skills highlighting the importance of the inclusion of multiple perspectives in transition planning. Although ratings varied, agreement between adolescents with autism spectrum disorder, parents, and teachers across the highest and lowest rated skills suggests the need to broaden the focus on critical transition skills to include problem-solving, planning for life after high school, and self-advocacy.

Tutor-led teaching of procedural skills in the skills lab: Complexity, relevance and teaching competence from the medical teacher, tutor and student perspective.

PubMed

Lauter, Jan; Branchereau, Sylvie; Herzog, Wolfgang; Bugaj, Till Johannes; Nikendei, Christoph

2017-05-01

In current medical curricula, the transfer of procedural skills has received increasing attention. Skills lab learning and tutor-led teaching have become an inherent part of all medical curricula at German medical faculties. In 2011, the initial basis for the classification of clinical skills in medical school was created by the German Association for Medical Education (GMA) Committee's consensus statement on procedural skills. As a recommendation for medical curricula, the National Competency-based Catalogue of Learning Objectives (NKLM, 2015) lists procedural skills according to their curriculum integration and competency level. However, classification in regard to the perceived complexity, relevance, or teaching competency is still lacking. The present study aimed to investigate procedural skills taught at the Medical Faculty of Heidelberg in regard to their complexity, relevance, and required teaching skills. To achieve this aim (1) the specific procedural skills in terms of complexity, that is, the degree of difficulty, and (2) the perceived relevance of taught procedural skills for studying and subsequent medical profession as well as (3) the personal preparation and required teaching skills were assessed in medical teachers, tutors and students. During the winter semester 2014/2015, the evaluations of all medical teachers, student tutors, and medical students in the skills lab teaching departments of internal medicine, surgery, pediatrics, gynecology, and otorhinolaryngology at the Medical Faculty of Heidelberg were assessed via a quantitative cross-sectional questionnaire survey using 7-point Likert scales. The questionnaire comprised four item sets concerning 1) demographic details, 2) procedural skill complexity, 3) practical relevance, and 4) required preparation and teaching skills. Descriptive, quantitative analysis was used for questionnaire data. The survey included the data from 17 of 20 physicians (return rate: 85 %), 10 of 10 student tutors

Issues in Basic Skills Assessment and Placement in the California Community Colleges

ERIC Educational Resources Information Center

Academic Senate for California Community Colleges, 2004

2004-01-01

When the Academic Senate for California Community Colleges compiled best practices for serving basic skills students in 2002-2003, assessment practices were notably absent. In this paper, problems with current assessment and placement practices with regards to basic skills are explored. The paper begins with a review of the matriculation process…

Predicting Children's Reading and Mathematics Achievement from Early Quantitative Knowledge and Domain-General Cognitive Abilities

PubMed Central

Chu, Felicia W.; vanMarle, Kristy; Geary, David C.

2016-01-01

One hundred children (44 boys) participated in a 3-year longitudinal study of the development of basic quantitative competencies and the relation between these competencies and later mathematics and reading achievement. The children's preliteracy knowledge, intelligence, executive functions, and parental educational background were also assessed. The quantitative tasks assessed a broad range of symbolic and nonsymbolic knowledge and were administered four times across 2 years of preschool. Mathematics achievement was assessed at the end of each of 2 years of preschool, and mathematics and word reading achievement were assessed at the end of kindergarten. Our goals were to determine how domain-general abilities contribute to growth in children's quantitative knowledge and to determine how domain-general and domain-specific abilities contribute to children's preschool mathematics achievement and kindergarten mathematics and reading achievement. We first identified four core quantitative competencies (e.g., knowledge of the cardinal value of number words) that predict later mathematics achievement. The domain-general abilities were then used to predict growth in these competencies across 2 years of preschool, and the combination of domain-general abilities, preliteracy skills, and core quantitative competencies were used to predict mathematics achievement across preschool and mathematics and word reading achievement at the end of kindergarten. Both intelligence and executive functions predicted growth in the four quantitative competencies, especially across the first year of preschool. A combination of domain-general and domain-specific competencies predicted preschoolers' mathematics achievement, with a trend for domain-specific skills to be more strongly related to achievement at the beginning of preschool than at the end of preschool. Preschool preliteracy skills, sensitivity to the relative quantities of collections of objects, and cardinal knowledge predicted

Predicting Children's Reading and Mathematics Achievement from Early Quantitative Knowledge and Domain-General Cognitive Abilities.

PubMed

Chu, Felicia W; vanMarle, Kristy; Geary, David C

2016-01-01

One hundred children (44 boys) participated in a 3-year longitudinal study of the development of basic quantitative competencies and the relation between these competencies and later mathematics and reading achievement. The children's preliteracy knowledge, intelligence, executive functions, and parental educational background were also assessed. The quantitative tasks assessed a broad range of symbolic and nonsymbolic knowledge and were administered four times across 2 years of preschool. Mathematics achievement was assessed at the end of each of 2 years of preschool, and mathematics and word reading achievement were assessed at the end of kindergarten. Our goals were to determine how domain-general abilities contribute to growth in children's quantitative knowledge and to determine how domain-general and domain-specific abilities contribute to children's preschool mathematics achievement and kindergarten mathematics and reading achievement. We first identified four core quantitative competencies (e.g., knowledge of the cardinal value of number words) that predict later mathematics achievement. The domain-general abilities were then used to predict growth in these competencies across 2 years of preschool, and the combination of domain-general abilities, preliteracy skills, and core quantitative competencies were used to predict mathematics achievement across preschool and mathematics and word reading achievement at the end of kindergarten. Both intelligence and executive functions predicted growth in the four quantitative competencies, especially across the first year of preschool. A combination of domain-general and domain-specific competencies predicted preschoolers' mathematics achievement, with a trend for domain-specific skills to be more strongly related to achievement at the beginning of preschool than at the end of preschool. Preschool preliteracy skills, sensitivity to the relative quantities of collections of objects, and cardinal knowledge predicted

Employability Skills Assessment: Measuring Work Ethic for Research and Learning

ERIC Educational Resources Information Center

Park, HwaChoon; Hill, Roger B.

2016-01-01

The Employability Skills Assessment (ESA) was developed by Hill (1995) to provide an alternative measure of work ethic needed for success in employment. This study tested goodness-of-fit for a model used to interpret ESA results. The model had three factors: interpersonal skills, initiative, and dependability. Confirmatory factor analysis results…

Assessment of structured physical examination skills training using a retro-pre-questionnaire.

PubMed

Piryani, Rano Mal; Shankar, P Ravi; Piryani, Suneel; Thapa, Trilok Pati; Karki, Balmansingh; Khakurel, Mahesh Prasad; Bhandary, Shital

2013-01-01

The effectiveness of physical examination skills (PES) training is very rarely assessed using the "post-then-pre" approach. In this study, a retro-pre-questionnaire was used to study the effect of structured physical examination skills training (SPEST) imparted to second-year undergraduate medical students. KIST Medical College (KISTMC) affiliated to Tribhuvan University Nepal admitted its first batch of MBBS students in November 2008. The university curriculum recommends the involvement of Medicine and Surgery Departments in PES training, but the methods for teaching and assessment are not well defined. KISTMC has made training more structured and involved the Medicine, Surgery, Gynaecology and Obstetrics, Orthopaedics, ENT, Ophthalmology, Paediatrics, and Family Medicine Departments. SPEST includes the teaching/learning of basic PES for 210 minutes once a week for 28 weeks. Self-assessment is done by using a retro-pre-questionnaire at the end of the last session of training, and these data are analysed using SPSS. Out of 100 students, 98 participated in the objective structured clinical examination (OSCE); 82 completed the retro-pre-questionnaire. Forty-six skills representing various systems were selected for inclusion in the retro-pre-questionnaire from among the many skills taught in different departments. The average perceived skills score (maximum score, 46×4=184) before training was 15.9 and increased to 116.5 after training. The increase was statistically significant upon the application of a paired t-test. The students perceived that their level of skills improved after the training. The retro-pre- instrument seems to be useful for assessing the learners' self-reported changes in PES after training if a large number of skills need to be assessed. However, it should be noted that although a retro-pre-questionnaire may reveal valuable information, it is not a substitute for an objective measure or gold standard.

Aligning CASAS Competencies and Assessments to Basic Skills Content Standards. Second Edition

ERIC Educational Resources Information Center

CASAS - Comprehensive Adult Student Assessment Systems (NJ1), 2009

2009-01-01

Since its inception, the Comprehensive Adult Student Assessment System (CASAS) has focused on teaching and assessing basic skills in contexts that are relevant and important to adult learners. CASAS has developed and continues to refine a highly formalized hierarchy of competencies, the application of basic skills that adults need to be fully…

Ability-versus skill-based assessment of emotional intelligence.

PubMed

Bradberry, Travis R; Su, Lac D

2006-01-01

Emotional intelligence has received an intense amount of attention in leadership circles during the last decade and continuing debate exists concerning the best method for measuring this construct. This study analyzed leader emotional intelligence scores, measured via skill and ability methodologies, against leader job performance. Two hundred twelve employees from three organizations participated in this study. Scores on the Emotional Intelligence Appraisal, a skill-based assessment, were positively, though not significantly, correlated with scores on the MSCEIT, an ability-based assessment of emotional intelligence. Scores on the MSCEIT did not have a significant relationship with job performance in this study, whereas, scores on the Emotional Intelligence Appraisal had a strong link to leader job performance. The four subcomponents of the Emotional Intelligence Appraisal were examined against job performance. Relationship management was a stronger predictor of leader job performance than the other three subcomponents. Social awareness was the single emotional intelligence skill that did not have a significant link to leader job performance. Factor analyses yielded a two-component model of emotional intelligence encompassing personal and social competence, rather than confirmation of a four-part taxonomy.

The Intra-Rater Reliability of Nine Content-Validated Technical Skill Assessment Instruments (TSAI) for Athletic Taping Skills

ERIC Educational Resources Information Center

Lagumen, Niko G.; Butterwick, Dale J.; Paskevich, David M.; Fung, Tak S.; Donnon, Tyrone L.

2008-01-01

Objective: To establish the intra-rater reliability of nine content-validated Technical Skill Assessment Instruments (TSAI) for the skills of athletic taping. Setting: University of Calgary. Subjects: Canadian Certified Athletic Therapists, CAT(C), with a mean ± SD of 9.6 ± 10.8 years as a CAT(C), 7.8 ± 10.9 years as a Supervisory Athletic…

Extended score interval in the assessment of basic surgical skills.

PubMed

Acosta, Stefan; Sevonius, Dan; Beckman, Anders

2015-01-01

The Basic Surgical Skills course uses an assessment score interval of 0-3. An extended score interval, 1-6, was proposed by the Swedish steering committee of the course. The aim of this study was to analyze the trainee scores in the current 0-3 scored version compared to a proposed 1-6 scored version. Sixteen participants, seven females and nine males, were evaluated in the current and proposed assessment forms by instructors, observers, and learners themselves during the first and second day. In each assessment form, 17 tasks were assessed. The inter-rater reliability between the current and the proposed score sheets were evaluated with intraclass correlation (ICC) with 95% confidence intervals (CI). The distribution of scores for 'knot tying' at the last time point and 'bowel anastomosis side to side' given by the instructors in the current assessment form showed that the highest score was given in 31 and 62%, respectively. No ceiling effects were found in the proposed assessment form. The overall ICC between the current and proposed score sheets after assessment by the instructors increased from 0.38 (95% CI 0.77-0.78) on Day 1 to 0.83 (95% CI 0.51-0.94) on Day 2. A clear ceiling effect of scores was demonstrated in the current assessment form, questioning its validity. The proposed score sheet provides more accurate scores and seems to be a better feedback instrument for learning technical surgical skills in the Basic Surgical Skills course.

Multi-model global assessment of subseasonal prediction skill of atmospheric rivers

NASA Astrophysics Data System (ADS)

Deflorio, M. J.

2017-12-01

Atmospheric rivers (ARs) are global phenomena that are characterized by long, narrow plumes of water vapor transport. They are most often observed in the midlatitudes near climatologically active storm track regions. Because of their frequent association with floods, landslides, and other hydrological impacts on society, there is significant incentive at the intersection of academic research, water management, and policymaking to understand the skill with which state-of-the-art operational weather models can predict ARs weeks-to-months in advance. We use the newly assembled Subseasonal-to-Seasonal (S2S) database, which includes extensive hindcast records of eleven operational weather models, to assess global prediction skill of atmospheric rivers on S2S timescales. We develop a metric to assess AR skill that is suitable for S2S timescales by counting the total number of AR days which occur over each model and observational grid cell during a 2-week time window. This "2-week AR occurrence" metric is suitable for S2S prediction skill assessment because it does not consider discrete hourly or daily AR objects, but rather a smoothed representation of AR occurrence over a longer period of time. Our results indicate that several of the S2S models, especially the ECMWF model, show useful prediction skill in the 2-week forecast window, with significant interannual variation in some regions. We also present results from an experimental forecast of S2S AR prediction skill using the ECMWF and NCEP models.

Assessment of time management skills: psychometric properties of the Swedish version.

PubMed

Janeslätt, Gunnel Kristina; Holmqvist, Kajsa Lidström; White, Suzanne; Holmefur, Marie

2018-05-01

Persons with impaired time management skills are often in need of occupational therapy. Valid and reliable instruments to assess time management and organizational skills are needed for the evaluation of intervention. The purpose of this study was to evaluate the psychometric properties of a Swedish version of the Assessment of Time Management Skills (ATMS-S) for persons with and without impaired time management skills. A total of 238 persons participated in the study, of whom 94 had self-reported impaired time management skills due to mental disorders such as schizophrenic spectrum or neurodevelopmental disorders such as attention deficit/hyperactivity disorder (ADHD), autism spectrum disorder (ASD) and mild intellectual disabilities, and 144 persons had no reported impaired time management skills. Rasch analysis was used to analyze data. Three subscales were detected: the time management subscale with 11 items, the organization & planning subscale with 11 items, and the subscale of regulation of emotions with 5 items, with excellent to acceptable psychometric properties. The conclusions were that: ATMS-S is a valid instrument for self-rating of time management, organization & planning and for the regulation of emotions. ATMS-S can be useful for persons with mental disorders including mild neurodevelopmental disorders.

Development and validation of trauma surgical skills metrics: Preliminary assessment of performance after training.

PubMed

Shackelford, Stacy; Garofalo, Evan; Shalin, Valerie; Pugh, Kristy; Chen, Hegang; Pasley, Jason; Sarani, Babak; Henry, Sharon; Bowyer, Mark; Mackenzie, Colin F

2015-07-01

Maintaining trauma-specific surgical skills is an ongoing challenge for surgical training programs. An objective assessment of surgical skills is needed. We hypothesized that a validated surgical performance assessment tool could detect differences following a training intervention. We developed surgical performance assessment metrics based on discussion with expert trauma surgeons, video review of 10 experts and 10 novice surgeons performing three vascular exposure procedures and lower extremity fasciotomy on cadavers, and validated the metrics with interrater reliability testing by five reviewers blinded to level of expertise and a consensus conference. We tested these performance metrics in 12 surgical residents (Year 3-7) before and 2 weeks after vascular exposure skills training in the Advanced Surgical Skills for Exposure in Trauma (ASSET) course. Performance was assessed in three areas as follows: knowledge (anatomic, management), procedure steps, and technical skills. Time to completion of procedures was recorded, and these metrics were combined into a single performance score, the Trauma Readiness Index (TRI). Wilcoxon matched-pairs signed-ranks test compared pretraining/posttraining effects. Mean time to complete procedures decreased by 4.3 minutes (from 13.4 minutes to 9.1 minutes). The performance component most improved by the 1-day skills training was procedure steps, completion of which increased by 21%. Technical skill scores improved by 12%. Overall knowledge improved by 3%, with 18% improvement in anatomic knowledge. TRI increased significantly from 50% to 64% with ASSET training. Interrater reliability of the surgical performance assessment metrics was validated with single intraclass correlation coefficient of 0.7 to 0.98. A trauma-relevant surgical performance assessment detected improvements in specific procedure steps and anatomic knowledge taught during a 1-day course, quantified by the TRI. ASSET training reduced time to complete vascular

A Virtual Reality Simulator Prototype for Learning and Assessing Phaco-sculpting Skills

NASA Astrophysics Data System (ADS)

Choi, Kup-Sze

This paper presents a virtual reality based simulator prototype for learning phacoemulsification in cataract surgery, with focus on the skills required for making a cross-shape trench in cataractous lens by an ultrasound probe during the phaco-sculpting procedure. An immersive virtual environment is created with 3D models of the lens and surgical tools. Haptic device is also used as 3D user interface. Phaco-sculpting is simulated by interactively deleting the constituting tetrahedrons of the lens model. Collisions between the virtual probe and the lens are effectively identified by partitioning the space containing the lens hierarchically with an octree. The simulator can be programmed to collect real-time quantitative user data for reviewing and assessing trainee's performance in an objective manner. A game-based learning environment can be created on top of the simulator by incorporating gaming elements based on the quantifiable performance metrics.

Communication and Research Skills in the Information Systems Curriculum: A Method of Assessment

ERIC Educational Resources Information Center

Lazarony, Paul J.; Driscoll, Donna A.

2010-01-01

Assessment of learning goals has become the norm in business programs in higher education across the country. This paper offers a methodology for the assessment of both communication skills and research skills within a curriculum of the Bachelor of Science in Information Systems Program. Program level learning goals assessed in this paper are: (1)…

Comparison of performance on process- and product-oriented assessments of fundamental motor skills across childhood.

PubMed

Logan, Samuel W; Barnett, Lisa M; Goodway, Jacqueline D; Stodden, David F

2017-04-01

Process-oriented motor competence (MC) assessments evaluate how a movement is performed. Product-oriented assessments evaluate the outcome of a movement. Determining the concurrent validity of process and product assessments is important to address the predictive utility of motor competence for health. The current study aimed to: (1) compare process and product assessments of the standing long jump, hop and throw across age groups and (2) determine the capacity of process assessments to classify levels of MC. Participants included 170 children classified into three age groups: 4-5, 7-8 and 10-11 years old. Participants' skills were examined concurrently using three process assessments ((Test of Gross Motor Development-2nd edition [TGMD-2]), Get Skilled; Get Active, and developmental sequences) and one product measure (throw speed, jump and hop distance). Results indicate moderate to strong correlations between (1) process assessments across skills and age groups (r range = .37-70) and (2) process and product assessments across skills and age groups (r range = .26-.88). In general, sensitivity to detect advanced skill level is lowest for TGMD-2 and highest for developmental sequences for all three skills. The use of process and product assessments is suggested to comprehensively capture levels of MC in human movement.

The Effects of Performance-Based Assessment Criteria on Student Performance and Self-Assessment Skills

ERIC Educational Resources Information Center

Fastre, Greet Mia Jos; van der Klink, Marcel R.; van Merrienboer, Jeroen J. G.

2010-01-01

This study investigated the effect of performance-based versus competence-based assessment criteria on task performance and self-assessment skills among 39 novice secondary vocational education students in the domain of nursing and care. In a performance-based assessment group students are provided with a preset list of performance-based…

Evaluation of a training manual for the acquisition of behavioral assessment interviewing skills.

PubMed Central

Miltenberger, R G; Fuqua, R W

1985-01-01

Two procedures were used to teach behavioral assessment interviewing skills: a training manual and one-to-one instruction that included modeling, rehearsal, and feedback. Two graduate students and two advanced undergraduates were trained with each procedure. Interviewing skills were recorded in simulated assessment interviews conducted by each student across baseline and treatment conditions. Each training procedure was evaluated in a multiple baseline across students design. The results showed that both procedures were effective for training behavioral interviewing skills, with all students reaching a level of 90%-100% correct responding. Finally, a group of experts in behavior analysis rated each interviewing skill as relevant to the conduct of an assessment interview and a group of behavioral clinicians socially validated the outcomes of the two procedures. PMID:4086413

A Quantitative Assessment of Skills and Competencies in Graduates of At-Risk High Schools

ERIC Educational Resources Information Center

Williams, Charles Jerry, Jr.

2012-01-01

The willingness to work and a high school diploma were once all that was needed to start one's career. The problem is that on the twentieth anniversary of the reports by the Secretary's Commission on Achieving Necessary Skills (SCANS), high school graduates may still lack what business and education leaders require for success in entry-level…

Implementing and Evaluating a National Certification Technical Skills Examination: The Colorectal Objective Structured Assessment of Technical Skill.

PubMed

de Montbrun, Sandra; Roberts, Patricia L; Satterthwaite, Lisa; MacRae, Helen

2016-07-01

To implement the Colorectal Objective Structured Assessment of Technical skill (COSATS) into American Board of Colon and Rectal Surgery (ABCRS) certification and build evidence of validity for the interpretation of the scores of this high stakes assessment tool. Currently, technical skill assessment is not a formal component of board certification. With the technical demands of surgical specialties, documenting competence in technical skill at the time of certification with a valid tool is ideal. In September 2014, the COSATS was a mandatory component of ABCRS certification. Seventy candidates took the examination, with their performance evaluated by expert colorectal surgeons using a task-specific checklist, global rating scale, and overall performance scale. Passing scores were set and compared using 2 standard setting methodologies, using a compensatory and conjunctive model. Inter-rater reliability and the reliability of the pass/fail decision were calculated using Cronbach alpha and Subkoviak methodology, respectively. Overall COSATS scores and pass/fail status were compared with results on the ABCRS oral examination. The pass rate ranged from 85.7% to 90%. Inter-rater reliability (0.85) and reliability of the pass/fail decision (0.87 and 0.84) were high. A low positive correlation (r= 0.25) was seen between the COSATS and oral examination. All individuals who failed the COSATS passed the ABCRS oral examination. COSATS is the first technical skill examination used in national surgical board certification. This study suggests that the current certification process may be failing to identify individuals who have demonstrated technical deficiencies on this standardized assessment tool.

The assessment of surgical skills as a complement to the training method. Revision.

PubMed

Sánchez-Fernández, J; Bachiller-Burgos, J; Serrano-Pascual, Á; Cózar-Olmo, J M; Díaz-Güemes Martín-Portugués, I; Pérez-Duarte, F J; Hernández-Hurtado, L; Álvarez-Ossorio, J L; Sánchez-Margallo, F M

2016-01-01

The acquisition and improvement of surgical skills constitute a fundamental element in the training of any practitioner. At present, however, the assessment of these skills is a scarcely developed area of research. The aim of this study was to analyse the peculiarities of the various assessment systems and establish the minimum criteria that a skills and knowledge assessment system should meet as a method for assessing surgical skills in urological surgery. Scientific literature review aimed at the various currently available assessment systems for skills and competencies (technical and nontechnical), with a special focus on the systematic reviews and prospective studies. After conducting the review, we found that the various assessment systems for surgical competence have, in our opinion, a number of shortcomings. There is a certain degree of subjectivity in the assessment of surgeons by the evaluators. The assessment of nontechnical competencies is not formally recorded. There is no description of a follow-up assessment or any basic parameters associated with healthcare quality. There is no registration of associated competencies associated with the various surgical techniques. There is also no ranking of these competencies and the specific peculiarities for their application. We believe that the development of a new assessment system for surgical competencies (technical and nontechnical) aimed at assessing urologists in the various surgical techniques is necessary. To this end, our team has worked on developing the Evaluation System for Surgical Competencies on Laparoscopy, which is based on the definition, ranking and assessment of competencies demonstrated by surgeons. Copyright © 2015 AEU. Publicado por Elsevier España, S.L.U. All rights reserved.

Physiologic basis for understanding quantitative dehydration assessment.

PubMed

Cheuvront, Samuel N; Kenefick, Robert W; Charkoudian, Nisha; Sawka, Michael N

2013-03-01

Dehydration (body water deficit) is a physiologic state that can have profound implications for human health and performance. Unfortunately, dehydration can be difficult to assess, and there is no single, universal gold standard for decision making. In this article, we review the physiologic basis for understanding quantitative dehydration assessment. We highlight how phenomenologic interpretations of dehydration depend critically on the type (dehydration compared with volume depletion) and magnitude (moderate compared with severe) of dehydration, which in turn influence the osmotic (plasma osmolality) and blood volume-dependent compensatory thresholds for antidiuretic and thirst responses. In particular, we review new findings regarding the biological variation in osmotic responses to dehydration and discuss how this variation can help provide a quantitative and clinically relevant link between the physiology and phenomenology of dehydration. Practical measures with empirical thresholds are provided as a starting point for improving the practice of dehydration assessment.

Validity and reliability of the robotic objective structured assessment of technical skills

PubMed Central

Siddiqui, Nazema Y.; Galloway, Michael L.; Geller, Elizabeth J.; Green, Isabel C.; Hur, Hye-Chun; Langston, Kyle; Pitter, Michael C.; Tarr, Megan E.; Martino, Martin A.

2015-01-01

Objective Objective structured assessments of technical skills (OSATS) have been developed to measure the skill of surgical trainees. Our aim was to develop an OSATS specifically for trainees learning robotic surgery. Study Design This is a multi-institutional study in eight academic training programs. We created an assessment form to evaluate robotic surgical skill through five inanimate exercises. Obstetrics/gynecology, general surgery, and urology residents, fellows, and faculty completed five robotic exercises on a standard training model. Study sessions were recorded and randomly assigned to three blinded judges who scored performance using the assessment form. Construct validity was evaluated by comparing scores between participants with different levels of surgical experience; inter- and intra-rater reliability were also assessed. Results We evaluated 83 residents, 9 fellows, and 13 faculty, totaling 105 participants; 88 (84%) were from obstetrics/gynecology. Our assessment form demonstrated construct validity, with faculty and fellows performing significantly better than residents (mean scores: 89 ± 8 faculty; 74 ± 17 fellows; 59 ± 22 residents, p<0.01). In addition, participants with more robotic console experience scored significantly higher than those with fewer prior console surgeries (p<0.01). R-OSATS demonstrated good inter-rater reliability across all five drills (mean Cronbach's α: 0.79 ± 0.02). Intra-rater reliability was also high (mean Spearman's correlation: 0.91 ± 0.11). Conclusions We developed an assessment form for robotic surgical skill that demonstrates construct validity, inter- and intra-rater reliability. When paired with standardized robotic skill drills this form may be useful to distinguish between levels of trainee performance. PMID:24807319

An Assessment of Need for Developing and Implementing Technical and Skilled Worker Training for the Solar Energy Industry.

ERIC Educational Resources Information Center

Orsak, Charles G.; And Others

A Navarro College, Texas, study determined the quantitative and qualitative needs for developing skilled manpower for the solar industry and secondarily identified the (present) solar industry manpower populations and tasks performed by solar technical and skilled workers. Results from three initial working groups addressing equipment, market…

Assessing study skills among university students: an Iranian survey.

PubMed

Didarloo, Alireza; Khalkhali, Hamid Reza

2014-05-05

Numerous studies have revealed that study skills have a constructive role on the academic performance of students, in addition to educational quality, student' intelligence, and their affective characteristics. This study aims to examine study skills and the factors influencing them among the health sciences students of Urmia University of Medical Sciences in Iran. This was a cross-sectional study carried out from May to November 2013. A total of 340 Urmia health sciences students were selected using a simple sampling method. Data were collected using the Study Skills Assessment Questionnaire of Counseling Center of Houston University and analyzed with descriptive and analytical statistics. The mean and standard deviation of the students' study skills were 172.5±23.2, out of a total score of 240. Around 1.2% of the study skills were weak; 86.8%, moderate; and 12%, good. Among the study skills, the scores of time management, and memory and concentration were better than the others. Also, there was a significant positive correlation between study skills scores and the students' family housing status and academic level (P<0.05). Although the majority of the participants had moderate study skills, these were not sufficient and far from good. Improving and promoting the study skills of university students require the designing and implementing of education programs for study strategies. Therefore, decision makers and planners in the educational areas of universities should consider the topic described above.

The Effect of Communication Skills and Interpersonal Problem Solving Skills on Social Self-Efficacy

ERIC Educational Resources Information Center

Erozkan, Atilgan

2013-01-01

The purpose of this study was to examine communication skills, interpersonal problem solving skills, and social self-efficacy perception of adolescents and the predictive role of communication skills and interpersonal problem solving skills on social self-efficacy. This study is a quantitative and relational study aimed at examining the…

Online interactive teaching modules enhance quantitative proficiency of introductory biology students.

PubMed

Thompson, Katerina V; Nelson, Kären C; Marbach-Ad, Gili; Keller, Michael; Fagan, William F

2010-01-01

There is widespread agreement within the scientific and education communities that undergraduate biology curricula fall short in providing students with the quantitative and interdisciplinary problem-solving skills they need to obtain a deep understanding of biological phenomena and be prepared fully to contribute to future scientific inquiry. MathBench Biology Modules were designed to address these needs through a series of interactive, Web-based modules that can be used to supplement existing course content across the biological sciences curriculum. The effect of the modules was assessed in an introductory biology course at the University of Maryland. Over the course of the semester, students showed significant increases in quantitative skills that were independent of previous math course work. Students also showed increased comfort with solving quantitative problems, whether or not they ultimately arrived at the correct answer. A survey of spring 2009 graduates indicated that those who had experienced MathBench in their course work had a greater appreciation for the role of mathematics in modern biology than those who had not used MathBench. MathBench modules allow students from diverse educational backgrounds to hone their quantitative skills, preparing them for more complex mathematical approaches in upper-division courses.

Quantitative Evaluation of Performance in Interventional Neuroradiology: An Integrated Curriculum Featuring Theoretical and Practical Challenges

PubMed Central

Ernst, Marielle; Kriston, Levente; Romero, Javier M.; Frölich, Andreas M.; Jansen, Olav; Fiehler, Jens; Buhk, Jan-Hendrik

2016-01-01

Purpose We sought to develop a standardized curriculum capable of assessing key competencies in Interventional Neuroradiology by the use of models and simulators in an objective, quantitative, and efficient way. In this evaluation we analyzed the associations between the practical experience, theoretical knowledge, and the skills lab performance of interventionalists. Materials and Methods We evaluated the endovascular skills of 26 participants of the Advanced Course in Endovascular Interventional Neuroradiology of the European Society of Neuroradiology with a set of three tasks (aneurysm coiling and thrombectomy in a virtual simulator and placement of an intra-aneurysmal flow disruptor in a flow model). Practical experience was assessed by a survey. Participants completed a written and oral examination to evaluate theoretical knowledge. Bivariate and multivariate analyses were performed. Results In multivariate analysis knowledge of materials and techniques in Interventional Neuroradiology was moderately associated with skills in aneurysm coiling and thrombectomy. Experience in mechanical thrombectomy was moderately associated with thrombectomy skills, while age was negatively associated with thrombectomy skills. We found no significant association between age, sex, or work experience and skills in aneurysm coiling. Conclusion Our study gives an example of how an integrated curriculum for reasonable and cost-effective assessment of key competences of an interventional neuroradiologist could look. In addition to traditional assessment of theoretical knowledge practical skills are measured by the use of endovascular simulators yielding objective, quantitative, and constructive data for the evaluation of the current performance status of participants as well as the evolution of their technical competency over time. PMID:26848840

Assessment of musculoskeletal physical examination skills and attitudes of orthopaedic residents.

PubMed

Beran, Matthew C; Awan, Hisham; Rowley, David; Samora, Julie Balch; Griesser, Michael J; Bishop, Julie Y

2012-03-21

Although the musculoskeletal physical examination is an essential part of patient encounters, we believe that it is underemphasized in residency education and that residents' physical examination skills may be lacking. We sought to assess attitudes regarding teaching of the physical examination in orthopaedic residencies, to assess physical examination knowledge and skills among residents, and to develop a method to track the skill level of residents in order to improve our physical examination curriculum. We created a thirty-question multiple-choice musculoskeletal physical examination test and administered it to our residents. We created a five-question survey assessing attitudes toward physical examination teaching in orthopaedic residencies and distributed it to U.S. orthopaedic department chairs We developed an Objective Structured Clinical Examination (OSCE), in which standardized patients enact four clinical scenarios, to observe and assess physical examination skills. The mean score on the multiple-choice physical examination test was 76% despite the fact that our residents consistently scored above 90% on the Orthopaedic In-Training Examination. Department chairs and residents agreed that, although learning to perform the physical examination is important, there is not enough time in the clinical setting to observe and critique a resident's patient examination. The overall score of our residents on the OSCE was 66%. We have exposed a deficiency in the physical examination knowledge and skills of our residents. Although the musculoskeletal physical examination is a vital practice component, our data indicate that it is likely underemphasized in training. Clinic time alone is likely insufficient for the teaching and learning of the musculoskeletal physical examination.

Extended score interval in the assessment of basic surgical skills.

PubMed

Acosta, Stefan; Sevonius, Dan; Beckman, Anders

2015-01-01

Introduction The Basic Surgical Skills course uses an assessment score interval of 0-3. An extended score interval, 1-6, was proposed by the Swedish steering committee of the course. The aim of this study was to analyze the trainee scores in the current 0-3 scored version compared to a proposed 1-6 scored version. Methods Sixteen participants, seven females and nine males, were evaluated in the current and proposed assessment forms by instructors, observers, and learners themselves during the first and second day. In each assessment form, 17 tasks were assessed. The inter-rater reliability between the current and the proposed score sheets were evaluated with intraclass correlation (ICC) with 95% confidence intervals (CI). Results The distribution of scores for 'knot tying' at the last time point and 'bowel anastomosis side to side' given by the instructors in the current assessment form showed that the highest score was given in 31 and 62%, respectively. No ceiling effects were found in the proposed assessment form. The overall ICC between the current and proposed score sheets after assessment by the instructors increased from 0.38 (95% CI 0.77-0.78) on Day 1 to 0.83 (95% CI 0.51-0.94) on Day 2. Discussion A clear ceiling effect of scores was demonstrated in the current assessment form, questioning its validity. The proposed score sheet provides more accurate scores and seems to be a better feedback instrument for learning technical surgical skills in the Basic Surgical Skills course.

Assessing family planning service-delivery skills in Kenya.

PubMed

Valadez, J J; Transgrud, R; Mbugua, M; Smith, T

1997-06-01

This report demonstrates the use of Lot Quality Assurance Sampling (LQAS) to evaluate the technical competence of two cohorts of family planning service providers in Kenya trained with a new curriculum. One cohort had just finished training within two months of the study. The other cohort was the first group trained with the new curriculum about one year before the study. LQAS was adapted from industrial and other public health applications to assess both the individual competence of 30 service providers and the competence of each cohort. Results show that Cohorts One and Two did not differ markedly in the number of tasks needing improvement. However, both cohorts exhibited more tasks needing improvement in counseling skills as compared with physical examination skills or with all other skills. Care-givers who were not currently providing services accounted for most service-delivery problems. This result suggests that providers' use of their skills explains their ability to retain service-delivery skills learned in training to a greater degree than does the amount of time elapsed since they were trained. LQAS proved to be a rapid, easy-to-use empirical method for management decisionmaking for improvement of a family planning training curriculum and services.

Assessment of email communication skills of rheumatology fellows: a pilot study

PubMed Central

Dhuper, Sonal; Siva, Chokkalingam; Fresen, John L; Petruc, Marius; Velázquez, Celso R

2010-01-01

Physician–patient email communication is gaining popularity. However, a formal assessment of physicians' email communication skills has not been described. We hypothesized that the email communication skills of rheumatology fellows can be measured in an objective structured clinical examination (OSCE) setting using a novel email content analysis instrument which has 18 items. During an OSCE, we asked 50 rheumatology fellows to respond to a simulated patient email. The content of the responses was assessed using our instrument. The majority of rheumatology fellows wrote appropriate responses scoring a mean (±SD) of 10.6 (±2.6) points (maximum score 18), with high inter-rater reliability (0.86). Most fellows were concise (74%) and courteous (68%) but not formal (22%). Ninety-two percent of fellows acknowledged that the patient's condition required urgent medical attention, but only 30% took active measures to contact the patient. No one encrypted their messages. The objective assessment of email communication skills is possible using simulated emails in an OSCE setting. The variable email communication scores and incidental patient safety gaps identified, suggest a need for further training and defined proficiency standards for physicians' email communication skills. PMID:20962134

Validation of Modified Soft Skills Assessment Instrument (MOSSAI) for Use in Nigeria

ERIC Educational Resources Information Center

Aworanti, O. A.; Taiwo, M. B.; Iluobe, O. I.

2015-01-01

Currently, it has become an accepted norm nearly all over the globe to teach and assess soft skills. However, in Nigeria, it is an emerging area of interest that needs to be addressed squarely. In the light of the fore-going, this study validated a modified version of Measuring and Assessment Soft Skills (MASS) (an instrument developed and used by…

Rubric Assessment on Science and Creative Thinking Skills of Students

NASA Astrophysics Data System (ADS)

Ratnasusanti, H.; Ana, A.; Nurafiati, P.; Umusyaadah, L.

2018-02-01

The result of the monitoring and evaluation of the latest Indonesian curriculum (the 2013 curriculum) implementation at junior high school level year of 2014 showed that one of the difficult things that learners had in implementation 2013 curriculum is doing the result. The characteristic of applying the 2013 curriculum is to emphasize the modern pedagogic dimension of learning, which is using scientific approach, which requires learners to have highlevel thinking skills, one of which is creative thinking skills. The aims of this research is to implement performance assessment in measuring the creative thinking of junior high school students on subject Prakarya. The form of the main performance assessment is the task and assessment criteria. The experimental method that been used is the Quasi Experiment with Non-Equivalent Design Group Research. Population in this study is the students of VIII class of junior high school in Bandung, Indonesia which consists of six classes. And two classes are selected for the sample from that six classes and VIII A class were chosen, while VIII F class has been chosen as control class. The result of this research showed that the rubics of performance assessment can be measure or identify the creative thinking skill, its prove by the result of pre-test dan post-test are more dominant. In material of identification student’s creative thinking skills are reached an average 85 compare 79 with the control class. while in the presentation the experimental class got an average of 85 bigger than the control class which only reached 79.

Facilitating the implementation of the American College of Surgeons/Association of Program Directors in Surgery phase III skills curriculum: training faculty in the assessment of team skills.

PubMed

Hull, Louise; Arora, Sonal; Stefanidis, Dimitrios; Sevdalis, Nick

2015-11-01

Effective teamwork is critical to safety in the operating room; however, implementation of phase III of the American College of Surgeons (ACS) and Association of Program Directors in Surgery (APDS) Curriculum that focuses on team-based skills remains worryingly low. Training and assessing the complexities of teamwork is challenging. The objective of this study was to establish guidelines and recommendations for training faculty in assessing/debriefing team skills. A multistage survey-based consensus study was completed by 108 experts responsible for training and assessing surgical residents from the ACS Accredited Educational Institutes. Experts agreed that a program to teach faculty to assess team-based skills should include training in the recognition of teamwork skills, practice rating these skills, and training in the provision of feedback/debriefing. Agreement was reached that faculty responsible for conducting team-based skills assessment should be revalidated every 2 years and stringent proficiency criteria should be met. Faculty development is critical to ensure high-quality, standardized training and assessment. Training faculty to assess team-based skills has the potential to facilitate the effective implementation of phase III of the ACS and APDS Curriculum. Copyright © 2015 Elsevier Inc. All rights reserved.

Assessing the reporting of categorised quantitative variables in observational epidemiological studies.

PubMed

Mabikwa, Onkabetse V; Greenwood, Darren C; Baxter, Paul D; Fleming, Sarah J

2017-03-14

One aspect to consider when reporting results of observational studies in epidemiology is how quantitative risk factors are analysed. The STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines recommend that researchers describe how they handle quantitative variables when analysing data. For categorised quantitative variables, the authors are required to provide reasons and justifications informing their practice. We investigated and assessed the practices and reporting of categorised quantitative variables in epidemiology. The assessment was based on five medical journals that publish epidemiological research. Observational studies published between April and June 2015 and investigating the relationships between quantitative exposures (or risk factors) and the outcomes were considered for assessment. A standard form was used to collect the data, and the reporting patterns amongst eligible studies were quantified and described. Out of 61 articles assessed for eligibility, 23 observational studies were included in the assessment. Categorisation of quantitative exposures occurred in 61% of these studies and reasons informing the practice were rarely provided. Only one article explained the choice of categorisation in the analysis. Transformation of quantitative exposures into four or five groups was common and dominant amongst studies using equally spaced categories. Dichotomisation was not popular; the practice featured in one article. Overall, the majority (86%) of the studies preferred ordered or arbitrary group categories. Other criterions used to decide categorical boundaries were based on established guidelines such as consensus statements and WHO standards. Categorisation of continuous variables remains a dominant practice in epidemiological studies. The reasons informing the practice of categorisation within published work are limited and remain unknown in most articles. The existing STROBE guidelines could provide stronger

Assessment of medical communication skills by computer: assessment method and student experiences.

PubMed

Hulsman, R L; Mollema, E D; Hoos, A M; de Haes, J C J M; Donnison-Speijer, J D

2004-08-01

A computer-assisted assessment (CAA) program for communication skills designated ACT was developed using the objective structured video examination (OSVE) format. This method features assessment of cognitive scripts underlying communication behaviour, a broad range of communication problems covered in 1 assessment, highly standardised assessment and rating procedures, and large group assessments without complex organisation. The Academic Medical Centre (AMC) at the University of Amsterdam, the Netherlands. Aims To describe the development of the AMC Communication Test (ACT); to describe our experiences with the examination and rating procedures; to present test score descriptives, and to present the students' opinions of ACT. The ACT presents films on history taking, breaking bad news and shared decision making. Each film is accompanied by 3 types of short essay questions derived from our assessment model: "knows", "knows why/when" and "knows how". Evaluation questions about ACT were integrated into the assessment. Participants A total of 210 third year medical undergraduates were assessed. This study reports on the 110 (53%) students who completed all evaluation questions. Marking 210 examinations took about 17 days. The test score matched a normal distribution and showed a good level of discrimination of the students. About 75% passed the examination. Some support for the validity of our assessment model was found in the students' differential performance on the 3 types of questions. The ACT was well received. Student evaluations confirmed our efforts to develop realistic films that related well to the communication training programme. The ACT is a useful assessment method which complements interpersonal assessment methods for the evaluation of the medical communication skills of undergraduates.

Quantitative Courses in a Liberal Education Program: A Case Study

ERIC Educational Resources Information Center

Wismath, Shelly L.; Mackay, D. Bruce

2012-01-01

This essay argues for the importance of quantitative reasoning skills as part of a liberal education and describes the successful introduction of a mathematics-based quantitative skills course at a small Canadian university. Today's students need quantitative problem-solving skills, to function as adults, professionals, consumers, and citizens in…

Laparoscopic skills assessment: an additional modality for pediatric surgery fellowship selection.

PubMed

Hazboun, Rajaie; Rodriguez, Samuel; Thirumoorthi, Arul; Baerg, Joanne; Moores, Donald; Tagge, Edward P

2017-12-01

The Pediatric Surgery fellow selection is a multi-layered process which has not included assessment of surgical dexterity. Data was collected prospectively as part of the 2016 Pediatric Surgery Match interview process. Applicants completed a questionnaire to document laparoscopic experience and fine motor skills activities. Actual laparoscopic skills were assessed using a simulator. Time to complete an intracorporeal knot was tabulated. An initial rank list was formulated based only on the ERAS application and interview scores. The rank list was re-formulated following the laparoscopic assessment. Un-paired T-test and regression were utilized to analyze the data. Forty applicants were interviewed with 18 matched (45%). The mean knot tying time was 201.31s for matched and 202.35s for unmatched applicants. Playing a musical instrument correlated with faster knot tying (p=0.03). No correlation was identified between knot tying time and either video game experience (p=0.4) or passing the FLS exam (p=0.78). Laparoscopic skills assessment lead to significant reordering of rank list (p=0.01). Laparoscopic skills performance significantly impacted ranking. Playing a musical instrument correlated with faster knot tying. No correlation was identified between laparoscopic performance and passing the FLS exam or other activities traditionally believed to improve technical ability. Prospective study. Level II. Copyright © 2017 Elsevier Inc. All rights reserved.

Social Validation of the New England Center for Children-Core Skills Assessment

ERIC Educational Resources Information Center

Dickson, Chata A.; MacDonald, Rebecca P. F.; Mansfield, Renee; Guilhardi, Paulo; Johnson, Cammarie; Ahearn, William H.

2014-01-01

We investigated the social validity of the NECC Core Skills Assessment (NECC-CSA) with parents and professionals as participants. The NECC-CSA is a measurement tool consisting of direct and indirect measures of skills important to all individuals with autism, across the lifespan. Participants (N = 245) were provided with a list of 66 skills, 47 of…

Do quantitative decadal forecasts from GCMs provide decision relevant skill?

NASA Astrophysics Data System (ADS)

Suckling, E. B.; Smith, L. A.

2012-04-01

It is widely held that only physics-based simulation models can capture the dynamics required to provide decision-relevant probabilistic climate predictions. This fact in itself provides no evidence that predictions from today's GCMs are fit for purpose. Empirical (data-based) models are employed to make probability forecasts on decadal timescales, where it is argued that these 'physics free' forecasts provide a quantitative 'zero skill' target for the evaluation of forecasts based on more complicated models. It is demonstrated that these zero skill models are competitive with GCMs on decadal scales for probability forecasts evaluated over the last 50 years. Complications of statistical interpretation due to the 'hindcast' nature of this experiment, and the likely relevance of arguments that the lack of hindcast skill is irrelevant as the signal will soon 'come out of the noise' are discussed. A lack of decision relevant quantiative skill does not bring the science-based insights of anthropogenic warming into doubt, but it does call for a clear quantification of limits, as a function of lead time, for spatial and temporal scales on which decisions based on such model output are expected to prove maladaptive. Failing to do so may risk the credibility of science in support of policy in the long term. The performance amongst a collection of simulation models is evaluated, having transformed ensembles of point forecasts into probability distributions through the kernel dressing procedure [1], according to a selection of proper skill scores [2] and contrasted with purely data-based empirical models. Data-based models are unlikely to yield realistic forecasts for future climate change if the Earth system moves away from the conditions observed in the past, upon which the models are constructed; in this sense the empirical model defines zero skill. When should a decision relevant simulation model be expected to significantly outperform such empirical models? Probability

Skills Needs Assessment Process To Support Economic Development. HRD & ALL Research Series. Paper 90-01.

ERIC Educational Resources Information Center

Geroy, Gary D.; Kaman, Vickie

The purpose of skills-training needs-assessment is to identify what type of skills and knowledge are needed to support economic strategies and, as a result, improve organizational decisionmaking related to training investments and general work force development. This paper presents findings of 11 skills-retraining, needs-assessment,…

Self-assessment and goal-setting is associated with an improvement in interviewing skills.

PubMed

Hanley, Kathleen; Zabar, Sondra; Charap, Joseph; Nicholson, Joseph; Disney, Lindsey; Kalet, Adina; Gillespie, Colleen

2014-01-01

Purpose Describe the relationship between medical students' self-assessment and goal-setting (SAGS) skills and development of interviewing skills during the first-year doctoring course. Method 157 first-year medical students completed three two-case standardized patient (SP) interviews. After each of the first two, students viewed videotapes of their interview, completed a SAGS worksheet, and reviewed a selected tape segment in a seminar. SAGS was categorized into good and poor quality and interviewing skills were rated by trained raters. Results SAGS improved over time (37% good week 1 vs. 61% good week 10). Baseline SAGS and interviewing skills were not associated. Initial SAGS quality was associated with change in interviewing skills - those with poor-quality SAGS demonstrated a decrease and those with good-quality SAGS demonstrated an increase in scores by 17 weeks (ANOVA F=4.16, p=0.024). For students whose SAGS skills were good at both week 1 and 10, interviewing skills declined in weeks 1-10 and then increased significantly at week 17. For those whose SAGS remained 'poor' in weeks 1-10, interviewing skills declined in weeks 10-17. Conclusions In general, the quality of students' SAGS improved over time. Poor baseline SAGS skills and failure to improve were associated with a decrease in interviewing skills at 17 weeks. For students with better SAGS, interviewing skills increased at week 17. Improvement in SAGS skills was not associated with improved interviewing skills. Understanding structured self-assessment skills helps identify student characteristics that influence progressive mastery of communication skills and therefore may inform curriculum and remediation tailoring.

Self-assessment and goal-setting is associated with an improvement in interviewing skills.

PubMed

Hanley, Kathleen; Zabar, Sondra; Charap, Joseph; Nicholson, Joseph; Disney, Lindsey; Kalet, Adina; Gillespie, Colleen

2014-01-01

Describe the relationship between medical students' self-assessment and goal-setting (SAGS) skills and development of interviewing skills during the first-year doctoring course. 157 first-year medical students completed three two-case standardized patient (SP) interviews. After each of the first two, students viewed videotapes of their interview, completed a SAGS worksheet, and reviewed a selected tape segment in a seminar. SAGS was categorized into good and poor quality and interviewing skills were rated by trained raters. SAGS improved over time (37% good week 1 vs. 61% good week 10). Baseline SAGS and interviewing skills were not associated. Initial SAGS quality was associated with change in interviewing skills - those with poor-quality SAGS demonstrated a decrease and those with good-quality SAGS demonstrated an increase in scores by 17 weeks (ANOVA F=4.16, p=0.024). For students whose SAGS skills were good at both week 1 and 10, interviewing skills declined in weeks 1-10 and then increased significantly at week 17. For those whose SAGS remained 'poor' in weeks 1-10, interviewing skills declined in weeks 10-17. In general, the quality of students' SAGS improved over time. Poor baseline SAGS skills and failure to improve were associated with a decrease in interviewing skills at 17 weeks. For students with better SAGS, interviewing skills increased at week 17. Improvement in SAGS skills was not associated with improved interviewing skills. Understanding structured self-assessment skills helps identify student characteristics that influence progressive mastery of communication skills and therefore may inform curriculum and remediation tailoring.

Goodness of Fit of Skills Assessment Approaches: Insights from Patterns of Real vs. Synthetic Data Sets

ERIC Educational Resources Information Center

Beheshti, Behzad; Desmarais, Michel C.

2015-01-01

This study investigates the issue of the goodness of fit of different skills assessment models using both synthetic and real data. Synthetic data is generated from the different skills assessment models. The results show wide differences of performances between the skills assessment models over synthetic data sets. The set of relative performances…

Needs Assessment for the Construction Industry in B.C. & the Yukon. Skill Plan.

ERIC Educational Resources Information Center

Lewe, Glenda

A basic skills needs assessment developed a picture of the basic skills levels and needs of the construction industry in British Columbia and the Yukon. The three parts of the assessment were interviews with business managers and managers of other programs provided through joint labor/management initiatives, a questionnaire administered to…

Psychometric Characteristics of Role-Play Assessments of Social Skill in Schizophrenia

ERIC Educational Resources Information Center

Bellack, Alan S.; Brown, Clayton H.; Thomas-Lohrman, Shannon

2006-01-01

There is an extensive literature documenting that people with schizophrenia have marked impairments in social role functioning and social skill. One of the most widely employed strategies for assessing social skill has been role-play tests: simulated social interactions that are videotaped for subsequent behavioral coding. There has been…

The Promotion of Critical Thinking Skills in School-Based Assessment (SBA)

ERIC Educational Resources Information Center

Kamarulzaman, Wirawani; Kamarulzaman, Wirawahida

2016-01-01

The new curriculum introduced in the Malaysian primary students; the Primary School Standard Curriculum (Kurikulum Standard Sekolah Rendah-KSSR) together with the school-based assessment (SBA) is a step taken by the Malaysian government to encourage thinking skills to students, specifically critical thinking skills. The study explores teachers'…

[Helping relationship skills in nurses: the validation of a measurement instrument].

PubMed

de Melo, Rosa Cândida Carvalho Pereira; Silva, Maria Júlia Paes; Parreira, Pedro Miguel Dinis; Ferreira, Manuela Maria Conceição

2011-12-01

Considering the importance of assessing nurses' helping relationship skills, it was necessary to use reliable and context-adapted instruments. Thus, the objective of this study was to assess the psychometric properties of the Helping Relationship Skills Inventory (Inventário de Competências Relacionais de Ajuda, ICRA), by conducting reliability and validity studies to increase the level of confidence or accuracy of the data obtained using this instrument. This quantitative study was conducted on a sample of 690 nurses who worked in six hospitals and eight health centres in Portugal. The results indicate a multidimensional structure of helping relationship skills, divided into four different dimensions (generic, empathetic, communication and contact skills) with a positive correlation between them. Cronbach's alpha for each dimension was higher than .79, showing a good internal consistency of the items within each factor.

The use of physical assessment skills by registered nurses in Australia: issues for nursing education.

PubMed

Birks, Melanie; Cant, Robyn; James, Ainsley; Chung, Catherine; Davis, Jenny

2013-01-01

The purpose of pre-service nursing education programs is to prepare competent graduates who are able to function as safe, professional registered nurses. An extensive element of these programs is the teaching of physical assessment skills, with most programs educating students to perform over 120 such skills. Previous research from North America suggests that the majority of skills taught to nurses in their pre-service programs are not used in practice. As part of a larger study, an online survey was used to explore use of 121 physical assessment skills by Australian nurses. Recruitment occurred via mailed invitation to members of the Australian Nursing Federation. Data were extracted from 1220 completed questionnaires returned by nurses who were mostly employed in New South Wales, were female and experienced nurses. Respondents indicated that they used only 34% of skills routinely. Results reinforce evidence found in the literature that many of the skills taught to nurses are either not used at all (35.5%) or are used rarely (31%). These findings have implications for the teaching of physical assessment skills in pre-service nursing programs, and raise questions about the value of extensive skills teaching in the context of contemporary health care. Further research into barriers to the use of physical assessment skills in nursing and the need for comprehensive skills preparation for the generalist nurse is likely to offer some solutions to these questions.

Reliability of the ECHOWS Tool for Assessment of Patient Interviewing Skills.

PubMed

Boissonnault, Jill S; Evans, Kerrie; Tuttle, Neil; Hetzel, Scott J; Boissonnault, William G

2016-04-01

History taking is an important component of patient/client management. Assessment of student history-taking competency can be achieved via a standardized tool. The ECHOWS tool has been shown to be valid with modest intrarater reliability in a previous study but did not demonstrate sufficient power to definitively prove its stability. The purposes of this study were: (1) to assess the reliability of the ECHOWS tool for student assessment of patient interviewing skills and (2) to determine whether the tool discerns between novice and experienced skill levels. A reliability and construct validity assessment was conducted. Three faculty members from the United States and Australia scored videotaped histories from standardized patients taken by students and experienced clinicians from each of these countries. The tapes were scored twice, 3 to 6 weeks apart. Reliability was assessed using interclass correlation coefficients (ICCs) and repeated measures. Analysis of variance models assessed the ability of the tool to discern between novice and experienced skill levels. The ECHOWS tool showed excellent intrarater reliability (ICC [3,1]=.74-.89) and good interrater reliability (ICC [2,1]=.55) as a whole. The summary of performance (S) section showed poor interrater reliability (ICC [2,1]=.27). There was no statistical difference in performance on the tool between novice and experienced clinicians. A possible ceiling effect may occur when standardized patients are not coached to provide complex and obtuse responses to interviewer questions. Variation in familiarity with the ECHOWS tool and in use of the online training may have influenced scoring of the S section. The ECHOWS tool demonstrates excellent intrarater reliability and moderate interrater reliability. Sufficient training with the tool prior to student assessment is recommended. The S section must evolve in order to provide a more discerning measure of interviewing skills. © 2016 American Physical Therapy

The consequences of using advanced physical assessment skills in medical and surgical nursing: A hermeneutic pragmatic study.

PubMed

Zambas, Shelaine I; Smythe, Elizabeth A; Koziol-Mclain, Jane

2016-01-01

The aim of this study was to explore the consequences of the nurse's use of advanced assessment skills on medical and surgical wards. Appropriate, accurate, and timely assessment by nurses is the cornerstone of maintaining patient safety in hospitals. The inclusion of "advanced" physical assessment skills such as auscultation, palpation, and percussion is thought to better prepare nurses for complex patient presentations within a wide range of clinical situations. This qualitative study used a hermeneutic pragmatic approach. Unstructured interviews were conducted with five experienced medical and surgical nurses to obtain 13 detailed narratives of assessment practice. Narratives were analyzed using Van Manen's six-step approach to identify the consequences of the nurse's use of advanced assessment skills. The consequences of using advanced assessment skills include looking for more, challenging interpretations, and perseverance. The use of advanced assessment skills directs what the nurse looks for, what she sees, interpretation of the findings, and her response. It is the interpretation of what is seen, heard, or felt within the full context of the patient situation, which is the advanced skill. Advanced assessment skill is the means to an accurate interpretation of the clinical situation and contributes to appropriate diagnosis and medical management in complex patient situations. The nurse's use of advanced assessment skills enables her to contribute to diagnostic reasoning within the acute medical and surgical setting.

Taxonomy of Trauma Leadership Skills: A Framework for Leadership Training and Assessment.

PubMed

Leenstra, Nico F; Jung, Oliver C; Johnson, Addie; Wendt, Klaus W; Tulleken, Jaap E

2016-02-01

Good leadership is essential for optimal trauma team performance, and targeted training of leadership skills is necessary to achieve such leadership proficiency. To address the need for a taxonomy of leadership skills that specifies the skill components to be learned and the behaviors by which they can be assessed across the five phases of trauma care, the authors developed the Taxonomy of Trauma Leadership Skills (TTLS). Critical incident interviews were conducted with trauma team leaders and members from different specialties-emergency physicians, trauma surgeons, anesthesiologists, and emergency ward nurses-at three teaching hospitals in the Netherlands during January-June 2013. Data were iteratively analyzed for examples of excellent leadership skills at each phase of trauma care. Using the grounded theory approach, elements of excellent leadership skills were identified and classified. Elements and behavioral markers were sorted and categorized using multiple raters. In a two-round verification process in late 2013, the taxonomy was reviewed and rated by trauma team leaders and members from the multiple specialties for its coverage of essential items. Data were gathered from 28 interviews and 14 raters. The TTLS details 5 skill categories (information coordination, decision making, action coordination, communication management, and coaching and team development) and 37 skill elements. The skill elements are captured by 67 behavioral markers. The three-level taxonomy is presented according to five phases of trauma care. The TTLS provides a framework for teaching, learning, and assessing team leadership skills in trauma care and other complex, acute care situations.

Examining construct validity of a new naturalistic observational assessment of hand skills for preschool- and school-age children.

PubMed

Chien, Chi-Wen; Brown, Ted; McDonald, Rachael

2012-04-01

The Assessment of Children's Hand Skills is a new assessment that utilises a naturalistic observational method to capture children's real-life hand skill performance when engaged at various types of daily activities in everyday living contexts. The Assessment of Children's Hand Skills is designed for use with 2- to 12-year-old children with a range of disabilities or health conditions. The study aimed to investigate construct validity of the Assessment of Children's Hand Skills in Australian children. Rasch analysis was used to examine internal construct validity of the Assessment of Children's Hand Skills in a mixed sample of 53 children with disabilities (including autism spectrum disorder, developmental/genetic disorders and physical disabilities) and 85 typically developing children. External construct validity was examined by correlating with three questionnaires evaluating daily living skills and hand skills. Rasch goodness-of-fit analysis suggested that all 22 activity items and 19 of 20 hand skill items in the Assessment of Children's Hand Skills measured a single construct. The Assessment of Children's Hand Skills items were placed in a clinically meaningful hierarchy from easy to hard, and the difficulty range of the items also matched the majority of children with disabilities and typically developing preschool-aged children. Moderate to high correlations (0.59 ≤ Spearman's ρ coefficients ≤ 0.89, P < 0.01) were found with the assessments of daily living and fine motor skills. This study provided preliminary evidence supporting the construct validity of the Assessment of Children's Hand Skills for its clinical application in assessing children's real-life hand skill performance in Australian contexts. © 2012 The Authors Australian Occupational Therapy Journal © 2012 Occupational Therapy Australia.

E-learning and nursing assessment skills and knowledge - An integrative review.

PubMed

McDonald, Ewan W; Boulton, Jessica L; Davis, Jacqueline L

2018-07-01

This review examines the current evidence on the effectiveness of digital technologies or e-based learning for enhancing the skills and knowledge of nursing students in nursing assessment. This integrative review identifies themes emerging from e-learning and 'nursing assessment' literature. Literature reviews have been undertaken in relation to digital learning and nursing education, including clinical skills, clinical case studies and the nurse-educator role. Whilst perceptions of digital learning are well covered, a gap in knowledge persists for understanding the effectiveness of e-learning on nursing assessment skills and knowledge. This is important as comprehensive assessment skills and knowledge are a key competency for newly qualified nurses. The MEDLINE, CINAHL, Cochrane Library and ProQuest Nursing and Allied Health Source electronic databases were searched for the period 2006 to 2016. Hand searching in bibliographies was also undertaken. Selection criteria for this review included: FINDINGS: Twenty articles met the selection criteria for this review, and five major themes for e-based learning were identified (a) students become self-evaluators; (b) blend and scaffold learning; (c) measurement of clinical reasoning; (d) mobile technology and Facebook are effective; and (e) training and preparation is vital. Although e-based learning programs provide a flexible teaching method, evidence suggests e-based learning alone does not exceed face-to-face patient simulation. This is particularly the case where nursing assessment learning is not scaffolded. This review demonstrates that e-based learning and traditional teaching methods used in conjunction with each other create a superior learning style. Copyright © 2018 Elsevier Ltd. All rights reserved.

Identification of the students' critical thinking skills through biochemistry laboratory work report

NASA Astrophysics Data System (ADS)

Anwar, Yunita Arian Sani; Senam, Laksono, Endang W.

2017-08-01

This work aims to (1) identify the critical thinking skills of student based on their ability to set up laboratory work reports, and (2) analyze the implementation of biochemistry laboratory work. The method of quantitative content analysis was employed. Quantitative data were in the form of critical thinking skills through the assessment of students' laboratory work reports and questionnaire data. Hoyo rubric was used to measure critical thinking skills with 10 indicators, namely clarity, accuracy, precision, consistency, relevance, evidence, reason, depth, breadth, and fairness. The research sample consisted of 105 students (35 male, 70 female) of Mataram University who took a Biochemistry course and 2 lecturers of Biochemistry course. The results showed students' critical thinking skills through laboratory work reports were still weak. Analysis of the questionnaire showed that three indicators become the biggest problems during the laboratory work implementation, namely, lecturers' involved in laboratory work implementation, the integration of laboratory work implementation of learning in the classroom has not been done optimally and laboratory work implementation as an effort to train critical thinking skills is not optimal yet.

1, 2, 3, 4: infusing quantitative literacy into introductory biology.

PubMed

Speth, Elena Bray; Momsen, Jennifer L; Moyerbrailean, Gregory A; Ebert-May, Diane; Long, Tammy M; Wyse, Sara; Linton, Debra

2010-01-01

Biology of the twenty-first century is an increasingly quantitative science. Undergraduate biology education therefore needs to provide opportunities for students to develop fluency in the tools and language of quantitative disciplines. Quantitative literacy (QL) is important for future scientists as well as for citizens, who need to interpret numeric information and data-based claims regarding nearly every aspect of daily life. To address the need for QL in biology education, we incorporated quantitative concepts throughout a semester-long introductory biology course at a large research university. Early in the course, we assessed the quantitative skills that students bring to the introductory biology classroom and found that students had difficulties in performing simple calculations, representing data graphically, and articulating data-driven arguments. In response to students' learning needs, we infused the course with quantitative concepts aligned with the existing course content and learning objectives. The effectiveness of this approach is demonstrated by significant improvement in the quality of students' graphical representations of biological data. Infusing QL in introductory biology presents challenges. Our study, however, supports the conclusion that it is feasible in the context of an existing course, consistent with the goals of college biology education, and promotes students' development of important quantitative skills.

1, 2, 3, 4: Infusing Quantitative Literacy into Introductory Biology

PubMed Central

Momsen, Jennifer L.; Moyerbrailean, Gregory A.; Ebert-May, Diane; Long, Tammy M.; Wyse, Sara; Linton, Debra

2010-01-01

Biology of the twenty-first century is an increasingly quantitative science. Undergraduate biology education therefore needs to provide opportunities for students to develop fluency in the tools and language of quantitative disciplines. Quantitative literacy (QL) is important for future scientists as well as for citizens, who need to interpret numeric information and data-based claims regarding nearly every aspect of daily life. To address the need for QL in biology education, we incorporated quantitative concepts throughout a semester-long introductory biology course at a large research university. Early in the course, we assessed the quantitative skills that students bring to the introductory biology classroom and found that students had difficulties in performing simple calculations, representing data graphically, and articulating data-driven arguments. In response to students' learning needs, we infused the course with quantitative concepts aligned with the existing course content and learning objectives. The effectiveness of this approach is demonstrated by significant improvement in the quality of students' graphical representations of biological data. Infusing QL in introductory biology presents challenges. Our study, however, supports the conclusion that it is feasible in the context of an existing course, consistent with the goals of college biology education, and promotes students' development of important quantitative skills. PMID:20810965

Creating Trails: Tool for Real-Time Assessment of Information Literacy Skills

ERIC Educational Resources Information Center

Schloman, Barbara F.; Gedeon, Julie A.

2007-01-01

This article describes Tool for Real-time Assessment of Information Literacy Skills (TRAILS), a freely available, online tool designed to measure the information literacy skills of high school students. It is based on information literacy competencies for ninth-graders found in the "Ohio Academic Content Standards" (Ohio Department of Education…

Peer Assessment of Soft Skills and Hard Skills

ERIC Educational Resources Information Center

Zhang, Aimao

2012-01-01

Both the information technology (IT) industry and the Accreditation Board for Engineering and Technology (ABET) demand soft-skill training in higher education and require IT graduates to demonstrate competence in interpersonal communication, teamwork, and conflict management. Group projects provide teamwork environment for soft-skill training, but…

Virtual humans and formative assessment to train diagnostic skills in bulimia nervosa.

PubMed

Gutiérrez-Maldonado, José; Ferrer-Garcia, Marta; Pla, Joana; Andrés-Pueyo, Antonio

2014-01-01

Carrying out a diagnostic interview requires skills that need to be taught in a controlled environment. Virtual Reality (VR) environments are increasingly used in the training of professionals, as they offer the most realistic alternative while not requiring students to face situations for which they are yet unprepared. The results of the training of diagnostic skills can also be generalized to any other situation in which effective communication skills play a major role. Our aim with this study has been to develop a procedure of formative assessment in order to increment the effectiveness of virtual learning simulation systems and then to assess their efficacy.

External validation of Global Evaluative Assessment of Robotic Skills (GEARS).

PubMed

Aghazadeh, Monty A; Jayaratna, Isuru S; Hung, Andrew J; Pan, Michael M; Desai, Mihir M; Gill, Inderbir S; Goh, Alvin C

2015-11-01

We demonstrate the construct validity, reliability, and utility of Global Evaluative Assessment of Robotic Skills (GEARS), a clinical assessment tool designed to measure robotic technical skills, in an independent cohort using an in vivo animal training model. Using a cross-sectional observational study design, 47 voluntary participants were categorized as experts (>30 robotic cases completed as primary surgeon) or trainees. The trainee group was further divided into intermediates (≥5 but ≤30 cases) or novices (<5 cases). All participants completed a standardized in vivo robotic task in a porcine model. Task performance was evaluated by two expert robotic surgeons and self-assessed by the participants using the GEARS assessment tool. Kruskal-Wallis test was used to compare the GEARS performance scores to determine construct validity; Spearman's rank correlation measured interobserver reliability; and Cronbach's alpha was used to assess internal consistency. Performance evaluations were completed on nine experts and 38 trainees (14 intermediate, 24 novice). Experts demonstrated superior performance compared to intermediates and novices overall and in all individual domains (p < 0.0001). In comparing intermediates and novices, the overall performance difference trended toward significance (p = 0.0505), while the individual domains of efficiency and autonomy were significantly different between groups (p = 0.0280 and 0.0425, respectively). Interobserver reliability between expert ratings was confirmed with a strong correlation observed (r = 0.857, 95 % CI [0.691, 0.941]). Experts and participant scoring showed less agreement (r = 0.435, 95 % CI [0.121, 0.689] and r = 0.422, 95 % CI [0.081, 0.0672]). Internal consistency was excellent for experts and participants (α = 0.96, 0.98, 0.93). In an independent cohort, GEARS was able to differentiate between different robotic skill levels, demonstrating excellent construct validity. As a standardized

Assessing competence in communication and interpersonal skills: the Kalamazoo II report.

PubMed

Duffy, F Daniel; Gordon, Geoffrey H; Whelan, Gerald; Cole-Kelly, Kathy; Frankel, Richard; Buffone, Natalie; Lofton, Stephanie; Wallace, MaryAnne; Goode, Leslie; Langdon, Lynn

2004-06-01

Accreditation of residency programs and certification of physicians requires assessment of competence in communication and interpersonal skills. Residency and continuing medical education program directors seek ways to teach and evaluate these competencies. This report summarizes the methods and tools used by educators, evaluators, and researchers in the field of physician-patient communication as determined by the participants in the "Kalamazoo II" conference held in April 2002. Communication and interpersonal skills form an integrated competence with two distinct parts. Communication skills are the performance of specific tasks and behaviors such as obtaining a medical history, explaining a diagnosis and prognosis, giving therapeutic instructions, and counseling. Interpersonal skills are inherently relational and process oriented; they are the effect communication has on another person such as relieving anxiety or establishing a trusting relationship. This report reviews three methods for assessment of communication and interpersonal skills: (1) checklists of observed behaviors during interactions with real or simulated patients; (2) surveys of patients' experience in clinical interactions; and (3) examinations using oral, essay, or multiple-choice response questions. These methods are incorporated into educational programs to assess learning needs, create learning opportunities, or guide feedback for learning. The same assessment tools, when administered in a standardized way, rated by an evaluator other than the teacher, and using a predetermined passing score, become a summative evaluation. The report summarizes the experience of using these methods in a variety of educational and evaluation programs and presents an extensive bibliography of literature on the topic. Professional conversation between patients and doctors shapes diagnosis, initiates therapy, and establishes a caring relationship. The degree to which these activities are successful depends, in

Comparison of Parent Report and Direct Assessment of Child Skills in Toddlers.

PubMed

Miller, Lauren E; Perkins, Kayla A; Dai, Yael G; Fein, Deborah A

2017-09-01

There are unique challenges associated with measuring development in early childhood. Two primary sources of information are used: parent report and direct assessment. Each approach has strengths and weaknesses, particularly when used to identify and diagnose developmental delays. The present study aimed to evaluate consistency between parent report and direct assessment of child skills in toddlers with and without Autism Spectrum Disorder (ASD) across receptive language, expressive language, and fine motor domains. 109 children were evaluated at an average age of two years; data on child skills were collected via parent report and direct assessment. Children were classified into three groups (i.e., ASD, Other Developmental Disorder, or Typical Development) based on DSM-IV-TR diagnosis. Mixed design ANOVAs, with data source as a within subjects factor and diagnostic group as a between subjects factor, were used to assess agreement. Chi square tests of agreement were then used to examine correspondence at the item level. Results suggested that parent report of language and fine motor skills did not significantly differ from direct assessment, and this finding held across diagnostic groups. Item level analyses revealed that, in most cases of significant disagreement, parents reported a skill as present, but it was not seen on direct testing. Results indicate that parents are generally reliable reporters of child language and fine motor abilities in toddlerhood, even when their children have developmental disorders such as ASD. However, the fullest picture may be obtained by using both parent report and direct assessment.

Impact of teaching and assessment format on electrocardiogram interpretation skills.

PubMed

Raupach, Tobias; Hanneforth, Nathalie; Anders, Sven; Pukrop, Tobias; Th J ten Cate, Olle; Harendza, Sigrid

2010-07-01

Interpretation of the electrocardiogram (ECG) is a core clinical skill that should be developed in undergraduate medical education. This study assessed whether small-group peer teaching is more effective than lectures in enhancing medical students' ECG interpretation skills. In addition, the impact of assessment format on study outcome was analysed. Two consecutive cohorts of Year 4 medical students (n=335) were randomised to receive either traditional ECG lectures or the same amount of small-group, near-peer teaching during a 6-week cardiorespiratory course. Before and after the course, written assessments of ECG interpretation skills were undertaken. Whereas this final assessment yielded a considerable amount of credit points for students in the first cohort, it was merely formative in nature for the second cohort. An unannounced retention test was applied 8 weeks after the end of the cardiovascular course. A significant advantage of near-peer teaching over lectures (effect size 0.33) was noted only in the second cohort, whereas, in the setting of a summative assessment, both teaching formats appeared to be equally effective. A summative instead of a formative assessment doubled the performance increase (Cohen's d 4.9 versus 2.4), mitigating any difference between teaching formats. Within the second cohort, the significant difference between the two teaching formats was maintained in the retention test (p=0.017). However, in both cohorts, a significant decrease in student performance was detected during the 8 weeks following the cardiovascular course. Assessment format appeared to be more powerful than choice of instructional method in enhancing student learning. The effect observed in the second cohort was masked by an overriding incentive generated by the summative assessment in the first cohort. This masking effect should be considered in studies assessing the effectiveness of different teaching methods.

Getting Started on Assessment: Developing a Voluntary System of Assessment and Certification Based on Skill Standards.

ERIC Educational Resources Information Center

National Skill Standards Board (DOL/ETA), Washington, DC.

This manual provides practical advice for voluntary partnerships that, since 1994, are part of the effort to build a voluntary national system of skill standards, assessment, and certification. Intended to be used with guidance from the National Skill Standards Board, it is designed for the voluntary partnerships that have completed the standards…

Self-assessment and goal-setting is associated with an improvement in interviewing skills

PubMed Central

Hanley, Kathleen; Zabar, Sondra; Charap, Joseph; Nicholson, Joseph; Disney, Lindsey; Kalet, Adina; Gillespie, Colleen

2014-01-01

Purpose Describe the relationship between medical students’ self-assessment and goal-setting (SAGS) skills and development of interviewing skills during the first-year doctoring course. Method 157 first-year medical students completed three two-case standardized patient (SP) interviews. After each of the first two, students viewed videotapes of their interview, completed a SAGS worksheet, and reviewed a selected tape segment in a seminar. SAGS was categorized into good and poor quality and interviewing skills were rated by trained raters. Results SAGS improved over time (37% good week 1 vs. 61% good week 10). Baseline SAGS and interviewing skills were not associated. Initial SAGS quality was associated with change in interviewing skills – those with poor-quality SAGS demonstrated a decrease and those with good-quality SAGS demonstrated an increase in scores by 17 weeks (ANOVA F=4.16, p=0.024). For students whose SAGS skills were good at both week 1 and 10, interviewing skills declined in weeks 1–10 and then increased significantly at week 17. For those whose SAGS remained ‘poor’ in weeks 1–10, interviewing skills declined in weeks 10–17. Conclusions In general, the quality of students’ SAGS improved over time. Poor baseline SAGS skills and failure to improve were associated with a decrease in interviewing skills at 17 weeks. For students with better SAGS, interviewing skills increased at week 17. Improvement in SAGS skills was not associated with improved interviewing skills. Understanding structured self-assessment skills helps identify student characteristics that influence progressive mastery of communication skills and therefore may inform curriculum and remediation tailoring. PMID:25059835

Assessing instrument handling and operative consequences simultaneously: a simple method for creating synced multicamera videos for endosurgical or microsurgical skills assessments.

PubMed

Jabbour, Noel; Sidman, James

2011-10-01

There has been an increasing interest in assessment of technical skills in most medical and surgical disciplines. Many of these assessments involve microscopy or endoscopy and are thus amenable to video recording for post hoc review. An ideal skills assessment video would provide the reviewer with a simultaneous view of the examinee's instrument handling and the operative field. Ideally, a reviewer should be blinded to the identity of the examinee and whether the assessment was performed as a pretest or posttest examination, when given in conjunction with an educational intervention. We describe a simple method for reliably creating deidentified, multicamera, time-synced videos, which may be used in technical skills assessments. We pilot tested this method in a pediatric airway endoscopy Objective Assessment of Technical Skills (OSATS). Total video length was compared with the OSATS administration time. Thirty-nine OSATS were administered. There were no errors encountered in time-syncing the videos using this method. Mean duration of OSATS videos was 11 minutes and 20 seconds, which was significantly less than the time needed for an expert to be present at the administration of each 30-minute OSATS (P < 0.001). The described method for creating time-synced, multicamera skills assessment videos is reliable and may be used in endosurgical or microsurgical skills assessments. Compared with live review, post hoc video review using this method can save valuable expert reviewer time. Most importantly, this method allows a reviewer to simultaneously evaluate an examinee's instrument handling and the operative field while being blinded to the examinee's identity and timing of examination administration.

[Objective surgery -- advanced robotic devices and simulators used for surgical skill assessment].

PubMed

Suhánszki, Norbert; Haidegger, Tamás

2014-12-01

Robotic assistance became a leading trend in minimally invasive surgery, which is based on the global success of laparoscopic surgery. Manual laparoscopy requires advanced skills and capabilities, which is acquired through tedious learning procedure, while da Vinci type surgical systems offer intuitive control and advanced ergonomics. Nevertheless, in either case, the key issue is to be able to assess objectively the surgeons' skills and capabilities. Robotic devices offer radically new way to collect data during surgical procedures, opening the space for new ways of skill parameterization. This may be revolutionary in MIS training, given the new and objective surgical curriculum and examination methods. The article reviews currently developed skill assessment techniques for robotic surgery and simulators, thoroughly inspecting their validation procedure and utility. In the coming years, these methods will become the mainstream of Western surgical education.

Measuring non-technical skills in medical emergency care: a review of assessment measures

PubMed Central

Cooper, Simon; Endacott, Ruth; Cant, Robyn

2010-01-01

Aim To review the literature on non-technical skills and assessment methods relevant to emergency care. Background Non-technical skills (NTS) include leadership, teamwork, decision making and situation awareness, all of which have an impact on healthcare outcomes. Significant concerns have been raised about the rates of adverse medical events, many of which are attributed to NTS failures. Methods Ovid, Medline, ProQUEST, PsycINFO and specialty websites were searched for NTS measures using applicable access strategies, inclusion and exclusion criteria. Publications identified were assessed for relevance. Results A range of non-technical skill measures relevant to emergency care was identified: leadership (n = 5), teamwork (n = 7), personality/behavior (n = 3) and situation awareness tools (n = 1). Of these, 9 have been used with emergency care populations/clinicians. All had varying degrees of reliability and validity. In the last decade there has been some development of teamwork measures specific to emergency care with a predominantly global and collective rating of broad skills. Conclusion A variety of non-technical skill measures are available; only a few have been used in the emergency care arena. There is a need for an increase in the focused assessment of teamwork skills for a greater understanding of team performance to enhance patient safety in medical emergency care. PMID:27147832

A Clinic-Based Assessment for Evaluating Job-Related Social Skills in Adolescents and Adults with Autism.

PubMed

Lerman, Dorothea C; White, Bridgette; Grob, Carolyn; Laudont, Courtney

2017-12-01

Many individuals with autism spectrum disorder (ASD) have difficulties obtaining and maintaining employment, yet little research has evaluated methods for assessing and improving critical vocational skills. In this study, we evaluated an assessment of job-related social skills for individuals with ASD by arranging conditions that simulated on-the-job experiences in a clinic setting. The experimenter contrived situations to assess a variety of social skills, including asking for help, asking for more materials, and responding to corrective feedback. A total of eight individuals, aged 16 to 32 years, participated. Results suggested that the assessment was useful for identifying specific social skills that could be targeted for intervention to increase success in the work environment. These findings add to the current literature by demonstrating an objective method for assessing a variety of job-related social skills under controlled, naturalistic conditions.

Peer-assessment of medical communication skills: the impact of students' personality, academic and social reputation on behavioural assessment.

PubMed

Hulsman, Robert L; Peters, Joline F; Fabriek, Marcel

2013-09-01

Peer-assessment of communication skills may contribute to mastery of assessment criteria. When students develop the capacity to judge their peers' performance, they might improve their capacity to examine their own clinical performance. In this study peer-assessment ratings are compared to teacher-assessment ratings. The aim of this paper is to explore the impact of personality and social reputation as source of bias in assessment of communication skills. Second year students were trained and assessed history taking communication skills. Peers rated the students' personality and academic and social reputation. Peer-assessment ratings were significantly correlated with teacher-ratings in a summative assessment of medical communication. Peers did not provide negative ratings on final scales but did provide negative ratings on subcategories. Peer- and teacher-assessments were both related to the students' personality and academic reputation. Peer-assessment cannot replace teacher-assessment if the assessment should result in high-stake decisions about students. Our data do not confirm the hypothesis that peers are overly biased by personality and reputation characteristics in peer-assessment of performance. Early introduction of peer-assessment in medical education would facilitate early acceptance of this mode of evaluation and would promote early on the habit of critical evaluation of professional clinical performance and acceptance of being evaluated critically by peers. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

The Computerized Perceptual Motor Skills Assessment: A new visual perceptual motor skills evaluation tool for children in early elementary grades.

PubMed

Howe, Tsu-Hsin; Chen, Hao-Ling; Lee, Candy Chieh; Chen, Ying-Dar; Wang, Tien-Ni

2017-10-01

Visual perceptual motor skills have been proposed as underlying courses of handwriting difficulties. However, there is no evaluation tool currently available to assess these skills comprehensively and to serve as a sensitive measure. The purpose of this study was to validate the Computerized Perceptual Motor Skills Assessment (CPMSA), a newly developed evaluation tool for children in early elementary grades. Its test-retest reliability, concurrent validity, discriminant validity, and responsiveness were examined in 43 typically developing children and 26 children with handwriting difficulty. The CPMSA demonstrated excellent reliability across all subtests with intra-class correlation coefficients (ICCs)≥0.80. Significant moderate correlations between the domains of the CPMSA and corresponding gold standards including Beery VMI, the TVPS-3, and the eye-hand coordination subtest of the DTVP-2 demonstrated good concurrent validity. In addition, the CPMSA showed evidence of discriminant validity in samples of children with and without handwriting difficulty. This article provides evidence in support of the CPMSA. The CPMSA is a reliable, valid, and promising measure of visual perceptual motor skills for children in early elementary grades. Directions for future study and improvements to the assessment are discussed. Copyright © 2017. Published by Elsevier Ltd.

The Impact of Adaptive Complex Assessment on the HOT Skill Development of Students

ERIC Educational Resources Information Center

Raiyn, Jamal; Tilchin, Oleg

2016-01-01

In this paper we propose a method for the adaptive complex assessment (ACA) of the higher-order thinking (HOT) skills needed by students for problem solving, and we examine the impact of the method on the development of HOT skills in a problem-based learning (PBL) environment. Complexity in the assessment is provided by initial, formative, and…

Cinemeducation: teaching family assessment skills using full-length movies.

PubMed

Wilson, Astrid H; Blake, Barbara J; Taylor, Gloria A; Hannings, Glenda

2013-05-01

A thorough family assessment provides a foundation for the nursing process when working with families. Therefore, nurses, along with other health care providers must develop expertise in conducting family assessments to provide the best possible care within the community. This article describes an innovative educational strategy using movies to teach family assessment skills and puts forth recommendations for future research to provide evidence to support this teaching modality. © 2013 Wiley Periodicals, Inc.

Using the Interpersonal Skills tool to assess interpersonal skills of internationally educated nurses.

PubMed

Shen, Jay J; Xu, Yu; Staples, Shelley; Bolstad, Anne L

2014-07-01

To assess interpersonal skills of internationally educated nurses (IEN) while interacting with standardized patients. Participants included 52 IEN at two community hospitals in the southwestern region of the USA. Standardized patients were used to create patient-nurse encounter. Seventeen items in four domains ("skills in interviewing and collecting information"; "skills in counseling and delivering information"; "rapport"; and "personal manner") in an Interpersonal Skills (IPS) instrument were measured by a Likert scale 1-4 with 4 indicating the best performance. The average composite score per domain and scores of the 17 items were compared across the domains. On 10 of the 17 items, the nurses received scores under 3. Counseling with an average score of 2.10 and closure with an average score of 2.44 in domain 2, small talk with an average score of 2.06 in domain 3, and physical exam with average score of 2.21 in domain 4 were below 2.5. The average composite score of domain 1 was 3.54, significantly higher than those of domains 2-4 (2.77, 2.81, and 2.71, respectively). Age was moderately related to the average score per domain with every 10 year increase in age resulting in a 0.1 increase in the average score. Sex and country of origin showed mixed results. The interpersonal skills of IEN in three of the four domains need improvement. Well-designed educational programs may facilitate the improvement, especially in areas of small talk, counseling, closure, and physical exam. Future research should examine relationships between the IPS and demographics factors. © 2013 The Authors. Japan Journal of Nursing Science © 2013 Japan Academy of Nursing Science.

Online Interactive Teaching Modules Enhance Quantitative Proficiency of Introductory Biology Students

PubMed Central

Nelson, Kären C.; Marbach-Ad, Gili; Keller, Michael; Fagan, William F.

2010-01-01

There is widespread agreement within the scientific and education communities that undergraduate biology curricula fall short in providing students with the quantitative and interdisciplinary problem-solving skills they need to obtain a deep understanding of biological phenomena and be prepared fully to contribute to future scientific inquiry. MathBench Biology Modules were designed to address these needs through a series of interactive, Web-based modules that can be used to supplement existing course content across the biological sciences curriculum. The effect of the modules was assessed in an introductory biology course at the University of Maryland. Over the course of the semester, students showed significant increases in quantitative skills that were independent of previous math course work. Students also showed increased comfort with solving quantitative problems, whether or not they ultimately arrived at the correct answer. A survey of spring 2009 graduates indicated that those who had experienced MathBench in their course work had a greater appreciation for the role of mathematics in modern biology than those who had not used MathBench. MathBench modules allow students from diverse educational backgrounds to hone their quantitative skills, preparing them for more complex mathematical approaches in upper-division courses. PMID:20810959

Reliability and Validity of 3 Methods of Assessing Orthopedic Resident Skill in Shoulder Surgery.

PubMed

Bernard, Johnathan A; Dattilo, Jonathan R; Srikumaran, Uma; Zikria, Bashir A; Jain, Amit; LaPorte, Dawn M

Traditional measures for evaluating resident surgical technical skills (e.g., case logs) assess operative volume but not level of surgical proficiency. Our goal was to compare the reliability and validity of 3 tools for measuring surgical skill among orthopedic residents when performing 3 open surgical approaches to the shoulder. A total of 23 residents at different stages of their surgical training were tested for technical skill pertaining to 3 shoulder surgical approaches using the following measures: Objective Structured Assessment of Technical Skills (OSATS) checklists, the Global Rating Scale (GRS), and a final pass/fail assessment determined by 3 upper extremity surgeons. Adverse events were recorded. The Cronbach α coefficient was used to assess reliability of the OSATS checklists and GRS scores. Interrater reliability was calculated with intraclass correlation coefficients. Correlations among OSATS checklist scores, GRS scores, and pass/fail assessment were calculated with Spearman ρ. Validity of OSATS checklists was determined using analysis of variance with postgraduate year (PGY) as a between-subjects factor. Significance was set at p < 0.05 for all tests. Criterion validity was shown between the OSATS checklists and GRS for the 3 open shoulder approaches. Checklist scores showed superior interrater reliability compared with GRS and subjective pass/fail measurements. GRS scores were positively correlated across training years. The incidence of adverse events was significantly higher among PGY-1 and PGY-2 residents compared with more experienced residents. OSATS checklists are a valid and reliable assessment of technical skills across 3 surgical shoulder approaches. However, checklist scores do not measure quality of technique. Documenting adverse events is necessary to assess quality of technique and ultimate pass/fail status. Multiple methods of assessing surgical skill should be considered when evaluating orthopedic resident surgical performance

Rock Slide Risk Assessment: A Semi-Quantitative Approach

NASA Astrophysics Data System (ADS)

Duzgun, H. S. B.

2009-04-01

Rock slides can be better managed by systematic risk assessments. Any risk assessment methodology for rock slides involves identification of rock slide risk components, which are hazard, elements at risk and vulnerability. For a quantitative/semi-quantitative risk assessment for rock slides, a mathematical value the risk has to be computed and evaluated. The quantitative evaluation of risk for rock slides enables comparison of the computed risk with the risk of other natural and/or human-made hazards and providing better decision support and easier communication for the decision makers. A quantitative/semi-quantitative risk assessment procedure involves: Danger Identification, Hazard Assessment, Elements at Risk Identification, Vulnerability Assessment, Risk computation, Risk Evaluation. On the other hand, the steps of this procedure require adaptation of existing or development of new implementation methods depending on the type of landslide, data availability, investigation scale and nature of consequences. In study, a generic semi-quantitative risk assessment (SQRA) procedure for rock slides is proposed. The procedure has five consecutive stages: Data collection and analyses, hazard assessment, analyses of elements at risk and vulnerability and risk assessment. The implementation of the procedure for a single rock slide case is illustrated for a rock slope in Norway. Rock slides from mountain Ramnefjell to lake Loen are considered to be one of the major geohazards in Norway. Lake Loen is located in the inner part of Nordfjord in Western Norway. Ramnefjell Mountain is heavily jointed leading to formation of vertical rock slices with height between 400-450 m and width between 7-10 m. These slices threaten the settlements around Loen Valley and tourists visiting the fjord during summer season, as the released slides have potential of creating tsunami. In the past, several rock slides had been recorded from the Mountain Ramnefjell between 1905 and 1950. Among them

Teaching Quantitative Reasoning for Nonscience Majors through Carbon Footprint Analysis

ERIC Educational Resources Information Center

Boose, David L.

2014-01-01

Quantitative reasoning is a key intellectual skill, applicable across disciplines and best taught in the context of authentic, relevant problems. Here, I describe and assess a laboratory exercise that has students calculate their "carbon footprint" and evaluate the impacts of various behavior choices on that footprint. Students gather…

Video self-assessment of basic suturing and knot tying skills by novice trainees.

PubMed

Hu, Yinin; Tiemann, Debbie; Michael Brunt, L

2013-01-01

Self-assessment is important to learning but few studies have utilized video self-assessment of basic surgical skills. We compared a video self-assessment of suturing and knot tying skills by novice trainees to the assessment by a senior attending surgeon. Sixteen senior medical students and 7 beginner surgical interns were video-recorded while performing five suturing and knot tying tasks. All videos were analyzed using an objective structured assessment of technical skills (OSATS) metrics (1-5 scale; 1 = novice, 5 = expert). Video self-assessment was carried out within 4 weeks of an instructional session and subsequently by one senior surgery instructor (blinded to the individual). Both a Global score and total combined OSATS scores were analyzed. Total possible OSATS scores were: interrupted suture-30, subcuticular closure-30, one and two-handed knot tying-25 each, tying in a restricted space 20; maximum combined score-130 points). Confidence levels in performing the tasks pre-test and the value of video self-assessment were rated on a 1-5 Likert scale (1 = low and 5 = high). Data are mean±SD and statistical significance was evaluated using Friedman's test. Self-assessment scoring was significantly higher than the assessment by a senior instructor for three tasks by global score and all five tasks by combined OSATS score (self-assessment 71.8±16.7 vs attending assessment 56.7±11.0, p = 0.007). Mean self-assessment Global scores ranged from 2.5 to 2.8 for all tasks performed compared to 1.8-2.3 for attending surgeon assessment (p<0.05). Confidence levels demonstrated no correlation to performance speed or proficiency. The video self-assessment was rated as a highly valuable (mean 4.3±0.8) component to skills training. Novice trainees over-estimate their basic technical skills performance compared to the assessment by a senior surgeon. Video self-assessment may be a valuable addition to a pre-residency and surgical internship preparatory curriculum in basic

E-learning teaches attendings "how to" objectively assess pediatric urology trainees' surgery skills for orchiopexy.

PubMed

Fernandez, Nicolas; Maizels, Max; Farhat, Walid; Smith, Edwin; Liu, Dennis; Chua, Michael; Bhanji, Yasin

2018-04-01

Established methods to train pediatric urology surgery by residency training programs require updating in response to administrative changes such as new, reduced trainee duty hours. Therefore, new objective methods must be developed to teach trainees. We approached this need by creating e-learning to teach attendings objective assessment of trainee skills using the Zwisch scale, an established assessment tool. The aim of this study was to identify whether or not e-learning is an appropriate platform for effective teaching of this assessment tool, by assessing inter-rater correlation of assessments made by the attendings after participation in the e-learning. Pediatric orchiopexy was used as the index case. An e-learning tool was created to teach attending surgeons objective assessment of trainees' surgical skills. First, e-learning content was created which showed the assessment method videotape of resident surgery done in the operating room. Next, attendings were enrolled to e-learn this method. Finally, the ability of enrollees to assess resident surgery skill performance was tested. Namely, test video was made showing a trainee performing inguinal orchiopexy. All enrollees viewed the same online videos. Assessments of surgical skills (Zwisch scale) were entered into an online survey. Data were analyzed by intercorrelation coefficient kappa analysis (strong correlation was ICC ≥ 0.7). A total of 11 attendings were enrolled. All accessed the online learning and then made assessments of surgical skills trainees showed on videotapes. The e-learning comprised three modules: 1. "Core concepts," in which users learned the assessment tool methods; 2. "Learn to assess," in which users learned how to assess by watching video clips, explaining the assessment method; and 3. "Test," in which users tested their skill at making assessments by watching video clips and then actively inputting their ratings of surgical and global skills as viewed in the video clips (Figure

Video observation of procedural skills for assessment of trabeculectomy performed by residents.

PubMed

Hassanpour, Narges; Chen, Rebecca; Baikpour, Masoud; Moghimi, Sasan

2016-06-01

The efficacy and sufficiency of a healthcare system is directly related to the knowledge and skills of graduates working in the system. In this regard, many different assessment methods have been proposed to evaluate various skills of the learners. Video Observation of Procedural Skills (VOPS) is one newly-proposed method. In this study we aimed to compare the results of the VOPS method with the more commonly used Direct Observation of Procedural Skills (DOPS). In this prospective study conducted in 2012, all 10 ophthalmology residents of post graduate year 4 were selected for participation. Three months into training in the glaucoma ward, these residents performed trabeculectomy surgery on patients, and their procedural skills were assessed in real time by an expert via the DOPS method. All surgeries were also recorded and later evaluated via the VOPS method by an expert. Bland-Altman plot also was used to compare the two methods and calculating the mean and 95% limit of agreement. Residents have been done a mean of 14.9 ± 3.5 (range 10-20) independent trabeculectomy before the assessments. DOPS grade was positively associated with number of independent trabeculectomy during glaucoma rotation (β=0.227, p = 0.004). The intra-observer reproducibility of VOPS measurements was 0.847 (95% CI: 0.634, 0.961). The mean VOPS grade was significantly lower than the mean DOPS grade (8.4 vs. 8.9, p = 0.02). However, a good correlation was observed between the grades of VOPS and DOPS (r = 0.89, p = 0.001). Bland-Altman analysis demonstrated that all data points fell within the 95% limits of agreement (-1.46, 0.46). The present study showed that VOPS might be considered a feasible, valid, and reliable assessment method for procedural skills of medical students and residents that can be used as an alternative to the DOPS method. However, VOPS might underestimate DOPS in evaluating surgical skills of residents.

Reliable and valid assessment of Lichtenstein hernia repair skills.

PubMed

Carlsen, C G; Lindorff-Larsen, K; Funch-Jensen, P; Lund, L; Charles, P; Konge, L

2014-08-01

Lichtenstein hernia repair is a common surgical procedure and one of the first procedures performed by a surgical trainee. However, formal assessment tools developed for this procedure are few and sparsely validated. The aim of this study was to determine the reliability and validity of an assessment tool designed to measure surgical skills in Lichtenstein hernia repair. Key issues were identified through a focus group interview. On this basis, an assessment tool with eight items was designed. Ten surgeons and surgical trainees were video recorded while performing Lichtenstein hernia repair, (four experts, three intermediates, and three novices). The videos were blindly and individually assessed by three raters (surgical consultants) using the assessment tool. Based on these assessments, validity and reliability were explored. The internal consistency of the items was high (Cronbach's alpha = 0.97). The inter-rater reliability was very good with an intra-class correlation coefficient (ICC) = 0.93. Generalizability analysis showed a coefficient above 0.8 even with one rater. The coefficient improved to 0.92 if three raters were used. One-way analysis of variance found a significant difference between the three groups which indicates construct validity, p < 0.001. Lichtenstein hernia repair skills can be assessed blindly by a single rater in a reliable and valid fashion with the new procedure-specific assessment tool. We recommend this tool for future assessment of trainees performing Lichtenstein hernia repair to ensure that the objectives of competency-based surgical training are met.

How can teachers assess reading skills of generation z learners in German language class?

NASA Astrophysics Data System (ADS)

Dirga, R. N.; Wijayati, P. H.

2018-01-01

Generation Z is a digital native generation who has unique characteristics on the daily basis includes reading. In order to assess their reading skills correctly, teachers need to take these characteristics into consideration. This paper aims to describe the process in developing an instrument to assess reading skills of Generation Z learners in German language class. This study used developmental method. The developed instrument has not only good quality but also consists of texts that are suitable for German learners of Generation Z. This instrument can be used as an example in assessing German learners’ reading skills in the 21st century.

Development and Application of a Stepwise Assessment Process for Rational Redesign of Sequential Skills-Based Courses.

PubMed

Gallimore, Casey E; Porter, Andrea L; Barnett, Susanne G

2016-10-25

Objective. To develop and apply a stepwise process to assess achievement of course learning objectives related to advanced pharmacy practice experiences (APPEs) preparedness and inform redesign of sequential skills-based courses. Design. Four steps comprised the assessment and redesign process: (1) identify skills critical for APPE preparedness; (2) utilize focus groups and course evaluations to determine student competence in skill performance; (3) apply course mapping to identify course deficits contributing to suboptimal skill performance; and (4) initiate course redesign to target exposed deficits. Assessment. Focus group participants perceived students were least prepared for skills within the Accreditation Council for Pharmacy Education's pre-APPE core domains of Identification and Assessment of Drug-related Problems and General Communication Abilities. Course mapping identified gaps in instruction, performance, and assessment of skills within aforementioned domains. Conclusions. A stepwise process that identified strengths and weaknesses of a course, was used to facilitate structured course redesign. Strengths of the process included input and corroboration from both preceptors and students. Limitations included feedback from a small number of pharmacy preceptors and increased workload on course coordinators.

Barriers of physical assessment skills among nursing students in Arab Peninsula

PubMed Central

Alamri, Majed Sulaiman; Almazan, Joseph U.

2018-01-01

Objective: There is a growing demand for health-care nursing services in several health care institutions. Understanding barriers to physical assessment among nursing students create a more detailed assessment in the development of quality patient’s care in nursing practice. This study examined the barriers to physical assessment skills among nursing students in a government university in Arab Peninsula. Methods: A cross-sectional research survey design of 206 nursing students using a standardized questionnaire was used. The questionnaire is composed of 7 subscales in evaluating the barriers to physical assessment skills between the classroom and clinical setting. Independent Samples t-test was used in comparing the gender mean of the nursing students about the barriers to physical assessment. Paired t-test was also used in determining the differences between perceived barriers to physical assessment in the classroom and clinical setting. Results: Subscale “reliance on others and technology,” ward culture, “lack of influence on patient care” have significant differences between perceived barriers in physical assessment among classroom settings and clinical setting. Conclusion: Although nursing students were oriented and educated about physical assessment in the nursing curriculum, this is not often practiced in clinical settings. The point that is if nursing students are incorrectly performing the patient assessment, then no amount of critical thinking could lead to better clinical decisions. Continuous exposure and enhancing the quality of planning and promotion of the nursing students could develop necessary skills. In addition, increasing self-confidence is vital to assess the patient’s health status effectively and minimize the barriers to performing the physical assessment. PMID:29896073

Gender-fair assessment of young gifted students' scientific thinking skills

NASA Astrophysics Data System (ADS)

Dori, Y. J.; Zohar, A.; Fischer-Shachor, D.; Kohan-Mass, J.; Carmi, M.

2018-04-01

This paper describes an Israeli national-level research examining the extent to which admissions of elementary school students to the gifted programmes based on standardised tests are gender-fair. In the research, the gifted students consisted of 275 boys, 128 girls, and additional 80 girls who were admitted to the gifted programme through affirmative action (AA). To assess these young students' scientific thinking skills, also referred to as science practices, open-ended questions of case-based questionnaires were developed. The investigated scientific thinking skills were question posing, explanation, graphing, inquiry, and metacognition. Analysis of the students' responses revealed that gifted girls who entered the programmes through AA performed at the same level as the other gifted students. We found significant differences between the three research groups in question posing and graphing skills. We suggest increasing gender-fairness by revising the standard national testing system to include case-based narratives followed by open-ended questions that assess gifted students' scientific thinking skills. This may diminish the gender inequity expressed by the different number of girls and boys accepted to the gifted programmes. We show that open-ended tools for analysing students' scientific thinking might better serve both research and practice by identifying gifted girls and boys equally well.

Haptic feedback can provide an objective assessment of arthroscopic skills.

PubMed

Chami, George; Ward, James W; Phillips, Roger; Sherman, Kevin P

2008-04-01

The outcome of arthroscopic procedures is related to the surgeon's skills in arthroscopy. Currently, evaluation of such skills relies on direct observation by a surgeon trainer. This type of assessment, by its nature, is subjective and time-consuming. The aim of our study was to identify whether haptic information generated from arthroscopic tools could distinguish between skilled and less skilled surgeons. A standard arthroscopic probe was fitted with a force/torque sensor. The probe was used by five surgeons with different levels of experience in knee arthroscopy performing 11 different tasks in 10 standard knee arthroscopies. The force/torque data from the hand and tool interface were recorded and synchronized with a video recording of the procedure. The torque magnitude and patterns generated were analyzed and compared. A computerized system was used to analyze the force/torque signature based on general principles for quality of performance using such measures as economy in movement, time efficiency, and consistency in performance. The results showed a considerable correlation between three haptic parameters and the surgeon's experience, which could be used in an automated objective assessment system for arthroscopic surgery. Level II, diagnostic study. See the Guidelines for Authors for a complete description of levels of evidence.

Workplace-based assessment of communication skills: A pilot project addressing feasibility, acceptance and reliability

PubMed Central

Weyers, Simone; Jemi, Iman; Karger, André; Raski, Bianca; Rotthoff, Thomas; Pentzek, Michael; Mortsiefer, Achim

2016-01-01

Background: Imparting communication skills has been given great importance in medical curricula. In addition to standardized assessments, students should communicate with real patients in actual clinical situations during workplace-based assessments and receive structured feedback on their performance. The aim of this project was to pilot a formative testing method for workplace-based assessment. Our investigation centered in particular on whether or not physicians view the method as feasible and how high acceptance is among students. In addition, we assessed the reliability of the method. Method: As part of the project, 16 students held two consultations each with chronically ill patients at the medical practice where they were completing GP training. These consultations were video-recorded. The trained mentoring physician rated the student’s performance and provided feedback immediately following the consultations using the Berlin Global Rating scale (BGR). Two impartial, trained raters also evaluated the videos using BGR. For qualitative and quantitative analysis, information on how physicians and students viewed feasibility and their levels of acceptance was collected in written form in a partially standardized manner. To test for reliability, the test-retest reliability was calculated for both of the overall evaluations given by each rater. The inter-rater reliability was determined for the three evaluations of each individual consultation. Results: The formative assessment method was rated positively by both physicians and students. It is relatively easy to integrate into daily routines. Its significant value lies in the personal, structured and recurring feedback. The two overall scores for each patient consultation given by the two impartial raters correlate moderately. The degree of uniformity among the three raters in respect to the individual consultations is low. Discussion: Within the scope of this pilot project, only a small sample of physicians and

Workplace-based assessment of communication skills: A pilot project addressing feasibility, acceptance and reliability.

PubMed

Weyers, Simone; Jemi, Iman; Karger, André; Raski, Bianca; Rotthoff, Thomas; Pentzek, Michael; Mortsiefer, Achim

2016-01-01

Background: Imparting communication skills has been given great importance in medical curricula. In addition to standardized assessments, students should communicate with real patients in actual clinical situations during workplace-based assessments and receive structured feedback on their performance. The aim of this project was to pilot a formative testing method for workplace-based assessment. Our investigation centered in particular on whether or not physicians view the method as feasible and how high acceptance is among students. In addition, we assessed the reliability of the method. Method: As part of the project, 16 students held two consultations each with chronically ill patients at the medical practice where they were completing GP training. These consultations were video-recorded. The trained mentoring physician rated the student's performance and provided feedback immediately following the consultations using the Berlin Global Rating scale (BGR). Two impartial, trained raters also evaluated the videos using BGR. For qualitative and quantitative analysis, information on how physicians and students viewed feasibility and their levels of acceptance was collected in written form in a partially standardized manner. To test for reliability, the test-retest reliability was calculated for both of the overall evaluations given by each rater. The inter-rater reliability was determined for the three evaluations of each individual consultation. Results: The formative assessment method was rated positively by both physicians and students. It is relatively easy to integrate into daily routines. Its significant value lies in the personal, structured and recurring feedback. The two overall scores for each patient consultation given by the two impartial raters correlate moderately. The degree of uniformity among the three raters in respect to the individual consultations is low. Discussion: Within the scope of this pilot project, only a small sample of physicians and

Some suggested future directions of quantitative resource assessments

USGS Publications Warehouse

Singer, D.A.

2001-01-01

Future quantitative assessments will be expected to estimate quantities, values, and locations of undiscovered mineral resources in a form that conveys both economic viability and uncertainty associated with the resources. Historically, declining metal prices point to the need for larger deposits over time. Sensitivity analysis demonstrates that the greatest opportunity for reducing uncertainty in assessments lies in lowering uncertainty associated with tonnage estimates. Of all errors possible in assessments, those affecting tonnage estimates are by far the most important. Selecting the correct deposit model is the most important way of controlling errors because the dominance of tonnage-deposit models are the best known predictor of tonnage. Much of the surface is covered with apparently barren rocks and sediments in many large regions. Because many exposed mineral deposits are believed to have been found, a prime concern is the presence of possible mineralized rock under cover. Assessments of areas with resources under cover must rely on extrapolation from surrounding areas, new geologic maps of rocks under cover, or analogy with other well-explored areas that can be considered training tracts. Cover has a profound effect on uncertainty and on methods and procedures of assessments because geology is seldom known and geophysical methods typically have attenuated responses. Many earlier assessment methods were based on relationships of geochemical and geophysical variables to deposits learned from deposits exposed on the surface-these will need to be relearned based on covered deposits. Mineral-deposit models are important in quantitative resource assessments for two reasons: (1) grades and tonnages of most deposit types are significantly different, and (2) deposit types are present in different geologic settings that can be identified from geologic maps. Mineral-deposit models are the keystone in combining the diverse geoscience information on geology, mineral

Toward Quantitative Small Animal Pinhole SPECT: Assessment of Quantitation Accuracy Prior to Image Compensations

PubMed Central

Chen, Chia-Lin; Wang, Yuchuan; Lee, Jason J. S.; Tsui, Benjamin M. W.

2011-01-01

Purpose We assessed the quantitation accuracy of small animal pinhole single photon emission computed tomography (SPECT) under the current preclinical settings, where image compensations are not routinely applied. Procedures The effects of several common image-degrading factors and imaging parameters on quantitation accuracy were evaluated using Monte-Carlo simulation methods. Typical preclinical imaging configurations were modeled, and quantitative analyses were performed based on image reconstructions without compensating for attenuation, scatter, and limited system resolution. Results Using mouse-sized phantom studies as examples, attenuation effects alone degraded quantitation accuracy by up to −18% (Tc-99m or In-111) or −41% (I-125). The inclusion of scatter effects changed the above numbers to −12% (Tc-99m or In-111) and −21% (I-125), respectively, indicating the significance of scatter in quantitative I-125 imaging. Region-of-interest (ROI) definitions have greater impacts on regional quantitation accuracy for small sphere sources as compared to attenuation and scatter effects. For the same ROI, SPECT acquisitions using pinhole apertures of different sizes could significantly affect the outcome, whereas the use of different radii-of-rotation yielded negligible differences in quantitation accuracy for the imaging configurations simulated. Conclusions We have systematically quantified the influence of several factors affecting the quantitation accuracy of small animal pinhole SPECT. In order to consistently achieve accurate quantitation within 5% of the truth, comprehensive image compensation methods are needed. PMID:19048346

Critical Emergency Medicine Procedural Skills: A Comparative Study of Methods for Teaching and Assessment.

ERIC Educational Resources Information Center

Chapman, Dane M.; And Others

Three critical procedural skills in emergency medicine were evaluated using three assessment modalities--written, computer, and animal model. The effects of computer practice and previous procedure experience on skill competence were also examined in an experimental sequential assessment design. Subjects were six medical students, six residents,…

Computer-enhanced visual learning method: a paradigm to teach and document surgical skills.

PubMed

Maizels, Max; Mickelson, Jennie; Yerkes, Elizabeth; Maizels, Evelyn; Stork, Rachel; Young, Christine; Corcoran, Julia; Holl, Jane; Kaplan, William E

2009-09-01

Changes in health care are stimulating residency training programs to develop new methods for teaching surgical skills. We developed Computer-Enhanced Visual Learning (CEVL) as an innovative Internet-based learning and assessment tool. The CEVL method uses the educational procedures of deliberate practice and performance to teach and learn surgery in a stylized manner. CEVL is a learning and assessment tool that can provide students and educators with quantitative feedback on learning a specific surgical procedure. Methods involved examine quantitative data of improvement in surgical skills. Herein, we qualitatively describe the method and show how program directors (PDs) may implement this technique in their residencies. CEVL allows an operation to be broken down into teachable components. The process relies on feedback and remediation to improve performance, with a focus on learning that is applicable to the next case being performed. CEVL has been shown to be effective for teaching pediatric orchiopexy and is being adapted to additional adult and pediatric procedures and to office examination skills. The CEVL method is available to other residency training programs.

Computer-Enhanced Visual Learning Method: A Paradigm to Teach and Document Surgical Skills

PubMed Central

Maizels, Max; Mickelson, Jennie; Yerkes, Elizabeth; Maizels, Evelyn; Stork, Rachel; Young, Christine; Corcoran, Julia; Holl, Jane; Kaplan, William E.

2009-01-01

Innovation Changes in health care are stimulating residency training programs to develop new methods for teaching surgical skills. We developed Computer-Enhanced Visual Learning (CEVL) as an innovative Internet-based learning and assessment tool. The CEVL method uses the educational procedures of deliberate practice and performance to teach and learn surgery in a stylized manner. Aim of Innovation CEVL is a learning and assessment tool that can provide students and educators with quantitative feedback on learning a specific surgical procedure. Methods involved examine quantitative data of improvement in surgical skills. Herein, we qualitatively describe the method and show how program directors (PDs) may implement this technique in their residencies. Results CEVL allows an operation to be broken down into teachable components. The process relies on feedback and remediation to improve performance, with a focus on learning that is applicable to the next case being performed. CEVL has been shown to be effective for teaching pediatric orchiopexy and is being adapted to additional adult and pediatric procedures and to office examination skills. The CEVL method is available to other residency training programs. PMID:21975716

Distinguishing nanomaterial particles from background airborne particulate matter for quantitative exposure assessment

NASA Astrophysics Data System (ADS)

Ono-Ogasawara, Mariko; Serita, Fumio; Takaya, Mitsutoshi

2009-10-01

As the production of engineered nanomaterials quantitatively expands, the chance that workers involved in the manufacturing process will be exposed to nanoparticles also increases. A risk management system is needed for workplaces in the nanomaterial industry based on the precautionary principle. One of the problems in the risk management system is difficulty of exposure assessment. In this article, examples of exposure assessment in nanomaterial industries are reviewed with a focus on distinguishing engineered nanomaterial particles from background nanoparticles in workplace atmosphere. An approach by JNIOSH (Japan National Institute of Occupational Safety and Health) to quantitatively measure exposure to carbonaceous nanomaterials is also introduced. In addition to real-time measurements and qualitative analysis by electron microscopy, quantitative chemical analysis is necessary for quantitatively assessing exposure to nanomaterials. Chemical analysis is suitable for quantitative exposure measurement especially at facilities with high levels of background NPs.

Assessing 21st Century Skills: A Guide to Evaluating Mastery and Authentic Learning

ERIC Educational Resources Information Center

Greenstein, Laura

2012-01-01

The Common Core State Standards clearly define the skills students need for success in college and the 21st century workplace. The question is, how can you measure student mastery of skills like creativity, problem solving, and use of technology? Laura Greenstein demonstrates how teachers can teach and assess 21st century skills using authentic…

Teaching and assessment of communication skills in undergraduate dental education - a survey in German-speaking countries.

PubMed

Rüttermann, S; Sobotta, A; Hahn, P; Kiessling, C; Härtl, A

2017-08-01

Teaching communication is perceived to be of importance in dental education. Several reports have been published worldwide in the educational literature describing modifications of the dental curriculum by implementing the teaching of communication skills. Surveys which evaluate the current state of training and assessment of communication skills in dental education in different countries exist already in some countries, but little information is available about German-speaking countries. In a cross-sectional study with the aim of a census, all 36 dental schools in Germany (30), Austria (3), and Switzerland (3) were surveyed. The present survey revealed that at 26 of the 34 dental schools (76%), communication skills training has been implemented. Training of communication skills mainly takes place between the 6th and the 9th semester. Ten schools were able to implement a partly longitudinal curriculum, while the other sites only offer stand-alone courses. Of the 34 dental schools, six assess communication skills in a summative way. Three of those schools also use formative assessments for their students. Another seven sites only use formative assessment. From the various formats of assessment, OSCE is mentioned most frequently. The necessity to train and assess communication skills has reached German-speaking dental schools. The present survey allows an overview of the training and assessment of communication skills in undergraduate dental education in German-speaking Europe. © 2016 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Identifying Academic Skill and Performance Deficits: The Experimental Analysis of Brief Assessments of Academic Skills. General Articles

ERIC Educational Resources Information Center

Duhon, Gary J.; Noell, George H.; Witt, Joseph C.; Freeland, Jennifer T.; Dufrene, Brad A.; Gilbertson, Donna N.

2004-01-01

This study examined an approach for assessing and intervening with academic concerns that is conceptually derived from the distinction between skill deficits and performance deficits. A class-wide assessment was used to describe the students' functioning in the context of their peers and to help select a target concern for intervention. An…

To Assess Students' Attitudes, Skills and Competencies in Mathematical Modeling

ERIC Educational Resources Information Center

Lingefjard, Thomas; Holmquist, Mikael

2005-01-01

Peer-to-peer assessment, take-home exams and a mathematical modeling survey were used to monitor and assess students' attitudes, skills and competencies in mathematical modeling. The students were all in a secondary mathematics, teacher education program with a comprehensive amount of mathematics studies behind them. Findings indicate that…

A Multidimensional Needs Assessment of Social Emotional Learning Skill Areas

ERIC Educational Resources Information Center

Yopp, Ashley; McKim, Billy R.; Moore, Lori L.; Odom, Summer F.; Hanagriff, Roger

2017-01-01

Social and Emotional Learning (SEL) has often been an umbrella term for a wide range of competencies, including emotional processes, social and interpersonal skills, and cognitive regulation (Jones, Bouffard, & Weissbourd, 2013). We used the Borich (1980) needs assessment model to assess the professional development needs of Texas agricultural…

Nursing physical assessment for patient safety in general wards: reaching consensus on core skills.

PubMed

Douglas, Clint; Booker, Catriona; Fox, Robyn; Windsor, Carol; Osborne, Sonya; Gardner, Glenn

2016-07-01

To determine consensus across acute care specialty areas on core physical assessment skills necessary for early recognition of changes in patient status in general wards. Current approaches to physical assessment are inconsistent and have not evolved to meet increased patient and system demands. New models of nursing assessment are needed in general wards that ensure a proactive and patient safety approach. A modified Delphi study. Focus group interviews with 150 acute care registered nurses at a large tertiary referral hospital generated a framework of core skills that were developed into a web-based survey. We then sought consensus with a panel of 35 senior acute care registered nurses following a classical Delphi approach over three rounds. Consensus was predefined as at least 80% agreement for each skill across specialty areas. Content analysis of focus group transcripts identified 40 discrete core physical assessment skills. In the Delphi rounds, 16 of these were consensus validated as core skills and were conceptually aligned with the primary survey: (Airway) Assess airway patency; (Breathing) Measure respiratory rate, Evaluate work of breathing, Measure oxygen saturation; (Circulation) Palpate pulse rate and rhythm, Measure blood pressure by auscultation, Assess urine output; (Disability) Assess level of consciousness, Evaluate speech, Assess for pain; (Exposure) Measure body temperature, Inspect skin integrity, Inspect and palpate skin for signs of pressure injury, Observe any wounds, dressings, drains and invasive lines, Observe ability to transfer and mobilise, Assess bowel movements. Among a large and diverse group of experienced acute care registered nurses consensus was achieved on a structured core physical assessment to detect early changes in patient status. Although further research is needed to refine the model, clinical application should promote systematic assessment and clinical reasoning at the bedside. © 2016 John Wiley & Sons Ltd.

ASK Standards: Assessment, Skills, and Knowledge Content Standards for Student Affairs Practitioners and Scholars

ERIC Educational Resources Information Center

ACPA College Student Educators International, 2011

2011-01-01

The Assessment Skills and Knowledge (ASK) standards seek to articulate the areas of content knowledge, skill and dispositions that student affairs professionals need in order to perform as practitioner-scholars to assess the degree to which students are mastering the learning and development outcomes the professionals intend. Consistent with…

Using Model-Tracing to Conduct Performance Assessment of Students' Inquiry Skills within a Microworld

ERIC Educational Resources Information Center

Gobert, Janice D.; Koedinger, Kenneth R.

2011-01-01

The National frameworks for science emphasize inquiry skills (NRC, 1996), however, in typical classroom practice, science learning often focuses on rote learning in part because science process skills are difficult to assess (Fadel, Honey, & Pasnick, 2007) and rote knowledge is prioritized on high-stakes tests. Short answer assessments of…

Computational Skills for Biology Students

ERIC Educational Resources Information Center

Gross, Louis J.

2008-01-01

This interview with Distinguished Science Award recipient Louis J. Gross highlights essential computational skills for modern biology, including: (1) teaching concepts listed in the Math & Bio 2010 report; (2) illustrating to students that jobs today require quantitative skills; and (3) resources and materials that focus on computational skills.

Assessing Quantitative Literacy in Higher Education: An Overview of Existing Research and Assessments with Recommendations for Next-Generation Assessment. Research Report. ETS RR-14-22

ERIC Educational Resources Information Center

Roohr, Katrina Crotts; Graf, Edith Aurora; Liu, Ou Lydia

2014-01-01

Quantitative literacy has been recognized as an important skill in the higher education and workforce communities, focusing on problem solving, reasoning, and real-world application. As a result, there is a need by various stakeholders in higher education and workforce communities to evaluate whether college students receive sufficient training on…

The effect of video review of resident laparoscopic surgical skills measured by self- and external assessment.

PubMed

Herrera-Almario, Gabriel E; Kirk, Katherine; Guerrero, Veronica T; Jeong, Kwonho; Kim, Sara; Hamad, Giselle G

2016-02-01

Video review of surgical skills is an educational modality that allows trainees to reflect on self-performance. The purpose of this study was to determine whether resident and attending assessments of a resident's laparoscopic performance differ and whether video review changes assessments. Third-year surgery residents were invited to participate. Elective laparoscopic procedures were video recorded. The Global Operative Assessment of Laparoscopic Skills evaluation was completed immediately after the procedure and again 7 to 10 days later by both resident and attending. Scores were compared using t tests. Nine residents participated and 76 video reviews were completed. Residents scored themselves significantly lower than the faculty scores both before and after video review. Resident scores did not change significantly after video review. Attending and resident self-assessment of laparoscopic skills differs and subsequent video review does not significantly affect Global Operative Assessment of Laparoscopic Skills scores. Further studies should evaluate the impact of video review combined with verbal feedback on skill acquisition and assessment. Copyright © 2016 Elsevier Inc. All rights reserved.

Project RAILS: Lessons Learned about Rubric Assessment of Information Literacy Skills

ERIC Educational Resources Information Center

Belanger, Jackie; Zou, Ning; Mills, Jenny Rushing; Holmes, Claire; Oakleaf, Megan

2015-01-01

Rubric assessment of information literacy is an important tool for librarians seeking to show evidence of student learning. The authors, who collaborated on the Rubric Assessment of Informational Literacy Skills (RAILS) research project, draw from their shared experience to present practical recommendations for implementing rubric assessment in a…

A Calculus of Occupational Skill Attainment: Building More Validity into a Valid Assessment System

ERIC Educational Resources Information Center

Munyofu, Paul; Kohr, Richard

2009-01-01

This study investigated several aspects of occupational skill assessment as implemented in one state: (1) What is the extent to which student achievement on the cognitive component was related to their achievement on the psychomotor component of the technical skill assessments? (2) How efficiently was their overall composite attainment calculated?…

Development of a Social Skills Assessment Screening Scale for Psychiatric Rehabilitation Settings: A Pilot Study.

PubMed

Bhola, Poornima; Basavarajappa, Chethan; Guruprasad, Deepti; Hegde, Gayatri; Khanam, Fatema; Thirthalli, Jagadisha; Chaturvedi, Santosh K

2016-01-01

Deficits in social skills may present in a range of psychiatric disorders, particularly in the more serious and persistent conditions, and have an influence on functioning across various domains. This pilot study aimed at developing a brief measure, for structured evaluation and screening for social skills deficits, which can be easily integrated into routine clinical practice. The sample consisted of 380 inpatients and their accompanying caregivers, referred to Psychiatric Rehabilitation Services at a tertiary care government psychiatric hospital. The evaluation included an Inpatient intake Proforma and the 20-item Social Skills Assessment Screening Scale (SSASS). Disability was assessed using the Indian Disability Evaluation and Assessment Scale (IDEAS) for a subset of 94 inpatients. The analysis included means and standard deviations, frequency and percentages, Cronbach's alpha to assess internal consistency, t -tests to assess differences in social skills deficits between select subgroups, and correlation between SSASS and IDEAS scores. The results indicated the profile of social skills deficits assessed among the inpatients with varied psychiatric diagnoses. The "psychosis" group exhibited significantly higher deficits than the "mood disorder" group. Results indicated high internal consistency of the SSASS and adequate criterion validity demonstrated by correlations with select IDEAS domains. Modifications were made to the SSASS following the pilot study. The SSASS has potential value as a measure for screening and individualised intervention plans for social skills training in mental health and rehabilitation settings. The implications for future work on the psychometric properties and clinical applications are discussed.

The development of a rubric for peer assessment of individual teamwork skills in undergraduate midwifery students.

PubMed

Hastie, Carolyn; Fahy, Kathleen; Parratt, Jenny

2014-09-01

Poor teamwork is cited as one of the major root causes of adverse events in healthcare. Bullying, resulting in illness for staff, is an expression of poor teamwork skills. Despite this knowledge, poor teamwork persists in healthcare and teamwork skills are rarely the focus of teaching and assessment in undergraduate health courses. To develop and implement an assessment tool for use in facilitating midwifery students' learning of teamwork skills. This paper describes how the TeamUP rubric tool was developed. A review of the literature found no research reports on how to teach and assess health students' teamwork skills in standing teams. The literature, however, gives guidance about how university educators should evaluate individual students using peer assessment. The developmental processes of the rubric were grounded in the theoretical literature and feminist collaborative conversations. The rubric incorporates five domains of teamwork skills: Fostering a Team Climate; Project Planning; Facilitating Teams; Managing Conflict and Quality Individual Contribution. The process and outcomes of student and academic content validation are described. The TeamUP rubric is useful for articulating, teaching and assessing teamwork skills for health professional students. The TeamUP rubric is a robust, theoretically grounded model that defines and details effective teamwork skills and related behaviours. If these skills are mastered, we predict that graduates will be more effective in teams. Our assumption is that graduates, empowered by having these skills, are more likely to manage conflict effectively and less likely to engage in bullying behaviours. Crown Copyright © 2014. Published by Elsevier Ltd. All rights reserved.

The Impact of Diagnosing Skill Deficiencies and Assessment-Based Communication Training on Managerial Performance.

ERIC Educational Resources Information Center

Papa, Michael J.; Graham, Elizabeth E.

1991-01-01

Evaluates an organizational diagnosis program that assesses managerial communication skills and provides the frame for follow-up training programs. Finds that managers participating in follow-up communication skills training performed significantly higher on interpersonal skills, problem-solving ability, and productivity over three long-term…

Outcomes of Individualized Formative Assessments in a Pharmacy Skills Laboratory.

PubMed

Gums, Tyler Harris; Kleppinger, Erika L; Urick, Benjamin Y

2014-11-15

To determine the effectiveness of an individualized teaching method in a pharmacy skills laboratory. All third-year students enrolled in an Accreditation Council for Pharmacy Education (ACPE) accredited doctor of pharmacy program (n=150) received an individual formative assessment from clinical pharmacists on communication skills and clinical competency after the students counseled standardized mock glaucoma patients during a laboratory focused on alternative dosing formulations. Objective structured clinical examination (OSCE) scores for this station from the 2012 and 2013 classes were compared before and after the intervention. Ophthalmic OSCE station scores were higher after the individual formative feedback intervention. Students in 2013 had a mean score of 83.2 ± 8.3% compared to a mean of 74.3 ± 12.9% in 2012 for this OSCE station. The percentage of students receiving an "A" on the OSCE station increased from 8.1% to 31.3% after the intervention. Individualized formative teaching methods benefited students in both their communication skills and clinical assessment. Future research should focus on wider implementation and overcoming obstacles, such as increased facilitator needs.

Development of a tool to support holistic generic assessment of clinical procedure skills.

PubMed

McKinley, Robert K; Strand, Janice; Gray, Tracey; Schuwirth, Lambert; Alun-Jones, Tom; Miller, Helen

2008-06-01

The challenges of maintaining comprehensive banks of valid checklists make context-specific checklists for assessment of clinical procedural skills problematic. This paper reports the development of a tool which supports generic holistic assessment of clinical procedural skills. We carried out a literature review, focus groups and non-participant observation of assessments with interview of participants, participant evaluation of a pilot objective structured clinical examination (OSCE), a national modified Delphi study with prior definitions of consensus and an OSCE. Participants were volunteers from a large acute teaching trust, a teaching primary care trust and a national sample of National Health Service staff. Results In total, 86 students, trainees and staff took part in the focus groups, observation of assessments and pilot OSCE, 252 in the Delphi study and 46 candidates and 50 assessors in the final OSCE. We developed a prototype tool with 5 broad categories amongst which were distributed 38 component competencies. There was > 70% agreement (our prior definition of consensus) at the first round of the Delphi study for inclusion of all categories and themes and no consensus for inclusion of additional categories or themes. Generalisability was 0.76. An OSCE based on the instrument has a predicted reliability of 0.79 with 12 stations and 1 assessor per station or 10 stations and 2 assessors per station. This clinical procedural skills assessment tool enables reliable assessment and has content and face validity for the assessment of clinical procedural skills. We have designated it the Leicester Clinical Procedure Assessment Tool (LCAT).

Assessing cultural competency skills in gastroenterology fellowship training.

PubMed

Balzora, Sophie; Abiri, Benjamin; Wang, Xiao-Jing; McKeever, James; Poles, Michael; Zabar, Sondra; Gillespie, Colleen; Weinshel, Elizabeth

2015-02-14

To assess and teach cultural competency skills at the fellowship training level through the use of objective structured clinical examinations (OSCEs). We revised four scenarios to infuse a specific focus on cross-cultural care, and to render them appropriate for gastroenterology fellows. Three are discussed here: (1) Poor Health Literacy; (2) Disclosing/Apologizing for a Complication to a Patient Who Mistrusts the Healthcare System; and (3) Breaking Bad News to a Fatalistic Patient. A fourth case emphasizing shared decision-making will be described elsewhere. Four stations were completed by fellows and observed live by four faculty members, and the fellows' performance was assessed. Eleven fellows from four programs participated in the four OSCE. In the "Poor Health Literacy" case, 18% (2/11) of participants recognized that the standardized patient (SP) had below-basic health literacy. None successfully evaluated the SP's reading skills in a culturally-sensitive manner. In "Disclosing/Apologizing for a Complication", 4/11 (36%) personally apologized for the complication. 1/11 recognized the SP's mistrust of the medical system. With "Breaking Bad News", 27% (3/11) explored the patient's values to identify her fatalistic beliefs. OSCEs can be used to assess deficiencies in culturally-competent care at the fellowship level. OSCEs also afford fellowships the opportunity to inform future training curricula.

Two approaches to improving mental health care: positivist/quantitative versus skill-based/qualitative.

PubMed

Luchins, Daniel

2012-01-01

The quality improvement model currently used in medicine and mental health was adopted from industry, where it developed out of early 20th-century efforts to apply a positivist/quantitative agenda to improving manufacturing. This article questions the application of this model to mental health care. It argues that (1) developing "operational definitions" for something as value-laden as "quality" risks conflating two realms, what we measure with what we value; (2) when measurements that are tied to individuals are aggregated to establish benchmarks and goals, unwarranted mathematical assumptions are made; (3) choosing clinical outcomes is problematic; (4) there is little relationship between process measures and clinical outcomes; and (5) since changes in quality indices do not relate to improved clinical care, management's reliance on such indices provides an illusory sense of control. An alternative model is the older, skill-based/qualitative approach to knowing, which relies on "implicit/ expert" knowledge. These two approaches offer a series of contrasts: quality versus excellence, competence versus expertise, management versus leadership, extrinsic versus intrinsic rewards. The article concludes that we need not totally dispense with the current quality improvement model, but rather should balance quantitative efforts with the older qualitative approach in a mixed methods model.

Validation of the Family Meeting Behavioral Skills Checklist. An Instrument to Assess Fellows' Communication Skills.

PubMed

Gustin, Jillian L; Way, David P; Wells-Di Gregorio, Sharla; McCallister, Jennifer W

2016-08-01

Fellows in pulmonary and critical care medicine are required to show competency in facilitating family meetings for critically ill patients. There are many assessment measures available for evaluating physician-patient communication (e.g., the SEGUE Framework [Set the stage, Elicit information, Give information, Understand the patient's perspective, End the encounter]) and some designed for family meetings. However, no validated measure exists that is specifically designed to assess communication skills during family meetings with surrogate decision makers in intensive care settings. We developed the Family Meeting Behavioral Skills Checklist (FMBSC) to measure advanced communication skills of fellows in family meetings of critically ill patients based on a literature review and consensus of an interdisciplinary group of communications experts. We evaluated the psychometric properties of the FMBSC. We digitally recorded 16 pulmonary/critical care fellows performing a simulated family meeting for a critically ill patient at the end of 1 year of fellowship training. Two clinical health psychologists evaluated each recording independently using the FMBSC Rating Scale and the SEGUE Framework. Judges recorded the number of skills performed using the checklist and employed a summary rating scale to judge the level of performance for each of nine subsets of skills. Each instrument was scored and converted to percentage scores. The FMBSC and SEGUE Framework items were summed and converted to percentage scores for each category and as a total for each instrument. The rating scale items on the FMBSC were also summed and converted to a percentage score. Four primary analyses were conducted to evaluate interjudge reliability, internal consistency, and concurrent validity. Interrater reliability was higher for the FMBSC (intraclass correlation [ICC2,2] = 0.57) than for the SEGUE instrument (ICC2,2 = 0.32) or the FMBSC Rating Scale (ICC2,2 = 0.23). The FMBSC

Assessment of communication, professionalism, and surgical skills in an objective structured performance-related examination (OSPRE): a psychometric study.

PubMed

Ponton-Carss, Alicia; Hutchison, Carol; Violato, Claudio

2011-10-01

The purpose of this study was to investigate the reliability and validity of a performance assessment of communication, professionalism, and surgical skills competencies for surgery residents. Fourteen residents from the general surgery program of the University of Calgary were assessed in 7 surgical simulation stations that included communication and professionalism skills. The internal consistency reliability of the checklists and global rating scales combined was adequate for communication (α = .75-.92) and surgical skills (α = .86-.96), but not for professionalism (α = 0). There was evidence of validity as surgical skills performance improved as a function of postgraduate year level but not for the professionalism checklist. Surgical skills and communication correlated in the 2 stations assessed (r = .55 and .57; P < .05). There is evidence for both reliability and validity for simultaneously assessing surgical skills and communication skills. Further instrument development is required to assess professionalism in a structured examination context. Copyright © 2011 Elsevier Inc. All rights reserved.

Possibilities of the ErgoScope high fidelity work simulator in skill assessment, skill development and vocational aptitude tests of physically disabled persons.

PubMed

Izsó, Lajos; Székely, Ildikó; Dános, László

2015-01-01

The aim of this paper - based on the extensive experiences of the authors gained by using one particular work simulator - is to present some promising possibilities of the application of this (and any other similar) work simulator in the field of skill assessment, skill development and vocational aptitude tests of physically disabled persons. During skill assessment and development, as parts of the therapy, the focus is on the disabled functions. During vocational aptitude tests, however, the focus is already on the functions that remained intact and therefore can be the basis of returning to work. Some factual examples are provided to realize the proposed possibilities in practice.

A Comparative Study of Student Math Skills: Perceptions, Validation, and Recommendations

ERIC Educational Resources Information Center

Jones, Thomas W.; Price, Barbara A.; Randall, Cindy H.

2011-01-01

A study was conducted at a southern university in sophomore level production classes to assess skills such as the order of arithmetic operations, decimal and percent conversion, solving of algebraic expressions, and evaluation of formulas. The study was replicated using business statistics and quantitative analysis classes at a southeastern…

Competency-based achievement system: using formative feedback to teach and assess family medicine residents' skills.

PubMed

Ross, Shelley; Poth, Cheryl N; Donoff, Michel; Humphries, Paul; Steiner, Ivan; Schipper, Shirley; Janke, Fred; Nichols, Darren

2011-09-01

Family medicine residency programs require innovative means to assess residents' competence in "soft" skills (eg, patient-centred care, communication, and professionalism) and to identify residents who are having difficulty early enough in their residency to provide remedial training. To develop a method to assess residents' competence in various skills and to identify residents who are having difficulty. The Competency-Based Achievement System (CBAS) was designed to measure competence using 3 main principles: formative feedback, guided self-assessment, and regular face-to-face meetings. The CBAS is resident driven and provides a framework for meaningful interactions between residents and advisors. Residents use the CBAS to organize and review their feedback, to guide their own assessment of their progress, and to discern their future learning needs. Advisors use the CBAS to monitor, guide, and verify residents' knowledge of and competence in important skills. By focusing on specific skills and behaviour, the CBAS enables residents and advisors to make formative assessments and to communicate their findings. Feedback indicates that the CBAS is a user-friendly and helpful system to assess competence.

Toward an objective assessment of technical skills: a national survey of surgical program directors in Saudi Arabia.

PubMed

Alkhayal, Abdullah; Aldhukair, Shahla; Alselaim, Nahar; Aldekhayel, Salah; Alhabdan, Sultan; Altaweel, Waleed; Magzoub, Mohi Elden; Zamakhshary, Mohammed

2012-01-01

After almost a decade of implementing competency-based programs in postgraduate training programs, the assessment of technical skills remains more subjective than objective. National data on the assessment of technical skills during surgical training are lacking. We conducted this study to document the assessment tools for technical skills currently used in different surgical specialties, their relationship with remediation, the recommended tools from the program directors' perspective, and program directors' attitudes toward the available objective tools to assess technical skills. This study was a cross-sectional survey of surgical program directors (PDs). The survey was initially developed using a focus group and was then sent to 116 PDs. The survey contains demographic information about the program, the objective assessment tools used, and the reason for not using assessment tools. The last section discusses the recommended tools to be used from the PDs' perspective and the PDs' attitude and motivation to apply these tools in each program. The associations between the responses to the assessment questions and remediation were statistically evaluated. Seventy-one (61%) participants responded. Of the respondents, 59% mentioned using only nonstandardized, subjective, direct observation for technical skills assessment. Sixty percent use only summative evaluation, whereas 15% perform only formative evaluations of their residents, and the remaining 22% conduct both summative and formative evaluations of their residents' technical skills. Operative portfolios are kept by 53% of programs. The percentage of programs with mechanisms for remediation is 29% (19 of 65). The survey showed that surgical training programs use different tools to assess surgical skills competency. Having a clear remediation mechanism was highly associated with reporting remediation, which reflects the capability to detect struggling residents. Surgical training leadership should invest more in

A sampling strategy for promoting and assessing medical student retention of physical examination skills.

PubMed

Williams, Reed G; Klamen, Debra L; Mayer, David; Valaski, Maureen; Roberts, Nicole K

2007-10-01

Skill acquisition and maintenance requires spaced deliberate practice. Assessing medical students' physical examination performance ability is resource intensive. The authors assessed the nature and size of physical examination performance samples necessary to accurately estimate total physical examination skill. Physical examination assessment data were analyzed from second year students at the University of Illinois College of Medicine at Chicago in 2002, 2003, and 2004 (N = 548). Scores on subgroups of physical exam maneuvers were compared with scores on the total physical exam, to identify sound predictors of total test performance. Five exam subcomponents were sufficiently correlated to overall test performance and provided adequate sensitivity and specificity to serve as a means to prompt continued student review and rehearsal of physical examination technical skills. Selection and administration of samples of the total physical exam provide a resource-saving approach for promoting and estimating overall physical examination skills retention.

Teachers' Perceptions of a Fundamental Movement Skill (FMS) Assessment Battery in a School Setting

ERIC Educational Resources Information Center

Lander, Natalie; Morgan, Philip J.; Salmon, Jo; Barnett, Lisa M.

2016-01-01

Fundamental movement skills (FMS) competence is low in adolescent girls. An assessment tool for teachers is needed to monitor FMS in this demographic. The present study explored whether the Canadian Agility and Movement Skill Assessment (CAMSA) is feasible for use by physical education (PE) teachers of Australian Year 7 girls in a school setting.…

Moving beyond Assessment to Improving Students' Critical Thinking Skills: A Model for Implementing Change

ERIC Educational Resources Information Center

Haynes, Ada; Lisic, Elizabeth; Goltz, Michele; Stein, Barry; Harris, Kevin

2016-01-01

This research examines how the use of the CAT (Critical thinking Assessment Test) and involvement in CAT-Apps (CAT Applications within the discipline) training can serve as an important part of a faculty development model that assists faculty in the assessment of students' critical thinking skills and in the development of these skills within…

Information Literacy and Adult Learners: Using Authentic Assessment to Determine Skill Gaps

ERIC Educational Resources Information Center

Rapchak, Marcia E.; Lewis, Leslie A.; Motyka, Julie K.; Balmert, Margaret

2015-01-01

Information literacy (IL) skills are essential for adult learners in higher education, especially those unfamiliar with information systems. Citing a lack of literature assessing such skills in adult learners, this article examines the IL abilities of adult learners in an IL course. Using a rubric and annotated bibliographies from study…

Assessing neurosurgical non-technical skills: an exploratory study of a new behavioural marker system.

PubMed

Michinov, Estelle; Jamet, Eric; Dodeler, Virginie; Haegelen, Claire; Jannin, Pierre

2014-10-01

The management of non-technical skills is a major factor affecting teamwork quality and patient safety. This article presents a behavioural marker system for assessing neurosurgical non-technical skills (BMS-NNTS). We tested the BMS during deep brain stimulation surgery. We developed the BMS in three stages. First, we drew up a provisional assessment tool based on the literature and observation tools developed for other surgical specialties. We then analysed videos made in an operating room (OR) during deep brain stimulation operations in order to ensure there were no significant omissions from the skills list. Finally, we used five videos of operations to identify the behavioural markers of non-technical skills in verbal communications. Analyses of more than six hours of observations revealed 3515 behaviours from which we determined the neurosurgeon's non-technical skills behaviour pattern. The neurosurgeon frequently engaged in explicit coordination, situation awareness and leadership behaviours. In addition, the neurosurgeon's behaviours differed according to the stage of the operation and the OR staff members with whom she was communicating. Our behavioural marker system provides a structured approach to assessing non-technical skills in the field of neurosurgery. It can also be transferred to other surgical specialties and used in surgeon training curricula. © 2014 John Wiley & Sons, Ltd.

Assessing Writing Skills

NASA Astrophysics Data System (ADS)

Narayanan, M.

2013-12-01

This is an encore presentation of what was presented at the 2012 AGU International Conference. It was entitled: 'ASSESSING CORE COMPETENCIES.' The poster presentation, however, has been redesigned and reorganized with new, revised perspectives. The importance of ASSESSMENT principles has been emphasized. Catherine Palomba and Trudy Banta offer the following definition of assessment, adapted from one provided by Marchese in 1987. Assessment is the systematic collection, review, and use of information about educational programs undertaken for the purpose of improving student learning and development. (Palomba and Banta 1999). Educational institutions are committing substantial resources to the establishment of dedicated technology-based laboratories, so that they will be able to accommodate and fulfill students' desire to master certain of these specific skills. This type of technology-based instruction may raise some fundamental questions about the core competencies of the student learner. Some of the most important questions are : 1. Is the utilization of these fast high-powered computers and user-friendly software programs creating a totally non-challenging instructional environment for the student learner ? 2. Can technology itself all too easily overshadow the learning outcomes intended ? 3. Are the educational institutions simply training students how to use technology rather than educating them in the appropriate field ? 4. Are we still teaching content-driven courses and analysis oriented subject matter ? 5. Are these sophisticated modern era technologies contributing to a decline in the Critical Thinking Capabilities of the 21st century technology-savvy students ? The author tries to focus on technology as a tool and not on the technology itself. He further argues that students must demonstrate that they have the have the ability to think critically before they make an attempt to use technology in a chosen application-specific environment. The author further

Self-assessment of intercultural communication skills: a survey of physicians and medical students in Geneva, Switzerland.

PubMed

Hudelson, Patricia; Perron, Noelle Junod; Perneger, Thomas

2011-09-01

Physicians working with multicultural populations need to know how to elicit the patient's understanding of the illness; determine the patient's sociocultural context and identify any issues that might affect care; communicate effectively across patient-provider social and cultural differences; and collaborate effectively with an interpreter. Skills self-assessment can contribute to identifying training needs and monitoring skills development in these areas. As part of a larger study exploring the knowledge, attitudes and practices of Geneva physicians and medical students regarding the care of immigrant patients, we asked respondents to self-rate their ability to perform a range of common yet challenging intercultural communication tasks. Overall, respondents rated themselves less competent at intercultural tasks than at basic medical skills and less competent at specific intercultural communication skills than at general intercultural skills. Qualified doctors (as opposed to students), those with greater interest in caring for immigrants, and those who rarely encountered difficulties with immigrants rated themselves significantly more competent for all clinical tasks. Having a higher percentage of immigrant patients and previous cultural competence training predicted greater self-rated intercultural communication skills. Our self-assessment results suggest that students and physicians should be provided with the opportunity to practice intercultural skills with immigrant patients as part of their cultural competence training. To strengthen the validity of self-assessment measures, they should ideally be combined with more objective methods to assess actual skills.

Development of a Social Skills Assessment Screening Scale for Psychiatric Rehabilitation Settings: A Pilot Study

PubMed Central

Bhola, Poornima; Basavarajappa, Chethan; Guruprasad, Deepti; Hegde, Gayatri; Khanam, Fatema; Thirthalli, Jagadisha; Chaturvedi, Santosh K.

2016-01-01

Context: Deficits in social skills may present in a range of psychiatric disorders, particularly in the more serious and persistent conditions, and have an influence on functioning across various domains. Aims: This pilot study aimed at developing a brief measure, for structured evaluation and screening for social skills deficits, which can be easily integrated into routine clinical practice. Settings and Design: The sample consisted of 380 inpatients and their accompanying caregivers, referred to Psychiatric Rehabilitation Services at a tertiary care government psychiatric hospital. Materials and Methods: The evaluation included an Inpatient intake Proforma and the 20-item Social Skills Assessment Screening Scale (SSASS). Disability was assessed using the Indian Disability Evaluation and Assessment Scale (IDEAS) for a subset of 94 inpatients. Statistical Analysis Used: The analysis included means and standard deviations, frequency and percentages, Cronbach's alpha to assess internal consistency, t-tests to assess differences in social skills deficits between select subgroups, and correlation between SSASS and IDEAS scores. Results: The results indicated the profile of social skills deficits assessed among the inpatients with varied psychiatric diagnoses. The “psychosis” group exhibited significantly higher deficits than the “mood disorder” group. Results indicated high internal consistency of the SSASS and adequate criterion validity demonstrated by correlations with select IDEAS domains. Modifications were made to the SSASS following the pilot study. Conclusions: The SSASS has potential value as a measure for screening and individualised intervention plans for social skills training in mental health and rehabilitation settings. The implications for future work on the psychometric properties and clinical applications are discussed. PMID:27833220

Documenting Student Competence through Effective Performance Assessment: Employability Skills. Workshop Summary.

ERIC Educational Resources Information Center

Ohio State Univ., Columbus. Agricultural Curriculum Materials Service.

This report contains 26 performance assessments for documenting student employability skills. Each performance assessment consists of the following: a competency; a terminal performance objective (outcome); competency builders and pupil performance objectives (criteria for documenting mastery of the objective); applied academic competencies;…

A framework for organizing and selecting quantitative approaches for benefit-harm assessment.

PubMed

Puhan, Milo A; Singh, Sonal; Weiss, Carlos O; Varadhan, Ravi; Boyd, Cynthia M

2012-11-19

Several quantitative approaches for benefit-harm assessment of health care interventions exist but it is unclear how the approaches differ. Our aim was to review existing quantitative approaches for benefit-harm assessment and to develop an organizing framework that clarifies differences and aids selection of quantitative approaches for a particular benefit-harm assessment. We performed a review of the literature to identify quantitative approaches for benefit-harm assessment. Our team, consisting of clinicians, epidemiologists, and statisticians, discussed the approaches and identified their key characteristics. We developed a framework that helps investigators select quantitative approaches for benefit-harm assessment that are appropriate for a particular decisionmaking context. Our framework for selecting quantitative approaches requires a concise definition of the treatment comparison and population of interest, identification of key benefit and harm outcomes, and determination of the need for a measure that puts all outcomes on a single scale (which we call a benefit and harm comparison metric). We identified 16 quantitative approaches for benefit-harm assessment. These approaches can be categorized into those that consider single or multiple key benefit and harm outcomes, and those that use a benefit-harm comparison metric or not. Most approaches use aggregate data and can be used in the context of single studies or systematic reviews. Although the majority of approaches provides a benefit and harm comparison metric, only four approaches provide measures of uncertainty around the benefit and harm comparison metric (such as a 95 percent confidence interval). None of the approaches considers the actual joint distribution of benefit and harm outcomes, but one approach considers competing risks when calculating profile-specific event rates. Nine approaches explicitly allow incorporating patient preferences. The choice of quantitative approaches depends on the

A framework for organizing and selecting quantitative approaches for benefit-harm assessment

PubMed Central

2012-01-01

Background Several quantitative approaches for benefit-harm assessment of health care interventions exist but it is unclear how the approaches differ. Our aim was to review existing quantitative approaches for benefit-harm assessment and to develop an organizing framework that clarifies differences and aids selection of quantitative approaches for a particular benefit-harm assessment. Methods We performed a review of the literature to identify quantitative approaches for benefit-harm assessment. Our team, consisting of clinicians, epidemiologists, and statisticians, discussed the approaches and identified their key characteristics. We developed a framework that helps investigators select quantitative approaches for benefit-harm assessment that are appropriate for a particular decisionmaking context. Results Our framework for selecting quantitative approaches requires a concise definition of the treatment comparison and population of interest, identification of key benefit and harm outcomes, and determination of the need for a measure that puts all outcomes on a single scale (which we call a benefit and harm comparison metric). We identified 16 quantitative approaches for benefit-harm assessment. These approaches can be categorized into those that consider single or multiple key benefit and harm outcomes, and those that use a benefit-harm comparison metric or not. Most approaches use aggregate data and can be used in the context of single studies or systematic reviews. Although the majority of approaches provides a benefit and harm comparison metric, only four approaches provide measures of uncertainty around the benefit and harm comparison metric (such as a 95 percent confidence interval). None of the approaches considers the actual joint distribution of benefit and harm outcomes, but one approach considers competing risks when calculating profile-specific event rates. Nine approaches explicitly allow incorporating patient preferences. Conclusion The choice of