From Perception to Practice: The Impact of Teachers' Scoring Experience on Performance-based Instruction and Classroom Assessment

ERIC Educational Resources Information Center

Goldberg, Gail Lynn; Roswell, Barbara Sherr

2000-01-01

Studied the impact of experience scoring the Maryland School Performance Assessment tasks on teachers' instructional and classroom assessment practice. Interview data, questionnaires, classroom observation, and classroom artifacts from approximately 5 teacher-scorers demonstrated that teachers' appropriation of performance-based instruction may be…

Putting the Focus on Student Engagement: The Benefits of Performance-Based Assessment

ERIC Educational Resources Information Center

Barlowe, Avram; Cook, Ann

2016-01-01

For more than two decades, the New York Performance Standards Consortium, a coalition of 38 public high schools, has steered clear of high-stakes testing, which superficially assess student learning. Instead, the consortium's approach relies on performance-based assessments--essays, research papers, science experiments, and high-level mathematical…

Influence of Strategies-Based Feedback in Students' Oral Performance

ERIC Educational Resources Information Center

Sisquiarco, Angie; Rojas, Santiago Sánchez; Abad, José Vicente

2018-01-01

This article reports on an action research study that assessed the influence of cognitive and metacognitive strategies-based feedback in the oral performance of a group of 6th grade students at a public school in Medellin, Colombia. Researchers analyzed students' oral performance through assessment and self-assessment rubrics, applied inventories…

Examiners and Content and Site: Oh My! a National Organization's Investigation of Score Variation in Large-Scale Performance Assessments

ERIC Educational Resources Information Center

Sebok, Stefanie S.; Roy, Marguerite; Klinger, Don A.; De Champlain, André F.

2015-01-01

Examiner effects and content specificity are two well known sources of construct irrelevant variance that present great challenges in performance-based assessments. National medical organizations that are responsible for large-scale performance based assessments experience an additional challenge as they are responsible for administering…

Student Emotions in Conversation-Based Assessments

ERIC Educational Resources Information Center

Lehman, Blair A.; Zapata-Rivera, Diego

2018-01-01

Students can experience a variety of emotions while completing assessments. Some emotions can get in the way of students performing their best (e.g., anxiety, frustration), whereas other emotions can facilitate student performance (e.g., engagement). Many new, non-traditional assessments, such as automated conversation-based assessments (CBA), are…

Impact of workplace based assessment on doctors' education and performance: a systematic review.

PubMed

Miller, Alice; Archer, Julian

2010-09-24

To investigate the literature for evidence that workplace based assessment affects doctors' education and performance. Systematic review. The primary data sources were the databases Journals@Ovid, Medline, Embase, CINAHL, PsycINFO, and ERIC. Evidence based reviews (Bandolier, Cochrane Library, DARE, HTA Database, and NHS EED) were accessed and searched via the Health Information Resources website. Reference lists of relevant studies and bibliographies of review articles were also searched. Review methods Studies of any design that attempted to evaluate either the educational impact of workplace based assessment, or the effect of workplace based assessment on doctors' performance, were included. Studies were excluded if the sampled population was non-medical or the study was performed with medical students. Review articles, commentaries, and letters were also excluded. The final exclusion criterion was the use of simulated patients or models rather than real life clinical encounters. Sixteen studies were included. Fifteen of these were non-comparative descriptive or observational studies; the other was a randomised controlled trial. Study quality was mixed. Eight studies examined multisource feedback with mixed results; most doctors felt that multisource feedback had educational value, although the evidence for practice change was conflicting. Some junior doctors and surgeons displayed little willingness to change in response to multisource feedback, whereas family physicians might be more prepared to initiate change. Performance changes were more likely to occur when feedback was credible and accurate or when coaching was provided to help subjects identify their strengths and weaknesses. Four studies examined the mini-clinical evaluation exercise, one looked at direct observation of procedural skills, and three were concerned with multiple assessment methods: all these studies reported positive results for the educational impact of workplace based assessment tools. However, there was no objective evidence of improved performance with these tools. Considering the emphasis placed on workplace based assessment as a method of formative performance assessment, there are few published articles exploring its impact on doctors' education and performance. This review shows that multisource feedback can lead to performance improvement, although individual factors, the context of the feedback, and the presence of facilitation have a profound effect on the response. There is no evidence that alternative workplace based assessment tools (mini-clinical evaluation exercise, direct observation of procedural skills, and case based discussion) lead to improvement in performance, although subjective reports on their educational impact are positive.

Predictors of performance-based measures of instrumental activities of daily living in nondemented patients with Parkinson's disease.

PubMed

Pirogovsky, Eva; Martinez-Hannon, Mercedes; Schiehser, Dawn M; Lessig, Stephanie L; Song, David D; Litvan, Irene; Filoteo, J Vincent

2013-01-01

Few studies have examined instrumental activities of daily living (iADLs) in nondemented Parkinson's disease (PD), and the majority of these studies have used report-based measures, which can have limited validity. The present study had two main goals: (a) to examine the performance of nondemented PD patients on two performance-based measures of iADLs, which are considered more objective functional measures, and (b) to examine the cognitive, motor, and psychiatric correlates of iADL impairment in PD. Ninety-eight nondemented PD patients and 47 healthy older adults were administered performance-based measures that assess the ability to manage medications (Medication Management Ability Assessment) and finances (University of California, San Diego, UCSD, Performance-based Skills Assessment), the Mattis Dementia Rating Scale to assess global cognitive functioning, the Unified Parkinson's Disease Rating Scale Part III to assess motor symptom severity, and the Geriatric Depression Scale to assess depressive symptoms. Nondemented PD patients demonstrated significantly impaired scores relative to the healthy comparison group on the performance-based measure of financial management, but there were no significant group differences in medication management. Global cognitive functioning, motor severity, and depressive symptoms did not correlate with scores on either of the functional measures, except for a small correlation between depressive symptoms and financial management. The two performance-based measures of iADL functioning did not correlate with one another. These findings suggest that medication and financial management may not be predicted based on global cognitive functioning and that iADLs may not be represented by a single construct. Furthermore, these findings suggest the potential need for a multidimensional approach to assessing iADLs.

Motor Skill Assessment of Children: Is There an Association between Performance-Based, Child-Report, and Parent-Report Measures of Children's Motor Skills?

ERIC Educational Resources Information Center

Kennedy, Johanna; Brown, Ted; Chien, Chi-Wen

2012-01-01

Client-centered practice requires therapists to actively seek the perspectives of children and families. Several assessment tools are available to facilitate this process. However, when evaluating motor skill performance, therapists typically concentrate on performance-based assessment. To improve understanding of the information provided by the…

Evaluation of the role of incentive structure on student participation and performance in active learning strategies: A comparison of case-based and team-based learning.

PubMed

Carrasco, Gonzalo A; Behling, Kathryn C; Lopez, Osvaldo J

2018-04-01

Student participation is important for the success of active learning strategies, but participation is often linked to the level of preparation. At our institution, we use two types of active learning activities, a modified case-based learning exercise called active learning groups (ALG) and team-based learning (TBL). These strategies have different assessment and incentive structures for participation. Non-cognitive skills are assessed in ALG using a subjective five-point Likert scale. In TBL, assessment of individual student preparation is based on a multiple choice quiz conducted at the beginning of each session. We studied first-year medical student participation and performance in ALG and TBL as well as performance on course final examinations. Student performance in TBL, but not in ALG, was strongly correlated with final examination scores. Additionally, in students who performed in the upper 33rd percentile on the final examination, there was a positive correlation between final examination performance and participation in TBL and ALG. This correlation was not seen in students who performed in the lower 33rd percentile on the final examinations. Our results suggest that assessments of medical knowledge during active learning exercises could supplement non-cognitive assessments and could be good predictors of performance on summative examinations.

A Web-Based Peer-Assessment Approach to Improving Junior High School Students' Performance, Self-Efficacy and Motivation in Performing Arts Courses

ERIC Educational Resources Information Center

Hsia, Lu-Ho; Huang, Iwen; Hwang, Gwo-Jen

2016-01-01

In this paper, a web-based peer-assessment approach is proposed for conducting performing arts activities. A peer-assessment system was implemented and applied to a junior high school performing arts course to evaluate the effectiveness of the proposed approach. A total of 163 junior high students were assigned to an experimental group and a…

Issues in developing valid assessments of speech pathology students' performance in the workplace.

PubMed

McAllister, Sue; Lincoln, Michelle; Ferguson, Alison; McAllister, Lindy

2010-01-01

Workplace-based learning is a critical component of professional preparation in speech pathology. A validated assessment of this learning is seen to be 'the gold standard', but it is difficult to develop because of design and validation issues. These issues include the role and nature of judgement in assessment, challenges in measuring quality, and the relationship between assessment and learning. Valid assessment of workplace-based performance needs to capture the development of competence over time and account for both occupation specific and generic competencies. This paper reviews important conceptual issues in the design of valid and reliable workplace-based assessments of competence including assessment content, process, impact on learning, measurement issues, and validation strategies. It then goes on to share what has been learned about quality assessment and validation of a workplace-based performance assessment using competency-based ratings. The outcomes of a four-year national development and validation of an assessment tool are described. A literature review of issues in conceptualizing, designing, and validating workplace-based assessments was conducted. Key factors to consider in the design of a new tool were identified and built into the cycle of design, trialling, and data analysis in the validation stages of the development process. This paper provides an accessible overview of factors to consider in the design and validation of workplace-based assessment tools. It presents strategies used in the development and national validation of a tool COMPASS, used in an every speech pathology programme in Australia, New Zealand, and Singapore. The paper also describes Rasch analysis, a model-based statistical approach which is useful for establishing validity and reliability of assessment tools. Through careful attention to conceptual and design issues in the development and trialling of workplace-based assessments, it has been possible to develop the world's first valid and reliable national assessment tool for the assessment of performance in speech pathology.

Simulation-Based Performance Assessment: An Innovative Approach to Exploring Understanding of Physical Science Concepts

ERIC Educational Resources Information Center

Gale, Jessica; Wind, Stefanie; Koval, Jayma; Dagosta, Joseph; Ryan, Mike; Usselman, Marion

2016-01-01

This paper illustrates the use of simulation-based performance assessment (PA) methodology in a recent study of eighth-grade students' understanding of physical science concepts. A set of four simulation-based PA tasks were iteratively developed to assess student understanding of an array of physical science concepts, including net force,…

Investigation of Formative Assessment of Learning (INFORMAL): The Performance Indicator Tool (PIT)

ERIC Educational Resources Information Center

Dutton, Philip J.; Bickerstaff, Helen E.; Rymer, Janice M.; Webb, Mary E.; Ballinger-Mills, Deborah; Greenough, Anne; Reynolds, Patricia A.

2017-01-01

This pilot study evaluated the student user experience of a performance indicator tool (PIT) for formative assessment of a cohort of fourth year medical undergraduates undertaking a 12 week rotation. Scoping and elaboration for the standalone web-based tool was based on the existing paper-based formative assessment system for which training and…

Effect of online formative assessment on summative performance in integrated musculoskeletal system module.

PubMed

Mitra, Nilesh Kumar; Barua, Ankur

2015-03-03

The impact of web-based formative assessment practices on performance of undergraduate medical students in summative assessments is not widely studied. This study was conducted among third-year undergraduate medical students of a designated university in Malaysia to compare the effect, on performance in summative assessment, of repeated computer-based formative assessment with automated feedback with that of single paper-based formative assessment with face-to face feedback. This quasi-randomized trial was conducted among two groups of undergraduate medical students who were selected by stratified random technique from a cohort undertaking the Musculoskeletal module. The control group C (n = 102) was subjected to a paper-based formative MCQ test. The experimental group E (n = 65) was provided three online formative MCQ tests with automated feedback. The summative MCQ test scores for both these groups were collected after the completion of the module. In this study, no significant difference was observed between the mean summative scores of the two groups. However, Band 1 students from group E with higher entry qualification showed higher mean score in the summative assessment. A trivial, but significant and positive correlation (r(2) = +0.328) was observed between the online formative test scores and summative assessment scores of group E. The proportionate increase of performance in group E was found to be almost double than group C. The use of computer based formative test with automated feedback improved the performance of the students with better academic background in the summative assessment. Computer-based formative test can be explored as an optional addition to the curriculum of pre-clinical integrated medical program to improve the performance of the students with higher academic ability.

Methodological Issues in Curriculum-Based Reading Assessment.

ERIC Educational Resources Information Center

Fuchs, Lynn S.; And Others

1984-01-01

Three studies involving elementary students examined methodological issues in curriculum-based reading assessment. Results indicated that (1) whereas sample duration did not affect concurrent validity, increasing duration reduced performance instability and increased performance slopes and (2) domain size was related inversely to performance slope…

Puzzle-based versus traditional lecture: comparing the effects of pedagogy on academic performance in an undergraduate human anatomy and physiology II lab.

PubMed

Stetzik, Lucas; Deeter, Anthony; Parker, Jamie; Yukech, Christine

2015-06-23

A traditional lecture-based pedagogy conveys information and content while lacking sufficient development of critical thinking skills and problem solving. A puzzle-based pedagogy creates a broader contextual framework, and fosters critical thinking as well as logical reasoning skills that can then be used to improve a student's performance on content specific assessments. This paper describes a pedagogical comparison of traditional lecture-based teaching and puzzle-based teaching in a Human Anatomy and Physiology II Lab. Using a single subject/cross-over design half of the students from seven sections of the course were taught using one type of pedagogy for the first half of the semester, and then taught with a different pedagogy for the second half of the semester. The other half of the students were taught the same material but with the order of the pedagogies reversed. Students' performance on quizzes and exams specific to the course, and in-class assignments specific to this study were assessed for: learning outcomes (the ability to form the correct conclusion or recall specific information), and authentic academic performance as described by (Am J Educ 104:280-312, 1996). Our findings suggest a significant improvement in students' performance on standard course specific assessments using a puzzle-based pedagogy versus a traditional lecture-based teaching style. Quiz and test scores for students improved by 2.1 and 0.4% respectively in the puzzle-based pedagogy, versus the traditional lecture-based teaching. Additionally, the assessments of authentic academic performance may only effectively measure a broader conceptual understanding in a limited set of contexts, and not in the context of a Human Anatomy and Physiology II Lab. In conclusion, a puzzle-based pedagogy, when compared to traditional lecture-based teaching, can effectively enhance the performance of students on standard course specific assessments, even when the assessments only test a limited conceptual understanding of the material.

Evaluating the Effect of Learning Style and Student Background on Self-Assessment Accuracy

ERIC Educational Resources Information Center

Alaoutinen, Satu

2012-01-01

This study evaluates a new taxonomy-based self-assessment scale and examines factors that affect assessment accuracy and course performance. The scale is based on Bloom's Revised Taxonomy and is evaluated by comparing students' self-assessment results with course performance in a programming course. Correlation has been used to reveal possible…

The Utility of Curriculum-Based Measurement within a Multitiered Framework: Establishing Cut Scores as Predictors of Student Performance on the Alaska Standards-Based Assessment

ERIC Educational Resources Information Center

Legg, David E.

2013-01-01

The purpose of this study was to explore the relationship between student performance on Reading Curriculum-based Measures (R-CBM) and student performance on the Alaska's standards based assessment (SBA) administered to students in Studied School District (SSD) Grade 3 through Grade 5 students in the Studied School District as required by…

Advanced Engineering Technology for Measuring Performance.

PubMed

Rutherford, Drew N; D'Angelo, Anne-Lise D; Law, Katherine E; Pugh, Carla M

2015-08-01

The demand for competency-based assessments in surgical training is growing. Use of advanced engineering technology for clinical skills assessment allows for objective measures of hands-on performance. Clinical performance can be assessed in several ways via quantification of an assessee's hand movements (motion tracking), direction of visual attention (eye tracking), levels of stress (physiologic marker measurements), and location and pressure of palpation (force measurements). Innovations in video recording technology and qualitative analysis tools allow for a combination of observer- and technology-based assessments. Overall the goal is to create better assessments of surgical performance with robust validity evidence. Copyright © 2015 Elsevier Inc. All rights reserved.

Seismic performance assessment of base-isolated safety-related nuclear structures

USGS Publications Warehouse

Huang, Y.-N.; Whittaker, A.S.; Luco, N.

2010-01-01

Seismic or base isolation is a proven technology for reducing the effects of earthquake shaking on buildings, bridges and infrastructure. The benefit of base isolation has been presented in terms of reduced accelerations and drifts on superstructure components but never quantified in terms of either a percentage reduction in seismic loss (or percentage increase in safety) or the probability of an unacceptable performance. Herein, we quantify the benefits of base isolation in terms of increased safety (or smaller loss) by comparing the safety of a sample conventional and base-isolated nuclear power plant (NPP) located in the Eastern U.S. Scenario- and time-based assessments are performed using a new methodology. Three base isolation systems are considered, namely, (1) Friction Pendulum??? bearings, (2) lead-rubber bearings and (3) low-damping rubber bearings together with linear viscous dampers. Unacceptable performance is defined by the failure of key secondary systems because these systems represent much of the investment in a new build power plant and ensure the safe operation of the plant. For the scenario-based assessments, the probability of unacceptable performance is computed for an earthquake with a magnitude of 5.3 at a distance 7.5 km from the plant. For the time-based assessments, the annual frequency of unacceptable performance is computed considering all potential earthquakes that may occur. For both assessments, the implementation of base isolation reduces the probability of unacceptable performance by approximately four orders of magnitude for the same NPP superstructure and secondary systems. The increase in NPP construction cost associated with the installation of seismic isolators can be offset by substantially reducing the required seismic strength of secondary components and systems and potentially eliminating the need to seismically qualify many secondary components and systems. ?? 2010 John Wiley & Sons, Ltd.

Simulation-based assessment to identify critical gaps in safe anesthesia resident performance.

PubMed

Blum, Richard H; Boulet, John R; Cooper, Jeffrey B; Muret-Wagstaff, Sharon L

2014-01-01

Valid methods are needed to identify anesthesia resident performance gaps early in training. However, many assessment tools in medicine have not been properly validated. The authors designed and tested use of a behaviorally anchored scale, as part of a multiscenario simulation-based assessment system, to identify high- and low-performing residents with regard to domains of greatest concern to expert anesthesiology faculty. An expert faculty panel derived five key behavioral domains of interest by using a Delphi process (1) Synthesizes information to formulate a clear anesthetic plan; (2) Implements a plan based on changing conditions; (3) Demonstrates effective interpersonal and communication skills with patients and staff; (4) Identifies ways to improve performance; and (5) Recognizes own limits. Seven simulation scenarios spanning pre-to-postoperative encounters were used to assess performances of 22 first-year residents and 8 fellows from two institutions. Two of 10 trained faculty raters blinded to trainee program and training level scored each performance independently by using a behaviorally anchored rating scale. Residents, fellows, facilitators, and raters completed surveys. Evidence supporting the reliability and validity of the assessment scores was procured, including a high generalizability coefficient (ρ = 0.81) and expected performance differences between first-year resident and fellow participants. A majority of trainees, facilitators, and raters judged the assessment to be useful, realistic, and representative of critical skills required for safe practice. The study provides initial evidence to support the validity of a simulation-based performance assessment system for identifying critical gaps in safe anesthesia resident performance early in training.

Development of Internet-Based Tasks for the Executive Function Performance Test.

PubMed

Rand, Debbie; Lee Ben-Haim, Keren; Malka, Rachel; Portnoy, Sigal

The Executive Function Performance Test (EFPT) is a reliable and valid performance-based tool to assess executive functions (EFs). This study's objective was to develop and verify two Internet-based tasks for the EFPT. A cross-sectional study assessed the alternate-form reliability of the Internet-based bill-paying and telephone-use tasks in healthy adults and people with subacute stroke (Study 1). It also sought to establish the tasks' criterion reliability for assessing EF deficits by correlating performance with that on the Trail Making Test in five groups: healthy young adults, healthy older adults, people with subacute stroke, people with chronic stroke, and young adults with attention deficit hyperactivity disorder (Study 2). The alternative-form reliability and initial construct validity for the Internet-based bill-paying task were verified. Criterion validity was established for both tasks. The Internet-based tasks are comparable to the original EFPT tasks and can be used for assessment of EF deficits. Copyright © 2018 by the American Occupational Therapy Association, Inc.

A BEME (Best Evidence in Medical Education) systematic review of the use of workplace-based assessment in identifying and remediating poor performance among postgraduate medical trainees.

PubMed

Barrett, Aileen; Galvin, Rose; Steinert, Yvonne; Scherpbier, Albert; O'Shaughnessy, Ann; Horgan, Mary; Horsley, Tanya

2015-05-08

Workplace-based assessments were designed to facilitate observation and structure feedback on the performance of trainees in real-time clinical settings and scenarios. Research in workplace-based assessments has primarily centred on understanding psychometric qualities and performance improvement impacts of trainees generally. An area that is far less understood is the use of workplace-based assessments for trainees who may not be performing at expected or desired standards, referred to within the literature as trainees 'in difficulty' or 'underperforming'. In healthcare systems that increasingly depend on service provided by junior doctors, early detection (and remediation) of poor performance is essential. However, barriers to successful implementation of workplace-based assessments (WBAs) in this context include a misunderstanding of the use and purpose of these formative assessment tools. This review aims to explore the impact - or effectiveness - of workplace-based assessment on the identification of poor performance and to determine those conditions that support and enable detection, i.e. whether by routine or targeted use where poor performance is suspected. The review also aims to explore what effect (if any) the use of WBA may have on remediation or on changing clinical practice. The personal impact of the detection of poor performance on trainees and/or trainers may also be explored. Using BEME (Best Evidence in Medical Education) Collaboration review guidelines, nine databases will be searched for English-language records. Studies examining interventions for workplace-based assessment either routinely or in relation to poor performance will be included. Independent agreement (kappa .80) will be achieved using a randomly selected set of records prior to commencement of screening and data extraction using a BEME coding sheet modified as applicable (Buckley et al., Med Teach 31:282-98, 2009) as this has been used in previous WBA systematic reviews (Miller and Archer, BMJ doi:10.1136/bmj.c5064, 2010) allowing for more rigorous comparisons with the published literature. Educational outcomes will be evaluated using Kirkpatrick's framework of educational outcomes using Barr's adaptations (Barr et al., Evaluations of interprofessional education; a United Kingdom review of health and social care, 2000) for medical education research. Our study will contribute to an ongoing international debate regarding the applicability of workplace-based assessments as a meaningful formative assessment approach within the context of postgraduate medical education. The review has been registered by the BEME Collaboration www.bemecollaboration.org .

Sensors vs. experts - a performance comparison of sensor-based fall risk assessment vs. conventional assessment in a sample of geriatric patients.

PubMed

Marschollek, Michael; Rehwald, Anja; Wolf, Klaus-Hendrik; Gietzelt, Matthias; Nemitz, Gerhard; zu Schwabedissen, Hubertus Meyer; Schulze, Mareike

2011-06-28

Fall events contribute significantly to mortality, morbidity and costs in our ageing population. In order to identify persons at risk and to target preventive measures, many scores and assessment tools have been developed. These often require expertise and are costly to implement. Recent research investigates the use of wearable inertial sensors to provide objective data on motion features which can be used to assess individual fall risk automatically. So far it is unknown how well this new method performs in comparison with conventional fall risk assessment tools. The aim of our research is to compare the predictive performance of our new sensor-based method with conventional and established methods, based on prospective data. In a first study phase, 119 inpatients of a geriatric clinic took part in motion measurements using a wireless triaxial accelerometer during a Timed Up&Go (TUG) test and a 20 m walk. Furthermore, the St. Thomas Risk Assessment Tool in Falling Elderly Inpatients (STRATIFY) was performed, and the multidisciplinary geriatric care team estimated the patients' fall risk. In a second follow-up phase of the study, 46 of the participants were interviewed after one year, including a fall and activity assessment. The predictive performances of the TUG, the STRATIFY and team scores are compared. Furthermore, two automatically induced logistic regression models based on conventional clinical and assessment data (CONV) as well as sensor data (SENSOR) are matched. Among the risk assessment scores, the geriatric team score (sensitivity 56%, specificity 80%) outperforms STRATIFY and TUG. The induced logistic regression models CONV and SENSOR achieve similar performance values (sensitivity 68%/58%, specificity 74%/78%, AUC 0.74/0.72, +LR 2.64/2.61). Both models are able to identify more persons at risk than the simple scores. Sensor-based objective measurements of motion parameters in geriatric patients can be used to assess individual fall risk, and our prediction model's performance matches that of a model based on conventional clinical and assessment data. Sensor-based measurements using a small wearable device may contribute significant information to conventional methods and are feasible in an unsupervised setting. More prospective research is needed to assess the cost-benefit relation of our approach.

Sensors vs. experts - A performance comparison of sensor-based fall risk assessment vs. conventional assessment in a sample of geriatric patients

PubMed Central

2011-01-01

Background Fall events contribute significantly to mortality, morbidity and costs in our ageing population. In order to identify persons at risk and to target preventive measures, many scores and assessment tools have been developed. These often require expertise and are costly to implement. Recent research investigates the use of wearable inertial sensors to provide objective data on motion features which can be used to assess individual fall risk automatically. So far it is unknown how well this new method performs in comparison with conventional fall risk assessment tools. The aim of our research is to compare the predictive performance of our new sensor-based method with conventional and established methods, based on prospective data. Methods In a first study phase, 119 inpatients of a geriatric clinic took part in motion measurements using a wireless triaxial accelerometer during a Timed Up&Go (TUG) test and a 20 m walk. Furthermore, the St. Thomas Risk Assessment Tool in Falling Elderly Inpatients (STRATIFY) was performed, and the multidisciplinary geriatric care team estimated the patients' fall risk. In a second follow-up phase of the study, 46 of the participants were interviewed after one year, including a fall and activity assessment. The predictive performances of the TUG, the STRATIFY and team scores are compared. Furthermore, two automatically induced logistic regression models based on conventional clinical and assessment data (CONV) as well as sensor data (SENSOR) are matched. Results Among the risk assessment scores, the geriatric team score (sensitivity 56%, specificity 80%) outperforms STRATIFY and TUG. The induced logistic regression models CONV and SENSOR achieve similar performance values (sensitivity 68%/58%, specificity 74%/78%, AUC 0.74/0.72, +LR 2.64/2.61). Both models are able to identify more persons at risk than the simple scores. Conclusions Sensor-based objective measurements of motion parameters in geriatric patients can be used to assess individual fall risk, and our prediction model's performance matches that of a model based on conventional clinical and assessment data. Sensor-based measurements using a small wearable device may contribute significant information to conventional methods and are feasible in an unsupervised setting. More prospective research is needed to assess the cost-benefit relation of our approach. PMID:21711504

Can Online Course-Based Assessment Methods Be Fair and Equitable? Relationships between Students' Preferences and Performance within Online and Offline Assessments

ERIC Educational Resources Information Center

Hewson, C.

2012-01-01

To address concerns raised regarding the use of online course-based summative assessment methods, a quasi-experimental design was implemented in which students who completed a summative assessment either online or offline were compared on performance scores when using their self-reported "preferred" or "non-preferred" modes.…

Performance of Fourth-Grade Students in the 2012 NAEP Computer-Based Writing Pilot Assessment: Scores, Text Length, and Use of Editing Tools. Working Paper Series. NCES 2015-119

ERIC Educational Resources Information Center

White, Sheida; Kim, Young Yee; Chen, Jing; Liu, Fei

2015-01-01

This study examined whether or not fourth-graders could fully demonstrate their writing skills on the computer and factors associated with their performance on the National Assessment of Educational Progress (NAEP) computer-based writing assessment. The results suggest that high-performing fourth-graders (those who scored in the upper 20 percent…

The reliability and validity of the Complex Task Performance Assessment: A performance-based assessment of executive function.

PubMed

Wolf, Timothy J; Dahl, Abigail; Auen, Colleen; Doherty, Meghan

2017-07-01

The objective of this study was to evaluate the inter-rater reliability, test-retest reliability, concurrent validity, and discriminant validity of the Complex Task Performance Assessment (CTPA): an ecologically valid performance-based assessment of executive function. Community control participants (n = 20) and individuals with mild stroke (n = 14) participated in this study. All participants completed the CTPA and a battery of cognitive assessments at initial testing. The control participants completed the CTPA at two different times one week apart. The intra-class correlation coefficient (ICC) for inter-rater reliability for the total score on the CTPA was .991. The ICCs for all of the sub-scores of the CTPA were also high (.889-.977). The CTPA total score was significantly correlated to Condition 4 of the DKEFS Color-Word Interference Test (p = -.425), and the Wechsler Test of Adult Reading (p  = -.493). Finally, there were significant differences between control subjects and individuals with mild stroke on the total score of the CTPA (p = .007) and all sub-scores except interpretation failures and total items incorrect. These results are also consistent with other current executive function performance-based assessments and indicate that the CTPA is a reliable and valid performance-based measure of executive function.

Measuring Primary Students' Graph Interpretation Skills Via a Performance Assessment: A case study in instrument development

NASA Astrophysics Data System (ADS)

Peterman, Karen; Cranston, Kayla A.; Pryor, Marie; Kermish-Allen, Ruth

2015-11-01

This case study was conducted within the context of a place-based education project that was implemented with primary school students in the USA. The authors and participating teachers created a performance assessment of standards-aligned tasks to examine 6-10-year-old students' graph interpretation skills as part of an exploratory research project. Fifty-five students participated in a performance assessment interview at the beginning and end of a place-based investigation. Two forms of the assessment were created and counterbalanced within class at pre and post. In situ scoring was conducted such that responses were scored as correct versus incorrect during the assessment's administration. Criterion validity analysis demonstrated an age-level progression in student scores. Tests of discriminant validity showed that the instrument detected variability in interpretation skills across each of three graph types (line, bar, dot plot). Convergent validity was established by correlating in situ scores with those from the Graph Interpretation Scoring Rubric. Students' proficiency with interpreting different types of graphs matched expectations based on age and the standards-based progression of graphs across primary school grades. The assessment tasks were also effective at detecting pre-post gains in students' interpretation of line graphs and dot plots after the place-based project. The results of the case study are discussed in relation to the common challenges associated with performance assessment. Implications are presented in relation to the need for authentic and performance-based instructional and assessment tasks to respond to the Common Core State Standards and the Next Generation Science Standards.

The development of performance-based practical assessment model at civil engineering workshop in state polytechnic

NASA Astrophysics Data System (ADS)

Kristinayanti, W. S.; Mas Pertiwi, I. G. A. I.; Evin Yudhi, S.; Lokantara, W. D.

2018-01-01

Assessment is an important element in education that shall oversees students’ competence not only in terms of cognitive aspect, but alsothe students’ psychomotorin a comprehensive way. Civil Engineering Department at Bali State Polytechnic,as a vocational education institution, emphasizes on not only the theoretical foundation of the study, but also the application throughpracticum in workshop-based learning. We are aware of a need for performance-based assessment for these students, which would be essential for the student’s all-round performance in their studies.We try to develop a performance-based practicum assessment model that is needed to assess student’s ability in workshop-based learning. This research was conducted in three stages, 1) learning needs analysis, 2) instruments development, and 3) testing of instruments. The study uses rubrics set-up to test students’ competence in the workshop and test the validity. We obtained 34-point valid statement out of 35, and resulted in value of Cronbach’s alpha equal to 0.977. In expert test we obtained a value of CVI = 0.75 which means that the drafted assessment is empirically valid within thetrial group.

Development of self and peer performance assessment on iodometric titration experiment

NASA Astrophysics Data System (ADS)

Nahadi; Siswaningsih, W.; Kusumaningtyas, H.

2018-05-01

This study aims to describe the process in developing of reliable and valid assessment to measure students’ performance on iodometric titration and the effect of the self and peer assessment on students’ performance. The self and peer-instrument provides valuable feedback for the student performance improvement. The developed assessment contains rubric and task for facilitating self and peer assessment. The participants are 24 students at the second-grade student in certain vocational high school in Bandung. The participants divided into two groups. The first 12 students involved in the validity test of the developed assessment, while the remain 12 students participated for the reliability test. The content validity was evaluated based on the judgment experts. Test result of content validity based on judgment expert show that the developed performance assessment instrument categorized as valid on each task with the realibity classified as very good. Analysis of the impact of the self and peer assessment implementation showed that the peer instrument supported the self assessment.

Assessment! The Foundation to This Performance/Field-Based Teacher Preparation Program.

ERIC Educational Resources Information Center

Schmitz, Steve; Veilbig, Cindy; Fitch, Lanny; Alawiye, Osman

This report describes the assessment practices of the undergraduate performance and field-based teacher preparation programs of Central Washington University. The undergraduate program involves 2-year programs for elementary school teacher candidates in 3 school districts. Portfolios are used in this program as a means of authentic assessment and…

Mobile Formative Assessment Tool Based on Data Mining Techniques for Supporting Web-Based Learning

ERIC Educational Resources Information Center

Chen, Chih-Ming; Chen, Ming-Chuan

2009-01-01

Current trends clearly indicate that online learning has become an important learning mode. However, no effective assessment mechanism for learning performance yet exists for e-learning systems. Learning performance assessment aims to evaluate what learners learned during the learning process. Traditional summative evaluation only considers final…

The Utility of Curriculum-Based Measurement and Performance Assessment as Alternatives to Traditional Intelligence and Achievement Tests.

ERIC Educational Resources Information Center

Elliott, Stephen N.; Fuchs, Lynn S.

1997-01-01

Curriculum-based measurement and performance assessments can provide valuable data for making special-education eligibility decisions. Reviews applied research on these assessment approaches and discusses the practical context of treatment validation and decisions about instructional services for students with diverse academic needs. (Author/JDM)

Descriptive and Criterion-Referenced Self-Assessment with L2 Readers

ERIC Educational Resources Information Center

Brantmeier, Cindy; Vanderplank, Robert

2008-01-01

Brantmeier [Brantmeier, C., 2006. "Advanced L2 learners and reading placement: self-assessment, computer-based testing, and subsequent performance." 'System 34" (1), 15-35] found that self-assessment (SA) of second language (L2) reading ability is not an accurate predictor for computer-based testing or subsequent classroom performance. With 359…

Specifying and Refining a Measurement Model for a Simulation-Based Assessment. CSE Report 619.

ERIC Educational Resources Information Center

Levy, Roy; Mislevy, Robert J.

2004-01-01

The challenges of modeling students' performance in simulation-based assessments include accounting for multiple aspects of knowledge and skill that arise in different situations and the conditional dependencies among multiple aspects of performance in a complex assessment. This paper describes a Bayesian approach to modeling and estimating…

Application of process mining to assess the data quality of routinely collected time-based performance data sourced from electronic health records by validating process conformance.

PubMed

Perimal-Lewis, Lua; Teubner, David; Hakendorf, Paul; Horwood, Chris

2016-12-01

Effective and accurate use of routinely collected health data to produce Key Performance Indicator reporting is dependent on the underlying data quality. In this research, Process Mining methodology and tools were leveraged to assess the data quality of time-based Emergency Department data sourced from electronic health records. This research was done working closely with the domain experts to validate the process models. The hospital patient journey model was used to assess flow abnormalities which resulted from incorrect timestamp data used in time-based performance metrics. The research demonstrated process mining as a feasible methodology to assess data quality of time-based hospital performance metrics. The insight gained from this research enabled appropriate corrective actions to be put in place to address the data quality issues. © The Author(s) 2015.

Mining Formative Evaluation Rules Using Web-Based Learning Portfolios for Web-Based Learning Systems

ERIC Educational Resources Information Center

Chen, Chih-Ming; Hong, Chin-Ming; Chen, Shyuan-Yi; Liu, Chao-Yu

2006-01-01

Learning performance assessment aims to evaluate what knowledge learners have acquired from teaching activities. Objective technical measures of learning performance are difficult to develop, but are extremely important for both teachers and learners. Learning performance assessment using learning portfolios or web server log data is becoming an…

Performance assessments of Android-powered military applications operating on tactical handheld devices

NASA Astrophysics Data System (ADS)

Weiss, Brian A.; Fronczek, Lisa; Morse, Emile; Kootbally, Zeid; Schlenoff, Craig

2013-05-01

Transformative Apps (TransApps) is a Defense Advanced Research Projects Agency (DARPA) funded program whose goal is to develop a range of militarily-relevant software applications ("apps") to enhance the operational-effectiveness of military personnel on (and off) the battlefield. TransApps is also developing a military apps marketplace to facilitate rapid development and dissemination of applications to address user needs by connecting engaged communities of endusers with development groups. The National Institute of Standards and Technology's (NIST) role in the TransApps program is to design and implement evaluation procedures to assess the performance of: 1) the various software applications, 2) software-hardware interactions, and 3) the supporting online application marketplace. Specifically, NIST is responsible for evaluating 50+ tactically-relevant applications operating on numerous Android™-powered platforms. NIST efforts include functional regression testing and quantitative performance testing. This paper discusses the evaluation methodologies employed to assess the performance of three key program elements: 1) handheld-based applications and their integration with various hardware platforms, 2) client-based applications and 3) network technologies operating on both the handheld and client systems along with their integration into the application marketplace. Handheld-based applications are assessed using a combination of utility and usability-based checklists and quantitative performance tests. Client-based applications are assessed to replicate current overseas disconnected (i.e. no network connectivity between handhelds) operations and to assess connected operations envisioned for later use. Finally, networked applications are assessed on handhelds to establish baselines of performance for when connectivity will be common usage.

Sidney Blatt's Contributions to Personality Assessment.

PubMed

Auerbach, John S

2016-01-01

Over a long, distinguished career, Sidney Blatt contributed to theory and research in personality development, personality assessment, and psychotherapy. Best known for his 2-configurations model of personality and author or co-author of more than 250 articles and 18 books and monographs, Blatt was also a master clinician, a psychoanalyst who was awarded the 1989 Bruno J. Klopfer Award by the Society for Personality Assessment (SPA) for his contributions to both self-report and performance-based assessment. He was also the president of SPA from 1984 to 1986. This special series contains papers by writers who participated in all aspects of Blatt's contributions to personality assessment, both self-report and performance-based. Topics covered include Blatt's 2-configurations model of personality, development, and psychopathology; boundary disturbance and psychosis in performance-based assessment; the interaction of gender and personality on narrative assessments; and the Object Relations Inventory and differentiation relatedness, especially as these relate to therapeutic outcome.

Validation of a Performance Assessment Instrument in Problem-Based Learning Tutorials Using Two Cohorts of Medical Students

ERIC Educational Resources Information Center

Lee, Ming; Wimmers, Paul F.

2016-01-01

Although problem-based learning (PBL) has been widely used in medical schools, few studies have attended to the assessment of PBL processes using validated instruments. This study examined reliability and validity for an instrument assessing PBL performance in four domains: Problem Solving, Use of Information, Group Process, and Professionalism.…

Engaging with Faculty to Develop, Implement, and Pilot Electronic Performance Assessments of Student Teachers Using Mobile Devices

ERIC Educational Resources Information Center

Haughton, Noela A.; Keil, Virginia L.

2009-01-01

This article discusses the development and implementation of a technology-supported student teacher performance assessment that supports integration with a larger electronic assessment system. The authors spearheaded a multidisciplinary team to develop a comprehensive performance assessment based on the Pathwise framework. The team collaborated…

Exploring Proficiency-Based vs. Performance-Based Items with Elicited Imitation Assessment

ERIC Educational Resources Information Center

Cox, Troy L.; Bown, Jennifer; Burdis, Jacob

2015-01-01

This study investigates the effect of proficiency- vs. performance-based elicited imitation (EI) assessment. EI requires test-takers to repeat sentences in the target language. The accuracy at which test-takers are able to repeat sentences highly correlates with test-takers' language proficiency. However, in EI, the factors that render an item…

Psychometric Properties of Performance-based Measurements of Functional Capacity: Test-Retest Reliability, Practice Effects, and Potential Sensitivity to Change

PubMed Central

Leifker, Feea R.; Patterson, Thomas L.; Bowie, Christopher R.; Mausbach, Brent T.; Harvey, Philip D.

2010-01-01

Performance-based measures of the ability to perform social and everyday living skills are being more widely used to assess functional capacity in people with serious mental illnesses such as schizophrenia and bipolar disorder. Since they are also being used as outcome measures in pharmacological and cognitive remediation studies aimed at cognitive impairments in schizophrenia, understanding their measurement properties and potential sensitivity to change is important. In this study, the test-retest reliability, practice effects, and reliable change indices of two different performance-based functional capacity measures, the UCSD Performance-based skills assessment (UPSA) and Social skills performance assessment (SSPA) were examined over several different retest intervals in two different samples of people with schizophrenia (n’s=238 and 116) and a healthy comparison sample (n=109). These psychometric properties were compared to those of a neuropsychological assessment battery. Test-retest reliabilities of the long form of the UPSA ranged from r=.63 to r=.80 over follow-up periods up to 36 months in people with schizophrenia, while brief UPSA reliabilities ranged from r=.66 to r=.81. Test-retest reliability of the NP performance scores ranged from r=.77 to r=.79. Test-retest reliabilities of the UPSA were lower in healthy controls, while NP performance was slightly more reliable. SSPA test-retest reliability was lower. Practice effect sizes ranged from .05 to .16 for the UPSA and .07 to .19 for the NP assessment in patients, with HC having more practice effects. Reliable change intervals were consistent across NP and both FC measures, indicating equal potential for detection of change. These performance-based measures of functional capacity appear to have similar potential to be sensitive to change compared to NP performance in people with schizophrenia. PMID:20399613

Comparison of performance-based assessment and real world skill in people with serious mental illness: Ecological validity of the Test of Grocery Shopping Skills.

PubMed

Faith, Laura A; Rempfer, Melisa V

2018-05-07

Valid functional measures are essential for clinical and research efforts that address recovery and community functioning in people with serious mental illness. Although there is a great deal of interest in functional assessment, there is limited research supporting how well current evaluation methods provide a true assessment of real world functioning or naturalistic behavior. To address this gap in the literature, the present study examined the performance of individuals with serious mental illness (i.e., diagnosis of schizophrenia-spectrum, bipolar disorder, or other depression/anxiety diagnoses and accompanying functional disability) on the Test of Grocery Shopping Skills (TOGSS), a performance-based naturalistic task. We compared TOGSS performance to two dimensions of real world functioning: directly observed real world grocery shopping and ratings of community functioning. Results indicated that the TOGSS was significantly associated with real life grocery shopping, in terms of both shopping accuracy (r = 0.424) and time (r = 0.491). Further, self-report and observer-rated methods of assessing real world shopping behaviors were significantly correlated (r = 0.455). To our knowledge, this is one of the first studies to directly compare a performance-based naturalistic skill assessment with carefully observed real world performance of that skill in people with serious mental illness. These findings support the feasibility and ecological validity of performance-based naturalistic assessment with the TOGSS. Copyright © 2018 Elsevier B.V. All rights reserved.

A meta-analytic review of self-reported, clinician-rated, and performance-based motivation measures in schizophrenia: Are we measuring the same "stuff"?

PubMed

Luther, Lauren; Firmin, Ruth L; Lysaker, Paul H; Minor, Kyle S; Salyers, Michelle P

2018-04-07

An array of self-reported, clinician-rated, and performance-based measures has been used to assess motivation in schizophrenia; however, the convergent validity evidence for these motivation assessment methods is mixed. The current study is a series of meta-analyses that summarize the relationships between methods of motivation measurement in 45 studies of people with schizophrenia. The overall mean effect size between self-reported and clinician-rated motivation measures (r = 0.27, k = 33) was significant, positive, and approaching medium in magnitude, and the overall effect size between performance-based and clinician-rated motivation measures (r = 0.21, k = 11) was positive, significant, and small in magnitude. The overall mean effect size between self-reported and performance-based motivation measures was negligible and non-significant (r = -0.001, k = 2), but this meta-analysis was underpowered. Findings suggest modest convergent validity between clinician-rated and both self-reported and performance-based motivation measures, but additional work is needed to clarify the convergent validity between self-reported and performance-based measures. Further, there is likely more variability than similarity in the underlying construct that is being assessed across the three methods, particularly between the performance-based and other motivation measurement types. These motivation assessment methods should not be used interchangeably, and measures should be more precisely described as the specific motivational construct or domain they are capturing. Copyright © 2018 Elsevier Ltd. All rights reserved.

Assessing Therapist Competence: Development of a Performance-Based Measure and Its Comparison With a Web-Based Measure.

PubMed

Cooper, Zafra; Doll, Helen; Bailey-Straebler, Suzanne; Bohn, Kristin; de Vries, Dian; Murphy, Rebecca; O'Connor, Marianne E; Fairburn, Christopher G

2017-10-31

Recent research interest in how best to train therapists to deliver psychological treatments has highlighted the need for rigorous, but scalable, means of measuring therapist competence. There are at least two components involved in assessing therapist competence: the assessment of their knowledge of the treatment concerned, including how and when to use its strategies and procedures, and an evaluation of their ability to apply such knowledge skillfully in practice. While the assessment of therapists' knowledge has the potential to be completed efficiently on the Web, the assessment of skill has generally involved a labor-intensive process carried out by clinicians, and as such, may not be suitable for assessing training outcome in certain circumstances. The aims of this study were to develop and evaluate a role-play-based measure of skill suitable for assessing training outcome and to compare its performance with a highly scalable Web-based measure of applied knowledge. Using enhanced cognitive behavioral therapy (CBT-E) for eating disorders as an exemplar, clinical scenarios for role-play assessment were developed and piloted together with a rating scheme for assessing trainee therapists' performance. These scenarios were evaluated by examining the performance of 93 therapists from different professional backgrounds and at different levels of training in implementing CBT-E. These therapists also completed a previously developed Web-based measure of applied knowledge, and the ability of the Web-based measure to efficiently predict competence on the role-play measure was investigated. The role-play measure assessed performance at implementing a range of CBT-E procedures. The majority of the therapists rated their performance as moderately or closely resembling their usual clinical performance. Trained raters were able to achieve good-to-excellent reliability for averaged competence, with intraclass correlation coefficients ranging from .653 to 909. The measure was also sensitive to change, with scores being significantly higher after training than before as might be expected (mean difference 0.758, P<.001) even when taking account of repeated data (mean difference 0.667, P<.001). The major shortcoming of the role-play measure was that it required considerable time and resources. This shortcoming is inherent in the method. Given this, of most interest for assessing training outcome, scores on the Web-based measure efficiently predicted therapist competence, as judged by the role-play measure (with the Web-based measure having a positive predictive value of 77% and specificity of 78%). The results of this study suggest that while it was feasible and acceptable to assess performance using the newly developed role-play measure, the highly scalable Web-based measure could be used in certain circumstances as a substitute for the more labor-intensive, and hence, more costly role-play method. ©Zafra Cooper, Helen Doll, Suzanne Bailey-Straebler, Kristin Bohn, Dian de Vries, Rebecca Murphy, Marianne E O'Connor, Christopher G Fairburn. Originally published in JMIR Mental Health (http://mental.jmir.org), 31.10.2017.

T59. VIRTUAL REALTY ASSESSMENT OF FUNCTIONAL CAPACITY IN EARLY SCHIZOPHRENIA: ASSOCIATIONS WITH NEUROCOGNITION, FUNCTIONAL CAPACITY PERFORMANCE, AND DAILY FUNCTIONING

PubMed Central

Ventura, Joseph; Welikson, Tamara; Subotnik, Kenneth L; Ered, Arielle; Keefe, Richard; Hellemann, Gerhard H; Nuechterlein, Keith H

2018-01-01

Abstract Background Research using virtual reality assessment of functional capacity has shown promise as a reliable and valid way to assess treatment response in patients with established schizophrenia. There has been little work on virtual reality based assessments of functional capacity for patients in the early phase of schizophrenia. We examined whether virtual reality based assessment methods reveal functional capacity deficits in young patients and relevant relationships with established measures of neurocognition, functional capacity performance, and daily functioning. Methods The sample consisted of UCLA Aftercare Research Program patients (n=42) who were diagnosed by trained raters administering the SCID and who met criteria for schizophrenia, schizoaffective disorder, or schizophreniform disorder, and screened normal control subjects (n=13). Patients were within 2 years of their first psychotic episode upon clinic entry, were an average of 23.2 years old, and had an average of 12.9 years of education. The Virtual Reality Functional Capacity Assessment Tool (VRFCAT) was the computer-based measure of functional capacity. We used the MATRICS Consensus Cognitive Battery (MCCB) as an objective measure of neurocognition and the UCSD Performance-Based Skills Assessment (UPSA) to assess functional capacity performance. The Global Functioning Scale: Role and Social, and the Role Functioning Scale were used to assess work and school performance, familial interactions, and social functioning. Results We were able to confirm that the deficit in functional capacity performance measured using VRFCAT is present in the early course of schizophrenia in that the patients were slower and committed more errors (M=830.41) as compared with normal controls (M=716.84; t=3.0, p<.01). Virtual reality based assessment of functional capacity was correlated with objective measures of neurocognition (MCCB Overall Composite), r=-.71, p=<.01, standard approaches to functional capacity assessment (UPSA), r=-.66, p=<.01, work and school functioning (r=-.52, p<.01), and level of social relationships (r=-.43, p=<.03), but not familial relationships (r=-.03, p=.87). Interestingly, neither neurocognition (MCCB) nor functional capacity performance (UPSA) were correlated with the level of familial relationships. Discussion We extend previous findings in that even patients in the early course of schizophrenia showed virtual reality based functional capacity performance deficits when compared with normal control subjects. Virtual reality based performance was correlated with neurocognition, suggesting that it may be sensitive to changes in cognition. Furthermore, correlations with everyday work/school and social functioning indicate promise as a co-primary measure to index change in functioning in response to treatment. Interestingly, none of our measures of functional capacity or neurocognition were correlated with familial relationships indicating that the determinates of family interactions might be driven by factors other than cognitive capacities.

Performance-based classrooms: A case study of two elementary teachers of mathematics and science

NASA Astrophysics Data System (ADS)

Jones, Kenneth W.

This case study depicts how two elementary teachers develop classrooms devoted to performance-based instruction in mathematics and science. The purpose is to develop empirical evidence of classroom practices that leads to a conceptual framework about the nature of performance-based instruction. Performance-based assessment and instruction are defined from the literature to entail involving students in tasks that are complex and engaging, requiring them to apply knowledge and skills in authentic contexts. In elementary mathematics and science, such an approach emphasizes problem solving, exploration, inquiry, and reasoning. The body of the work examines teacher beliefs, curricular orientations, instructional strategies, assessment approaches, management and organizational skills, and interpersonal relationships. The focus throughout is on those aspects that foster student performance in elementary mathematics and science. The resulting framework describes five characteristics that contribute to performance-based classrooms: a caring classroom community, a connectionist learning theory, a thinking and doing curriculum, diverse opportunities for learning, and ongoing assessment, feedback, and adjustment. The conclusion analyzes factors external to the classroom that support or constrain the development of performance-based classrooms and discusses the implications for educational policy and further research.

Is Teacher Assessment Reliable or Valid for High School Students under a Web-Based Portfolio Environment?

ERIC Educational Resources Information Center

Chang, Chi-Cheng; Wu, Bing-Hong

2012-01-01

This study explored the reliability and validity of teacher assessment under a Web-based portfolio assessment environment (or Web-based teacher portfolio assessment). Participants were 72 eleventh graders taking the "Computer Application" course. The students perform portfolio creation, inspection, self- and peer-assessment using the Web-based…

Rater Training to Support High-Stakes Simulation-Based Assessments

PubMed Central

Feldman, Moshe; Lazzara, Elizabeth H.; Vanderbilt, Allison A.; DiazGranados, Deborah

2013-01-01

Competency-based assessment and an emphasis on obtaining higher-level outcomes that reflect physicians’ ability to demonstrate their skills has created a need for more advanced assessment practices. Simulation-based assessments provide medical education planners with tools to better evaluate the 6 Accreditation Council for Graduate Medical Education (ACGME) and American Board of Medical Specialties (ABMS) core competencies by affording physicians opportunities to demonstrate their skills within a standardized and replicable testing environment, thus filling a gap in the current state of assessment for regulating the practice of medicine. Observational performance assessments derived from simulated clinical tasks and scenarios enable stronger inferences about the skill level a physician may possess, but also introduce the potential of rater errors into the assessment process. This article reviews the use of simulation-based assessments for certification, credentialing, initial licensure, and relicensing decisions and describes rater training strategies that may be used to reduce rater errors, increase rating accuracy, and enhance the validity of simulation-based observational performance assessments. PMID:23280532

Game-Based Assessment: Investigating the Impact on Test Anxiety and Exam Performance

ERIC Educational Resources Information Center

Mavridis, A.; Tsiatsos, T.

2017-01-01

The aim of this study is to assess the impact of a 3D educational computer game on students' test anxiety and exam performance when used in evaluative situations as compared to the traditional method of examination. The participants of the study were students in tertiary education who were examined using game-based assessment and traditional…

A practical approach to competency assessment.

PubMed

Claflin, N

1997-01-01

Assessing clinical performance is difficult. Members of the Nursing Service Clinical Practice Committee at the Carl T. Hayden Veterans Affairs Medical Center in Phoenix developed a comprehensive program of competency assessment based on performance measures. This article describes the committee's process of developing and implementing the program and includes a blueprint for competency assessment and selected performance measures for all nursing staff who provide patient care. The approach to competency assessment includes performance measures specific to patients' ages.

Performance Assessment Links in Science.

ERIC Educational Resources Information Center

Quellmalz, Edys; Schank, Patricia; Hinojosa, Thomas; Padilla, Christine

1999-01-01

Describes work in progress at SRI International in the Performance Assessment Links in Science (PALS) project, which is developing an online, standards-based, interactive resource bank of science performance assessments. Coupled with the development of the resource bank is a program of research on effective use of these resources. (SLD)

Metrics for Performance Evaluation of Patient Exercises during Physical Therapy.

PubMed

Vakanski, Aleksandar; Ferguson, Jake M; Lee, Stephen

2017-06-01

The article proposes a set of metrics for evaluation of patient performance in physical therapy exercises. Taxonomy is employed that classifies the metrics into quantitative and qualitative categories, based on the level of abstraction of the captured motion sequences. Further, the quantitative metrics are classified into model-less and model-based metrics, in reference to whether the evaluation employs the raw measurements of patient performed motions, or whether the evaluation is based on a mathematical model of the motions. The reviewed metrics include root-mean square distance, Kullback Leibler divergence, log-likelihood, heuristic consistency, Fugl-Meyer Assessment, and similar. The metrics are evaluated for a set of five human motions captured with a Kinect sensor. The metrics can potentially be integrated into a system that employs machine learning for modelling and assessment of the consistency of patient performance in home-based therapy setting. Automated performance evaluation can overcome the inherent subjectivity in human performed therapy assessment, and it can increase the adherence to prescribed therapy plans, and reduce healthcare costs.

Evaluation of the performance of existing non-laboratory based cardiovascular risk assessment algorithms

PubMed Central

2013-01-01

Background The high burden and rising incidence of cardiovascular disease (CVD) in resource constrained countries necessitates implementation of robust and pragmatic primary and secondary prevention strategies. Many current CVD management guidelines recommend absolute cardiovascular (CV) risk assessment as a clinically sound guide to preventive and treatment strategies. Development of non-laboratory based cardiovascular risk assessment algorithms enable absolute risk assessment in resource constrained countries. The objective of this review is to evaluate the performance of existing non-laboratory based CV risk assessment algorithms using the benchmarks for clinically useful CV risk assessment algorithms outlined by Cooney and colleagues. Methods A literature search to identify non-laboratory based risk prediction algorithms was performed in MEDLINE, CINAHL, Ovid Premier Nursing Journals Plus, and PubMed databases. The identified algorithms were evaluated using the benchmarks for clinically useful cardiovascular risk assessment algorithms outlined by Cooney and colleagues. Results Five non-laboratory based CV risk assessment algorithms were identified. The Gaziano and Framingham algorithms met the criteria for appropriateness of statistical methods used to derive the algorithms and endpoints. The Swedish Consultation, Framingham and Gaziano algorithms demonstrated good discrimination in derivation datasets. Only the Gaziano algorithm was externally validated where it had optimal discrimination. The Gaziano and WHO algorithms had chart formats which made them simple and user friendly for clinical application. Conclusion Both the Gaziano and Framingham non-laboratory based algorithms met most of the criteria outlined by Cooney and colleagues. External validation of the algorithms in diverse samples is needed to ascertain their performance and applicability to different populations and to enhance clinicians’ confidence in them. PMID:24373202

Does feedback matter? Practice-based learning for medical students after a multi-institutional clinical performance examination.

PubMed

Srinivasan, Malathi; Hauer, Karen E; Der-Martirosian, Claudia; Wilkes, Michael; Gesundheit, Neil

2007-09-01

Achieving competence in 'practice-based learning' implies that doctors can accurately self- assess their clinical skills to identify behaviours that need improvement. This study examines the impact of receiving feedback via performance benchmarks on medical students' self-assessment after a clinical performance examination (CPX). The authors developed a practice-based learning exercise at 3 institutions following a required 8-station CPX for medical students at the end of Year 3. Standardised patients (SPs) scored students after each station using checklists developed by experts. Students assessed their own performance immediately after the CPX (Phase 1). One month later, students watched their videotaped performance and reassessed (Phase 2). Some students received performance benchmarks (their scores, plus normative class data) before the video review. Pearson's correlations between self-ratings and SP ratings were calculated for overall performance and specific skill areas (history taking, physical examination, doctor-patient communication) for Phase 1 and Phase 2. The 2 correlations were then compared for each student group (i.e. those who received and those who did not receive feedback). A total of 280 students completed both study phases. Mean CPX scores ranged from 51% to 71% of items correct overall and for each skill area. Phase 1 self-assessment correlated weakly with SP ratings of student performance (r = 0.01-0.16). Without feedback, Phase 2 correlations remained weak (r = 0.13-0.18; n = 109). With feedback, Phase 2 correlations improved significantly (r = 0.26-0.47; n = 171). Low-performing students showed the greatest improvement after receiving feedback. The accuracy of student self-assessment was poor after a CPX, but improved significantly with performance feedback (scores and benchmarks). Videotape review alone (without feedback) did not improve self-assessment accuracy. Practice-based learning exercises that incorporate feedback to medical students hold promise to improve self-assessment skills.

Performance-Based Assessment Resource Guide.

ERIC Educational Resources Information Center

Gilbert, Judith C.; Burger, Patricia

This resource guide reviews a variety of performance-based student assessment strategies, and provides examples of, and references for, the strategies. Strategies include anecdotal records, interviews, peer report and group evaluations, and portfolios. Materials in the guide include: (1) a resource guide update form for teachers; (2) department…

Performance Based Assessment and Instructional Activities in Communication Arts for Marketing Education. Application Activities for Communications in Marketing, Employment and Advancement, and Selling.

ERIC Educational Resources Information Center

Broeker, Arlene M.

This document contains performance-based assessment and instructional activities for Missouri high school teachers to use in teaching the communications arts needed by marketing education students. The activities included were developed to reflect Missouri's new Show-Me Standards, which are knowledge (content) and performance (process) standards…

University Students' Attainment and Perceptions of Computer Delivered Assessment; A Comparison between Computer-Based and Traditional Tests in a "High-Stakes" Examination

ERIC Educational Resources Information Center

Escudier, M. P.; Newton, T. J.; Cox, M. J.; Reynolds, P. A.; Odell, E. W.

2011-01-01

This study compared higher education dental undergraduate student performance in online assessments with performance in traditional paper-based tests and investigated students' perceptions of the fairness and acceptability of online tests, and showed performance to be comparable. The project design involved two parallel cross-over trials, one in…

Strategies for increasing the feasibility of performance assessments during competency-based education: Subjective and objective evaluations correlate in the operating room.

PubMed

Szasz, Peter; Louridas, Marisa; Harris, Kenneth A; Grantcharov, Teodor P

2017-08-01

Competency-based education necessitates assessments that determine whether trainees have acquired specific competencies. The evidence on the ability of internal raters (staff surgeons) to provide accurate assessments is mixed; however, this has not yet been directly explored in the operating room. This study's objective is to compare the ratings given by internal raters vs an expert external rater (independent to the training process) in the operating room. Raters assessed general surgery residents during a laparoscopic cholecystectomy for their technical and nontechnical performance. Fifteen cases were observed. There was a moderately positive correlation (r s = .618, P = .014) for technical performance and a strong positive correlation (r s = .731, P = .002) for nontechnical performance. The internal raters were less stringent for technical (mean rank 3.33 vs 8.64, P = .007) and nontechnical (mean rank 3.83 vs 8.50, P = .01) performances. This study provides evidence to help operationalize competency-based assessments. Copyright © 2016 Elsevier Inc. All rights reserved.

360-degree physician performance assessment.

PubMed

Dubinsky, Isser; Jennings, Kelly; Greengarten, Moshe; Brans, Amy

2010-01-01

Few jurisdictions have a robust common approach to assessing the quantitative and qualitative dimensions of physician performance. In this article, we examine the need for 360-degree physician performance assessment and review the literature supporting comprehensive physician assessment. An evidence-based, "best practice" approach to the development of a 360-degree physician performance assessment framework is presented, including an overview of a tool kit to support implementation. The focus of the framework is to support physician career planning and to enhance the quality of patient care. Finally, the legal considerations related to implementing 360-degree physician performance assessment are explored.

Performance Level Assessment: Developing Quality and Consistency through Research Partnerships.

ERIC Educational Resources Information Center

Sanderson, Kay; Pearce, Ron

Performance level assessment (PLA) is a type of supplementary evidence system for competency-based assessment that is being pilot tested at five technical and further education (TAFE) institutes throughout Queensland, Australia. The PLA system uses the following three levels of performance: competency achieved; competency achieved with credit; and…

High-speed engine/component performance assessment using exergy and thrust-based methods

NASA Technical Reports Server (NTRS)

Riggins, D. W.

1996-01-01

This investigation summarizes a comparative study of two high-speed engine performance assessment techniques based on energy (available work) and thrust-potential (thrust availability). Simple flow-fields utilizing Rayleigh heat addition and one-dimensional flow with friction are used to demonstrate the fundamental inability of conventional energy techniques to predict engine component performance, aid in component design, or accurately assess flow losses. The use of the thrust-based method on these same examples demonstrates its ability to yield useful information in all these categories. Energy and thrust are related and discussed from the stand-point of their fundamental thermodynamic and fluid dynamic definitions in order to explain the differences in information obtained using the two methods. The conventional definition of energy is shown to include work which is inherently unavailable to an aerospace Brayton engine. An engine-based energy is then developed which accurately accounts for this inherently unavailable work; performance parameters based on this quantity are then shown to yield design and loss information equivalent to the thrust-based method.

Exploring the Role of Performance Assessment in Competency-Based Education: Recommendations from the National Summit on K-12 Competency-Based Education

ERIC Educational Resources Information Center

Gagnon, Laurie

2017-01-01

Based on reports created for the 2017 National Summit on K-12 Competency-Based Education, this article explores how educators can take action to build momentum for and implement competency education and performance assessment. As a guide to taking action, the article provides the following: (1) links to the four framing reports from the Summit;…

A Team, Case-based Examination and Its Impact on Student Performance in a Patient Safety and Informatics Course

PubMed Central

Etheridge, Kierstan; DeLellis, Teresa

2017-01-01

Objective. To describe the redesigned assessment plan for a patient safety and informatics course and assess student pharmacist performance and perceptions. Methods. The final examination of a patient safety course was redesigned from traditional multiple choice and short answer to team-based, open-ended, and case-based. Faculty for each class session developed higher level activities, focused on developing key skills or attitudes deemed essential for practice, for a progressive patient case consisting of nine activities. Student performance and perceptions were analyzed with pre- and post-surveys using 5-point scales. Results. Mean performance on the examination was 93.6%; median scores for each assessed course outcome ranged from 90% to 100%. Eighty-five percent of students completed both surveys. Confidence performing skills and demonstrating attitudes improved for each item on post-survey compared with pre-survey. Eighty-one percent of students indicated the experience of taking the examination was beneficial for their professional development. Conclusion. A team, case-based examination was associated with high student performance and improved self-confidence in performing medication safety-related skills. PMID:28970618

Do Expert Clinical Teachers Have a Shared Understanding of What Constitutes a Competent Reasoning Performance in Case-Based Teaching?

ERIC Educational Resources Information Center

Gauthier, Geneviève; Lajoie, Susanne P.

2014-01-01

To explore the assessment challenge related to case based learning we study how experienced clinical teachers--i.e., those who regularly teach and assess case-based learning--conceptualize the notion of competent reasoning performance for specific teaching cases. Through an in-depth qualitative case study of five expert teachers, we investigate…

A randomized controlled study to evaluate the role of video-based coaching in training laparoscopic skills.

PubMed

Singh, Pritam; Aggarwal, Rajesh; Tahir, Muaaz; Pucher, Philip H; Darzi, Ara

2015-05-01

This study evaluates whether video-based coaching can enhance laparoscopic surgical skills performance. Many professions utilize coaching to improve performance. The sports industry employs video analysis to maximize improvement from every performance. Laparoscopic novices were baseline tested and then trained on a validated virtual reality (VR) laparoscopic cholecystectomy (LC) curriculum. After competence, subjects were randomized on a 1:1 ratio and each performed 5 VRLCs. After each LC, intervention group subjects received video-based coaching by a surgeon, utilizing an adaptation of the GROW (Goals, Reality, Options, Wrap-up) coaching model. Control subjects viewed online surgical lectures. All subjects then performed 2 porcine LCs. Performance was assessed by blinded video review using validated global rating scales. Twenty subjects were recruited. No significant differences were observed between groups in baseline performance and in VRLC1. For each subsequent repetition, intervention subjects significantly outperformed controls on all global rating scales. Interventions outperformed controls in porcine LC1 [Global Operative Assessment of Laparoscopic Skills: (20.5 vs 15.5; P = 0.011), Objective Structured Assessment of Technical Skills: (21.5vs 14.5; P = 0.001), and Operative Performance Rating System: (26 vs 19.5; P = 0.001)] and porcine LC2 [Global Operative Assessment of Laparoscopic Skills: (28 vs 17.5; P = 0.005), Objective Structured Assessment of Technical Skills: (30 vs 16.5; P < 0.001), and Operative Performance Rating System: (36 vs 21; P = 0.004)]. Intervention subjects took significantly longer than controls in porcine LC1 (2920 vs 2004 seconds; P = 0.009) and LC2 (2297 vs 1683; P = 0.003). Despite equivalent exposure to practical laparoscopic skills training, video-based coaching enhanced the quality of laparoscopic surgical performance on both VR and porcine LCs, although at the expense of increased time. Video-based coaching is a feasible method of maximizing performance enhancement from every clinical exposure.

Workplace-based assessment: raters' performance theories and constructs.

PubMed

Govaerts, M J B; Van de Wiel, M W J; Schuwirth, L W T; Van der Vleuten, C P M; Muijtjens, A M M

2013-08-01

Weaknesses in the nature of rater judgments are generally considered to compromise the utility of workplace-based assessment (WBA). In order to gain insight into the underpinnings of rater behaviours, we investigated how raters form impressions of and make judgments on trainee performance. Using theoretical frameworks of social cognition and person perception, we explored raters' implicit performance theories, use of task-specific performance schemas and the formation of person schemas during WBA. We used think-aloud procedures and verbal protocol analysis to investigate schema-based processing by experienced (N = 18) and inexperienced (N = 16) raters (supervisor-raters in general practice residency training). Qualitative data analysis was used to explore schema content and usage. We quantitatively assessed rater idiosyncrasy in the use of performance schemas and we investigated effects of rater expertise on the use of (task-specific) performance schemas. Raters used different schemas in judging trainee performance. We developed a normative performance theory comprising seventeen inter-related performance dimensions. Levels of rater idiosyncrasy were substantial and unrelated to rater expertise. Experienced raters made significantly more use of task-specific performance schemas compared to inexperienced raters, suggesting more differentiated performance schemas in experienced raters. Most raters started to develop person schemas the moment they began to observe trainee performance. The findings further our understanding of processes underpinning judgment and decision making in WBA. Raters make and justify judgments based on personal theories and performance constructs. Raters' information processing seems to be affected by differences in rater expertise. The results of this study can help to improve rater training, the design of assessment instruments and decision making in WBA.

Human behavior and human performance: Psychomotor demands

NASA Technical Reports Server (NTRS)

1992-01-01

The results of several experiments are presented in abstract form. These studies are critical for the interpretation and acceptance of flight based science to be conducted by the Behavior and Performance project. Some representative titles are as follow: External audio for IBM/PC compatible computers; A comparative assessment of psychomotor performance (target prediction by humans and macaques); Response path (a dependent measure for computer maze solving and other tasks); Behavioral asymmetries of psychomotor performance in Rhesus monkey (a dissociation between hand preference and skill); Testing primates with joystick based automated apparatus; and Environmental enrichment and performance assessment for ground or flight based research with primates;

Effects of Selected Task Performance Criteria at Initiating Adaptive Task Real locations

NASA Technical Reports Server (NTRS)

Montgomery, Demaris A.

2001-01-01

In the current report various performance assessment methods used to initiate mode transfers between manual control and automation for adaptive task reallocation were tested. Participants monitored two secondary tasks for critical events while actively controlling a process in a fictional system. One of the secondary monitoring tasks could be automated whenever operators' performance was below acceptable levels. Automation of the secondary task and transfer of the secondary task back to manual control were either human- or machine-initiated. Human-initiated transfers were based on the operator's assessment of the current task demands while machine-initiated transfers were based on the operators' performance. Different performance assessment methods were tested in two separate experiments.

Performance of a computer-based assessment of cognitive function measures in two cohorts of seniors

USDA-ARS?s Scientific Manuscript database

Computer-administered assessment of cognitive function is being increasingly incorporated in clinical trials, however its performance in these settings has not been systematically evaluated. The Seniors Health and Activity Research Program (SHARP) pilot trial (N=73) developed a computer-based tool f...

Specifying and Refining a Measurement Model for a Computer-Based Interactive Assessment

ERIC Educational Resources Information Center

Levy, Roy; Mislevy, Robert J.

2004-01-01

The challenges of modeling students' performance in computer-based interactive assessments include accounting for multiple aspects of knowledge and skill that arise in different situations and the conditional dependencies among multiple aspects of performance. This article describes a Bayesian approach to modeling and estimating cognitive models…

Workplace-Based Assessment: Raters' Performance Theories and Constructs

ERIC Educational Resources Information Center

Govaerts, M. J. B.; Van de Wiel, M. W. J.; Schuwirth, L. W. T.; Van der Vleuten, C. P. M.; Muijtjens, A. M. M.

2013-01-01

Weaknesses in the nature of rater judgments are generally considered to compromise the utility of workplace-based assessment (WBA). In order to gain insight into the underpinnings of rater behaviours, we investigated how raters form impressions of and make judgments on trainee performance. Using theoretical frameworks of social cognition and…

Exploring the Impact of Mental Workload on Rater-Based Assessments

ERIC Educational Resources Information Center

Tavares, Walter; Eva, Kevin W.

2013-01-01

When appraising the performance of others, assessors must acquire relevant information and process it in a meaningful way in order to translate it effectively into ratings, comments, or judgments about how well the performance meets appropriate standards. Rater-based assessment strategies in health professional education, including scale and…

Teachers' Reactions towards Performance-Based Language Assessment

ERIC Educational Resources Information Center

Chinda, Bordin

2014-01-01

This research aims at examining the reactions of tertiary EFL teachers towards the use of performance-based language assessment. The study employed a mixed-method research methodology. For the quantitative method, 36 teachers responded to a questionnaire survey. In addition, four teachers participated in the in-depth interviews which were…

Construct and face validity of the educational computer-based environment (ECE) assessment scenarios for basic endoneurosurgery skills.

PubMed

Cagiltay, Nergiz Ercil; Ozcelik, Erol; Sengul, Gokhan; Berker, Mustafa

2017-11-01

In neurosurgery education, there is a paradigm shift from time-based training to criterion-based model for which competency and assessment becomes very critical. Even virtual reality simulators provide alternatives to improve education and assessment in neurosurgery programs and allow for several objective assessment measures, there are not many tools for assessing the overall performance of trainees. This study aims to develop and validate a tool for assessing the overall performance of participants in a simulation-based endoneurosurgery training environment. A training program was developed in two levels: endoscopy practice and beginning surgical practice based on four scenarios. Then, three experiments were conducted with three corresponding groups of participants (Experiment 1, 45 (32 beginners, 13 experienced), Experiment 2, 53 (40 beginners, 13 experienced), and Experiment 3, 26 (14 novices, 12 intermediate) participants). The results analyzed to understand the common factors among the performance measurements of these experiments. Then, a factor capable of assessing the overall skill levels of surgical residents was extracted. Afterwards, the proposed measure was tested to estimate the experience levels of the participants. Finally, the level of realism of these educational scenarios was assessed. The factor formed by time, distance, and accuracy on simulated tasks provided an overall performance indicator. The prediction correctness was very high for the beginners than the one for experienced surgeons in Experiments 1 and 2. When non-dominant hand is used in a surgical procedure-based scenario, skill levels of surgeons can be better predicted. The results indicate that the scenarios in Experiments 1 and 2 can be used as an assessment tool for the beginners, and scenario-2 in Experiment 3 can be used as an assessment tool for intermediate and novice levels. It can be concluded that forming the balance between perceived action capacities and skills is critical for better designing and developing skill assessment surgical simulation tools.

Data-Based Performance Assessments for the DOE Hydropower Advancement Project

DOE Office of Scientific and Technical Information (OSTI.GOV)

March, Patrick; Wolff, Dr. Paul; Smith, Brennan T

2012-01-01

The U. S. Department of Energy s Hydropower Advancement Project (HAP) was initiated to characterize and trend hydropower asset conditions across the U.S.A. s existing hydropower fleet and to identify and evaluate the upgrading opportunities. Although HAP includes both detailed performance assessments and condition assessments of existing hydropower plants, this paper focuses on the performance assessments. Plant performance assessments provide a set of statistics and indices that characterize the historical extent to which each plant has converted the potential energy at a site into electrical energy for the power system. The performance metrics enable benchmarking and trending of performance acrossmore » many projects in a variety contexts (e.g., river systems, power systems, and water availability). During FY2011 and FY2012, assessments will be performed on ten plants, with an additional fifty plants scheduled for FY2013. This paper focuses on the performance assessments completed to date, details the performance assessment process, and describes results from the performance assessments.« less

Brief Report: Suitability of the Social Skills Performance Assessment (SSPA) for the Assessment of Social Skills in Adults with Autism Spectrum Disorders

ERIC Educational Resources Information Center

Verhoeven, E. W. M.; Smeekens, I.; Didden, R.

2013-01-01

The present study aims at examining whether the "Social Skills Performance Assessment" (SSPA; Patterson et al. in "Schizophr Res" 48(2-3):351-360, 2001) is a suitable performance-based measure to assess social skills in adults with autism spectrum disorders (ASD). For this purpose, social skills of individuals with ASD and…

High fidelity simulation based team training in urology: a preliminary interdisciplinary study of technical and nontechnical skills in laparoscopic complications management.

PubMed

Lee, Jason Y; Mucksavage, Phillip; Canales, Cecilia; McDougall, Elspeth M; Lin, Sharon

2012-04-01

Simulation based team training provides an opportunity to develop interdisciplinary communication skills and address potential medical errors in a high fidelity, low stakes environment. We evaluated the implementation of a novel simulation based team training scenario and assessed the technical and nontechnical performance of urology and anesthesiology residents. Urology residents were randomly paired with anesthesiology residents to participate in a simulation based team training scenario involving the management of 2 scripted critical events during laparoscopic radical nephrectomy, including the vasovagal response to pneumoperitoneum and renal vein injury during hilar dissection. A novel kidney surgical model and a high fidelity mannequin simulator were used for the simulation. A debriefing session followed each simulation based team training scenario. Assessments of technical and nontechnical performance were made using task specific checklists and global rating scales. A total of 16 residents participated, of whom 94% rated the simulation based team training scenario as useful for communication skill training. Also, 88% of urology residents believed that the kidney surgical model was useful for technical skill training. Urology resident training level correlated with technical performance (p=0.004) and blood loss during renal vein injury management (p=0.022) but not with nontechnical performance. Anesthesia resident training level correlated with nontechnical performance (p=0.036). Urology residents consistently rated themselves higher on nontechnical performance than did faculty (p=0.033). Anesthesia residents did not differ in the self-assessment of nontechnical performance compared to faculty assessments. Residents rated the simulation based team training scenario as useful for interdisciplinary communication skill training. Urology resident training level correlated with technical performance but not with nontechnical performance. Urology residents consistently overestimated their nontechnical performance. Copyright © 2012 American Urological Association Education and Research, Inc. Published by Elsevier Inc. All rights reserved.

What Makes a Change Unsuccessful through the Eyes of Teachers

ERIC Educational Resources Information Center

Cimer, Sabiha Odabasi

2018-01-01

Over the past two decades, Turkey has initiated a reform movement to change her classroom assessment system to accommodate performance-based alternative assessment methods in schools. However, research investigating the impact of assessment reform on learning and teaching in schools report that performance assessment approaches have not been…

Assessing Minimum Competencies of Beginning Teachers: Instrumentation, Measurement Issues, Legal Concerns.

ERIC Educational Resources Information Center

Ellett, Chad D.

An overview is presented of a performance-based assessment system, Teacher Performance Assessment Instruments (TPAI), developed by the Teacher Assessment Project at the University of Georgia to measure competencies of beginning teachers for initial professional certification. To clearly separate the preparation and certification functions within…

Stakeholder perspectives on workplace-based performance assessment: towards a better understanding of assessor behaviour.

PubMed

de Jonge, Laury P J W M; Timmerman, Angelique A; Govaerts, Marjan J B; Muris, Jean W M; Muijtjens, Arno M M; Kramer, Anneke W M; van der Vleuten, Cees P M

2017-12-01

Workplace-Based Assessment (WBA) plays a pivotal role in present-day competency-based medical curricula. Validity in WBA mainly depends on how stakeholders (e.g. clinical supervisors and learners) use the assessments-rather than on the intrinsic qualities of instruments and methods. Current research on assessment in clinical contexts seems to imply that variable behaviours during performance assessment of both assessors and learners may well reflect their respective beliefs and perspectives towards WBA. We therefore performed a Q methodological study to explore perspectives underlying stakeholders' behaviours in WBA in a postgraduate medical training program. Five different perspectives on performance assessment were extracted: Agency, Mutuality, Objectivity, Adaptivity and Accountability. These perspectives reflect both differences and similarities in stakeholder perceptions and preferences regarding the utility of WBA. In comparing and contrasting the various perspectives, we identified two key areas of disagreement, specifically 'the locus of regulation of learning' (i.e., self-regulated versus externally regulated learning) and 'the extent to which assessment should be standardised' (i.e., tailored versus standardised assessment). Differing perspectives may variously affect stakeholders' acceptance, use-and, consequently, the effectiveness-of assessment programmes. Continuous interaction between all stakeholders is essential to monitor, adapt and improve assessment practices and to stimulate the development of a shared mental model. Better understanding of underlying stakeholder perspectives could be an important step in bridging the gap between psychometric and socio-constructivist approaches in WBA.

An ecologically valid performance-based social functioning assessment battery for schizophrenia.

PubMed

Shi, Chuan; He, Yi; Cheung, Eric F C; Yu, Xin; Chan, Raymond C K

2013-12-30

Psychiatrists pay more attention to the social functioning outcome of schizophrenia nowadays. How to evaluate the real world function among schizophrenia is a challenging task due to culture difference, there is no such kind of instrument in terms of the Chinese setting. This study aimed to report the validation of an ecologically valid performance-based everyday functioning assessment for schizophrenia, namely the Beijing Performance-based Functional Ecological Test (BJ-PERFECT). Fifty community-dwelling adults with schizophrenia and 37 healthy controls were recruited. Fifteen of the healthy controls were re-tested one week later. All participants were administered the University of California, San Diego, Performance-based Skill Assessment-Brief version (UPSA-B) and the MATRICS Consensus Cognitive Battery (MCCB). The finalized assessment included three subdomains: transportation, financial management and work ability. The test-retest and inter-rater reliabilities were good. The total score significantly correlated with the UPSA-B. The performance of individuals with schizophrenia was significantly more impaired than healthy controls, especially in the domain of work ability. Among individuals with schizophrenia, functional outcome was influenced by premorbid functioning, negative symptoms and neurocognition such as processing speed, visual learning and attention/vigilance. © 2013 Elsevier Ireland Ltd. All rights reserved.

Transabdominal ultrasound to assess pelvic floor muscle performance during abdominal curl in exercising women.

PubMed

Barton, Amanda; Serrao, Chloe; Thompson, Judith; Briffa, Kathy

2015-12-01

The aim of this study was to assess pelvic floor muscle (PFM) function using transabdominal ultrasound (TAUS) in women attending group exercise classes. Specific aims were to: (1) identify the ability to perform a correct elevating PFM contraction and (2) assess bladder-base movement during an abdominal curl exercise. Ninety women participating in group exercise were recruited to complete a survey and TAUS assessment performed by two qualified Continence and Women's Health physiotherapists with clinical experience in ultrasound scanning. The assessment comprised three attempts of a PFM contraction and an abdominal curl exercise in crook lying. Bladder-base displacement was measured to determine correct or incorrect activation patterns. Twenty-five percent (n = 23) of women were unable to demonstrate an elevating PFM contraction, and all women displayed bladder-base depression on abdominal curl (range 0.33-31.2 mm). Parous women displayed, on average, significantly more bladder-base depression than did nulliparous women [15.5 (7.3) mm vs 11.4 (5.8) mm, p < 0.009). Sixty percent (n = 54) reported stress urinary incontinence (SUI). There was no association between SUI and the inability to perform an elevating PFM contraction (p = 0.278) or the amount of bladder-base depression with abdominal curl [14.1 (7.6) mm SUI vs 14.2 (6.7) mm non-SUI]. TAUS identified that 25 % of women who participated in group exercise were unable to perform a correctly elevating PFM contraction, and all depressed the bladder-base on abdominal curl. Therefore, exercising women may be at risk of PFM dysfunction when performing abdominal curl activities.

Towards a Uniform Metrological Assessment of Grating-Based Optical Fiber Sensors: From Refractometers to Biosensors

PubMed Central

Chiavaioli, Francesco; Gouveia, Carlos A. J.; Jorge, Pedro A. S.; Baldini, Francesco

2017-01-01

A metrological assessment of grating-based optical fiber sensors is proposed with the aim of providing an objective evaluation of the performance of this sensor category. Attention was focused on the most common parameters, used to describe the performance of both optical refractometers and biosensors, which encompassed sensitivity, with a distinction between volume or bulk sensitivity and surface sensitivity, resolution, response time, limit of detection, specificity (or selectivity), reusability (or regenerability) and some other parameters of generic interest, such as measurement uncertainty, accuracy, precision, stability, drift, repeatability and reproducibility. Clearly, the concepts discussed here can also be applied to any resonance-based sensor, thus providing the basis for an easier and direct performance comparison of a great number of sensors published in the literature up to now. In addition, common mistakes present in the literature made for the evaluation of sensor performance are highlighted, and lastly a uniform performance assessment is discussed and provided. Finally, some design strategies will be proposed to develop a grating-based optical fiber sensing scheme with improved performance. PMID:28635665

Towards a Uniform Metrological Assessment of Grating-Based Optical Fiber Sensors: From Refractometers to Biosensors.

PubMed

Chiavaioli, Francesco; Gouveia, Carlos A J; Jorge, Pedro A S; Baldini, Francesco

2017-06-21

A metrological assessment of grating-based optical fiber sensors is proposed with the aim of providing an objective evaluation of the performance of this sensor category. Attention was focused on the most common parameters, used to describe the performance of both optical refractometers and biosensors, which encompassed sensitivity, with a distinction between volume or bulk sensitivity and surface sensitivity, resolution, response time, limit of detection, specificity (or selectivity), reusability (or regenerability) and some other parameters of generic interest, such as measurement uncertainty, accuracy, precision, stability, drift, repeatability and reproducibility. Clearly, the concepts discussed here can also be applied to any resonance-based sensor, thus providing the basis for an easier and direct performance comparison of a great number of sensors published in the literature up to now. In addition, common mistakes present in the literature made for the evaluation of sensor performance are highlighted, and lastly a uniform performance assessment is discussed and provided. Finally, some design strategies will be proposed to develop a grating-based optical fiber sensing scheme with improved performance.

Performance-based alternative assessments as a means of eliminating gender achievement differences on science tests

NASA Astrophysics Data System (ADS)

Brown, Norman Merrill

1998-09-01

Historically, researchers have reported an achievement difference between females and males on standardized science tests. These differences have been reported to be based upon science knowledge, abstract reasoning skills, mathematical abilities, and cultural and social phenomena. This research was designed to determine how mastery of specific science content from public school curricula might be evaluated with performance-based assessment models, without producing gender achievement differences. The assessment instruments used were Harcourt Brace Educational Measurement's GOALSsp°ler: A Performance-Based Measure of Achievement and the performance-based portion of the Stanford Achievement Testspcopyright, Ninth Edition. The identified independent variables were test, gender, ethnicity, and grade level. A 2 x 2 x 6 x 12 (test x gender x ethnicity x grade) factorial experimental design was used to organize the data. A stratified random sample (N = 2400) was selected from a national pool of norming data: N = 1200 from the GOALSsp°ler group and N = 1200 from the SAT9spcopyright group. The ANOVA analysis yielded mixed results. The factors of test, gender, ethnicity by grade, gender by grade, and gender by grade by ethnicity failed to produce significant results (alpha = 0.05). The factors yielding significant results were ethnicity, grade, and ethnicity by grade. Therefore, no significant differences were found between female and male achievement on these performance-based assessments.

Stakeholder Perspectives on Workplace-Based Performance Assessment: Towards a Better Understanding of Assessor Behaviour

ERIC Educational Resources Information Center

de Jonge, Laury P. J. W. M.; Timmerman, Angelique A.; Govaerts, Marjan J. B.; Muris, Jean W. M.; Muijtjens, Arno M. M.; Kramer, Anneke W. M.; van der Vleuten, Cees P. M.

2017-01-01

Workplace-Based Assessment (WBA) plays a pivotal role in present-day competency-based medical curricula. Validity in WBA mainly depends on how stakeholders (e.g. clinical supervisors and learners) use the assessments--rather than on the intrinsic qualities of instruments and methods. Current research on assessment in clinical contexts seems to…

Is Learner Self-Assessment Reliable and Valid in a Web-Based Portfolio Environment for High School Students?

ERIC Educational Resources Information Center

Chang, Chi-Cheng; Liang, Chaoyun; Chen, Yi-Hui

2013-01-01

This study explored the reliability and validity of Web-based portfolio self-assessment. Participants were 72 senior high school students enrolled in a computer application course. The students created learning portfolios, viewed peers' work, and performed self-assessment on the Web-based portfolio assessment system. The results indicated: 1)…

Competency Based Assessment in Fashion Design

NASA Astrophysics Data System (ADS)

Russanti, Irma; Nurlaela, Lutfiyah; Basuki, Ismet; Munoto

2018-04-01

Professional certification is a form of stipulation on certain competency standards provided by one professional organization to the performance of a person through assessment. For that an assessment needs to be standardized so that there exists a general standardized scale to measure competence. In the professional certification of fashion design department, an instrument of competency based assessment is essential to be developed. The purpose of this review is to know the application of competency based assessment in the field of fashion design. The literature reviews were found by journal searching with keywords competency based assessment and fashion design in Google scholar, of which was gotten over 20 journals from 2006 to 2016. Afterwards, the search of the free-downloaded e-books in libgen was conducted under competency based assessment and fashion design, which is then found some related references. The obtained literatures were used to review the definition, approach, and implementation of competency based assessment in the field of fashion design. Results show that it is important to develop an assessment sheet in the field of fashion design covering garment, apparel and embroidery sectors by patterning the criteria of performers along with the qualifications.

Graduate performance of science education department in implementing conservation-based science teaching

NASA Astrophysics Data System (ADS)

Parmin; Savitri, E. N.; Amalia, A. V.; Pratama, M. R.

2018-04-01

This study aims to measure the performance of graduates in implementing conservation-based science teaching. The study employed a qualitative method by collecting the self-assessment data from alumni and the performance assessment from the headmasters of schools where the graduates are currently teaching. There are nine indicators of conservation insight examined in this study. The study concluded that the 78 alumni, who have become teachers when the study was conducted, perform well in implementing conservative science lessons.

Cooperation, Technology, and Performance: A Case Study.

ERIC Educational Resources Information Center

Cavanagh, Thomas; Dickenson, Sabrina; Brandt, Suzanne

1999-01-01

Describes the CTP (Cooperation, Technology, and Performance) model and explains how it is used by the Department of Veterans Affairs-Veteran's Benefit Administration (VBA) for training. Discusses task analysis; computer-based training; cooperative-based learning environments; technology-based learning; performance-assessment methods; courseware…

Towards a Better Understanding of the Legibility Bias in Performance Assessments: The Case of Gender-Based Inferences

ERIC Educational Resources Information Center

Greifeneder, Rainer; Zelt, Sarah; Seele, Tim; Bottenberg, Konstantin; Alt, Alexander

2012-01-01

Background: Handwriting legibility systematically biases evaluations in that highly legible handwriting results in more positive evaluations than less legible handwriting. Because performance assessments in educational contexts are not only based on computerized or multiple choice tests but often include the evaluation of handwritten work samples,…

Planetary Patterns. Grade 8 Science, Language Usage. Maryland School Performance Assessment Program (MSPAP): Resource Library. Public Release Task.

ERIC Educational Resources Information Center

Maryland State Dept. of Education. Baltimore. Div. of Planning, Results and Information Management.

One component of the Maryland School Performance Assessment Program (MSPAP) is the state's performance-based assessments, criterion-referenced tests that require students to apply what they know and can do to solve problems and display other higher-order thinking skills. This document helps parents, teachers, students, and other citizens…

Performance Measurement and Accommodation: Students with Visual Impairments on Pennsylvania's Alternate Assessment

ERIC Educational Resources Information Center

Zebehazy, Kim T.; Zigmond, Naomi; Zimmerman, George J.

2012-01-01

Introduction: This study investigated the use of accommodations and the performance of students with visual impairments and severe cognitive disabilities on the Pennsylvania Alternate System of Assessment (PASA)yCoan alternate performance-based assessment. Methods: Differences in test scores on the most basic level (level A) of the PASA of 286…

Keeping Student Performance Central: The New York Assessment Collection. Studies on Exhibitions.

ERIC Educational Resources Information Center

Allen, David; McDonald, Joseph

This report describes a computer tool used by the state of New York to assess student performance in elementary and secondary grades. Based on the premise that every assessment is a system of interacting elements, the tool examines students on six dimensions: vision, prompt, coaching context, performance, standards, and reflection. Vision, which…

Educational Assessment via a Web-Based Intelligent System

ERIC Educational Resources Information Center

Huang, Jingshan; He, Lei; Davidson-Shivers, Gayle V.

2011-01-01

Effective assessment is vital in educational activities. We propose IWAS (intelligent Web-based assessment system), an intelligent, generalized and real-time system to assess both learning and teaching. IWAS provides a foundation for more efficiency in instructional activities and, ultimately, students' performances. Our contributions are…

Performance-based management and quality of work: an empirical assessment.

PubMed

Falzon, Pierre; Nascimento, Adelaide; Gaudart, Corinne; Piney, Cécile; Dujarier, Marie-Anne; Germe, Jean-François

2012-01-01

In France, in the private sector as in the public sector, performance-based management tends to become a norm. Performance-based management is supposed to improve service quality, productivity and efficiency, transparency of allotted means and achieved results, and to better focus the activity of employees and of the whole organization. This text reports a study conducted for the French Ministry of Budget by a team of researchers in ergonomics, sociology and management science, in order to assess the impact of performance-based management on employees, on teams and on work organization. About 100 interviews were conducted with employees of all categories and 6 working groups were set up in order to discuss and validate or amend our first analyses. Results concern several aspects: workload and work intensification, indicators and performance management and the transformation of jobs induced by performance management.

Probabilistic Assessment of National Wind Tunnel

NASA Technical Reports Server (NTRS)

Shah, A. R.; Shiao, M.; Chamis, C. C.

1996-01-01

A preliminary probabilistic structural assessment of the critical section of National Wind Tunnel (NWT) is performed using NESSUS (Numerical Evaluation of Stochastic Structures Under Stress) computer code. Thereby, the capabilities of NESSUS code have been demonstrated to address reliability issues of the NWT. Uncertainties in the geometry, material properties, loads and stiffener location on the NWT are considered to perform the reliability assessment. Probabilistic stress, frequency, buckling, fatigue and proof load analyses are performed. These analyses cover the major global and some local design requirements. Based on the assumed uncertainties, the results reveal the assurance of minimum 0.999 reliability for the NWT. Preliminary life prediction analysis results show that the life of the NWT is governed by the fatigue of welds. Also, reliability based proof test assessment is performed.

Development of student performance assessment based on scientific approach for a basic physics practicum in simple harmonic motion materials

NASA Astrophysics Data System (ADS)

Serevina, V.; Muliyati, D.

2018-05-01

This research aims to develop students’ performance assessment instrument based on scientific approach is valid and reliable in assessing the performance of students on basic physics lab of Simple Harmonic Motion (SHM). This study uses the ADDIE consisting of stages: Analyze, Design, Development, Implementation, and Evaluation. The student performance assessment developed can be used to measure students’ skills in observing, asking, conducting experiments, associating and communicate experimental results that are the ‘5M’ stages in a scientific approach. Each grain of assessment in the instrument is validated by the instrument expert and the evaluation with the result of all points of assessment shall be eligible to be used with a 100% eligibility percentage. The instrument is then tested for the quality of construction, material, and language by panel (lecturer) with the result: 85% or very good instrument construction aspect, material aspect 87.5% or very good, and language aspect 83% or very good. For small group trial obtained instrument reliability level of 0.878 or is in the high category, where r-table is 0.707. For large group trial obtained instrument reliability level of 0.889 or is in the high category, where r-table is 0.320. Instruments declared valid and reliable for 5% significance level. Based on the result of this research, it can be concluded that the student performance appraisal instrument based on the developed scientific approach is declared valid and reliable to be used in assessing student skill in SHM experimental activity.

Performance-based measures and behavioral ratings of executive function in diagnosing attention-deficit/hyperactivity disorder in children.

PubMed

Tan, Alexander; Delgaty, Lauren; Steward, Kayla; Bunner, Melissa

2018-04-16

Deficits in real-world executive functioning (EF) are a frequent characteristic of attention-deficit/hyperactivity disorder (ADHD). However, the predictive value of using performance-based and behavioral rating measures of EF when diagnosing ADHD remains unclear. The current study investigates the use of performance-based EF measures and a parent-report questionnaire with established ecological validity and clinical utility when diagnosing ADHD. Participants included 21 healthy controls, 21 ADHD-primary inattentive, and 21 ADHD-combined type subjects aged 6-15 years. A brief neuropsychological battery was administered to each subject including common EF assessment measures. Significant differences were not found between groups on most performance-based EF measures, whereas significant differences (p < 0.05) were found on most parent-report behavioral rating scales. Furthermore, performance-based measures did not predict group membership above chance levels. Results further support differences in predictive value of EF performance-based measures compared to parent-report questionnaires when diagnosing ADHD. Further research must investigate the relationship between performance-based and behavioral rating measures when assessing EF in ADHD.

Assessing teamwork performance in obstetrics: A systematic search and review of validated tools.

PubMed

Fransen, Annemarie F; de Boer, Liza; Kienhorst, Dieneke; Truijens, Sophie E; van Runnard Heimel, Pieter J; Oei, S Guid

2017-09-01

Teamwork performance is an essential component for the clinical efficiency of multi-professional teams in obstetric care. As patient safety is related to teamwork performance, it has become an important learning goal in simulation-based education. In order to improve teamwork performance, reliable assessment tools are required. These can be used to provide feedback during training courses, or to compare learning effects between different types of training courses. The aim of the current study is to (1) identify the available assessment tools to evaluate obstetric teamwork performance in a simulated environment, and (2) evaluate their psychometric properties in order to identify the most valuable tool(s) to use. We performed a systematic search in PubMed, MEDLINE, and EMBASE to identify articles describing assessment tools for the evaluation of obstetric teamwork performance in a simulated environment. In order to evaluate the quality of the identified assessment tools the standards and grading rules have been applied as recommended by the Accreditation Council for Graduate Medical Education (ACGME) Committee on Educational Outcomes. The included studies were also assessed according to the Oxford Centre for Evidence Based Medicine (OCEBM) levels of evidence. This search resulted in the inclusion of five articles describing the following six tools: Clinical Teamwork Scale, Human Factors Rating Scale, Global Rating Scale, Assessment of Obstetric Team Performance, Global Assessment of Obstetric Team Performance, and the Teamwork Measurement Tool. Based on the ACGME guidelines we assigned a Class 3, level C of evidence, to all tools. Regarding the OCEBM levels of evidence, a level 3b was assigned to two studies and a level 4 to four studies. The Clinical Teamwork Scale demonstrated the most comprehensive validation, and the Teamwork Measurement Tool demonstrated promising results, however it is recommended to further investigate its reliability. Copyright © 2017. Published by Elsevier B.V.

Assessing Vocal Performances Using Analytical Assessment: A Case Study

ERIC Educational Resources Information Center

Gynnild, Vidar

2016-01-01

This study investigated ways to improve the appraisal of vocal performances within a national academy of music. Since a criterion-based assessment framework had already been adopted, the conceptual foundation of an assessment rubric was used as a guide in an action research project. The group of teachers involved wanted to explore thinking…

Laboratory- and field-based testing as predictors of skating performance in competitive-level female ice hockey.

PubMed

Henriksson, Tommy; Vescovi, Jason D; Fjellman-Wiklund, Anncristine; Gilenstam, Kajsa

2016-01-01

The purpose of this study was to examine whether field-based and/or laboratory-based assessments are valid tools for predicting key performance characteristics of skating in competitive-level female hockey players. Cross-sectional study. Twenty-three female ice hockey players aged 15-25 years (body mass: 66.1±6.3 kg; height: 169.5±5.5 cm), with 10.6±3.2 years playing experience volunteered to participate in the study. The field-based assessments included 20 m sprint, squat jump, countermovement jump, 30-second repeated jump test, standing long jump, single-leg standing long jump, 20 m shuttle run test, isometric leg pull, one-repetition maximum bench press, and one-repetition maximum squats. The laboratory-based assessments included body composition (dual energy X-ray absorptiometry), maximal aerobic power, and isokinetic strength (Biodex). The on-ice tests included agility cornering s-turn, cone agility skate, transition agility skate, and modified repeat skate sprint. Data were analyzed using stepwise multivariate linear regression analysis. Linear regression analysis was used to establish the relationship between key performance characteristics of skating and the predictor variables. Regression models (adj R (2)) for the on-ice variables ranged from 0.244 to 0.663 for the field-based assessments and from 0.136 to 0.420 for the laboratory-based assessments. Single-leg tests were the strongest predictors for key performance characteristics of skating. Single leg standing long jump alone explained 57.1%, 38.1%, and 29.1% of the variance in skating time during transition agility skate, agility cornering s-turn, and modified repeat skate sprint, respectively. Isokinetic peak torque in the quadriceps at 90° explained 42.0% and 32.2% of the variance in skating time during agility cornering s-turn and modified repeat skate sprint, respectively. Field-based assessments, particularly single-leg tests, are an adequate substitute to more expensive and time-consuming laboratory assessments if the purpose is to gain knowledge about key performance characteristics of skating.

Laboratory- and field-based testing as predictors of skating performance in competitive-level female ice hockey

PubMed Central

Henriksson, Tommy; Vescovi, Jason D; Fjellman-Wiklund, Anncristine; Gilenstam, Kajsa

2016-01-01

Objectives The purpose of this study was to examine whether field-based and/or laboratory-based assessments are valid tools for predicting key performance characteristics of skating in competitive-level female hockey players. Design Cross-sectional study. Methods Twenty-three female ice hockey players aged 15–25 years (body mass: 66.1±6.3 kg; height: 169.5±5.5 cm), with 10.6±3.2 years playing experience volunteered to participate in the study. The field-based assessments included 20 m sprint, squat jump, countermovement jump, 30-second repeated jump test, standing long jump, single-leg standing long jump, 20 m shuttle run test, isometric leg pull, one-repetition maximum bench press, and one-repetition maximum squats. The laboratory-based assessments included body composition (dual energy X-ray absorptiometry), maximal aerobic power, and isokinetic strength (Biodex). The on-ice tests included agility cornering s-turn, cone agility skate, transition agility skate, and modified repeat skate sprint. Data were analyzed using stepwise multivariate linear regression analysis. Linear regression analysis was used to establish the relationship between key performance characteristics of skating and the predictor variables. Results Regression models (adj R2) for the on-ice variables ranged from 0.244 to 0.663 for the field-based assessments and from 0.136 to 0.420 for the laboratory-based assessments. Single-leg tests were the strongest predictors for key performance characteristics of skating. Single leg standing long jump alone explained 57.1%, 38.1%, and 29.1% of the variance in skating time during transition agility skate, agility cornering s-turn, and modified repeat skate sprint, respectively. Isokinetic peak torque in the quadriceps at 90° explained 42.0% and 32.2% of the variance in skating time during agility cornering s-turn and modified repeat skate sprint, respectively. Conclusion Field-based assessments, particularly single-leg tests, are an adequate substitute to more expensive and time-consuming laboratory assessments if the purpose is to gain knowledge about key performance characteristics of skating. PMID:27574474

Taiwan industrial cooperation program technology transfer for low-level radioactive waste final disposal - phase I.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Knowlton, Robert G.; Cochran, John Russell; Arnold, Bill Walter

2007-01-01

Sandia National Laboratories and the Institute of Nuclear Energy Research, Taiwan have collaborated in a technology transfer program related to low-level radioactive waste (LLW) disposal in Taiwan. Phase I of this program included regulatory analysis of LLW final disposal, development of LLW disposal performance assessment capabilities, and preliminary performance assessments of two potential disposal sites. Performance objectives were based on regulations in Taiwan and comparisons to those in the United States. Probabilistic performance assessment models were constructed based on limited site data using software including GoldSim, BLT-MS, FEHM, and HELP. These software codes provided the probabilistic framework, container degradation, waste-formmore » leaching, groundwater flow, radionuclide transport, and cover infiltration simulation capabilities in the performance assessment. Preliminary performance assessment analyses were conducted for a near-surface disposal system and a mined cavern disposal system at two representative sites in Taiwan. Results of example calculations indicate peak simulated concentrations to a receptor within a few hundred years of LLW disposal, primarily from highly soluble, non-sorbing radionuclides.« less

Visual Arts (9-12): Performance-Based Assessment. Course of Study.

ERIC Educational Resources Information Center

Idaho State Department of Education, 2004

2004-01-01

In response to a need for curricular support, the Idaho State Department of Education gathered teams of exemplary educators from throughout the state to write courses of study related to all subject areas within Idaho?s Achievement Standards. A need arose to assess areas not tested statewide in a performance-based manner. Teachers involved in…

Foreign Language: Year One (9-12). Performance-Based Assessment. Course of Study.

ERIC Educational Resources Information Center

Wenner, Peggy J.

2004-01-01

In response to a need for curricular support, the Idaho State Department of Education gathered teams of exemplary educators from throughout the state to write courses of study related to all subject areas within Idaho's Achievement Standards. A need arose to assess areas not tested statewide in a performance-based manner. Teachers involved in…

Correspondence between Video-Based Preference Assessment and Subsequent Community Job Performance

ERIC Educational Resources Information Center

Morgan, Robert L.; Horrocks, Erin L.

2011-01-01

Researchers identified high and low preference jobs using a video web-based assessment program with three young adults ages 18 to 19 with intellectual disabilities. Individual participants were then taught to perform high and low preference jobs in community locations. The order of 25-min high and low preference job sessions was randomized. A…

Scenario-based and scenario-neutral assessment of climate change impacts on operational performance of a multipurpose reservoir

Treesearch

Allison G. Danner; Mohammad Safeeq; Gordon E. Grant; Charlotte Wickham; Desirée Tullos; Mary V. Santelmann

2017-01-01

Scenario-based and scenario-neutral impacts assessment approaches provide complementary information about how climate change-driven effects on streamflow may change the operational performance of multipurpose dams. Examining a case study of Cougar Dam in Oregon, United States, we simulated current reservoir operations under scenarios of plausible future hydrology....

Performance-Based Assessment in an Online Course: Comparing Different Types of Information Literacy Instruction

ERIC Educational Resources Information Center

Mery, Yvonne; Newby, Jill; Peng, Ke

2012-01-01

This study investigates whether the type of instruction (a single face-to-face librarian-led instruction, instructor-led instruction, or an online IL course--the Online Research Lab) has an impact on student information literacy gains in a Freshman English Composition program. A performance-based assessment was carried out by analyzing…

Practitioner Review: Do Performance-Based Measures and Ratings of Executive Function Assess the Same Construct?

ERIC Educational Resources Information Center

Toplak, Maggie E.; West, Richard F.; Stanovich, Keith E.

2013-01-01

Background: Both performance-based and rating measures are commonly used to index executive function in clinical and neuropsychological assessments. They are intended to index the same broad underlying mental construct of executive function. The association between these two types of measures was investigated in the current article. Method and…

Development of a Criterion-Referenced, Performance-Based Assessment of Reading Comprehension in a Whole Literacy Program.

ERIC Educational Resources Information Center

Tibbetts, Katherine A.; And Others

This paper describes the development of a criterion-referenced, performance-based measure of third grade reading comprehension. The primary purpose of the assessment is to contribute unique and valid information for use in the formative evaluation of a whole literacy program. A secondary purpose is to supplement other program efforts to…

Formative, Informative, and Summative Assessment: The Relationship among Curriculum-Based Measurement of Reading, Classroom Engagement, and Reading Performance

ERIC Educational Resources Information Center

Marchand, Gwen C.; Furrer, Carrie J.

2014-01-01

This study explored the relationships among formative curriculum-based measures of reading (CBM-R), student engagement as an extra-academic indicator of student motivation, and summative performance on a high-stakes reading assessment. A diverse sample of third-, fourth-, and fifth-grade students and their teachers responded to questionnaires…

Computer-Based Assessment of Collaborative Problem Solving: Exploring the Feasibility of Human-to-Agent Approach

ERIC Educational Resources Information Center

Rosen, Yigal

2015-01-01

How can activities in which collaborative skills of an individual are measured be standardized? In order to understand how students perform on collaborative problem solving (CPS) computer-based assessment, it is necessary to examine empirically the multi-faceted performance that may be distributed across collaboration methods. The aim of this…

Why IEP Teams Assign Low Performers with Mild Disabilities to the Alternate Assessment Based on Alternate Achievement Standards

ERIC Educational Resources Information Center

Cho, Hyun-Jeong; Kingston, Neal

2013-01-01

The purpose of this case study was to determine teachers' rationales for assigning students with mild disabilities to alternate assessment based on alternate achievement standards (AA-AAS). In interviews, special educators stated that their primary considerations in making the assignments were low academic performance, student use of extended…

Salinity. Grade 5 Science Language Usage. Maryland School Maryland School Performance Assessment Program (MSPAP): Resource Library. Public Release Task.

ERIC Educational Resources Information Center

Maryland State Dept. of Education. Baltimore. Div. of Planning, Results and Information Management.

One component of the Maryland School Performance Assessment; Program (MSPAP) is the state's performance-based assessments, criterion-referenced tests that require students to apply what they know and can do to solve problems and display other higher-order thinking skills. This document helps parents, teachers, students, and other citizens…

Effectiveness of Integrating Simulation with Art-Based Teaching Strategies on Oncology Fellows' Performance Regarding Breaking Bad News.

PubMed

Yakhforoshha, Afsaneh; Emami, Seyed Amir Hossein; Shahi, Farhad; Shahsavari, Saeed; Cheraghi, Mohammadali; Mojtahedzadeh, Rita; Mahmoodi-Bakhtiari, Behrooz; Shirazi, Mandana

2018-02-21

The task of breaking bad news (BBN) may be improved by incorporating simulation with art-based teaching methods. The aim of the present study was to assess the effect of an integrating simulation with art-based teaching strategies, on fellows' performance regarding BBN, in Iran. The study was carried out using quasi-experimental methods, interrupted time series. The participants were selected from medical oncology fellows at two teaching hospitals of Tehran University of Medical Sciences (TUMS), Iran. Participants were trained through workshop, followed by engaging participants with different types of art-based teaching methods. In order to assess the effectiveness of the integrating model, fellows' performance was rated by two independent raters (standardized patients (SPs) and faculty members) using the BBN assessment checklist. This assessment tool measured seven different domains of BBN skill. Segmented regression was used to analyze the results of study. Performance of all oncology fellows (n = 19) was assessed for 228 time points during the study, by rating three time points before and three time points after the intervention by two raters. Based on SP ratings, fellows' performance scores in post-training showed significant level changes in three domains of BBN checklist (B = 1.126, F = 3.221, G = 2.241; p < 0.05). Similarly, the significant level change in fellows' score rated by faculty members in post-training was B = 1.091, F = 3.273, G = 1.724; p < 0.05. There was no significant change in trend of fellows' performance after the intervention. Our results showed that using an integrating simulation with art-based teaching strategies may help oncology fellows to improve their communication skills in different facets of BBN performance. Iranian Registry of Clinical Trials ID: IRCT2016011626039N1.

Redesigned High Schools for Transformed STEM Learning: Performance Assessment Pilot Outcome

ERIC Educational Resources Information Center

Ernst, Jeremy Vaughn; Glennie, Elizabeth J.

2015-01-01

This performance assessment pilot study was a major research component of the overall National Science Foundation funded Redesigned High Schools for Transformed STEM Learning Project. Secondary Earth/Environmental Science students' abilities to translate cognitive knowledge into demonstrable performance-based proficiencies were specifically…

Simulation-based team training improved the self-assessed ability of physicians, nurses and midwives to perform neonatal resuscitation.

PubMed

Malmström, B; Nohlert, E; Ewald, U; Widarsson, M

2017-08-01

The use of simulation-based team training in neonatal resuscitation has increased in Sweden during the last decade, but no formal evaluation of this training method has been performed. This study evaluated the effect of simulation-based team training on the self-assessed ability of personnel to perform neonatal resuscitation. We evaluated a full-day simulation-based team training course in neonatal resuscitation, by administering a questionnaire to 110 physicians, nurses and midwives before and after the training period. The questionnaire focused on four important domains: communication, leadership, confidence and technical skills. The study was carried out in Sweden from 2005 to 2007. The response rate was 84%. Improvements in the participants' self-assessed ability to perform neonatal resuscitation were seen in all four domains after training (p < 0.001). Professionally inexperienced personnel showed a significant improvement in the technical skills domain compared to experienced personnel (p = 0.001). No differences were seen between professions or time since training in any of the four domains. Personnel with less previous experience with neonatal resuscitation showed improved confidence (p = 0.007) and technical skills (p = 0.003). A full-day course on simulation-based team training with video-supported debriefing improved the participants' self-assessed ability to perform neonatal resuscitation. ©2017 Foundation Acta Paediatrica. Published by John Wiley & Sons Ltd.

The Influence of Test Mode and Visuospatial Ability on Mathematics Assessment Performance

ERIC Educational Resources Information Center

Logan, Tracy

2015-01-01

Mathematics assessment and testing are increasingly situated within digital environments with international tests moving to computer-based testing in the near future. This paper reports on a secondary data analysis which explored the influence the mode of assessment--computer-based (CBT) and pencil-and-paper based (PPT)--and visuospatial ability…

A Two-Stage Multi-Agent Based Assessment Approach to Enhance Students' Learning Motivation through Negotiated Skills Assessment

ERIC Educational Resources Information Center

Chadli, Abdelhafid; Bendella, Fatima; Tranvouez, Erwan

2015-01-01

In this paper we present an Agent-based evaluation approach in a context of Multi-agent simulation learning systems. Our evaluation model is based on a two stage assessment approach: (1) a Distributed skill evaluation combining agents and fuzzy sets theory; and (2) a Negotiation based evaluation of students' performance during a training…

A Comparative Analysis of the Consistency and Difference among Teacher-Assessment, Student Self-Assessment and Peer-Assessment in a Web-Based Portfolio Assessment Environment for High School Students

ERIC Educational Resources Information Center

Chang, Chi-Cheng; Tseng, Kuo-Hung; Lou, Shi-Jer

2012-01-01

This study explored the consistency and difference of teacher-, student self- and peer-assessment in the context of Web-based portfolio assessment. Participants were 72 senior high school students enrolled in a computer application course. Through the assessment system, the students performed portfolio creation, inspection, self- and…

Evaluation of high fidelity patient simulator in assessment of performance of anaesthetists.

PubMed

Weller, J M; Bloch, M; Young, S; Maze, M; Oyesola, S; Wyner, J; Dob, D; Haire, K; Durbridge, J; Walker, T; Newble, D

2003-01-01

There is increasing emphasis on performance-based assessment of clinical competence. The High Fidelity Patient Simulator (HPS) may be useful for assessment of clinical practice in anaesthesia, but needs formal evaluation of validity, reliability, feasibility and effect on learning. We set out to assess the reliability of a global rating scale for scoring simulator performance in crisis management. Using a global rating scale, three judges independently rated videotapes of anaesthetists in simulated crises in the operating theatre. Five anaesthetists then independently rated subsets of these videotapes. There was good agreement between raters for medical management, behavioural attributes and overall performance. Agreement was high for both the initial judges and the five additional raters. Using a global scale to assess simulator performance, we found good inter-rater reliability for scoring performance in a crisis. We estimate that two judges should provide a reliable assessment. High fidelity simulation should be studied further for assessing clinical performance.

Assessment of active methods for removal of LEO debris

NASA Astrophysics Data System (ADS)

Hakima, Houman; Emami, M. Reza

2018-03-01

This paper investigates the applicability of five active methods for removal of large low Earth orbit debris. The removal methods, namely net, laser, electrodynamic tether, ion beam shepherd, and robotic arm, are selected based on a set of high-level space mission constraints. Mission level criteria are then utilized to assess the performance of each redirection method in light of the results obtained from a Monte Carlo simulation. The simulation provides an insight into the removal time, performance robustness, and propellant mass criteria for the targeted debris range. The remaining attributes are quantified based on the models provided in the literature, which take into account several important parameters pertaining to each removal method. The means of assigning attributes to each assessment criterion is discussed in detail. A systematic comparison is performed using two different assessment schemes: Analytical Hierarchy Process and utility-based approach. A third assessment technique, namely the potential-loss analysis, is utilized to highlight the effect of risks in each removal methods.

CSHM: Web-based safety and health monitoring system for construction management.

PubMed

Cheung, Sai On; Cheung, Kevin K W; Suen, Henry C H

2004-01-01

This paper describes a web-based system for monitoring and assessing construction safety and health performance, entitled the Construction Safety and Health Monitoring (CSHM) system. The design and development of CSHM is an integration of internet and database systems, with the intent to create a total automated safety and health management tool. A list of safety and health performance parameters was devised for the management of safety and health in construction. A conceptual framework of the four key components of CSHM is presented: (a) Web-based Interface (templates); (b) Knowledge Base; (c) Output Data; and (d) Benchmark Group. The combined effect of these components results in a system that enables speedy performance assessment of safety and health activities on construction sites. With the CSHM's built-in functions, important management decisions can theoretically be made and corrective actions can be taken before potential hazards turn into fatal or injurious occupational accidents. As such, the CSHM system will accelerate the monitoring and assessing of performance safety and health management tasks.

Web-based application on employee performance assessment using exponential comparison method

NASA Astrophysics Data System (ADS)

Maryana, S.; Kurnia, E.; Ruyani, A.

2017-02-01

Employee performance assessment is also called a performance review, performance evaluation, or assessment of employees, is an effort to assess the achievements of staffing performance with the aim to increase productivity of employees and companies. This application helps in the assessment of employee performance using five criteria: Presence, Quality of Work, Quantity of Work, Discipline, and Teamwork. The system uses the Exponential Comparative Method and Weighting Eckenrode. Calculation results using graphs were provided to see the assessment of each employee. Programming language used in this system is written in Notepad++ and MySQL database. The testing result on the system can be concluded that this application is correspond with the design and running properly. The test conducted is structural test, functional test, and validation, sensitivity analysis, and SUMI testing.

Communication of Expectations between Principals and Entry-Year Instrumental Music Teachers: Implications for Music Teacher Assessment

ERIC Educational Resources Information Center

Edgar, Scott

2012-01-01

Assessment of arts educators, including music educators, has evolved into a high-stakes situation that drives teacher pay, promotion, and retention. This assessment process is driven by federal policy advocating for a value-added model based on student performance. Principals, who are often charged with assessing artistic musical performance,…

Comparing the Performance and Preference of Students Experiencing a Reading Aloud Accommodation to Those Who Do Not on a Virtual Science Assessment

ERIC Educational Resources Information Center

Shelton, Angela

2012-01-01

Many United States secondary students perform poorly on standardized summative science assessments. Situated Assessments using Virtual Environments (SAVE) Science is an innovative assessment project that seeks to capture students' science knowledge and understanding by contextualizing problems in a game-based virtual environment called…

Objective structured assessment of nontechnical skills: Reliability of a global rating scale for the in-training assessment in the operating room.

PubMed

Dedy, Nicolas J; Szasz, Peter; Louridas, Marisa; Bonrath, Esther M; Husslein, Heinrich; Grantcharov, Teodor P

2015-06-01

Nontechnical skills are critical for patient safety in the operating room (OR). As a result, regulatory bodies for accreditation and certification have mandated the integration of these competencies into postgraduate education. A generally accepted approach to the in-training assessment of nontechnical skills, however, is lacking. The goal of the present study was to develop an evidence-based and reliable tool for the in-training assessment of residents' nontechnical performance in the OR. The Objective Structured Assessment of Nontechnical Skills tool was designed as a 5-point global rating scale with descriptive anchors for each item, based on existing evidence-based frameworks of nontechnical skills, as well as resident training requirements. The tool was piloted on scripted videos and refined in an iterative process. The final version was used to rate residents' performance in recorded OR crisis simulations and during live observations in the OR. A total of 37 simulations and 10 live procedures were rated. Interrater agreement was good for total mean scores, both in simulation and in the real OR, with intraclass correlation coefficients >0.90 in all settings for average and single measures. Internal consistency of the scale was high (Cronbach's alpha = 0.80). The Objective Structured Assessment of Nontechnical Skills global rating scale was developed as an evidence-based tool for the in-training assessment of residents' nontechnical performance in the OR. Unique descriptive anchors allow for a criterion-referenced assessment of performance. Good reliability was demonstrated in different settings, supporting applications in research and education. Copyright © 2015 Elsevier Inc. All rights reserved.

Developing Web-Based Assessment Strategies for Facilitating Junior High School Students to Perform Self-Regulated Learning in an E-Learning Environment

ERIC Educational Resources Information Center

Wang, Tzu-Hua

2011-01-01

This research refers to the self-regulated learning strategies proposed by Pintrich (1999) in developing a multiple-choice Web-based assessment system, the Peer-Driven Assessment Module of the Web-based Assessment and Test Analysis system (PDA-WATA). The major purpose of PDA-WATA is to facilitate learner use of self-regulatory learning behaviors…

Initial construct validity evidence of a virtual human application for competency assessment in breaking bad news to a cancer patient.

PubMed

Guetterman, Timothy C; Kron, Frederick W; Campbell, Toby C; Scerbo, Mark W; Zelenski, Amy B; Cleary, James F; Fetters, Michael D

2017-01-01

Despite interest in using virtual humans (VHs) for assessing health care communication, evidence of validity is limited. We evaluated the validity of a VH application, MPathic-VR, for assessing performance-based competence in breaking bad news (BBN) to a VH patient. We used a two-group quasi-experimental design, with residents participating in a 3-hour seminar on BBN. Group A (n=15) completed the VH simulation before and after the seminar, and Group B (n=12) completed the VH simulation only after the BBN seminar to avoid the possibility that testing alone affected performance. Pre- and postseminar differences for Group A were analyzed with a paired t -test, and comparisons between Groups A and B were analyzed with an independent t -test. Compared to the preseminar result, Group A's postseminar scores improved significantly, indicating that the VH program was sensitive to differences in assessing performance-based competence in BBN. Postseminar scores of Group A and Group B were not significantly different, indicating that both groups performed similarly on the VH program. Improved pre-post scores demonstrate acquisition of skills in BBN to a VH patient. Pretest sensitization did not appear to influence posttest assessment. These results provide initial construct validity evidence that the VH program is effective for assessing BBN performance-based communication competence.

Initial construct validity evidence of a virtual human application for competency assessment in breaking bad news to a cancer patient

PubMed Central

Guetterman, Timothy C; Kron, Frederick W; Campbell, Toby C; Scerbo, Mark W; Zelenski, Amy B; Cleary, James F; Fetters, Michael D

2017-01-01

Background Despite interest in using virtual humans (VHs) for assessing health care communication, evidence of validity is limited. We evaluated the validity of a VH application, MPathic-VR, for assessing performance-based competence in breaking bad news (BBN) to a VH patient. Methods We used a two-group quasi-experimental design, with residents participating in a 3-hour seminar on BBN. Group A (n=15) completed the VH simulation before and after the seminar, and Group B (n=12) completed the VH simulation only after the BBN seminar to avoid the possibility that testing alone affected performance. Pre- and postseminar differences for Group A were analyzed with a paired t-test, and comparisons between Groups A and B were analyzed with an independent t-test. Results Compared to the preseminar result, Group A’s postseminar scores improved significantly, indicating that the VH program was sensitive to differences in assessing performance-based competence in BBN. Postseminar scores of Group A and Group B were not significantly different, indicating that both groups performed similarly on the VH program. Conclusion Improved pre–post scores demonstrate acquisition of skills in BBN to a VH patient. Pretest sensitization did not appear to influence posttest assessment. These results provide initial construct validity evidence that the VH program is effective for assessing BBN performance-based communication competence. PMID:28794664

Effectiveness of simulation-based learning on student nurses' self-efficacy and performance while learning fundamental nursing skills.

PubMed

Lin, Hsin-Hsin

2015-01-01

It was noted worldwide while learning fundamental skills and facing skills assessments, nursing students seemed to experience low confidence and high anxiety levels. Could simulation-based learning help to enhance students' self-efficacy and performance? Its effectiveness is mostly unidentified. This study was conducted to provide a shared experience to give nurse educators confidence and an insight into how simulation-based teaching can fit into nursing skills learning. A pilot study was completed with 50 second-year undergraduate nursing students, and the main study included 98 students where a pretest-posttest design was adopted. Data were gathered through four questionnaires and a performance assessment under scrutinized controls such as previous experiences, lecturers' teaching skills, duration of teaching, procedure of skills performance assessment and the inter-rater reliability. The results showed that simulation-based learning significantly improved students' self-efficacy regarding skills learning and the skills performance that nurse educators wish students to acquire. However, technology anxiety, examiners' critical attitudes towards students' performance and their unpredicted verbal and non-verbal expressions, have been found as possible confounding factors. The simulation-based learning proved to have a powerful positive effect on students' achievement outcomes. Nursing skills learning is one area that can benefit greatly from this kind of teaching and learning method.

Competency-based assessment in surgeon-performed head and neck ultrasonography: A validity study.

PubMed

Todsen, Tobias; Melchiors, Jacob; Charabi, Birgitte; Henriksen, Birthe; Ringsted, Charlotte; Konge, Lars; von Buchwald, Christian

2018-06-01

Head and neck ultrasonography (HNUS) increasingly is used as a point-of-care diagnostic tool by otolaryngologists. However, ultrasonography (US) is a very operator-dependent image modality. Hence, this study aimed to explore the diagnostic accuracy of surgeon-performed HNUS and to establish validity evidence for an objective structured assessment of ultrasound skills (OSAUS) used for competency-based assessment. A prospective experimental study. Six otolaryngologists and 11 US novices were included in a standardized test setup for which they had to perform focused HNUS of eight patients suspected for different head and neck lesions. Their diagnostic accuracy was calculated based on the US reports, and two blinded raters assessed the video-recorded US performance using the OSAUS scale. The otolaryngologists obtained a high diagnostic accuracy on 88% (range 63%-100%) compared to the US novices on 38% (range 0-63%); P < 0.001. The OSAUS score demonstrated good inter-case reliability (0.85) and inter-rater reliability (0.76), and significant discrimination between otolaryngologist and US novices; P < 0.001. A strong correlation between the OSAUS score and the diagnostic accuracy was found (Spearman's ρ, 0.85; P < P 0.001), and a pass/fail score was established at 2.8. Strong validity evidence supported the use of the OSAUS scale to assess HNUS competence with good reliability, significant discrimination between US competence levels, and a strong correlation of assessment score to diagnostic accuracy. An OSAUS pass/fail score was established and could be used for competence-based assessment in surgeon-performed HNUS. NA. Laryngoscope, 128:1346-1352, 2018. © 2017 The American Laryngological, Rhinological and Otological Society, Inc.

How to Assess Creative Thinking Skill in Making Products of Liquid Pressure?

NASA Astrophysics Data System (ADS)

Chasanah, L.; Kaniawati, I.; Hernani, H.

2017-09-01

The primary skills that must be possessed in the 21st century curriculum are learning and innovation skills. One of the learning strategies that can train students to innovate and improve creative thinking skills is by applying Science, Technology, Engineering and Mathematics (STEM) in learning. Based on an interview to one of the science teachers that learning that aims to train learning and innovation skills has not been applied to learning in the classroom because there is not enough time, learning materials and assessment instruments used. This study aims to determine the results of the implementation of performance assessment of creative thinking skills on STEM-based learning in junior high school for the material of liquid pressure. This research uses descriptive method. Participants in this study were junior high school students 8th in Kudus area. The research instrument consists of observation sheet, performance assessment and documentation. The result showed that creative thinking skills performance assessment can assess student’s creativity in making products of STEM-based learning for junior high school.

Neuropsychological predictors of performance-based measures of functional capacity and social skills in individuals with severe mental illness.

PubMed

Mahmood, Zanjbeel; Burton, Cynthia Z; Vella, Lea; Twamley, Elizabeth W

2018-04-13

Neuropsychological abilities may underlie successful performance of everyday functioning and social skills. We aimed to determine the strongest neuropsychological predictors of performance-based functional capacity and social skills performance across the spectrum of severe mental illness (SMI). Unemployed outpatients with SMI (schizophrenia, bipolar disorder, or major depression; n = 151) were administered neuropsychological (expanded MATRICS Consensus Cognitive Battery), functional capacity (UCSD Performance-Based Skills Assessment-Brief; UPSA-B), and social skills (Social Skills Performance Assessment; SSPA) assessments. Bivariate correlations between neuropsychological performance and UPSA-B and SSPA total scores showed that most neuropsychological tests were significantly associated with each performance-based measure. Forward entry stepwise regression analyses were conducted entering education, diagnosis, symptom severity, and neuropsychological performance as predictors of functional capacity and social skills. Diagnosis, working memory, sustained attention, and category and letter fluency emerged as significant predictors of functional capacity, in a model that explained 43% of the variance. Negative symptoms, sustained attention, and letter fluency were significant predictors of social skill performance, in a model explaining 35% of the variance. Functional capacity is positively associated with neuropsychological functioning, but diagnosis remains strongly influential, with mood disorder participants outperforming those with psychosis. Social skill performance appears to be positively associated with sustained attention and verbal fluency regardless of diagnosis; however, negative symptom severity strongly predicts social skills performance. Improving neuropsychological functioning may improve psychosocial functioning in people with SMI. Published by Elsevier Ltd.

Bureau of Indian Education 2010-2011 (Based on SY 2009-2010 Data) Special Education Indicator Performance

ERIC Educational Resources Information Center

Bureau of Indian Education, 2012

2012-01-01

This report presents Special Education Indicator Performance data tables for 174 Bureau of Indian Education schools. Indicators include: (1) Graduation Rate; (2) Dropouts; (3) Assessment Targets; (4) Assessment Targets--Reading; (5) Assessment Targets--Mathematics; (6) Suspensions and Expulsions--High School; (7) Suspensions and…

Bureau of Indian Education 2012-2013 (Based on SY 2011-2012 Data) Special Education Indicator Performance

ERIC Educational Resources Information Center

Bureau of Indian Education, 2013

2013-01-01

This report presents Special Education Indicator Performance data tables for 173 Bureau of Indian Education schools. Indicators include: (1) Graduation Rate; (2) Dropouts; (3) Assessment Targets; (4) Assessment Targets--Reading; (5) Assessment Targets--Mathematics; (6) Suspensions and Expulsions--High School; (7) Suspensions and…

Multilayered Word Structure Model for Assessing Spelling of Finnish Children in Shallow Orthography

ERIC Educational Resources Information Center

Kulju, Pirjo; Mäkinen, Marita

2017-01-01

This study explores Finnish children's word-level spelling by applying a linguistically based multilayered word structure model for assessing spelling performance. The model contributes to the analytical qualitative assessment approach in order to identify children's spelling performance for enhancing writing skills. The children (N = 105)…

Great Performances: Creating Classroom-Based Assessment Tasks. Second Edition

ERIC Educational Resources Information Center

Shoemaker, Betty; Lewin, Larry

2011-01-01

Get an in-depth understanding of how to create fun, engaging, and challenging performance assessments that require students to elaborate on content and demonstrate mastery of skills. This update of an ASCD (Association for Supervision and Curriculum Development) classic includes new scoring methods, reading assessments, and insights on navigating…

Performance and Cognitive Assessment in 3-D Modeling

ERIC Educational Resources Information Center

Fahrer, Nolan E.; Ernst, Jeremy V.; Branoff, Theodore J.; Clark, Aaron C.

2011-01-01

The purpose of this study was to investigate identifiable differences between performance and cognitive assessment scores in a 3-D modeling unit of an engineering drafting course curriculum. The study aimed to provide further investigation of the need of skill-based assessments in engineering/technical graphics courses to potentially increase…

Six Years of Comprehensive, Clinical, Performance-Based Assessment Using Standardized Patients at the Southern Illinois University School of Medicine.

ERIC Educational Resources Information Center

Vu, Nu Viet; And Others

1992-01-01

The use of a performance-based assessment of senior medical students' clinical skills utilizing standardized patients was evaluated, with 6,804 student-patient encounters involving 405 students over 6 years. Results provide evidence for test security, content validity, construct validity, reliability, and test ability to discriminate a wide range…

Participation and Performance Reporting for the Alternate Assessment Based on Modified Achievement Standards (AA-MAS). Technical Report 58

ERIC Educational Resources Information Center

Albus, Deb; Thurlow, Martha L.; Lazarus, Sheryl S.

2011-01-01

This report examines publicly reported participation and performance data for the alternate assessment based on modified achievement standards (AA-MAS). The authors' analysis of these data included all states publicly reporting AA-MAS data, regardless of whether they had received approval to use the results for Title I accountability calculations.…

On-Line Mathematics Assessment: The Impact of Mode on Performance and Question Answering Strategies

ERIC Educational Resources Information Center

Johnson, Martin; Green, Sylvia

2006-01-01

The transition from paper-based to computer-based assessment raises a number of important issues about how mode might affect children's performance and question answering strategies. In this project 104 eleven-year-olds were given two sets of matched mathematics questions, one set on-line and the other on paper. Facility values were analyzed to…

Ready for the Future: The Role of Performance Assessments in Shaping Graduates' Academic, Professional, and Personal Lives

ERIC Educational Resources Information Center

Gagnon, Laurie

2010-01-01

The perspectives of graduates offer a valuable source of understanding for educators and policy-makers on how to ensure high quality educational pathways that prepare all students for work and college. Based on in-depth interviews with graduates from three Boston Public Schools with well-established performance-based assessment systems, the study…

Teacher Perspectives on Performance Based Student Assessment in Career and Technical Education in West Virginia Public Schools

ERIC Educational Resources Information Center

Tuckwiller, Brenda L.

2012-01-01

The purpose of this study was to investigate career and technical education teachers' level of knowledge and use of performance based student assessment practices in West Virginia's secondary and post-secondary career education centers. In addition, this study sought to determine what relationships, if any, exist between levels of knowledge and…

Showing Up Is Half the Battle: Assessing Different Contextualized Learning Tools to Increase the Performance in Introductory Computer Science Courses

ERIC Educational Resources Information Center

Rolka, Christine; Remshagen, Anja

2015-01-01

Contextualized learning is considered beneficial for student success. In this article, we assess the impact of context-based learning tools on student grade performance in an introductory computer science course. In particular, we investigate two central questions: (1) does the use context-based learning tools, robots and animations, affect…

Use of Low-Fidelity Simulation Laboratory Training for Teaching Radiology Residents CT-Guided Procedures.

PubMed

Picard, Melissa; Nelson, Rachel; Roebel, John; Collins, Heather; Anderson, M Bret

2016-11-01

To determine the benefit of the addition of low-fidelity simulation-based training to the standard didactic-based training in teaching radiology residents common CT-guided procedures. This was a prospective study involving 24 radiology residents across all years in a university program. All residents underwent standard didactic lecture followed by low-fidelity simulation-based training on three common CT-guided procedures: random liver biopsy, lung nodule biopsy, and drain placement. Baseline knowledge, confidence, and performance assessments were obtained after the didactic session and before the simulation training session. Approximately 2 months later, all residents participated in a simulation-based training session covering all three of these procedures. Knowledge, confidence, and performance data were obtained afterward. These assessments covered topics related to preprocedure workup, intraprocedure steps, and postprocedure management. Knowledge data were collected based on a 15-question assessment. Confidence data were obtained based on a 5-point Likert-like scale. Performance data were obtained based on successful completion of predefined critical steps. There was significant improvement in knowledge (P = .005), confidence (P < .008), and tested performance (P < .043) after the addition of simulation-based training to the standard didactic curriculum for all procedures. This study suggests that the addition of low-fidelity simulation-based training to a standard didactic-based curriculum is beneficial in improving resident knowledge, confidence, and tested performance of common CT-guided procedures. Copyright © 2016 American College of Radiology. Published by Elsevier Inc. All rights reserved.

Video gaming and gender differences in digital and printed reading performance among 15-year-olds students in 26 countries.

PubMed

Borgonovi, Francesca

2016-04-01

Video games are a favorite leisure-time activity among teenagers worldwide. This study examines cross-national gender differences in reading achievement and video gaming and whether video gaming explains gender differences in reading achievement and differences in performance between paper-based and computer-based reading. We use data from a representative sample of 145,953 students from 26 countries who sat the PISA 2012 assessments and provided self-reports on use of video games. Although boys tend to have poorer results in both the computer-based and the paper-based reading assessments, boys' under achievement is smaller when the assessment is delivered on computer than when it is delivered on paper. Boys underperformance compared to girls in the two reading assessments is particularly pronounced among low-achieving students. Among both boys and girls moderate use of single-player games is associated with a performance advantage. However, frequent engagement with collaborative online games is generally associated with a steep reduction in achievement, particularly in the paper-based test and particularly among low-achieving students. Excessive gaming may hinder academic achievement, but moderate gaming can promote positive student outcomes. In many countries video gaming explains the difference in the gender gap in reading between the paper-based and the computer-based assessments. Copyright © 2016 The Foundation for Professionals in Services for Adolescents. Published by Elsevier Ltd. All rights reserved.

Assessing a VR-based learning environment for anatomy education.

PubMed

Hoffman, H; Murray, M; Hettinger, L; Viirre, E

1998-01-01

The purpose of the research proposed herein is to develop an empirical, methodological tool for the assessment of visual depth perception in virtual environments (VEs). Our goal is to develop and employ a behaviorally-based method for assessing the impact of VE design features on the perception of visual depth as indexed by the performance of fundamental perceptual-motor activities. Specifically, in this experiment we will assess the affect of two dimensions of VE system design--(1) viewing condition or "level of immersion", and (2) layout/design of the VE--on the performance of an engaging, game-like task. The characteristics of the task to be employed are as follows--(1) it places no demands on cognition in the form of problem solving, retrieval of previously learned information, or other analytic activity in order to assure that (2) variations in task performance can be exclusively attributed to the extent to which the experimental factors influence visual depth perception. Subjects' performance will be assessed in terms of the speed and accuracy of task performance, as well as underlying dimensions of performance such as workload, fatigue, and physiological well being (i.e., cybersickness). The results of this experiment will provide important information on the effect of VE immersion and other VE design issues on human perception and performance. Further development, refinement, and validation of this behaviorally-based methodology will be pursued to provide user-centered design criteria for the design and use of VE systems.

Assessing performance outcomes of new graduates utilizing simulation in a military transition program.

PubMed

Hughes, Robie V; Smith, Sherrill J; Sheffield, Clair M; Wier, Grady

2013-01-01

This multi-site, quasi-experimental study examined the performance outcomes of nurses (n = 152) in a military nurse transition program. A modified-performance instrument was used to assess participants in two high-fidelity simulation scenarios. Although results indicated a significant increase in scores posttraining, only moderate interrater reliability results were found for the new instrument. These findings have implications for nurse educators assessing performance-based outcomes of new nurses completing transition programs.

Psychometric Properties of Virtual Reality Vignette Performance Measures: A Novel Approach for Assessing Adolescents' Social Competency Skills

ERIC Educational Resources Information Center

Paschall, Mallie J.; Fishbein, Diana H.; Hubal, Robert C.; Eldreth, Diana

2005-01-01

This study examined the psychometric properties of performance measures for three novel, interactive virtual reality vignette exercises developed to assess social competency skills of at-risk adolescents. Performance data were collected from 117 African-American male 15-17 year olds. Data for 18 performance measures were obtained, based on…

Assessment.

ERIC Educational Resources Information Center

Andrews, Theodore E., Ed.

"The Role of the State in Performance-Based Teacher Education-Certification" by Robert Roth creates a context for viewing how state agencies are approaching performance education. Peter Airasian then explores the value questions that are at the heart of evaluation issues. Fred McDonald looks at "The State of the Art in Performance Assessment," and…

Assessing Procedural Competence: Validity Considerations.

PubMed

Pugh, Debra M; Wood, Timothy J; Boulet, John R

2015-10-01

Simulation-based medical education (SBME) offers opportunities for trainees to learn how to perform procedures and to be assessed in a safe environment. However, SBME research studies often lack robust evidence to support the validity of the interpretation of the results obtained from tools used to assess trainees' skills. The purpose of this paper is to describe how a validity framework can be applied when reporting and interpreting the results of a simulation-based assessment of skills related to performing procedures. The authors discuss various sources of validity evidence because they relate to SBME. A case study is presented.

Automated Clinical Assessment from Smart home-based Behavior Data

PubMed Central

Dawadi, Prafulla Nath; Cook, Diane Joyce; Schmitter-Edgecombe, Maureen

2016-01-01

Smart home technologies offer potential benefits for assisting clinicians by automating health monitoring and well-being assessment. In this paper, we examine the actual benefits of smart home-based analysis by monitoring daily behaviour in the home and predicting standard clinical assessment scores of the residents. To accomplish this goal, we propose a Clinical Assessment using Activity Behavior (CAAB) approach to model a smart home resident’s daily behavior and predict the corresponding standard clinical assessment scores. CAAB uses statistical features that describe characteristics of a resident’s daily activity performance to train machine learning algorithms that predict the clinical assessment scores. We evaluate the performance of CAAB utilizing smart home sensor data collected from 18 smart homes over two years using prediction and classification-based experiments. In the prediction-based experiments, we obtain a statistically significant correlation (r = 0.72) between CAAB-predicted and clinician-provided cognitive assessment scores and a statistically significant correlation (r = 0.45) between CAAB-predicted and clinician-provided mobility scores. Similarly, for the classification-based experiments, we find CAAB has a classification accuracy of 72% while classifying cognitive assessment scores and 76% while classifying mobility scores. These prediction and classification results suggest that it is feasible to predict standard clinical scores using smart home sensor data and learning-based data analysis. PMID:26292348

Ability-versus skill-based assessment of emotional intelligence.

PubMed

Bradberry, Travis R; Su, Lac D

2006-01-01

Emotional intelligence has received an intense amount of attention in leadership circles during the last decade and continuing debate exists concerning the best method for measuring this construct. This study analyzed leader emotional intelligence scores, measured via skill and ability methodologies, against leader job performance. Two hundred twelve employees from three organizations participated in this study. Scores on the Emotional Intelligence Appraisal, a skill-based assessment, were positively, though not significantly, correlated with scores on the MSCEIT, an ability-based assessment of emotional intelligence. Scores on the MSCEIT did not have a significant relationship with job performance in this study, whereas, scores on the Emotional Intelligence Appraisal had a strong link to leader job performance. The four subcomponents of the Emotional Intelligence Appraisal were examined against job performance. Relationship management was a stronger predictor of leader job performance than the other three subcomponents. Social awareness was the single emotional intelligence skill that did not have a significant link to leader job performance. Factor analyses yielded a two-component model of emotional intelligence encompassing personal and social competence, rather than confirmation of a four-part taxonomy.

Structural assessment of a Space Station solar dynamic heat receiver thermal energy storage canister

NASA Technical Reports Server (NTRS)

Tong, M. T.; Kerslake, T. W.; Thompson, R. L.

1988-01-01

This paper assesses the structural performance of a Space Station thermal energy storage (TES) canister subject to orbital solar flux variation and engine cold start-up operating conditions. The impact of working fluid temperature and salt-void distribution on the canister structure are assessed. Both analytical and experimental studies were conducted to determine the temperature distribution of the canister. Subsequent finite-element structural analyses of the canister were performed using both analytically and experimentally obtained temperatures. The Arrhenius creep law was incorporated into the procedure, using secondary creep data for the canister material, Haynes-188 alloy. The predicted cyclic creep strain accumulations at the hot spot were used to assess the structural performance of the canister. In addition, the structural performance of the canister based on the analytically-determined temperature was compared with that based on the experimentally-measured temperature data.

Structural assessment of a space station solar dynamic heat receiver thermal energy storage canister

NASA Technical Reports Server (NTRS)

Thompson, R. L.; Kerslake, T. W.; Tong, M. T.

1988-01-01

The structural performance of a space station thermal energy storage (TES) canister subject to orbital solar flux variation and engine cold start up operating conditions was assessed. The impact of working fluid temperature and salt-void distribution on the canister structure are assessed. Both analytical and experimental studies were conducted to determine the temperature distribution of the canister. Subsequent finite element structural analyses of the canister were performed using both analytically and experimentally obtained temperatures. The Arrhenius creep law was incorporated into the procedure, using secondary creep data for the canister material, Haynes 188 alloy. The predicted cyclic creep strain accumulations at the hot spot were used to assess the structural performance of the canister. In addition, the structural performance of the canister based on the analytically determined temperature was compared with that based on the experimentally measured temperature data.

Architecture, Design, and System; Performance Assessment and Development Methodology for Computer-Based Systems. Volume 1. Methodology Description, Discussion, and Assessment,

DTIC Science & Technology

1983-12-30

AD-Ri46 57? ARCHITECTURE DESIGN AND SYSTEM; PERFORMANCE ASSESSMENT i/i AND DEVELOPMENT ME..(U) NAVAL SURFACE WEAPONS CENTER SILYER SPRING MD J...AD-A 146 577 NSIWC TR 83-324 ARCHITECTURE , DESIGN , AND SYSTEM; PERFORMANCE ASSESSMENT AND DEVELOPMENT METHODOLOGY...REPORT NUMBER 12. GOVT ACCESSION NO.3. RECIPIENT’S CATALOG NUMBER NSWC TR 83-324 10- 1 1 51’ 4. ?ITLE (and subtitle) ARCHITECTURE , DESIGN , AND SYSTEM; S

Critical thinking skills in nursing students: comparison of simulation-based performance with metrics.

PubMed

Fero, Laura J; O'Donnell, John M; Zullo, Thomas G; Dabbs, Annette DeVito; Kitutu, Julius; Samosky, Joseph T; Hoffman, Leslie A

2010-10-01

This paper is a report of an examination of the relationship between metrics of critical thinking skills and performance in simulated clinical scenarios. Paper and pencil assessments are commonly used to assess critical thinking but may not reflect simulated performance. In 2007, a convenience sample of 36 nursing students participated in measurement of critical thinking skills and simulation-based performance using videotaped vignettes, high-fidelity human simulation, the California Critical Thinking Disposition Inventory and California Critical Thinking Skills Test. Simulation-based performance was rated as 'meeting' or 'not meeting' overall expectations. Test scores were categorized as strong, average, or weak. Most (75.0%) students did not meet overall performance expectations using videotaped vignettes or high-fidelity human simulation; most difficulty related to problem recognition and reporting findings to the physician. There was no difference between overall performance based on method of assessment (P = 0.277). More students met subcategory expectations for initiating nursing interventions (P ≤ 0.001) using high-fidelity human simulation. The relationship between videotaped vignette performance and critical thinking disposition or skills scores was not statistically significant, except for problem recognition and overall critical thinking skills scores (Cramer's V = 0.444, P = 0.029). There was a statistically significant relationship between overall high-fidelity human simulation performance and overall critical thinking disposition scores (Cramer's V = 0.413, P = 0.047). Students' performance reflected difficulty meeting expectations in simulated clinical scenarios. High-fidelity human simulation performance appeared to approximate scores on metrics of critical thinking best. Further research is needed to determine if simulation-based performance correlates with critical thinking skills in the clinical setting. © 2010 The Authors. Journal of Advanced Nursing © 2010 Blackwell Publishing Ltd.

Critical thinking skills in nursing students: comparison of simulation-based performance with metrics

PubMed Central

Fero, Laura J.; O’Donnell, John M.; Zullo, Thomas G.; Dabbs, Annette DeVito; Kitutu, Julius; Samosky, Joseph T.; Hoffman, Leslie A.

2018-01-01

Aim This paper is a report of an examination of the relationship between metrics of critical thinking skills and performance in simulated clinical scenarios. Background Paper and pencil assessments are commonly used to assess critical thinking but may not reflect simulated performance. Methods In 2007, a convenience sample of 36 nursing students participated in measurement of critical thinking skills and simulation-based performance using videotaped vignettes, high-fidelity human simulation, the California Critical Thinking Disposition Inventory and California Critical Thinking Skills Test. Simulation- based performance was rated as ‘meeting’ or ‘not meeting’ overall expectations. Test scores were categorized as strong, average, or weak. Results Most (75·0%) students did not meet overall performance expectations using videotaped vignettes or high-fidelity human simulation; most difficulty related to problem recognition and reporting findings to the physician. There was no difference between overall performance based on method of assessment (P = 0·277). More students met subcategory expectations for initiating nursing interventions (P ≤ 0·001) using high-fidelity human simulation. The relationship between video-taped vignette performance and critical thinking disposition or skills scores was not statistically significant, except for problem recognition and overall critical thinking skills scores (Cramer’s V = 0·444, P = 0·029). There was a statistically significant relationship between overall high-fidelity human simulation performance and overall critical thinking disposition scores (Cramer’s V = 0·413, P = 0·047). Conclusion Students’ performance reflected difficulty meeting expectations in simulated clinical scenarios. High-fidelity human simulation performance appeared to approximate scores on metrics of critical thinking best. Further research is needed to determine if simulation-based performance correlates with critical thinking skills in the clinical setting. PMID:20636471

Patient-reported outcome measures versus inertial performance-based outcome measures: A prospective study in patients undergoing primary total knee arthroplasty.

PubMed

Bolink, S A A N; Grimm, B; Heyligers, I C

2015-12-01

Outcome assessment of total knee arthroplasty (TKA) by subjective patient reported outcome measures (PROMs) may not fully capture the functional (dis-)abilities of relevance. Objective performance-based outcome measures could provide distinct information. An ambulant inertial measurement unit (IMU) allows kinematic assessment of physical performance and could potentially be used for routine follow-up. To investigate the responsiveness of IMU measures in patients following TKA and compare outcomes with conventional PROMs. Patients with end stage knee OA (n=20, m/f=7/13; age=67.4 standard deviation 7.7 years) were measured preoperatively and one year postoperatively. IMU measures were derived during gait, sit-stand transfers and block step-up transfers. PROMs were assessed by using the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) and Knee Society Score (KSS). Responsiveness was calculated by the effect size, correlations were calculated with Spearman's rho correlation coefficient. One year after TKA, patients performed significantly better at gait, sit-to-stand transfers and block step-up transfers. Measures of time and kinematic IMU measures demonstrated significant improvements postoperatively for each performance-based test. The largest improvement was found in block step-up transfers (effect size=0.56-1.20). WOMAC function score and KSS function score demonstrated moderate correlations (Spearman's rho=0.45-0.74) with some of the physical performance-based measures pre- and postoperatively. To characterize the changes in physical function after TKA, PROMs could be supplemented by performance-based measures, assessing function during different activities and allowing kinematic characterization with an ambulant IMU. Copyright © 2015 Elsevier B.V. All rights reserved.

Performance assessment in complex individual and team tasks

NASA Technical Reports Server (NTRS)

Eddy, Douglas R.

1992-01-01

Described here is an eclectic, performance based approach to assessing cognitive performance from multiple perspectives. The experience gained from assessing the effects of antihistamines and scenario difficulty on C (exp 2) decision making performance in Airborne Warning and Control Systems (AWACS) weapons director (WD) teams can serve as a model for realistic simulations in space operations. Emphasis is placed on the flexibility of measurement, hierarchical organization of measurement levels, data collection from multiple perspectives, and the difficulty of managing large amounts of data.

Job-Preference and Job-Matching Assessment Results and Their Association with Job Performance and Satisfaction among Young Adults with Developmental Disabilities

ERIC Educational Resources Information Center

Hall, Julie; Morgan, Robert L.; Salzberg, Charles L.

2014-01-01

We investigated the effects of preference and degree of match on job performance of four 19 to 20-year-old young adults with developmental disabilities placed in community-based job conditions. We identified high-preference, high-matched and low-preference, low-matched job tasks using a video web-based assessment program. The job matching…

Performance assessment of conventional and base-isolated nuclear power plants for earthquake and blast loadings

NASA Astrophysics Data System (ADS)

Huang, Yin-Nan

Nuclear power plants (NPPs) and spent nuclear fuel (SNF) are required by code and regulations to be designed for a family of extreme events, including very rare earthquake shaking, loss of coolant accidents, and tornado-borne missile impacts. Blast loading due to malevolent attack became a design consideration for NPPs and SNF after the terrorist attacks of September 11, 2001. The studies presented in this dissertation assess the performance of sample conventional and base isolated NPP reactor buildings subjected to seismic effects and blast loadings. The response of the sample reactor building to tornado-borne missile impacts and internal events (e.g., loss of coolant accidents) will not change if the building is base isolated and so these hazards were not considered. The sample NPP reactor building studied in this dissertation is composed of containment and internal structures with a total weight of approximately 75,000 tons. Four configurations of the reactor building are studied, including one conventional fixed-base reactor building and three base-isolated reactor buildings using Friction Pendulum(TM), lead rubber and low damping rubber bearings. The seismic assessment of the sample reactor building is performed using a new procedure proposed in this dissertation that builds on the methodology presented in the draft ATC-58 Guidelines and the widely used Zion method, which uses fragility curves defined in terms of ground-motion parameters for NPP seismic probabilistic risk assessment. The new procedure improves the Zion method by using fragility curves that are defined in terms of structural response parameters since damage and failure of NPP components are more closely tied to structural response parameters than to ground motion parameters. Alternate ground motion scaling methods are studied to help establish an optimal procedure for scaling ground motions for the purpose of seismic performance assessment. The proposed performance assessment procedure is used to evaluate the vulnerability of the conventional and base-isolated NPP reactor buildings. The seismic performance assessment confirms the utility of seismic isolation at reducing spectral demands on secondary systems. Procedures to reduce the construction cost of secondary systems in isolated reactor buildings are presented. A blast assessment of the sample reactor building is performed for an assumed threat of 2000 kg of TNT explosive detonated on the surface with a closest distance to the reactor building of 10 m. The air and ground shock waves produced by the design threat are generated and used for performance assessment. The air blast loading to the sample reactor building is computed using a Computational Fluid Dynamics code Air3D and the ground shock time series is generated using an attenuation model for soil/rock response. Response-history analysis of the sample conventional and base isolated reactor buildings to external blast loadings is performed using the hydrocode LS-DYNA. The spectral demands on the secondary systems in the isolated reactor building due to air blast loading are greater than those for the conventional reactor building but much smaller than those spectral demands associated with Safe Shutdown Earthquake shaking. The isolators are extremely effective at filtering out high acceleration, high frequency ground shock loading.

Kinespell: Kinesthetic Learning Activity and Assessment in a Digital Game-Based Learning Environment

NASA Astrophysics Data System (ADS)

Cariaga, Ada Angeli; Salvador, Jay Andrae; Solamo, Ma. Rowena; Feria, Rommel

Various approaches in learning are commonly classified into visual, auditory and kinesthetic (VAK) learning styles. One way of addressing the VAK learning styles is through game-based learning which motivates learners pursue knowledge holistically. The paper presents Kinespell, an unconventional method of learning through digital game-based learning. Kinespell is geared towards enhancing not only the learner’s spelling abilities but also the motor skills through utilizing wireless controllers. It monitors player’s performance through integrated assessment scheme. Results show that Kinespell may accommodate the VAK learning styles and is a promising alternative to established methods in learning and assessing students’ performance in Spelling.

Exploring a Framework for Consequential Validity for Performance-Based Assessments

ERIC Educational Resources Information Center

Kim, Su Jung

2017-01-01

This study explores a new comprehensive framework for understanding elements of validity, specifically for performance assessments that are administered within specific and dynamic contexts. The adoption of edTPA is a good empirical case for examining the concept of consequential validity because this assessment has been implemented at the state…

Bureau of Indian Education 2011-2012 (Based on SY 2010-2011 Data) Revised 06/01/2013 Special Education Indicator Performance

ERIC Educational Resources Information Center

Bureau of Indian Education, 2013

2013-01-01

This report presents Special Education Indicator Performance data tables for 173 Bureau of Indian Education schools. Indicators include: (1) Graduation Rate; (2) Dropouts; (3) Assessment Targets; (4) Assessment Targets--Reading; (5) Assessment Targets--Mathematics; (6) Suspensions and Expulsions--High School; (7) Suspensions and…

Promoting Physicians' Self-Assessment and Quality Improvement: The ABIM Diabetes Practice Improvement Module

ERIC Educational Resources Information Center

Holmboe, Eric S.; Meehan, Thomas P.; Lynn, Lorna; Doyle, Paula; Sherwin, Tierney; Duffy, F. Daniel

2006-01-01

Introduction: The American Board of Internal Medicine (ABIM) recognized that certification and recertification must be based on an assessment of performance in practice as well as an examination of medical knowledge. Physician self-assessment of practice performance is proposed as one method that certification boards may use to evaluate competence…

Improving Student Outcomes with mCLASS: Math, a Technology-Enhanced CBM and Diagnostic Interview Assessment

ERIC Educational Resources Information Center

Wang, Ye; Gushta, Matthew

2013-01-01

The No Child Left Behind Act resulted in increased school-level implementation of assessment-based school interventions that aim to improve student performance. Diagnostic assessments are included among these interventions, designed to help teachers use evidence about student performance to modify and differentiate instruction and improve student…

Cooperating Teachers: Stakeholders in the edTPA?

ERIC Educational Resources Information Center

Seymour, Clancy A.; Burns, Barbara A.; Henry, Julie J.

2018-01-01

The educative Teacher Performance Assessment (edTPA) is a performance-based assessment designed for beginning teachers to demonstrate their readiness to teach (SCALE, 2014). As more states come to adopt this assessment, many facets of its implementation need to be reviewed. One component is the role of the cooperating teacher in the implementation…

Risk Assessment

EPA Pesticide Factsheets

How the EPA conducts risk assessment to protect human health and the environment. Several assessments are included with the guidelines, models, databases, state-based RSL Tables, local contacts and framework documents used to perform these assessments.

The Future Value of Serious Games for Assessment: Where Do We Go Now?

ERIC Educational Resources Information Center

de Klerk, Sebastiaan; Kato, Pamela M.

2017-01-01

Game-based assessments will most likely be an increasing part of testing programs in future generations because they provide promising possibilities for more valid and reliable measurement of students' skills as compared to the traditional methods of assessment like paper-and-pencil tests or performance-based assessments. The current status of…

Operating Room Performance Improves after Proficiency-Based Virtual Reality Cataract Surgery Training.

PubMed

Thomsen, Ann Sofia Skou; Bach-Holm, Daniella; Kjærbo, Hadi; Højgaard-Olsen, Klavs; Subhi, Yousif; Saleh, George M; Park, Yoon Soo; la Cour, Morten; Konge, Lars

2017-04-01

To investigate the effect of virtual reality proficiency-based training on actual cataract surgery performance. The secondary purpose of the study was to define which surgeons benefit from virtual reality training. Multicenter masked clinical trial. Eighteen cataract surgeons with different levels of experience. Cataract surgical training on a virtual reality simulator (EyeSi) until a proficiency-based test was passed. Technical performance in the operating room (OR) assessed by 3 independent, masked raters using a previously validated task-specific assessment tool for cataract surgery (Objective Structured Assessment of Cataract Surgical Skill). Three surgeries before and 3 surgeries after the virtual reality training were video-recorded, anonymized, and presented to the raters in random order. Novices (non-independently operating surgeons) and surgeons having performed fewer than 75 independent cataract surgeries showed significant improvements in the OR-32% and 38%, respectively-after virtual reality training (P = 0.008 and P = 0.018). More experienced cataract surgeons did not benefit from simulator training. The reliability of the assessments was high with a generalizability coefficient of 0.92 and 0.86 before and after the virtual reality training, respectively. Clinically relevant cataract surgical skills can be improved by proficiency-based training on a virtual reality simulator. Novices as well as surgeons with an intermediate level of experience showed improvement in OR performance score. Copyright © 2017 American Academy of Ophthalmology. Published by Elsevier Inc. All rights reserved.

Fifteen-Year-Old Pupils' Variable Handling Performance in the Context of Scientific Investigations.

ERIC Educational Resources Information Center

Donnelly, J. F.

1987-01-01

Reports findings on variable-handling aspects of pupil performance in investigatory tasks, using data from the British Assessment of Performance Unit (APU) national survey program. Discusses the significance of these findings for assessment methodology and for understanding of 15-year-olds' approaches to the variable-based logic of investigation.…

Assessing the driving performance of older adult drivers: on-road versus simulated driving.

PubMed

Lee, Hoe C; Cameron, Don; Lee, Andy H

2003-09-01

To validate a laboratory-based driving simulator in measuring on-road driving performance, 129 older adult drivers were assessed with both the simulator and an on-road test. The driving performance of the participants was gauged by appropriate and reliable age-specific assessment criteria, which were found to be negatively correlated with age. Using principal component analysis, two performance indices were developed from the criteria to represent the overall performance in simulated driving and the on-road assessment. There was significant positive association between the two indices, with the simulated driving performance index explaining over two-thirds of the variability of the on-road driving performance index, after adjustment for age and gender of the drivers. The results supported the validity of the driving simulator and it is a safer and more economical method than the on-road testing to assess the driving performance of older adult drivers.

Comparability of Computer-Based and Paper-Based Science Assessments

ERIC Educational Resources Information Center

Herrmann-Abell, Cari F.; Hardcastle, Joseph; DeBoer, George E.

2018-01-01

We compared students' performance on a paper-based test (PBT) and three computer-based tests (CBTs). The three computer-based tests used different test navigation and answer selection features, allowing us to examine how these features affect student performance. The study sample consisted of 9,698 fourth through twelfth grade students from across…

A Paradigm Shift toward Evidence-Based Clinical Practice: Developing a Performance Assessment

ERIC Educational Resources Information Center

Wentworth, Nancy; Erickson, Lynnette B.; Lawrence, Barbara; Popham, J. Aaron; Korth, Byran

2009-01-01

The Clinical Practice Assessment System (CPAS), developed in response to teacher preparation program accreditation requirements, represents a paradigm shift of one university toward data-based decision-making in teacher education programs. The new assessment system is a scale aligned with INTASC Standards, which allows for observation and…

Predictors of Time-Based Prospective Memory in Children

ERIC Educational Resources Information Center

Mackinlay, Rachael J.; Kliegel, Matthias; Mantyla, Timo

2009-01-01

This study identified age differences in time-based prospective memory performance in school-aged children and explored possible cognitive correlates of age-related performance. A total of 56 7- to 12-year-olds performed a prospective memory task in which prospective memory accuracy, ongoing task performance, and time monitoring were assessed.…

Word Recall: Cognitive Performance Within Internet Surveys

PubMed Central

Craig, Benjamin M; Jim, Heather S

2015-01-01

Background The use of online surveys for data collection has increased exponentially, yet it is often unclear whether interview-based cognitive assessments (such as face-to-face or telephonic word recall tasks) can be adapted for use in application-based research settings. Objective The objective of the current study was to compare and characterize the results of online word recall tasks to those of the Health and Retirement Study (HRS) and determine the feasibility and reliability of incorporating word recall tasks into application-based cognitive assessments. Methods The results of the online immediate and delayed word recall assessment, included within the Women’s Health and Valuation (WHV) study, were compared to the results of the immediate and delayed recall tasks of Waves 5-11 (2000-2012) of the HRS. Results Performance on the WHV immediate and delayed tasks demonstrated strong concordance with performance on the HRS tasks (ρc=.79, 95% CI 0.67-0.91), despite significant differences between study populations (P<.001) and study design. Sociodemographic characteristics and self-reported memory demonstrated similar relationships with performance on both the HRS and WHV tasks. Conclusions The key finding of this study is that the HRS word recall tasks performed similarly when used as an online cognitive assessment in the WHV. Online administration of cognitive tests, which has the potential to significantly reduce participant and administrative burden, should be considered in future research studies and health assessments. PMID:26543924

Economic method for helical gear flank surface characterisation

NASA Astrophysics Data System (ADS)

Koulin, G.; Reavie, T.; Frazer, R. C.; Shaw, B. A.

2018-03-01

Typically the quality of a gear pair is assessed based on simplified geometric tolerances which do not always correlate with functional performance. In order to identify and quantify functional performance based parameters, further development of the gear measurement approach is required. Methodology for interpolation of the full active helical gear flank surface, from sparse line measurements, is presented. The method seeks to identify the minimum number of line measurements required to sufficiently characterise an active gear flank. In the form ground gear example presented, a single helix and three profile line measurements was considered to be acceptable. The resulting surfaces can be used to simulate the meshing engagement of a gear pair and therefore provide insight into functional performance based parameters. Therefore the assessment of the quality can be based on the predicted performance in the context of an application.

The Fishery Performance Indicators: A Management Tool for Triple Bottom Line Outcomes

PubMed Central

Anderson, James L.; Anderson, Christopher M.; Chu, Jingjie; Meredith, Jennifer; Asche, Frank; Sylvia, Gil; Smith, Martin D.; Anggraeni, Dessy; Arthur, Robert; Guttormsen, Atle; McCluney, Jessica K.; Ward, Tim; Akpalu, Wisdom; Eggert, Håkan; Flores, Jimely; Freeman, Matthew A.; Holland, Daniel S.; Knapp, Gunnar; Kobayashi, Mimako; Larkin, Sherry; MacLauchlin, Kari; Schnier, Kurt; Soboil, Mark; Tveteras, Sigbjorn; Uchida, Hirotsugu; Valderrama, Diego

2015-01-01

Pursuit of the triple bottom line of economic, community and ecological sustainability has increased the complexity of fishery management; fisheries assessments require new types of data and analysis to guide science-based policy in addition to traditional biological information and modeling. We introduce the Fishery Performance Indicators (FPIs), a broadly applicable and flexible tool for assessing performance in individual fisheries, and for establishing cross-sectional links between enabling conditions, management strategies and triple bottom line outcomes. Conceptually separating measures of performance, the FPIs use 68 individual outcome metrics—coded on a 1 to 5 scale based on expert assessment to facilitate application to data poor fisheries and sectors—that can be partitioned into sector-based or triple-bottom-line sustainability-based interpretative indicators. Variation among outcomes is explained with 54 similarly structured metrics of inputs, management approaches and enabling conditions. Using 61 initial fishery case studies drawn from industrial and developing countries around the world, we demonstrate the inferential importance of tracking economic and community outcomes, in addition to resource status. PMID:25946194

Practice Guidelines for Operative Performance Assessments.

PubMed

Williams, Reed G; Kim, Michael J; Dunnington, Gary L

2016-12-01

To provide recommended practice guidelines for assessing single operative performances and for combining results of operative performance assessments into estimates of overall operative performance ability. Operative performance is one defining characteristic of surgeons. Assessment of operative performance is needed to provide feedback with learning benefits to surgical residents in training and to assist in making progress decisions for residents. Operative performance assessment has been a focus of investigation over the past 20 years. This review is designed to integrate findings of this research into a set of recommended operative performance practices. Literature from surgery and from other pertinent research areas (psychology, education, business) was reviewed looking for evidence to inform practice guideline development. Guidelines were created along with a conceptual and scientific foundation for each guideline. Ten guidelines are provided for assessing individual operative performances and 10 are provided for combing data from individual operative performances into overall judgments of operative performance ability. The practice guidelines organize available information to be immediately useful to program directors, to support surgical training, and to provide a conceptual framework upon which to build as the base of pertinent knowledge expands through future research and development efforts.

Effect of exposure to good vs poor medical trainee performance on attending physician ratings of subsequent performances.

PubMed

Yeates, Peter; O'Neill, Paul; Mann, Karen; Eva, Kevin W

2012-12-05

Competency-based models of education require assessments to be based on individuals' capacity to perform, yet the nature of human judgment may fundamentally limit the extent to which such assessment is accurately possible. To determine whether recent observations of the Mini Clinical Evaluation Exercise (Mini-CEX) performance of postgraduate year 1 physicians influence raters' scores of subsequent performances, consistent with either anchoring bias (scores biased similar to previous experience) or contrast bias (scores biased away from previous experience). Internet-based randomized, blinded experiment using videos of Mini-CEX assessments of postgraduate year 1 trainees interviewing new internal medicine patients. Participants were 41 attending physicians from England and Wales experienced with the Mini-CEX, with 20 watching and scoring 3 good trainee performances and 21 watching and scoring 3 poor performances. All then watched and scored the same 3 borderline video performances. The study was completed between July and November 2011. The primary outcome was scores assigned to the borderline videos, using a 6-point Likert scale (anchors included: 1, well below expectations; 3, borderline; 6, well above expectations). Associations were tested in a multivariable analysis that included participants' sex, years of practice, and the stringency index (within-group z score of initial 3 ratings). The mean rating scores assigned by physicians who viewed borderline video performances following exposure to good performances was 2.7 (95% CI, 2.4-3.0) vs 3.4 (95% CI, 3.1-3.7) following exposure to poor performances (difference of 0.67 [95% CI, 0.28-1.07]; P = .001). Borderline videos were categorized as consistent with failing scores in 33 of 60 assessments (55%) in those exposed to good performances and in 15 of 63 assessments (24%) in those exposed to poor performances (P < .001). They were categorized as consistent with passing scores in 5 of 60 assessments (8.3%) in those exposed to good performances compared with 25 of 63 assessments (39.5%) in those exposed to poor performances (P < .001). Sex and years of attending practice were not associated with scores. The priming condition (good vs poor performances) and the stringency index jointly accounted for 45% of the observed variation in raters' scores for the borderline videos (P < .001). In an experimental setting, attending physicians exposed to videos of good medical trainee performances rated subsequent borderline performances lower than those who had been exposed to poor performances, consistent with a contrast bias.

Assessment of School-Based Management. [Volume I: Findings and Conclusions.] Studies of Education Reform.

ERIC Educational Resources Information Center

Wohlstetter, Priscilla; Mohrman, Susan Albers

This document presents findings of the Assessment of School-Based Management Study, which identified the conditions in schools that promote high performance through school-based management (SBM). The study's conceptual framework was based on Edward E. Lawler's (1986) model. The high-involvement framework posits that four resources must spread…

Implementing the Science Assessment Standards: Developing and validating a set of laboratory assessment tasks in high school biology

NASA Astrophysics Data System (ADS)

Saha, Gouranga Chandra

Very often a number of factors, especially time, space and money, deter many science educators from using inquiry-based, hands-on, laboratory practical tasks as alternative assessment instruments in science. A shortage of valid inquiry-based laboratory tasks for high school biology has been cited. Driven by this need, this study addressed the following three research questions: (1) How can laboratory-based performance tasks be designed and developed that are doable by students for whom they are designed/written? (2) Do student responses to the laboratory-based performance tasks validly represent at least some of the intended process skills that new biology learning goals want students to acquire? (3) Are the laboratory-based performance tasks psychometrically consistent as individual tasks and as a set? To answer these questions, three tasks were used from the six biology tasks initially designed and developed by an iterative process of trial testing. Analyses of data from 224 students showed that performance-based laboratory tasks that are doable by all students require careful and iterative process of development. Although the students demonstrated more skill in performing than planning and reasoning, their performances at the item level were very poor for some items. Possible reasons for the poor performances have been discussed and suggestions on how to remediate the deficiencies have been made. Empirical evidences for validity and reliability of the instrument have been presented both from the classical and the modern validity criteria point of view. Limitations of the study have been identified. Finally implications of the study and directions for further research have been discussed.

The Compass Rose Effectiveness Model

ERIC Educational Resources Information Center

Spiers, Cynthia E.; Kiel, Dorothy; Hohenrink, Brad

2008-01-01

The effectiveness model focuses the institution on mission achievement through assessment and improvement planning. Eleven mission criteria, measured by key performance indicators, are aligned with the accountability interest of internal and external stakeholders. A Web-based performance assessment application supports the model, documenting the…

Embedded assessment algorithms within home-based cognitive computer game exercises for elders.

PubMed

Jimison, Holly; Pavel, Misha

2006-01-01

With the recent consumer interest in computer-based activities designed to improve cognitive performance, there is a growing need for scientific assessment algorithms to validate the potential contributions of cognitive exercises. In this paper, we present a novel methodology for incorporating dynamic cognitive assessment algorithms within computer games designed to enhance cognitive performance. We describe how this approach works for variety of computer applications and describe cognitive monitoring results for one of the computer game exercises. The real-time cognitive assessments also provide a control signal for adapting the difficulty of the game exercises and providing tailored help for elders of varying abilities.

PERFORMANCE OF A COMPUTER-BASED ASSESSMENT OF COGNITIVE FUNCTION MEASURES IN TWO COHORTS OF SENIORS

PubMed Central

Espeland, Mark A.; Katula, Jeffrey A.; Rushing, Julia; Kramer, Arthur F.; Jennings, Janine M.; Sink, Kaycee M.; Nadkarni, Neelesh K.; Reid, Kieran F.; Castro, Cynthia M.; Church, Timothy; Kerwin, Diana R.; Williamson, Jeff D.; Marottoli, Richard A.; Rushing, Scott; Marsiske, Michael; Rapp, Stephen R.

2013-01-01

Background Computer-administered assessment of cognitive function is being increasingly incorporated in clinical trials, however its performance in these settings has not been systematically evaluated. Design The Seniors Health and Activity Research Program (SHARP) pilot trial (N=73) developed a computer-based tool for assessing memory performance and executive functioning. The Lifestyle Interventions and Independence for Seniors (LIFE) investigators incorporated this battery in a full scale multicenter clinical trial (N=1635). We describe relationships that test scores have with those from interviewer-administered cognitive function tests and risk factors for cognitive deficits and describe performance measures (completeness, intra-class correlations). Results Computer-based assessments of cognitive function had consistent relationships across the pilot and full scale trial cohorts with interviewer-administered assessments of cognitive function, age, and a measure of physical function. In the LIFE cohort, their external validity was further demonstrated by associations with other risk factors for cognitive dysfunction: education, hypertension, diabetes, and physical function. Acceptable levels of data completeness (>83%) were achieved on all computer-based measures, however rates of missing data were higher among older participants (odds ratio=1.06 for each additional year; p<0.001) and those who reported no current computer use (odds ratio=2.71; p<0.001). Intra-class correlations among clinics were at least as low (ICC≤0.013) as for interviewer measures (ICC≤0.023), reflecting good standardization. All cognitive measures loaded onto the first principal component (global cognitive function), which accounted for 40% of the overall variance. Conclusion Our results support the use of computer-based tools for assessing cognitive function in multicenter clinical trials of older individuals. PMID:23589390

Linking Workplace Health Promotion Best Practices and Organizational Financial Performance: Tracking Market Performance of Companies With Highest Scores on the HERO Scorecard.

PubMed

Grossmeier, Jessica; Fabius, Ray; Flynn, Jennifer P; Noeldner, Steven P; Fabius, Dan; Goetzel, Ron Z; Anderson, David R

2016-01-01

The aim of the study was to evaluate the stock performance of publicly traded companies that received high scores on the HERO Employee Health Management Best Practices Scorecard in Collaboration with Mercer© based on their implementation of evidence-based workplace health promotion practices. A portfolio of companies that received high scores in a corporate health and wellness self-assessment was simulated based on past market performance and compared with past performance of companies represented on the Standard and Poor's (S&P) 500 Index. Stock values for a portfolio of companies that received high scores in a corporate health and wellness self-assessment appreciated by 235% compared with the S&P 500 Index appreciation of 159% over a 6-year simulation period. Robust investment in workforce health and well-being appears to be one of multiple practices pursued by high-performing, well-managed companies.

Reuse and recycle--considering the soil below constructions.

PubMed

Suer, Pascal; Wik, Ola; Erlandsson, Martin

2014-07-01

The European Construction Products Regulation provides a life cycle based framework for the environmental assessment of construction products. Harmonised European standards for the assessment of the release of dangerous substances and for declaration of environmental performance are in progress. Risk based limit values for the protection of soil and groundwater below construction works will still bet set nationally. In this paper we review the possibilities to expand the ongoing harmonisation to include risk assessment and life cycle assessment (LCA). Based on reviews of national European limit value models (LMVs) for assessment of release to soil and groundwater, two areas for harmonisation emerge: 1- The toxicological criteria. Toxicological endpoints to protect human health and environment are similar, and data from the same toxicological data sets are used to establish acceptance criteria. 2- The emission part of LMVs. We extracted six generic construction works for granular materials. These encompass the most common choices and span the different release scenarios applied. Harmonised emission models would also facilitate LCA and environmental product declaration (EPD). The immission or transport part of the LVMs is less promising for harmonisation. Locating the acceptance criteria point of compliance close to the construction works is advantageous from many aspects and would facilitate harmonisation of assessments. We have identified two different strategies to include recycling in the assessments: 1- Tiered procedure where assessment and declaration of performance are made for the intended primary use of the product only and renewed assessments are made whenever the construction works are demolished and the product is recovered. 2- Scenario based procedure where future recycling scenarios, into new products and construction works, are forecasted. In this case the initial assessment and declaration of environmental performance of a construction product is performed both for the intended primary use of the product and for the recycling scenarios. Copyright © 2014 Elsevier B.V. All rights reserved.

The Contribution of Rubrics to the Validity of Performance Assessment: A Study of the Conservation-Restoration and Design Undergraduate Degrees

ERIC Educational Resources Information Center

Menéndez-Varela, José-Luis; Gregori-Giralt, Eva

2016-01-01

Rubrics have attained considerable importance in the authentic and sustainable assessment paradigm; nevertheless, few studies have examined their contribution to validity, especially outside the domain of educational studies. This empirical study used a quantitative approach to analyse the validity of a rubrics-based performance assessment. Raters…

Moderation of Peer Assessment in Group Projects

ERIC Educational Resources Information Center

Bushell, Graeme

2006-01-01

It is shown here that a grade distribution scheme commonly used to moderate peer assessments where self assessment is excluded is based on a false premise and will give an erroneous ranking in the situation where the best performer in a student group ranks the second best performer much higher than the other group members. An alternative to…

Technology in Note Taking and Assessment: The Effects of Congruence on Student Performance

ERIC Educational Resources Information Center

Barrett, Matthew E.; Swan, Alexander B.; Mamikonian, Ani; Ghajoyan, Inna; Kramarova, Olga; Youmans, Robert J.

2014-01-01

This study examined the encoding specificity principle in relation to traditional and computer-based note taking and assessment formats in higher education. Students (N = 79) took lecture notes either by hand (n = 40) or by computer (n = 39) and then completed either a computer or a paper-based assessment. When note taking and assessment formats…

Integrated performance and reliability specification for digital avionics systems

NASA Technical Reports Server (NTRS)

Brehm, Eric W.; Goettge, Robert T.

1995-01-01

This paper describes an automated tool for performance and reliability assessment of digital avionics systems, called the Automated Design Tool Set (ADTS). ADTS is based on an integrated approach to design assessment that unifies traditional performance and reliability views of system designs, and that addresses interdependencies between performance and reliability behavior via exchange of parameters and result between mathematical models of each type. A multi-layer tool set architecture has been developed for ADTS that separates the concerns of system specification, model generation, and model solution. Performance and reliability models are generated automatically as a function of candidate system designs, and model results are expressed within the system specification. The layered approach helps deal with the inherent complexity of the design assessment process, and preserves long-term flexibility to accommodate a wide range of models and solution techniques within the tool set structure. ADTS research and development to date has focused on development of a language for specification of system designs as a basis for performance and reliability evaluation. A model generation and solution framework has also been developed for ADTS, that will ultimately encompass an integrated set of analytic and simulated based techniques for performance, reliability, and combined design assessment.

Performance Degradation Assessment of Rolling Element Bearings using Improved Fuzzy Entropy

NASA Astrophysics Data System (ADS)

Zhu, Keheng; Jiang, Xiaohui; Chen, Liang; Li, Haolin

2017-10-01

Rolling element bearings are an important unit in the rotating machines, and their performance degradation assessment is the basis of condition-based maintenance. Targeting the non-linear dynamic characteristics of faulty signals of rolling element bearings, a bearing performance degradation assessment approach based on improved fuzzy entropy (FuzzyEn) is proposed in this paper. FuzzyEn has less dependence on data length and achieves more freedom of parameter selection and more robustness to noise. However, it neglects the global trend of the signal when calculating similarity degree of two vectors, and thus cannot reflect the running state of the rolling element bearings accurately. Based on this consideration, the algorithm of FuzzyEn is improved in this paper and the improved FuzzyEn is utilized as an indicator for bearing performance degradation evaluation. The vibration data from run-to-failure test of rolling element bearings are used to validate the proposed method. The experimental results demonstrate that, compared with the traditional kurtosis and root mean square, the proposed method can detect the incipient fault in advance and can reflect the whole performance degradation process more clearly.

The Pediatrics Milestones Assessment Pilot: Development of Workplace-Based Assessment Content, Instruments, and Processes.

PubMed

Hicks, Patricia J; Margolis, Melissa; Poynter, Sue E; Chaffinch, Christa; Tenney-Soeiro, Rebecca; Turner, Teri L; Waggoner-Fountain, Linda; Lockridge, Robin; Clyman, Stephen G; Schwartz, Alan

2016-05-01

To report on the development of content and user feedback regarding the assessment process and utility of the workplace-based assessment instruments of the Pediatrics Milestones Assessment Pilot (PMAP). One multisource feedback instrument and two structured clinical observation instruments were developed and refined by experts in pediatrics and assessment to provide evidence for nine competencies based on the Pediatrics Milestones (PMs) and chosen to inform residency program faculty decisions about learners' readiness to serve as pediatric interns in the inpatient setting. During the 2012-2013 PMAP study, 18 U.S. pediatric residency programs enrolled interns and subinterns. Faculty, residents, nurses, and other observers used the instruments to assess learner performance through direct observation during a one-month rotation. At the end of the rotation, data were aggregated for each learner, milestone levels were assigned using a milestone classification form, and feedback was provided to learners. Learners and site leads were surveyed and/or interviewed about their experience as participants. Across the sites, 2,338 instruments assessing 239 learners were completed by 630 unique observers. Regarding end-of-rotation feedback, 93% of learners (128/137) agreed the assessments and feedback "helped me understand how those with whom I work perceive my performance," and 85% (117/137) agreed they were "useful for constructing future goals or identifying a developmental path." Site leads identified several benefits and challenges to the assessment process. PM-based instruments used in workplace-based assessment provide a meaningful and acceptable approach to collecting evidence of learner competency development. Learners valued feedback provided by PM-based assessment.

Using screen-based simulation to improve performance during pediatric resuscitation.

PubMed

Biese, Kevin J; Moro-Sutherland, Donna; Furberg, Robert D; Downing, Brian; Glickman, Larry; Murphy, Alison; Jackson, Cheryl L; Snyder, Graham; Hobgood, Cherri

2009-12-01

To assess the ability of a screen-based simulation-training program to improve emergency medicine and pediatric resident performance in critical pediatric resuscitation knowledge, confidence, and skills. A pre-post, interventional design was used. Three measures of performance were created and assessed before and after intervention: a written pre-course knowledge examination, a self-efficacy confidence score, and a skills-based high-fidelity simulation code scenario. For the high-fidelity skills assessment, independent physician raters recorded and reviewed subject performance. The intervention consisted of eight screen-based pediatric resuscitation scenarios that subjects had 4 weeks to complete. Upon completion of the scenarios, all three measures were repeated. For the confidence assessment, summary pre- and post-test summary confidence scores were compared using a t-test, and for the skills assessment, pre-scores were compared with post-test measures for each individual using McNemar's chi-square test for paired samples. Twenty-six of 35 (71.3%) enrolled subjects completed the institutional review board-approved study. Increases were observed in written test scores, confidence, and some critical interventions in high-fidelity simulation. The mean improvement in cumulative confidence scores for all residents was 10.1 (SD +/-4.9; range 0-19; p < 0.001), with no resident feeling less confident after the intervention. Although overall performance in simulated codes did not change significantly, with average scores of 6.65 (+/-1.76) to 7.04 (+/-1.37) out of 9 possible points (p = 0.58), improvement was seen in the administering of appropriate amounts of IV fluids (59-89%, p = 0.03). In this study, improvements in resident knowledge, confidence, and performance of certain skills in simulated pediatric cardiac arrest scenarios suggest that screen-based simulations may be an effective way to enhance resuscitation skills of pediatric providers. These results should be confirmed using a randomized design with an appropriate control group. (c) 2009 by the Society for Academic Emergency Medicine.

Measuring nursing competencies in the operating theatre: instrument development and psychometric analysis using Item Response Theory.

PubMed

Nicholson, Patricia; Griffin, Patrick; Gillis, Shelley; Wu, Margaret; Dunning, Trisha

2013-09-01

Concern about the process of identifying underlying competencies that contribute to effective nursing performance has been debated with a lack of consensus surrounding an approved measurement instrument for assessing clinical performance. Although a number of methodologies are noted in the development of competency-based assessment measures, these studies are not without criticism. The primary aim of the study was to develop and validate a Performance Based Scoring Rubric, which included both analytical and holistic scales. The aim included examining the validity and reliability of the rubric, which was designed to measure clinical competencies in the operating theatre. The fieldwork observations of 32 nurse educators and preceptors assessing the performance of 95 instrument nurses in the operating theatre were used in the calibration of the rubric. The Rasch model, a particular model among Item Response Models, was used in the calibration of each item in the rubric in an attempt at improving the measurement properties of the scale. This is done by establishing the 'fit' of the data to the conditions demanded by the Rasch model. Acceptable reliability estimates, specifically a high Cronbach's alpha reliability coefficient (0.940), as well as empirical support for construct and criterion validity for the rubric were achieved. Calibration of the Performance Based Scoring Rubric using Rasch model revealed that the fit statistics for most items were acceptable. The use of the Rasch model offers a number of features in developing and refining healthcare competency-based assessments, improving confidence in measuring clinical performance. The Rasch model was shown to be useful in developing and validating a competency-based assessment for measuring the competence of the instrument nurse in the operating theatre with implications for use in other areas of nursing practice. Crown Copyright © 2012. Published by Elsevier Ltd. All rights reserved.

Strengthening the afferent limb of rapid response systems: an educational intervention using web-based learning for early recognition and responding to deteriorating patients.

PubMed

Liaw, Sok Ying; Wong, Lai Fun; Ang, Sophia Bee Leng; Ho, Jasmine Tze Yin; Siau, Chiang; Ang, Emily Neo Kim

2016-06-01

The timely recognition and response to patients with clinical deteriorations constitute the afferent limb failure of a rapid response system (RRS). This area is a persistent problem in acute healthcare settings worldwide. In this study, we evaluated the effect of an educational programme on improving the nurses' knowledge and performances in recognising and responding to clinical deterioration. The interactive web-based programme addressed three areas: (1) early detection of changes in vital signs; (2) performance of nursing assessment and interventions using airway, breathing, circulation, disability and expose/examine and (3) reporting clinical deterioration using identity, situation, background, assessment and recommendation. Sixty-seven registered nurses participated in the randomised control study. The experimental group underwent a 3 h programme while the control group received no intervention. Pretests and post-tests, a mannequin-based assessment and a multiple-choice knowledge questionnaire were conducted. We evaluated the participants' performances in assessing, managing and reporting the deterioration of a patient using a validated performance tool. A significantly higher number of nurses from the experimental group than the control group monitored respiratory rates (48.2% vs 25%, p<0.05) and pulse rates (74.3% vs 37.5%, p<0.01) in the simulated environment, after the intervention. The post-test mean scores of the experimental group was significantly higher than the control group for knowledge (21.29 vs 18.28, p<0.001), performance in assessing and managing clinical deterioration (25.83 vs 19.50, p<0.001) and reporting clinical deterioration (12.83 vs 10.97, p<0.001). A web-based educational programme developed for hospital nurses to strengthen the afferent limb of the RRS significantly increased their knowledge and performances in assessing, managing and reporting clinical deterioration. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

Milestone-specific, Observed data points for evaluating levels of performance (MODEL) assessment strategy for anesthesiology residency programs.

PubMed

Nagy, Christopher J; Fitzgerald, Brian M; Kraus, Gregory P

2014-01-01

Anesthesiology residency programs will be expected to have Milestones-based evaluation systems in place by July 2014 as part of the Next Accreditation System. The San Antonio Uniformed Services Health Education Consortium (SAUSHEC) anesthesiology residency program developed and implemented a Milestones-based feedback and evaluation system a year ahead of schedule. It has been named the Milestone-specific, Observed Data points for Evaluating Levels of performance (MODEL) assessment strategy. The "MODEL Menu" and the "MODEL Blueprint" are tools that other anesthesiology residency programs can use in developing their own Milestones-based feedback and evaluation systems prior to ACGME-required implementation. Data from our early experience with the streamlined MODEL blueprint assessment strategy showed substantially improved faculty compliance with reporting requirements. The MODEL assessment strategy provides programs with a workable assessment method for residents, and important Milestones data points to programs for ACGME reporting.

Assessment without Testing: Using Performance Measures Embedded in a Technology-Based Instructional Program as Indicators of Reading Ability

ERIC Educational Resources Information Center

Mitchell, Alison; Baron, Lauren; Macaruso, Paul

2018-01-01

Screening and monitoring student reading progress can be costly and time consuming. Assessment embedded within the context of online instructional programs can capture ongoing student performance data while limiting testing time outside of instruction. This paper presents two studies that examined the validity of using performance measures from a…

Performance of bed-load transport equations relative to geomorphic significance: Predicting effective discharge and its transport rate

Treesearch

Jeffrey J. Barry; John M. Buffington; Peter Goodwin; John .G. King; William W. Emmett

2008-01-01

Previous studies assessing the accuracy of bed-load transport equations have considered equation performance statistically based on paired observations of measured and predicted bed-load transport rates. However, transport measurements were typically taken during low flows, biasing the assessment of equation performance toward low discharges, and because equation...

Assessing Scientific Performance.

ERIC Educational Resources Information Center

Weiner, John M.; And Others

1984-01-01

A method for assessing scientific performance based on relationships displayed numerically in published documents is proposed and illustrated using published documents in pediatric oncology for the period 1979-1982. Contributions of a major clinical investigations group, the Childrens Cancer Study Group, are analyzed. Twenty-nine references are…

Assessing healthcare quality using routine data: evaluating the performance of the national tuberculosis programme in South Africa.

PubMed

McLaren, Zoë M; Sharp, Alana R; Zhou, Jifang; Wasserman, Sean; Nanoo, Ananta

2017-02-01

To assess the performance of healthcare facilities by means of indicators based on guidelines for clinical care of TB, which is likely a good measure of overall facility quality. We assessed quality of care in all public health facilities in South Africa using graphical, correlation and locally weighted kernel regression analysis of routine TB test data. Facility performance falls short of national standards of care. Only 74% of patients with TB provided a second specimen for testing, 18% received follow-up testing and 14% received drug resistance testing. Only resistance testing rates improved over time, tripling between 2004 and 2011. National awareness campaigns and changes in clinical guidelines had only a transient impact on testing rates. The poorest performing facilities remained at the bottom of the rankings over the period of study. The optimal policy strategy requires both broad-based policies and targeted resources to poor performers. This approach to assessing facility quality of care can be adapted to other contexts and also provides a low-cost method for evaluating the effectiveness of proposed interventions. Devising targeted policies based on routine data is a cost-effective way to improve the quality of public health care provided. © 2016 John Wiley & Sons Ltd.

Task-Based Variability in Children's Singing Accuracy

ERIC Educational Resources Information Center

Nichols, Bryan E.

2013-01-01

The purpose of this study was to explore task-based variability in children's singing accuracy performance. The research questions were: Does children's singing accuracy vary based on the nature of the singing assessment employed? Is there a hierarchy of difficulty and discrimination ability among singing assessment tasks? What is the…

Cognitive skills assessment during robot-assisted surgery: separating the wheat from the chaff.

PubMed

Guru, Khurshid A; Esfahani, Ehsan T; Raza, Syed J; Bhat, Rohit; Wang, Katy; Hammond, Yana; Wilding, Gregory; Peabody, James O; Chowriappa, Ashirwad J

2015-01-01

To investigate the utility of cognitive assessment during robot-assisted surgery (RAS) to define skills in terms of cognitive engagement, mental workload, and mental state; while objectively differentiating between novice and expert surgeons. In all, 10 surgeons with varying operative experience were assigned to beginner (BG), combined competent and proficient (CPG), and expert (EG) groups based on the Dreyfus model. The participants performed tasks for basic, intermediate and advanced skills on the da Vinci Surgical System. Participant performance was assessed using both tool-based and cognitive metrics. Tool-based metrics showed significant differences between the BG vs CPG and the BG vs EG, in basic skills. While performing intermediate skills, there were significant differences only on the instrument-to-instrument collisions between the BG vs CPG (2.0 vs 0.2, P = 0.028), and the BG vs EG (2.0 vs 0.1, P = 0.018). There were no significant differences between the CPG and EG for both basic and intermediate skills. However, using cognitive metrics, there were significant differences between all groups for the basic and intermediate skills. In advanced skills, there were no significant differences between the CPG and the EG except time (1116 vs 599.6 s), using tool-based metrics. However, cognitive metrics revealed significant differences between both groups. Cognitive assessment of surgeons may aid in defining levels of expertise performing complex surgical tasks once competence is achieved. Cognitive assessment may be used as an adjunct to the traditional methods for skill assessment during RAS. © 2014 The Authors. BJU International © 2014 BJU International.

School-Based Functional Assessments for Children with Physical Disabilities in Grades K-12

ERIC Educational Resources Information Center

Johnson, Richard W.

2012-01-01

The purpose of this study was to develop three school-based assessments and determine the content validity for each assessment. The School Activities and Participation Analysis-Elementary (SAPA-E) measures functional movement performance in children with physical disability attending the elementary school, and the School Activities and…

Using a Sampling Strategy to Address Psychometric Challenges in Tutorial-Based Assessments

ERIC Educational Resources Information Center

Eva, Kevin W.; Solomon, Patty; Neville, Alan J.; Ladouceur, Michael; Kaufman, Karyn; Walsh, Allyn; Norman, Geoffrey R.

2007-01-01

Introduction: Tutorial-based assessment, despite providing a good match with the philosophy adopted by educational programmes that emphasize small group learning, remains one of the greatest challenges for educators working in this context. The current study was performed in an attempt to assess the psychometric characteristics of tutorial-based…

Virtual reality measures in neuropsychological assessment: a meta-analytic review.

PubMed

Neguț, Alexandra; Matu, Silviu-Andrei; Sava, Florin Alin; David, Daniel

2016-02-01

Virtual reality-based assessment is a new paradigm for neuropsychological evaluation, that might provide an ecological assessment, compared to paper-and-pencil or computerized neuropsychological assessment. Previous research has focused on the use of virtual reality in neuropsychological assessment, but no meta-analysis focused on the sensitivity of virtual reality-based measures of cognitive processes in measuring cognitive processes in various populations. We found eighteen studies that compared the cognitive performance between clinical and healthy controls on virtual reality measures. Based on a random effects model, the results indicated a large effect size in favor of healthy controls (g = .95). For executive functions, memory and visuospatial analysis, subgroup analysis revealed moderate to large effect sizes, with superior performance in the case of healthy controls. Participants' mean age, type of clinical condition, type of exploration within virtual reality environments, and the presence of distractors were significant moderators. Our findings support the sensitivity of virtual reality-based measures in detecting cognitive impairment. They highlight the possibility of using virtual reality measures for neuropsychological assessment in research applications, as well as in clinical practice.

Assessing physical functioning on pain management programmes: the unique contribution of directly assessed physical performance measures and their relationship to self-reports

PubMed Central

Guildford, Beth J; Jacobs, Clair M; Daly-Eichenhardt, Aisling; Scott, Whitney; McCracken, Lance M

2016-01-01

Physical functioning is a recommended outcome domain for pain management programmes. It can be assessed by self-report and by direct assessment of performance. Although physical performance measures may provide unique and useful information about patient functioning over and above self-report measures, it is not entirely clear which of the many possible performances to assess. This study investigated a battery of three directly assessed physical performance measures and their relationship to three currently used self-report measures of general health and functioning. The three performance measures were sensitive to treatment; patients performed significantly better on all three measures following completion of the pain management programme. The three performance measures were shown to represent a single underlying dimension, and there was a significant degree of overlap between them. The performance measures were shown to be relevant in explaining variation in the self-report measures, as well as to offer a clinically relevant different dimension of assessment to self-report. Future research could focus on developing performance-based measures that capture quality of movement and that are sensitive to relevant processes of therapeutic change. PMID:28386404

Predicting performance of junior doctors: Association of workplace based assessment with demographic characteristics, emotional intelligence, selection scores, and undergraduate academic performance.

PubMed

Carr, Sandra E; Celenza, Antonio; Mercer, Annette M; Lake, Fiona; Puddey, Ian B

2018-01-21

Predicting workplace performance of junior doctors from before entry or during medical school is difficult and has limited available evidence. This study explored the association between selected predictor variables and workplace based performance in junior doctors during their first postgraduate year. Two cohorts of medical students (n = 200) from one university in Western Australia participated in the longitudinal study. Pearson correlation coefficients and multivariate analyses utilizing linear regression were used to assess the relationships between performance on the Junior Doctor Assessment Tool (JDAT) and its sub-components with demographic characteristics, selection scores for medical school entry, emotional intelligence, and undergraduate academic performance. Grade Point Average (GPA) at the completion of undergraduate studies had the most significant association with better performance on the overall JDAT and each subscale. Increased age was a negative predictor for junior doctor performance on the Clinical management subscale and understanding emotion was a predictor for the JDAT Communication subscale. Secondary school performance measured by Tertiary Entry Rank on entry to medical school score predicted GPA but not junior doctor performance. The GPA as a composite measure of ability and performance in medical school is associated with junior doctor assessment scores. Using this variable to identify students at risk of difficulty could assist planning for appropriate supervision, support, and training for medical graduates transitioning to the workplace.

A web-based rapid assessment tool for production publishing solutions

NASA Astrophysics Data System (ADS)

Sun, Tong

2010-02-01

Solution assessment is a critical first-step in understanding and measuring the business process efficiency enabled by an integrated solution package. However, assessing the effectiveness of any solution is usually a very expensive and timeconsuming task which involves lots of domain knowledge, collecting and understanding the specific customer operational context, defining validation scenarios and estimating the expected performance and operational cost. This paper presents an intelligent web-based tool that can rapidly assess any given solution package for production publishing workflows via a simulation engine and create a report for various estimated performance metrics (e.g. throughput, turnaround time, resource utilization) and operational cost. By integrating the digital publishing workflow ontology and an activity based costing model with a Petri-net based workflow simulation engine, this web-based tool allows users to quickly evaluate any potential digital publishing solutions side-by-side within their desired operational contexts, and provides a low-cost and rapid assessment for organizations before committing any purchase. This tool also benefits the solution providers to shorten the sales cycles, establishing a trustworthy customer relationship and supplement the professional assessment services with a proven quantitative simulation and estimation technology.

Design definition study of a life/cruise fan technology V/STOL aircraft. Volume 2, addendum 2: Program risk assessment

NASA Technical Reports Server (NTRS)

1975-01-01

The results are presented of a risk assessment study conducted on two technology aircraft. The aircraft system components were reviewed and assessed for risk based on: (1) complexity relative to state-of-the-art, (2) manufacturing and qualification testing, (3) availability and delays, and (4) cost/schedule impact. These assessments were based on five risk nomenclatures: low, minor, moderate, high, and extreme. Each aircraft system was assigned an overall risk rating depending upon its contribution to the capability of the aircraft to achieve the performance goals. The slightly lower Sabreliner performance margin is due to the restricted flight envelope, the fixed landing gear, and internal fuel capacity. The Sabreliner with retractable gear and allowed to fly at its best speed and altitude would reflect performance margins similar to the New Airframe. These significant margins, inherent with the MCAIR three gas generator/three fan propulsion system, are major modifiers to risk assessment of both aircraft. The estimated risk and the associated key system and performance areas are tabulated.

Performance-based methodology for assessing seismic vulnerability and capacity of buildings

NASA Astrophysics Data System (ADS)

Shibin, Lin; Lili, Xie; Maosheng, Gong; Ming, Li

2010-06-01

This paper presents a performance-based methodology for the assessment of seismic vulnerability and capacity of buildings. The vulnerability assessment methodology is based on the HAZUS methodology and the improved capacitydemand-diagram method. The spectral displacement ( S d ) of performance points on a capacity curve is used to estimate the damage level of a building. The relationship between S d and peak ground acceleration (PGA) is established, and then a new vulnerability function is expressed in terms of PGA. Furthermore, the expected value of the seismic capacity index (SCev) is provided to estimate the seismic capacity of buildings based on the probability distribution of damage levels and the corresponding seismic capacity index. The results indicate that the proposed vulnerability methodology is able to assess seismic damage of a large number of building stock directly and quickly following an earthquake. The SCev provides an effective index to measure the seismic capacity of buildings and illustrate the relationship between the seismic capacity of buildings and seismic action. The estimated result is compared with damage surveys of the cities of Dujiangyan and Jiangyou in the M8.0 Wenchuan earthquake, revealing that the methodology is acceptable for seismic risk assessment and decision making. The primary reasons for discrepancies between the estimated results and the damage surveys are discussed.

Automated Portable Test System (APTS) - A performance envelope assessment tool

NASA Technical Reports Server (NTRS)

Kennedy, R. S.; Dunlap, W. P.; Jones, M. B.; Wilkes, R. L.; Bittner, A. C., Jr.

1985-01-01

The reliability and stability of microcomputer-based psychological tests are evaluated. The hardware, test programs, and system control of the Automated Portable Test System, which assesses human performance and subjective status, are described. Subjects were administered 11 pen-and-pencil and microcomputer-based tests for 10 sessions. The data reveal that nine of the 10 tests stabilized by the third administration; inertial correlations were high and consistent. It is noted that the microcomputer-based tests display good psychometric properties in terms of differential stability and reliability.

Application of Athletic Movement Tests that Predict Injury Risk in a Military Population: Development of Normative Data.

PubMed

Teyhen, Deydre S; Shaffer, Scott W; Butler, Robert J; Goffar, Stephen L; Kiesel, Kyle B; Rhon, Daniel I; Boyles, Robert E; McMillian, Daniel J; Williamson, Jared N; Plisky, Phillip J

2016-10-01

Performance on movement tests helps to predict injury risk in a variety of physically active populations. Understanding baseline measures for normal is an important first step. Determine differences in physical performance assessments and describe normative values for these tests based on military unit type. Assessment of power, balance, mobility, motor control, and performance on the Army Physical Fitness Test were assessed in a cohort of 1,466 soldiers. Analysis of variance was performed to compare the results based on military unit type (Rangers, Combat, Combat Service, and Combat Service Support) and analysis of covariance was performed to determine the influence of age and gender. Rangers performed the best on all performance and fitness measures (p < 0.05). Combat soldiers performed better than Combat Service and Service Support soldiers on several physical performance tests and the Army Physical Fitness Test (p < 0.05). Performance in Combat Service and Service Support soldiers was equivalent on most measures (p < 0.05). Functional performance and level of fitness varied significantly by military unit type. Understanding these differences will provide a foundation for future injury prediction and prevention strategies. Reprint & Copyright © 2016 Association of Military Surgeons of the U.S.

Effectiveness evaluation of objective and subjective weighting methods for aquifer vulnerability assessment in urban context

NASA Astrophysics Data System (ADS)

Sahoo, Madhumita; Sahoo, Satiprasad; Dhar, Anirban; Pradhan, Biswajeet

2016-10-01

Groundwater vulnerability assessment has been an accepted practice to identify the zones with relatively increased potential for groundwater contamination. DRASTIC is the most popular secondary information-based vulnerability assessment approach. Original DRASTIC approach considers relative importance of features/sub-features based on subjective weighting/rating values. However variability of features at a smaller scale is not reflected in this subjective vulnerability assessment process. In contrast to the subjective approach, the objective weighting-based methods provide flexibility in weight assignment depending on the variation of the local system. However experts' opinion is not directly considered in the objective weighting-based methods. Thus effectiveness of both subjective and objective weighting-based approaches needs to be evaluated. In the present study, three methods - Entropy information method (E-DRASTIC), Fuzzy pattern recognition method (F-DRASTIC) and Single parameter sensitivity analysis (SA-DRASTIC), were used to modify the weights of the original DRASTIC features to include local variability. Moreover, a grey incidence analysis was used to evaluate the relative performance of subjective (DRASTIC and SA-DRASTIC) and objective (E-DRASTIC and F-DRASTIC) weighting-based methods. The performance of the developed methodology was tested in an urban area of Kanpur City, India. Relative performance of the subjective and objective methods varies with the choice of water quality parameters. This methodology can be applied without/with suitable modification. These evaluations establish the potential applicability of the methodology for general vulnerability assessment in urban context.

Body surface assessment with 3D laser-based anthropometry: reliability, validation, and improvement of empirical surface formulae.

PubMed

Kuehnapfel, Andreas; Ahnert, Peter; Loeffler, Markus; Scholz, Markus

2017-02-01

Body surface area is a physiological quantity relevant for many medical applications. In clinical practice, it is determined by empirical formulae. 3D laser-based anthropometry provides an easy and effective way to measure body surface area but is not ubiquitously available. We used data from laser-based anthropometry from a population-based study to assess validity of published and commonly used empirical formulae. We performed a large population-based study on adults collecting classical anthropometric measurements and 3D body surface assessments (N = 1435). We determined reliability of the 3D body surface assessment and validity of 18 different empirical formulae proposed in the literature. The performance of these formulae is studied in subsets of sex and BMI. Finally, improvements of parameter settings of formulae and adjustments for sex and BMI were considered. 3D body surface measurements show excellent intra- and inter-rater reliability of 0.998 (overall concordance correlation coefficient, OCCC was used as measure of agreement). Empirical formulae of Fujimoto and Watanabe, Shuter and Aslani and Sendroy and Cecchini performed best with excellent concordance with OCCC > 0.949 even in subgroups of sex and BMI. Re-parametrization of formulae and adjustment for sex and BMI slightly improved results. In adults, 3D laser-based body surface assessment is a reliable alternative to estimation by empirical formulae. However, there are empirical formulae showing excellent results even in subgroups of sex and BMI with only little room for improvement.

Seating Arrangement, Group Composition and Competition-driven Interaction: Effects on Students' Performance in Physics

DOE Office of Scientific and Technical Information (OSTI.GOV)

Roxas, R. M.; Monterola, C.; Carreon-Monterola, S. L.

2010-07-28

We probe the effect of seating arrangement, group composition and group-based competition on students' performance in Physics using a teaching technique adopted from Mazur's peer instruction method. Ninety eight lectures, involving 2339 students, were conducted across nine learning institutions from February 2006 to June 2009. All the lectures were interspersed with student interaction opportunities (SIO), in which students work in groups to discuss and answer concept tests. Two individual assessments were administered before and after the SIO. The ratio of the post-assessment score to the pre-assessment score and the Hake factor were calculated to establish the improvement in student performance.more » Using actual assessment results and neural network (NN) modeling, an optimal seating arrangement for a class was determined based on student seating location. The NN model also provided a quantifiable method for sectioning students. Lastly, the study revealed that competition-driven interactions increase within-group cooperation and lead to higher improvement on the students' performance.« less

Using the International Classification of Functioning, Disability and Health in Assessment and Intervention of School-Aged Children With Language Impairments.

PubMed

Westby, Carol; Washington, Karla N

2017-07-26

The aim of this tutorial is to support speech-language pathologists' (SLPs') application of the International Classification of Functioning, Disability and Health (ICF) in assessment and treatment practices with children with language impairment. This tutorial reviews the framework of the ICF, describes the implications of the ICF for SLPs, distinguishes between students' capacity to perform a skill in a structured context and the actual performance of that skill in naturalistic contexts, and provides a case study of an elementary school child to demonstrate how the principles of the ICF can guide assessment and intervention. The Scope of Practice and Preferred Practice documents for the American Speech-Language-Hearing Association identify the ICF as the framework for practice in speech-language pathology. This tutorial will facilitate clinicians' ability to identify personal and environmental factors that influence students' skill capacity and skill performance, assess students' capacity and performance, and develop impairment-based and socially based language goals linked to Common Core State Standards that build students' language capacity and their communicative performance in naturalistic contexts.

Expectations, Observations, and the Cognitive Processes That Bind Them: Expert Assessment of Examinee Performance

ERIC Educational Resources Information Center

St-Onge, Christina; Chamberland, Martine; Lévesque, Annie; Varpio, Lara

2016-01-01

Performance-based assessment (PBA) is a valued assessment approach in medical education, be it in a clerkship, residency, or practice context. Raters are intrinsic to PBA and the increased use of PBA has lead to an increased interest in rater cognition. Although several researchers have tackled factors that may influence the variability in rater…

A Prototype Public Speaking Skills Assessment: An Evaluation of Human-Scoring Quality. Research Report. ETS RR-15-36

ERIC Educational Resources Information Center

Joe, Jilliam; Kitchen, Christopher; Chen, Lei; Feng, Gary

2015-01-01

The purpose of this paper is to summarize the evaluation of human-scoring quality for an assessment of public speaking skills. Videotaped performances given by 17 speakers on 4 tasks were scored by expert and nonexpert raters who had extensive experience scoring performance-based and constructed-response assessments. The Public Speaking Competence…

Creative group performances to assess core competencies in a first-year patient-centered medicine course

PubMed Central

Terregino, Carol A.; Saks, Norma S.

2010-01-01

Introduction A novel assessment of systems-based practice and practice-based learning and improvement learning objectives, implemented in a first-year patient-centered medicine course, is qualitatively described. Methods Student learning communities were asked to creatively demonstrate a problem and solution for health care delivery. Skits, filmed performances, plays, and documentaries were chosen by the students. Video recordings were reviewed for themes and the presence of course competencies. Results All performances demonstrated not only the index competencies of team work and facilitation of the learning of others, but many other core objectives of the course. The assignment was rated positively both by the faculty and the students, and has been added to the assessment modalities of the course. PMID:20174597

Creative group performances to assess core competencies in a first-year patient-centered medicine course.

PubMed

Terregino, Carol A; Saks, Norma S

2010-02-15

A novel assessment of systems-based practice and practice-based learning and improvement learning objectives, implemented in a first-year patient-centered medicine course, is qualitatively described. Student learning communities were asked to creatively demonstrate a problem and solution for health care delivery. Skits, filmed performances, plays, and documentaries were chosen by the students. Video recordings were reviewed for themes and the presence of course competencies. All performances demonstrated not only the index competencies of team work and facilitation of the learning of others, but many other core objectives of the course. The assignment was rated positively both by the faculty and the students, and has been added to the assessment modalities of the course.

Does the MCAT predict medical school and PGY-1 performance?

PubMed

Saguil, Aaron; Dong, Ting; Gingerich, Robert J; Swygert, Kimberly; LaRochelle, Jeffrey S; Artino, Anthony R; Cruess, David F; Durning, Steven J

2015-04-01

The Medical College Admissions Test (MCAT) is a high-stakes test required for entry to most U. S. medical schools; admissions committees use this test to predict future accomplishment. Although there is evidence that the MCAT predicts success on multiple choice-based assessments, there is little information on whether the MCAT predicts clinical-based assessments of undergraduate and graduate medical education performance. This study looked at associations between the MCAT and medical school grade point average (GPA), Medical Licensing Examination (USMLE) scores, observed patient care encounters, and residency performance assessments. This study used data collected as part of the Long-Term Career Outcome Study to determine associations between MCAT scores, USMLE Step 1, Step 2 clinical knowledge and clinical skill, and Step 3 scores, Objective Structured Clinical Examination performance, medical school GPA, and PGY-1 program director (PD) assessment of physician performance for students graduating 2010 and 2011. MCAT data were available for all students, and the PGY PD evaluation response rate was 86.2% (N = 340). All permutations of MCAT scores (first, last, highest, average) were weakly associated with GPA, Step 2 clinical knowledge scores, and Step 3 scores. MCAT scores were weakly to moderately associated with Step 1 scores. MCAT scores were not significantly associated with Step 2 clinical skills Integrated Clinical Encounter and Communication and Interpersonal Skills subscores, Objective Structured Clinical Examination performance or PGY-1 PD evaluations. MCAT scores were weakly to moderately associated with assessments that rely on multiple choice testing. The association is somewhat stronger for assessments occurring earlier in medical school, such as USMLE Step 1. The MCAT was not able to predict assessments relying on direct clinical observation, nor was it able to predict PD assessment of PGY-1 performance. Reprint & Copyright © 2015 Association of Military Surgeons of the U.S.

Assessment as Learning

ERIC Educational Resources Information Center

Gupta, Kavita

2016-01-01

In this article Kavita Gupta describes how she used assessment "as" learning, where students learn, self-correct, and collaborate "during" the assessment (Rowe 2012) in her AP chemistry class. In her class, she used these assessments as learning for "performance-based assessments" (PBAs). They share attributes of both…

A web-based micro-simulation program for self-learning BLS skills and the use of an AED. Can laypeople train themselves without a manikin?

PubMed

de Vries, Wiebe; Handley, Anthony J

2007-12-01

Various methods, including self-instruction, have been used to try to improve the acquisition of basic life support skills. This is a preliminary report of the effectiveness of a web-based self-training program for BLS and the use of an AED. Sixteen volunteers completed on-line training in their own time over a period of 8 weeks. The program included theory, scenario training and self-testing, but without practice on a manikin, or any instructor input. The volunteers were assessed, without prior warning, in a scenario setting. A recording manikin, expert assessors and video recording were used with a modified version of the Cardiff Test. All 16 volunteers performed the assessed skills in the use of an AED correctly. Most of the skills of BLS assessed were performed well. Chest compression depth and rate were performed less well (59% and 67% of participants, respectively, performed correctly). Opening the airway and lung inflation were performed poorly (38% and 13% of participants performed correctly), as was checking for safety (19% participants performed correctly). There was no significant correlation between the time a participant spent on-line and the quality of performance. Only 5 of the volunteers had ever attended a BLS course or used a resuscitation manikin before the assessment; their performance scores were not significantly better than those of the other 11 volunteers. These results suggest that it may be possible to train people in BLS and AED skills using a micro-simulation web-based interactive program but without any practice on a manikin. This seems to be particularly the case for the use of an AED, where performance achieved a uniformly high standard.

Language disturbance and functioning in first episode psychosis.

PubMed

Roche, Eric; Segurado, Ricardo; Renwick, Laoise; McClenaghan, Aisling; Sexton, Sarah; Frawley, Timothy; Chan, Carol K; Bonar, Maurice; Clarke, Mary

2016-01-30

Language disturbance has a central role in the presentation of psychotic disorders however its relationship with functioning requires further clarification, particularly in first episode psychosis (FEP). Both language disturbance and functioning can be evaluated with clinician-rated and performance-based measures. We aimed to investigate the concurrent association between clinician-rated and performance-based measures of language disturbance and functioning in FEP. We assessed 108 individuals presenting to an Early Intervention in Psychosis Service in Ireland. Formal thought disorder (FTD) dimensions and bizarre idiosyncratic thinking (BIT) were rated with structured assessment tools. Functioning was evaluated with a performance-based instrument, a clinician-rated measure and indicators of real-world functioning. The disorganisation dimension of FTD was significantly associated with clinician-rated measures of occupational and social functioning (Beta=-0.19, P<0.05 and Beta=-0.31, P<0.01, respectively). BIT was significantly associated with the performance-based measure of functioning (Beta=-0.22, P<0.05). Language disturbance was of less value in predicting real-world measures of functioning. Clinician-rated and performance-based assessments of language disturbance are complementary and each has differential associations with functioning. Communication disorders should be considered as a potential target for intervention in FEP, although further evaluation of the longitudinal relationship between language disturbance and functioning should be undertaken. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

Plans for Assessing Contractor Performance for the Camp Lemonnier Base Operations Support Contract Needed Improvement

DTIC Science & Technology

2015-08-27

ensure the functional assessment plans ( FAP ) for the three services contained all contractor work requiring assessment, measurable performance...review the FAPs after development and revisions to the FAPs , and did not require additional controls over the contractor assessment process in an...revised as follows: • Business Management System B-14.3 was updated to require the contract officer representative conduct a FAP review to ensure

Effects of gross motor function and manual function levels on performance-based ADL motor skills of children with spastic cerebral palsy.

PubMed

Park, Myoung-Ok

2017-02-01

[Purpose] The purpose of this study was to determine effects of Gross Motor Function Classification System and Manual Ability Classification System levels on performance-based motor skills of children with spastic cerebral palsy. [Subjects and Methods] Twenty-three children with cerebral palsy were included. The Assessment of Motor and Process Skills was used to evaluate performance-based motor skills in daily life. Gross motor function was assessed using Gross Motor Function Classification Systems, and manual function was measured using the Manual Ability Classification System. [Results] Motor skills in daily activities were significantly different on Gross Motor Function Classification System level and Manual Ability Classification System level. According to the results of multiple regression analysis, children categorized as Gross Motor Function Classification System level III scored lower in terms of performance based motor skills than Gross Motor Function Classification System level I children. Also, when analyzed with respect to Manual Ability Classification System level, level II was lower than level I, and level III was lower than level II in terms of performance based motor skills. [Conclusion] The results of this study indicate that performance-based motor skills differ among children categorized based on Gross Motor Function Classification System and Manual Ability Classification System levels of cerebral palsy.

ASSESSMENT OF TWO PHYSICALLY BASED WATERSHED MODELS BASED ON THEIR PERFORMANCES OF SIMULATING SEDIMENT MOVEMENT OVER SMALL WATERSHEDS

EPA Science Inventory

Abstract: Two physically based and deterministic models, CASC2-D and KINEROS are evaluated and compared for their performances on modeling sediment movement on a small agricultural watershed over several events. Each model has different conceptualization of a watershed. CASC...

ASSESSMENT OF TWO PHYSICALLY-BASED WATERSHED MODELS BASED ON THEIR PERFORMANCES OF SIMULATING WATER AND SEDIMENT MOVEMENT

EPA Science Inventory

Two physically based watershed models, GSSHA and KINEROS-2 are evaluated and compared for their performances on modeling flow and sediment movement. Each model has a different watershed conceptualization. GSSHA divides the watershed into cells, and flow and sediments are routed t...

Performance Based Teacher Education. What is the State of the Art?

ERIC Educational Resources Information Center

Elam, Stanley

This document clarifies the concepts of Performance Based Teacher Education, examines their potential, and identifies problems and questions. Five essential elements are identified: 1) teaching competencies to be demonstrated are role-derived, specified in behavioral terms, and made public; 2) assessment criteria are competency-based, specify…

Assessing the Relative Performance of Nurses Using Data Envelopment Analysis Matrix (DEAM).

PubMed

Vafaee Najar, Ali; Pooya, Alireza; Alizadeh Zoeram, Ali; Emrouznejad, Ali

2018-05-31

Assessing employee performance is one of the most important issue in healthcare management services. Because of their direct relationship with patients, nurses are also the most influential hospital staff who play a vital role in providing healthcare services. In this paper, a novel Data Envelopment Analysis Matrix (DEAM) approach is proposed for assessing the performance of nurses based on relative efficiency. The proposed model consists of five input variables (including type of employment, work experience, training hours, working hours and overtime hours) and eight output variables (the outputs are amount of hours each nurse spend on each of the eight activities including documentation, medical instructions, wound care and patient drainage, laboratory sampling, assessment and control care, follow-up and counseling and para-clinical measures, attendance during visiting and discharge suction) have been tested on 30 nurses from the heart department of a hospital in Iran. After determining the relative efficiency of each nurse based on the DEA model, the nurses' performance were evaluated in a DEAM format. As results the nurses were divided into four groups; superstars, potential stars, those who are needed to be trained effectively and question marks. Finally, based on the proposed approach, we have drawn some recommendations to policy makers in order to improve and maintain the performance of each of these groups. The proposed approach provides a practical framework for hospital managers so that they can assess the relative efficiency of nurses, plan and take steps to improve the quality of healthcare delivery.

A Cultured Learning Environment: Implementing a Problem- and Service-Based Microbiology Capstone Course to Assess Process- and Skill-Based Learning Objectives

ERIC Educational Resources Information Center

Watson, Rachel M.; Willford, John D.; Pfeifer, Mariel A.

2018-01-01

In this study, a problem-based capstone course was designed to assess the University of Wyoming Microbiology Program's skill-based and process-based student learning objectives. Students partnered with a local farm, a community garden, and a free downtown clinic in order to conceptualize, propose, perform, and present studies addressing problems…

Multi-criteria objective based climate change impact assessment for multi-purpose multi-reservoir systems

NASA Astrophysics Data System (ADS)

Müller, Ruben; Schütze, Niels

2014-05-01

Water resources systems with reservoirs are expected to be sensitive to climate change. Assessment studies that analyze the impact of climate change on the performance of reservoirs can be divided in two groups: (1) Studies that simulate the operation under projected inflows with the current set of operational rules. Due to non adapted operational rules the future performance of these reservoirs can be underestimated and the impact overestimated. (2) Studies that optimize the operational rules for best adaption of the system to the projected conditions before the assessment of the impact. The latter allows for estimating more realistically future performance and adaption strategies based on new operation rules are available if required. Multi-purpose reservoirs serve various, often conflicting functions. If all functions cannot be served simultaneously at a maximum level, an effective compromise between multiple objectives of the reservoir operation has to be provided. Yet under climate change the historically preferenced compromise may no longer be the most suitable compromise in the future. Therefore a multi-objective based climate change impact assessment approach for multi-purpose multi-reservoir systems is proposed in the study. Projected inflows are provided in a first step using a physically based rainfall-runoff model. In a second step, a time series model is applied to generate long-term inflow time series. Finally, the long-term inflow series are used as driving variables for a simulation-based multi-objective optimization of the reservoir system in order to derive optimal operation rules. As a result, the adapted Pareto-optimal set of diverse best compromise solutions can be presented to the decision maker in order to assist him in assessing climate change adaption measures with respect to the future performance of the multi-purpose reservoir system. The approach is tested on a multi-purpose multi-reservoir system in a mountainous catchment in Germany. A climate change assessment is performed for climate change scenarios based on the SRES emission scenarios A1B, B1 and A2 for a set of statistically downscaled meteorological data. The future performance of the multi-purpose multi-reservoir system is quantified and possible intensifications of trade-offs between management goals or reservoir utilizations are shown.

Assessing Preservice Teachers' Presentation Capabilities: Contrasting the Modes of Communication with the Constructed Impression

ERIC Educational Resources Information Center

Bower, Matt G.; Moloney, Robyn A.; Cavanagh, Michael S.; Sweller, Naomi

2013-01-01

A research-based understanding of how to develop and assess classroom presentation skills is vital for the effective development of pre-service teacher communication capabilities. This paper identifies and compares two different models of assessing pre-service teachers' presentation performance--one based on the Modes of Communication (voice,…

Installation Boundary Fence Replacement: Upland Areas: Environmental Assessment

DTIC Science & Technology

2004-05-27

Moody Air Force Base, GA Environmental Assessment Installation Boundary Fence Replacement INSTALLATION BOUNDARY...347 CES/CEV),3485 Georgia Street,Moody AFB, GA ,31699 8. PERFORMING ORGANIZATION REPORT NUMBER 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES...Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18 Moody Air Force Base, GA Environmental Assessment

Guide to Mathematics Released Items: Understanding Scoring. 2015

ERIC Educational Resources Information Center

Partnership for Assessment of Readiness for College and Careers, 2015

2015-01-01

The 2014-2015 administrations of the PARCC assessment included two separate test administration windows: the Performance-Based Assessment (PBA) and the End-of-Year (EOY), both of which were administered in paper-based and computer-based formats. The first window was for administration of the PBA, and the second window was for the administration of…

A Litmus Test for Performance Assessment.

ERIC Educational Resources Information Center

Finson, Kevin D.; Beaver, John B.

1992-01-01

Presents 10 guidelines for developing performance-based assessment items. Presents a sample activity developed from the guidelines. The activity tests students ability to observe, classify, and infer, using red and blue litmus paper, a pH-range finder, vinegar, ammonia, an unknown solution, distilled water, and paper towels. (PR)

NASA Earned Value Management (EVM) Update

NASA Technical Reports Server (NTRS)

Kerby, Jerald

2013-01-01

Earned Value Management (EVM) is an integrated management control system for assessing, understanding and qualifying what a project is achieving with the resoures. EVM integrates technical cost and schedules with risk management. It allows objective assessment and quantification of current project performance, and helps predict future performance-based trents.

Performance metrics for the assessment of satellite data products: an ocean color case study

EPA Science Inventory

Performance assessment of ocean color satellite data has generally relied on statistical metrics chosen for their common usage and the rationale for selecting certain metrics is infrequently explained. Commonly reported statistics based on mean squared errors, such as the coeffic...

ASSESSING UST CORRECTIVE ACTION TECHNOLOGIES: DIAGNOSTIC EVALUATION OF IN SITU SVE-BASED SYSTEM PERFORMANCE

EPA Science Inventory

In situ corrective action technologies are being proposed and installed at an increasing number of underground storage tank (LIST) sites contaminated with petroleum products in saturated and unsaturated zones. It is often difficult to accurately assess the performance of these sy...

Simulation-based Assessment of the Management of Critical Events by Board-certified Anesthesiologists.

PubMed

Weinger, Matthew B; Banerjee, Arna; Burden, Amanda R; McIvor, William R; Boulet, John; Cooper, Jeffrey B; Steadman, Randolph; Shotwell, Matthew S; Slagle, Jason M; DeMaria, Samuel; Torsher, Laurence; Sinz, Elizabeth; Levine, Adam I; Rask, John; Davis, Fred; Park, Christine; Gaba, David M

2017-09-01

We sought to determine whether mannequin-based simulation can reliably characterize how board-certified anesthesiologists manage simulated medical emergencies. Our primary focus was to identify gaps in performance and to establish psychometric properties of the assessment methods. A total of 263 consenting board-certified anesthesiologists participating in existing simulation-based maintenance of certification courses at one of eight simulation centers were video recorded performing simulated emergency scenarios. Each participated in two 20-min, standardized, high-fidelity simulated medical crisis scenarios, once each as primary anesthesiologist and first responder. Via a Delphi technique, an independent panel of expert anesthesiologists identified critical performance elements for each scenario. Trained, blinded anesthesiologists rated video recordings using standardized rating tools. Measures included the percentage of critical performance elements observed and holistic (one to nine ordinal scale) ratings of participant's technical and nontechnical performance. Raters also judged whether the performance was at a level expected of a board-certified anesthesiologist. Rater reliability for most measures was good. In 284 simulated emergencies, participants were rated as successfully completing 81% (interquartile range, 75 to 90%) of the critical performance elements. The median rating of both technical and nontechnical holistic performance was five, distributed across the nine-point scale. Approximately one-quarter of participants received low holistic ratings (i.e., three or less). Higher-rated performances were associated with younger age but not with previous simulation experience or other individual characteristics. Calling for help was associated with better individual and team performance. Standardized simulation-based assessment identified performance gaps informing opportunities for improvement. If a substantial proportion of experienced anesthesiologists struggle with managing medical emergencies, continuing medical education activities should be reevaluated.

The novel language-systematic aphasia screening SAPS: screening-based therapy in combination with computerised home training.

PubMed

Krzok, Franziska; Rieger, Verena; Niemann, Katharina; Nobis-Bosch, Ruth; Radermacher, Irmgard; Huber, Walter; Willmes, Klaus; Abel, Stefanie

2018-03-01

SAPS-'Sprachsystematisches Aphasiescreening'-is a novel language-systematic aphasia screening developed for the German language, which already had been positively evaluated. It offers a fast assessment of modality-specific psycholinguistic components at different levels of complexity and the derivation of impairment-based treatment foci from the individual performance profile. However, SAPS has not yet been evaluated in combination with the new SAPS-based treatment. To replicate the practicality of SAPS and to investigate the effectiveness of a SAPS-based face-to-face therapy combined with computerised home training in a feasibility study. To examine the soundness of the treatment design, to determine treatment-induced changes in patient performance as measured by SAPS, to assess parallel changes in communicative abilities, and to differentiate therapy effects achieved by face-to-face therapy versus add-on effects achieved by later home training. Sixteen participants with post-stroke aphasia (PWAs) were included into the study. They were administered the SAPS and communicative testing before and after the treatment regimen. Each PWA received one therapy session followed by home training per day, with the individual treatment foci being determined according to initial SAPS profile, and duration of treatment and possible change of focus dependent on performance assessed by continuous therapy monitoring. The combination of therapy and home training based on the SAPS was effective for all participants. We showed significant improvements for impairment-based SAPS performance and, with high inter-individual variability, in everyday communication. These two main targets of speech and language therapy were correlated and SAPS improvements after therapy were significantly higher than after home training. SAPS offers the assessment of an individual performance profile in order to derive sufficiently diversified, well-founded and specific treatment foci and to follow up changes in performance. The appending treatment regimen has shown to be effective for our participants. Thus, the study revealed feasibility of our approach. © 2017 Royal College of Speech and Language Therapists.

Identifying content for simulation-based curricula in urology: a national needs assessment.

PubMed

Nayahangan, Leizl Joy; Bølling Hansen, Rikke; Gilboe Lindorff-Larsen, Karen; Paltved, Charlotte; Nielsen, Bjørn Ulrik; Konge, Lars

2017-12-01

Simulation-based training is well recognized in the transforming field of urological surgery; however, integration into the curriculum is often unstructured. Development of simulation-based curricula should follow a stepwise approach starting with a needs assessment. This study aimed to identify technical procedures in urology that should be included in a simulation-based curriculum for residency training. A national needs assessment was performed using the Delphi method involving 56 experts with significant roles in the education of urologists. Round 1 identified technical procedures that newly qualified urologists should perform. Round 2 included a survey using an established needs assessment formula to explore: the frequency of procedures; the number of physicians who should be able to perform the procedure; the risk and/or discomfort to patients when a procedure is performed by an inexperienced physician; and the feasibility of simulation training. Round 3 involved elimination and reranking of procedures according to priority. The response rates for the three Delphi rounds were 70%, 55% and 67%, respectively. The 34 procedures identified in Round 1 were reduced to a final prioritized list of 18 technical procedures for simulation-based training. The five procedures that reached the highest prioritization were cystoscopy, transrectal ultrasound-guided biopsy of the prostate, placement of ureteral stent, insertion of urethral and suprapubic catheter, and transurethral resection of the bladder. The prioritized list of technical procedures in urology that were identified as highly suitable for simulation can be used as an aid in the planning and development of simulation-based training programs.

KSC management training system project

NASA Technical Reports Server (NTRS)

Sepulveda, Jose A.

1993-01-01

The stated objectives for the summer of 1993 were: to review the Individual Development Plan Surveys for 1994 in order to automate the analysis of the Needs Assessment effort; and to develop and implement evaluation methodologies to perform ongoing program-wide course-to-course assessment. This includes the following: to propose a methodology to develop and implement objective, performance-based assessment instruments for each training effort; to mechanize course evaluation forms and develop software to facilitate the data gathering, analysis, and reporting processes; and to implement the methodology, forms, and software in at lease one training course or seminar selected among those normally offered in the summer at KSC. Section two of this report addresses the work done in regard to the Individual Development Plan Surveys for 1994. Section three presents the methodology proposed to develop and implement objective, performance-based assessment instruments for each training course offered at KSC.

Assessing students' conceptual knowledge of electricity and magnetism

NASA Astrophysics Data System (ADS)

McColgan, Michele W.; Finn, Rose A.; Broder, Darren L.; Hassel, George E.

2017-12-01

We present the Electricity and Magnetism Conceptual Assessment (EMCA), a new assessment aligned with second-semester introductory physics courses. Topics covered include electrostatics, electric fields, circuits, magnetism, and induction. We have two motives for writing a new assessment. First, we find other assessments such as the Brief Electricity and Magnetism Assessment and the Conceptual Survey on Electricity and Magnetism not well aligned with the topics and content depth of our courses. We want to test introductory physics content at a level appropriate for our students. Second, we want the assessment to yield scores and gains comparable to the widely used Force Concept Inventory (FCI). After five testing and revision cycles, the assessment was finalized in early 2015 and is available online. We present performance results for a cohort of 225 students at Siena College who were enrolled in our algebra- and calculus-based physics courses during the spring 2015 and 2016 semesters. We provide pretest, post-test, and gain analyses, as well as individual question and whole test statistics to quantify difficulty and reliability. In addition, we compare EMCA and FCI scores and gains, and we find that students' FCI scores are strongly correlated with their performance on the EMCA. Finally, the assessment was piloted in an algebra-based physics course at George Washington University (GWU). We present performance results for a cohort of 130 GWU students and we find that their EMCA scores are comparable to the scores of students in our calculus-based physics course.

The Validation of a Case-Based, Cumulative Assessment and Progressions Examination

PubMed Central

Coker, Adeola O.; Copeland, Jeffrey T.; Gottlieb, Helmut B.; Horlen, Cheryl; Smith, Helen E.; Urteaga, Elizabeth M.; Ramsinghani, Sushma; Zertuche, Alejandra; Maize, David

2016-01-01

Objective. To assess content and criterion validity, as well as reliability of an internally developed, case-based, cumulative, high-stakes third-year Annual Student Assessment and Progression Examination (P3 ASAP Exam). Methods. Content validity was assessed through the writing-reviewing process. Criterion validity was assessed by comparing student scores on the P3 ASAP Exam with the nationally validated Pharmacy Curriculum Outcomes Assessment (PCOA). Reliability was assessed with psychometric analysis comparing student performance over four years. Results. The P3 ASAP Exam showed content validity through representation of didactic courses and professional outcomes. Similar scores on the P3 ASAP Exam and PCOA with Pearson correlation coefficient established criterion validity. Consistent student performance using Kuder-Richardson coefficient (KR-20) since 2012 reflected reliability of the examination. Conclusion. Pharmacy schools can implement internally developed, high-stakes, cumulative progression examinations that are valid and reliable using a robust writing-reviewing process and psychometric analyses. PMID:26941435

Ecological assessment of the environmental impacts of the kerosene burning in jet turbines and its improvement assessment.

PubMed

Geldermann, J; Gabriel, R; Rentz, O

1999-01-01

The burning of kerosene in jet turbines is investigated for two reference flights with a Boeing 747-400 and an Airbus A320-200, representing the typical Lufthansa planes for long and middle distance. The ecological evaluation is performed by Life Cycle Assessment (LCA). Formation of condensation trails, which is a specific environmental impact caused by air traffic, has to be considered in addition to established LCA impact categories. Based on the ecological assessment, an improvement assessment is performed. Environmental performance of diesel fuel during the combustion in car engines is analysed based on available publications. The relevant parameters for the environmental impact of the combustion of diesel (aromatics content, reduction of sulphur content, the reduction of the density and raising of the cetane number) are discussed with regard to improvements of the exhaust qualities of kerosene. A reduction of the aromatics content promises to improve the emission of soot which should be further investigated.

Managing patient deterioration: assessing teamwork and individual performance.

PubMed

Cooper, Simon; Cant, Robyn; Porter, Jo; Missen, Karen; Sparkes, Louise; McConnell-Henry, Tracy; Endacott, Ruth

2013-05-01

To assess the ability of rural Australian nurse teams to manage deteriorating patients. This quasi-experimental design used pre- and post-intervention assessments and observation to evaluate nurses' simulated clinical performance. Registered nurses (n=44) from two hospital wards completed a formative knowledge assessment and three team-based video recorded scenarios (Objective Structured Clinical Examinations (OSCE)). Trained patient actors simulated deteriorating patients. Skill performance and situation awareness were measured and team performance was rated using the Team Emergency Assessment Measure. Knowledge in relation to patient deterioration management varied (mean 63%, range 27-100%) with a median score of 64%. Younger nurses with a greater number of working hours scored the highest (p=0.001). OSCE performance was generally low with a mean performance of 54%, but performance was maintained despite the increasing complexity of the scenarios. Situation awareness was generally low (median 50%, mean 47%, range 17-83%, SD 14.03) with significantly higher levels in younger participants (r=-0.346, p=0.021). Teamwork ratings averaged 57% with significant associations between the subscales (Leadership, Teamwork and Task Management) (p<0.006), the global rating scale (p<0.001) and two of the OSCE measures (p<0.049). Feedback from participants following the programme indicated significant improvements in knowledge, confidence and competence (p<0.001). Despite a satisfactory knowledge base, the application of knowledge was low with notable performance deficits in these demanding and stressful situations. The identification and management of patient deterioration needs to be taught in professional development programmes incorporating high fidelity simulation techniques. The Team Emergency assessment tool proved to be a valid measure of team performance in patient deterioration scenarios.

Effects of Diurnal Variation and Caffeine Consumption on Test of Variables of Attention (TOVA) Performance in Healthy Young Adults

ERIC Educational Resources Information Center

Hunt, Melissa G.; Momjian, Ani J.; Wong, Keri K.

2011-01-01

The Test of Variables of Attention (TOVA) is a continuous performance test (CPT) that assesses attention, impulsivity, and processing speed. CPTs are used in the assessment of attention-deficit/hyperactivity disorder (ADHD) in children, but more young adults are being assessed for ADHD as well. The TOVA norms are based on a standardization sample…

Synchronous Discussion between Assessors and Assessees in Web-Based Peer Assessment: Impact on Writing Performance, Feedback Quality, Meta-Cognitive Awareness and Self-Efficacy

ERIC Educational Resources Information Center

Zheng, Lanqin; Cui, Panpan; Li, Xin; Huang, Ronghuai

2018-01-01

Research on peer assessment in higher education has been conducted for decades. However, ways to improve the quality of peer feedback and optimise student' benefits through peer assessment remain a puzzle. The present study aimed to examine the impacts of synchronous discussion between assessors and assessees on writing performance, qualitative…

Effects of Peer-Assessed Feedback, Goal Setting and a Group Contingency on Performance and Learning by 10-12-Year-Old Academy Soccer Players

ERIC Educational Resources Information Center

Holt, Josh E.; Kinchin, Gary; Clarke, Gill

2012-01-01

Background: Coaches developing young talent in team sports must maximise practice and learning of essential game skills and accurately and continuously assess the performance and potential of each player. Relative age effects highlight an erroneous process of initial and on-going player assessment, based largely on subjective opinions of game…

Photometer Performance Assessment in TESS SPOC Pipeline

NASA Astrophysics Data System (ADS)

Li, Jie; Caldwell, Douglas A.; Jenkins, Jon Michael; Twicken, Joseph D.; Wohler, Bill; Chen, Xiaolan; Rose, Mark; TESS Science Processing Operations Center

2018-06-01

This poster describes the Photometer Performance Assessment (PPA) software component in the Transiting Exoplanet Survey Satellite (TESS) Science Processing Operations Center (SPOC) pipeline, which is developed based on the Kepler science pipeline. The PPA component performs two tasks: the first task is to assess the health and performance of the instrument based on the science data sets collected during each observation sector, identifying out of bounds conditions and generating alerts. The second is to combine the astrometric data collected for each CCD readout channel to construct a high fidelity record of the pointing history for each of the 4 cameras and an attitude solution for the TESS spacecraft for each 2-min data collection interval. PPA is implemented with multiple pipeline modules: PPA Metrics Determination (PMD), PMD Aggregator (PAG), and PPA Attitude Determination (PAD). The TESS Mission is funded by NASA's Science Mission Directorate. The SPOC is managed and operated by NASA Ames Research Center.

Effects of team-based learning on perceived teamwork and academic performance in a health assessment subject.

PubMed

Park, Hyung-Ran; Kim, Chun-Ja; Park, Jee-Won; Park, Eunyoung

2015-01-01

The purpose of this study was to examine the effectiveness of team-based learning (a well-recognized learning and teaching strategy), applied in a health assessment subject, on nursing students' perceived teamwork (team-efficacy and team skills) and academic performance (individual and team readiness assurance tests, and examination scores). A prospective, one-group, pre- and post-test design enrolled a convenience sample of 74 second-year nursing students at a university in Suwon, Korea. Team-based learning was applied in a 2-credit health assessment subject over a 16-week semester. All students received written material one week before each class for readiness preparation. After administering individual- and team-readiness assurance tests consecutively, the subject instructor gave immediate feedback and delivered a mini-lecture to the students. Finally, students carried out skill based application exercises. The findings showed significant improvements in the mean scores of students' perceived teamwork after the introduction of team-based learning. In addition, team-efficacy was associated with team-adaptability skills and team-interpersonal skills. Regarding academic performance, team readiness assurance tests were significantly higher than individual readiness assurance tests over time. Individual readiness assurance tests were significantly related with examination scores, while team readiness assurance tests were correlated with team-efficacy and team-interpersonal skills. The application of team-based learning in a health assessment subject can enhance students' perceived teamwork and academic performance. This finding suggests that team-based learning may be an effective learning and teaching strategy for improving team-work of nursing students, who need to collaborate and effectively communicate with health care providers to improve patients' health.

Subjective and objective quantification of physician's workload and performance during radiation therapy planning tasks.

PubMed

Mazur, Lukasz M; Mosaly, Prithima R; Hoyle, Lesley M; Jones, Ellen L; Marks, Lawrence B

2013-01-01

To quantify, and compare, workload for several common physician-based treatment planning tasks using objective and subjective measures of workload. To assess the relationship between workload and performance to define workload levels where performance could be expected to decline. Nine physicians performed the same 3 tasks on each of 2 cases ("easy" vs "hard"). Workload was assessed objectively throughout the tasks (via monitoring of pupil size and blink rate), and subjectively at the end of each case (via National Aeronautics and Space Administration Task Load Index; NASA-TLX). NASA-TLX assesses the 6 dimensions (mental, physical, and temporal demands, frustration, effort, and performance); scores > or ≈ 50 are associated with reduced performance in other industries. Performance was measured using participants' stated willingness to approve the treatment plan. Differences in subjective and objective workload between cases, tasks, and experience were assessed using analysis of variance (ANOVA). The correlation between subjective and objective workload measures were assessed via the Pearson correlation test. The relationships between workload and performance measures were assessed using the t test. Eighteen case-wise and 54 task-wise assessments were obtained. Subjective NASA-TLX scores (P < .001), but not time-weighted averages of objective scores (P > .1), were significantly lower for the easy vs hard case. Most correlations between the subjective and objective measures were not significant, except between average blink rate and NASA-TLX scores (r = -0.34, P = .02), for task-wise assessments. Performance appeared to decline at NASA-TLX scores of ≥55. The NASA-TLX may provide a reasonable method to quantify subjective workload for broad activities, and objective physiologic eye-based measures may be useful to monitor workload for more granular tasks within activities. The subjective and objective measures, as herein quantified, do not necessarily track each other, and more work is needed to assess their utilities. From a series of controlled experiments, we found that performance appears to decline at subjective workload levels ≥55 (as measured via NASA-TLX), which is consistent with findings from other industries. Copyright © 2013 American Society for Radiation Oncology. Published by Elsevier Inc. All rights reserved.

Performance of residents and anesthesiologists in a simulation-based skill assessment.

PubMed

Murray, David J; Boulet, John R; Avidan, Michael; Kras, Joseph F; Henrichs, Bernadette; Woodhouse, Julie; Evers, Alex S

2007-11-01

Anesthesiologists and anesthesia residents are expected to acquire and maintain skills to manage a wide range of acute intraoperative anesthetic events. The purpose of this study was to determine whether an inventory of simulated intraoperative scenarios provided a reliable and valid measure of anesthesia residents' and anesthesiologists' skill. Twelve simulated acute intraoperative scenarios were designed to assess the performance of 64 residents and 35 anesthesiologists. The participants were divided into four groups based on their training and experience. There were 31 new CA-1, 12 advanced CA-1, and 22 CA-2/CA-3 residents as well as a group of 35 experienced anesthesiologists who participated in the assessment. Each participant managed a set of simulated events. The advanced CA-1 residents, CA-2/CA-3 residents, and 35 anesthesiologists managed 8 of 12 intraoperative simulation exercises. The 31 CA-1 residents each managed 3 intraoperative scenarios. The new CA-1 residents received lower scores on the simulated intraoperative events than the other groups of participants. The advanced CA-1 residents, CA-2/CA-3 residents, and anesthesiologists performed similarly on the overall assessment. There was a wide range of scores obtained by individuals in each group. A number of the exercises were difficult for the majority of participants to recognize and treat, but most events effectively discriminated among participants who achieved higher and lower overall scores. This simulation-based assessment provided a valid method to distinguish the skills of more experienced anesthesia residents and anesthesiologists from residents in early training. The overall score provided a reliable measure of a participant's ability to recognize and manage simulated acute intraoperative events. Additional studies are needed to determine whether these simulation-based assessments are valid measures of clinical performance.

Differences Between Japanese and U.S. Children's Performance on "Zoo U": A Game-Based Social Skills Assessment.

PubMed

Craig, Ashley B; DeRosier, Melissa E; Watanabe, Yayoi

2015-08-01

The purpose of this study was to test whether and how performance on a digital game-based social skills assessment tool, "Zoo U" (3C Institute, Durham, NC), differed for children in the United States and Japan across six core social skills. "Zoo U" was administered to 497 third and fourth grade children from the United States and Japan (46 percent Japanese) by teachers and researchers, respectively. U.S. children received the original version of "Zoo U," and Japanese children received a fully translated Japanese version of the program. Scoring of each of the six social skills is built into the "Zoo U" software, with specific scoring algorithms for each grade level that provide both a continuous scale score and cutoffs for three distinct performance categories: high, average, and low. A multivariate analysis of variance (MANOVA) was conducted to assess differences by cultural group on each of the six continuous social skill scores, controlling for gender and grade level. Results revealed significant differences on four of the six skills in expected directions. Chi-squared and odds ratios analyses were then conducted on the assignment of children into each of the performance categories by cultural group, revealing additional nuance to the cultural differences identified in the MANOVA consistent with existing literature. We were able to replicate known cultural differences between U.S. and Japanese children with a simple direct translation of a Web-based social skills assessment game, "Zoo U." Our results provide preliminary support for the potential of game-based assessment methods to provide efficient and valid social skill assessments to children around the world.

National Assessment and the Opportunity to Learn in Educational Reform

ERIC Educational Resources Information Center

Snyder, Conrad Wesley, Jr.; Mereku, Kofi D.; Amedahe, Francis K.; Etsey, Kofui; Adu, John

2013-01-01

Over two decades, national assessments in Ghana have revealed generally poor performances across curriculum-based tests for primary school (Grades 1-6). Various reform agendas have been applied to the education system, sometimes with isolated success, but the overall performances remained stable and low. Surveying teacher mentors in schools…

Phonological and Non-Phonological Language Skills as Predictors of Early Reading Performance

ERIC Educational Resources Information Center

Batson-Magnuson, LuAnn

2010-01-01

Accurate prediction of early childhood reading performance could help identify at-risk students, aid in the development of evidence-based intervention strategies, and further our theoretical understanding of reading development. This study assessed the validity of the Developmental Indicator for the Assessment of Learning (DIAL) language-based…

Performance Analysis of GAME: A Generic Automated Marking Environment

ERIC Educational Resources Information Center

Blumenstein, Michael; Green, Steve; Fogelman, Shoshana; Nguyen, Ann; Muthukkumarasamy, Vallipuram

2008-01-01

This paper describes the Generic Automated Marking Environment (GAME) and provides a detailed analysis of its performance in assessing student programming projects and exercises. GAME has been designed to automatically assess programming assignments written in a variety of languages based on the "structure" of the source code and the correctness…

Making It Personal: Performance-Based Assessments, Ubiquitous Technology, and Advanced Learners

ERIC Educational Resources Information Center

Arispe, Kelly; Burston, Jack

2017-01-01

This pedagogical implementation study advocates for performance-driven assessments to help learners become aware of and improve upon presentational speaking skills at the advanced level. A social media content creation tool, Adobe Spark Video, enabled learners to practice oral skills outside of class. The task design, implementation, and…

The Impact of Diagnosing Skill Deficiencies and Assessment-Based Communication Training on Managerial Performance.

ERIC Educational Resources Information Center

Papa, Michael J.; Graham, Elizabeth E.

1991-01-01

Evaluates an organizational diagnosis program that assesses managerial communication skills and provides the frame for follow-up training programs. Finds that managers participating in follow-up communication skills training performed significantly higher on interpersonal skills, problem-solving ability, and productivity over three long-term…

The role of simulation in continuing medical education for acute care physicians: a systematic review.

PubMed

Khanduja, P Kristina; Bould, M Dylan; Naik, Viren N; Hladkowicz, Emily; Boet, Sylvain

2015-01-01

We systematically reviewed the effectiveness of simulation-based education, targeting independently practicing qualified physicians in acute care specialties. We also describe how simulation is used for performance assessment in this population. Data source included: DataMEDLINE, Embase, Cochrane Database of Systematic Reviews, Cochrane CENTRAL Database of Controlled Trials, and National Health Service Economic Evaluation Database. The last date of search was January 31, 2013. All original research describing simulation-based education for independently practicing physicians in anesthesiology, critical care, and emergency medicine was reviewed. Data analysis was performed in duplicate with further review by a third author in cases of disagreement until consensus was reached. Data extraction was focused on effectiveness according to Kirkpatrick's model. For simulation-based performance assessment, tool characteristics and sources of validity evidence were also collated. Of 39 studies identified, 30 studies focused on the effectiveness of simulation-based education and nine studies evaluated the validity of simulation-based assessment. Thirteen studies (30%) targeted the lower levels of Kirkpatrick's hierarchy with reliance on self-reporting. Simulation was unanimously described as a positive learning experience with perceived impact on clinical practice. Of the 17 remaining studies, 10 used a single group or "no intervention comparison group" design. The majority (n = 17; 44%) were able to demonstrate both immediate and sustained improvements in educational outcomes. Nine studies reported the psychometric properties of simulation-based performance assessment as their sole objective. These predominantly recruited independent practitioners as a convenience sample to establish whether the tool could discriminate between experienced and inexperienced operators and concentrated on a single aspect of validity evidence. Simulation is perceived as a positive learning experience with limited evidence to support improved learning. Future research should focus on the optimal modality and frequency of exposure, quality of assessment tools and on the impact of simulation-based education beyond the individuals toward improved patient care.

Assessment of Age-Related Differences in Functional Capacity Using the Virtual Reality Functional Capacity Assessment Tool (VRFCAT)

PubMed Central

Atkins, A.S.; Stroescu, I.; Spagnola, N.B.; Davis, V.G.; Patterson, T.D.; Narasimhan, M.; Harvey, P.D.; Keefe, R.S.E.

2015-01-01

Clinical trials for primary prevention and early intervention in preclinical AD require measures of functional capacity with improved sensitivity to deficits in healthier, non-demented individuals. To this end, the Virtual Reality Functional Capacity Assessment Tool (VRFCAT) was developed as a direct performance-based assessment of functional capacity that is sensitive to changes in function across multiple populations. Using a realistic virtual reality environment, the VRFCAT assesses a subject's ability to complete instrumental activities associated with a shopping trip. The present investigation represents an initial evaluation of the VRFCAT as a potential co-primary measure of functional capacity in healthy aging and preclinical MCI/AD by examining test-retest reliability and associations with cognitive performance in healthy young and older adults. The VRFCAT was compared and contrasted with the UPSA-2-VIM, a traditional performance-based assessment utilizing physical props. Results demonstrated strong age-related differences in performance on each VRFCAT outcome measure, including total completion time, total errors, and total forced progressions. VRFCAT performance showed strong correlations with cognitive performance across both age groups. VRFCAT Total Time demonstrated good test-retest reliability (ICC=.80 in young adults; ICC=.64 in older adults) and insignificant practice effects, indicating the measure is suitable for repeated testing in healthy populations. Taken together, these results provide preliminary support for the VRFCAT as a potential measure of functionally relevant change in primary prevention and preclinical AD/MCI trials. PMID:26618145

Self-assessment in schizophrenia: Accuracy of evaluation of cognition and everyday functioning.

PubMed

Gould, Felicia; McGuire, Laura Stone; Durand, Dante; Sabbag, Samir; Larrauri, Carlos; Patterson, Thomas L; Twamley, Elizabeth W; Harvey, Philip D

2015-09-01

Self-assessment deficits, often referred to as impaired insight or unawareness of illness, are well established in people with schizophrenia. There are multiple levels of awareness, including awareness of symptoms, functional deficits, cognitive impairments, and the ability to monitor cognitive and functional performance in an ongoing manner. The present study aimed to evaluate the comparative predictive value of each aspect of awareness on the levels of everyday functioning in people with schizophrenia. We examined multiple aspects of self-assessment of functioning in 214 people with schizophrenia. We also collected information on everyday functioning rated by high contact clinicians and examined the importance of self-assessment for the prediction of real-world functional outcomes. The relative impact of performance-based measures of cognition, functional capacity, and metacognitive performance on everyday functioning was also examined. Misestimation of ability emerged as the strongest predictor of real-world functioning and exceeded the influences of cognitive performance, functional capacity performance, and performance-based assessment of metacognitive monitoring. The relative contribution of the factors other than self-assessment varied according to which domain of everyday functioning was being examined, but, in all cases, accounted for less predictive variance. These results underscore the functional impact of misestimating one's current functioning and relative level of ability. These findings are consistent with the use of insight-focused treatments and compensatory strategies designed to increase self-awareness in multiple functional domains. (c) 2015 APA, all rights reserved).

Self Assessment in Schizophrenia: Accuracy of Evaluation of Cognition and Everyday Functioning

PubMed Central

Gould, Felicia; McGuire, Laura Stone; Durand, Dante; Sabbag, Samir; Larrauri, Carlos; Patterson, Thomas L.; Twamley, Elizabeth W.; Harvey, Philip D.

2015-01-01

Objective Self-assessment deficits, often referred to as impaired insight or unawareness of illness, are well established in people with schizophrenia. There are multiple levels of awareness, including awareness of symptoms, functional deficits, cognitive impairments, and the ability to monitor cognitive and functional performance in an ongoing manner. The present study aimed to evaluate the comparative predictive value of each aspect of awareness on the levels of everyday functioning in people with schizophrenia. Method We examined multiple aspects of self-assessment of functioning in 214 people with schizophrenia. We also collected information on everyday functioning rated by high contact clinicians and examined the importance of self-assessment for the prediction of real world functional outcomes. The relative impact of performance based measures of cognition, functional capacity, and metacognitive performance on everyday functioning was also examined. Results Misestimation of ability emerged as the strongest predictor of real world functioning and exceeded the influences of cognitive performance, functional capacity performance, and performance-based assessment of metacognitive monitoring. The relative contribution of the factors other than self-assessment varied according to which domain of everyday functioning was being examined, but in all cases, accounted for less predictive variance. Conclusions These results underscore the functional impact of misestimating one’s current functioning and relative level of ability. These findings are consistent with the use of insight-focused treatments and compensatory strategies designed to increase self-awareness in multiple functional domains. PMID:25643212

Long-Term In-Service Monitoring and Performance Assessment of the Main Cables of Long-Span Suspension Bridges

PubMed Central

Deng, Yang; Liu, Yang; Chen, Suren

2017-01-01

Despite the recent developments in structural health monitoring, there remain great challenges for accurately, conveniently, and economically assessing the in-service performance of the main cables for long-span suspension bridges. A long-term structural health monitoring technique is developed to measure the tension force with a conventional sensing technology and further provide the in-service performance assessment strategy of the main cable. The monitoring system adopts conventional vibrating strings transducers to monitor the tension forces of separate cable strands of the main cable in the anchor span. The performance evaluation of the main cable is conducted based on the collected health monitoring data: (1) the measured strand forces are used to derive the overall tension force of a main cable, which is further translated into load bearing capacity assessment using the concept of safety factor; and (2) the proposed technique can also evaluate the uniformity of tension forces from different cable strands. The assessment of uniformity of strand forces of a main cable offers critical information in terms of potential risks of partial damage and performance deterioration of the main cable. The results suggest the proposed low-cost monitoring system is an option to provide approximate estimation of tension forces of main cables for suspension bridges. With the long-term monitoring data, the proposed monitoring-based evaluation methods can further provide critical information to assess the safety and serviceability performance of main cables. PMID:28621743

Long-Term In-Service Monitoring and Performance Assessment of the Main Cables of Long-Span Suspension Bridges.

PubMed

Deng, Yang; Liu, Yang; Chen, Suren

2017-06-16

Despite the recent developments in structural health monitoring, there remain great challenges for accurately, conveniently, and economically assessing the in-service performance of the main cables for long-span suspension bridges. A long-term structural health monitoring technique is developed to measure the tension force with a conventional sensing technology and further provide the in-service performance assessment strategy of the main cable. The monitoring system adopts conventional vibrating strings transducers to monitor the tension forces of separate cable strands of the main cable in the anchor span. The performance evaluation of the main cable is conducted based on the collected health monitoring data: (1) the measured strand forces are used to derive the overall tension force of a main cable, which is further translated into load bearing capacity assessment using the concept of safety factor; and (2) the proposed technique can also evaluate the uniformity of tension forces from different cable strands. The assessment of uniformity of strand forces of a main cable offers critical information in terms of potential risks of partial damage and performance deterioration of the main cable. The results suggest the proposed low-cost monitoring system is an option to provide approximate estimation of tension forces of main cables for suspension bridges. With the long-term monitoring data, the proposed monitoring-based evaluation methods can further provide critical information to assess the safety and serviceability performance of main cables.

Development and Application of a Stepwise Assessment Process for Rational Redesign of Sequential Skills-Based Courses.

PubMed

Gallimore, Casey E; Porter, Andrea L; Barnett, Susanne G

2016-10-25

Objective. To develop and apply a stepwise process to assess achievement of course learning objectives related to advanced pharmacy practice experiences (APPEs) preparedness and inform redesign of sequential skills-based courses. Design. Four steps comprised the assessment and redesign process: (1) identify skills critical for APPE preparedness; (2) utilize focus groups and course evaluations to determine student competence in skill performance; (3) apply course mapping to identify course deficits contributing to suboptimal skill performance; and (4) initiate course redesign to target exposed deficits. Assessment. Focus group participants perceived students were least prepared for skills within the Accreditation Council for Pharmacy Education's pre-APPE core domains of Identification and Assessment of Drug-related Problems and General Communication Abilities. Course mapping identified gaps in instruction, performance, and assessment of skills within aforementioned domains. Conclusions. A stepwise process that identified strengths and weaknesses of a course, was used to facilitate structured course redesign. Strengths of the process included input and corroboration from both preceptors and students. Limitations included feedback from a small number of pharmacy preceptors and increased workload on course coordinators.

Affordable, automatic quantitative fall risk assessment based on clinical balance scales and Kinect data.

PubMed

Colagiorgio, P; Romano, F; Sardi, F; Moraschini, M; Sozzi, A; Bejor, M; Ricevuti, G; Buizza, A; Ramat, S

2014-01-01

The problem of a correct fall risk assessment is becoming more and more critical with the ageing of the population. In spite of the available approaches allowing a quantitative analysis of the human movement control system's performance, the clinical assessment and diagnostic approach to fall risk assessment still relies mostly on non-quantitative exams, such as clinical scales. This work documents our current effort to develop a novel method to assess balance control abilities through a system implementing an automatic evaluation of exercises drawn from balance assessment scales. Our aim is to overcome the classical limits characterizing these scales i.e. limited granularity and inter-/intra-examiner reliability, to obtain objective scores and more detailed information allowing to predict fall risk. We used Microsoft Kinect to record subjects' movements while performing challenging exercises drawn from clinical balance scales. We then computed a set of parameters quantifying the execution of the exercises and fed them to a supervised classifier to perform a classification based on the clinical score. We obtained a good accuracy (~82%) and especially a high sensitivity (~83%).

Resident self-other assessor agreement: influence of assessor, competency, and performance level.

PubMed

Lipsett, Pamela A; Harris, Ilene; Downing, Steven

2011-08-01

To review the literature on self-assessment in the context of resident performance and to determine the correlation between self-assessment across competencies in high- and low-performing residents and assessments performed by raters from a variety of professional roles (peers, nurses, and faculty). Retrospective analysis of prospectively collected anonymous self-assessment and multiprofessional (360) performance assessments by competency and overall. University-based academic general surgical program. Sixty-two residents rotating in general surgery. Mean difference for each self-assessment dyad (self-peer, self-nurse, and self-attending physician) by resident performance quartile, adjusted for measurement error, correlation coefficients, and summed differences across all competencies. Irrespective of self-other dyad, residents asked to rate their global performance overestimated their skills. Residents in the upper quartile underestimated their specific skills while those in the lowest-performing quartile overestimated their abilities when compared with faculty, peers, and especially nurse raters. Moreover, overestimation was greatest in competencies related to interpersonal skills, communication, teamwork, and professionalism. Rater, level of performance, and the competency being assessed all influence the comparison of the resident's self-assessment and those of other raters. Self-assessment of competencies related to behavior may be inaccurate when compared with raters from various professions. Residents in the lowest-performing quartile are least able to identify their weakness. These data have important implications for residents, program directors, and the public and suggest that strategies that help the lowest-performing residents recognize areas in need of improvement are needed.

Assessment of the dose reduction potential of a model-based iterative reconstruction algorithm using a task-based performance metrology

DOE Office of Scientific and Technical Information (OSTI.GOV)

Samei, Ehsan, E-mail: samei@duke.edu; Richard, Samuel

2015-01-15

Purpose: Different computed tomography (CT) reconstruction techniques offer different image quality attributes of resolution and noise, challenging the ability to compare their dose reduction potential against each other. The purpose of this study was to evaluate and compare the task-based imaging performance of CT systems to enable the assessment of the dose performance of a model-based iterative reconstruction (MBIR) to that of an adaptive statistical iterative reconstruction (ASIR) and a filtered back projection (FBP) technique. Methods: The ACR CT phantom (model 464) was imaged across a wide range of mA setting on a 64-slice CT scanner (GE Discovery CT750 HD,more » Waukesha, WI). Based on previous work, the resolution was evaluated in terms of a task-based modulation transfer function (MTF) using a circular-edge technique and images from the contrast inserts located in the ACR phantom. Noise performance was assessed in terms of the noise-power spectrum (NPS) measured from the uniform section of the phantom. The task-based MTF and NPS were combined with a task function to yield a task-based estimate of imaging performance, the detectability index (d′). The detectability index was computed as a function of dose for two imaging tasks corresponding to the detection of a relatively small and a relatively large feature (1.5 and 25 mm, respectively). The performance of MBIR in terms of the d′ was compared with that of ASIR and FBP to assess its dose reduction potential. Results: Results indicated that MBIR exhibits a variability spatial resolution with respect to object contrast and noise while significantly reducing image noise. The NPS measurements for MBIR indicated a noise texture with a low-pass quality compared to the typical midpass noise found in FBP-based CT images. At comparable dose, the d′ for MBIR was higher than those of FBP and ASIR by at least 61% and 19% for the small feature and the large feature tasks, respectively. Compared to FBP and ASIR, MBIR indicated a 46%–84% dose reduction potential, depending on task, without compromising the modeled detection performance. Conclusions: The presented methodology based on ACR phantom measurements extends current possibilities for the assessment of CT image quality under the complex resolution and noise characteristics exhibited with statistical and iterative reconstruction algorithms. The findings further suggest that MBIR can potentially make better use of the projections data to reduce CT dose by approximately a factor of 2. Alternatively, if the dose held unchanged, it can improve image quality by different levels for different tasks.« less

Uncertainty estimates of purity measurements based on current information: toward a "live validation" of purity methods.

PubMed

Apostol, Izydor; Kelner, Drew; Jiang, Xinzhao Grace; Huang, Gang; Wypych, Jette; Zhang, Xin; Gastwirt, Jessica; Chen, Kenneth; Fodor, Szilan; Hapuarachchi, Suminda; Meriage, Dave; Ye, Frank; Poppe, Leszek; Szpankowski, Wojciech

2012-12-01

To predict precision and other performance characteristics of chromatographic purity methods, which represent the most widely used form of analysis in the biopharmaceutical industry. We have conducted a comprehensive survey of purity methods, and show that all performance characteristics fall within narrow measurement ranges. This observation was used to develop a model called Uncertainty Based on Current Information (UBCI), which expresses these performance characteristics as a function of the signal and noise levels, hardware specifications, and software settings. We applied the UCBI model to assess the uncertainty of purity measurements, and compared the results to those from conventional qualification. We demonstrated that the UBCI model is suitable to dynamically assess method performance characteristics, based on information extracted from individual chromatograms. The model provides an opportunity for streamlining qualification and validation studies by implementing a "live validation" of test results utilizing UBCI as a concurrent assessment of measurement uncertainty. Therefore, UBCI can potentially mitigate the challenges associated with laborious conventional method validation and facilitates the introduction of more advanced analytical technologies during the method lifecycle.

Advantages of integrated and sustainability based assessment for metabolism based strategic planning of urban water systems.

PubMed

Behzadian, Kourosh; Kapelan, Zoran

2015-09-15

Despite providing water-related services as the primary purpose of urban water system (UWS), all relevant activities require capital investments and operational expenditures, consume resources (e.g. materials and chemicals), and may increase negative environmental impacts (e.g. contaminant discharge, emissions to water and air). Performance assessment of such a metabolic system may require developing a holistic approach which encompasses various system elements and criteria. This paper analyses the impact of integration of UWS components on the metabolism based performance assessment for future planning using a number of intervention strategies. It also explores the importance of sustainability based criteria in the assessment of long-term planning. Two assessment approaches analysed here are: (1) planning for only water supply system (WSS) as a part of the UWS and (2) planning for an integrated UWS including potable water, stormwater, wastewater and water recycling. WaterMet(2) model is used to simulate metabolic type processes in the UWS and calculate quantitative performance indicators. The analysis is demonstrated on the problem of strategic level planning of a real-world UWS to where optional intervention strategies are applied. The resulting performance is assessed using the multiple criteria of both conventional and sustainability type; and optional intervention strategies are then ranked using the Compromise Programming method. The results obtained show that the high ranked intervention strategies in the integrated UWS are those supporting both water supply and stormwater/wastewater subsystems (e.g. rainwater harvesting and greywater recycling schemes) whilst these strategies are ranked low in the WSS and those targeting improvement of water supply components only (e.g. rehabilitation of clean water pipes and addition of new water resources) are preferred instead. Results also demonstrate that both conventional and sustainability type performance indicators are necessary for strategic planning in the UWS. Copyright © 2015 Elsevier B.V. All rights reserved.

Massive integration of diverse protein quality assessment methods to improve template based modeling in CASP11

PubMed Central

Cao, Renzhi; Bhattacharya, Debswapna; Adhikari, Badri; Li, Jilong; Cheng, Jianlin

2015-01-01

Model evaluation and selection is an important step and a big challenge in template-based protein structure prediction. Individual model quality assessment methods designed for recognizing some specific properties of protein structures often fail to consistently select good models from a model pool because of their limitations. Therefore, combining multiple complimentary quality assessment methods is useful for improving model ranking and consequently tertiary structure prediction. Here, we report the performance and analysis of our human tertiary structure predictor (MULTICOM) based on the massive integration of 14 diverse complementary quality assessment methods that was successfully benchmarked in the 11th Critical Assessment of Techniques of Protein Structure prediction (CASP11). The predictions of MULTICOM for 39 template-based domains were rigorously assessed by six scoring metrics covering global topology of Cα trace, local all-atom fitness, side chain quality, and physical reasonableness of the model. The results show that the massive integration of complementary, diverse single-model and multi-model quality assessment methods can effectively leverage the strength of single-model methods in distinguishing quality variation among similar good models and the advantage of multi-model quality assessment methods of identifying reasonable average-quality models. The overall excellent performance of the MULTICOM predictor demonstrates that integrating a large number of model quality assessment methods in conjunction with model clustering is a useful approach to improve the accuracy, diversity, and consequently robustness of template-based protein structure prediction. PMID:26369671

Concept for a Satellite-Based Advanced Air Traffic Management System : Volume 4. Operational Description and Qualitative Assessment.

DOT National Transportation Integrated Search

1974-02-01

The volume presents a description of how the Satellite-Based Advanced Air Traffic Management System (SAATMS) operates and a qualitative assessment of the system. The operational description includes the services, functions, and tasks performed by the...

An analysis of mathematical connection ability based on student learning style on visualization auditory kinesthetic (VAK) learning model with self-assessment

NASA Astrophysics Data System (ADS)

Apipah, S.; Kartono; Isnarto

2018-03-01

This research aims to analyze the quality of VAK learning with self-assessment toward the ability of mathematical connection performed by students and to analyze students’ mathematical connection ability based on learning styles in VAK learning model with self-assessment. This research applies mixed method type with concurrent embedded design. The subject of this research consists of VIII grade students from State Junior High School 9 Semarang who apply visual learning style, auditory learning style, and kinesthetic learning style. The data of learning style is collected by using questionnaires, the data of mathematical connection ability is collected by performing tests, and the data of self-assessment is collected by using assessment sheets. The quality of learning is qualitatively valued from planning stage, realization stage, and valuation stage. The result of mathematical connection ability test is analyzed quantitatively by mean test, conducting completeness test, mean differentiation test, and mean proportional differentiation test. The result of the research shows that VAK learning model results in well-qualified learning regarded from qualitative and quantitative sides. Students with visual learning style perform the highest mathematical connection ability, students with kinesthetic learning style perform average mathematical connection ability, and students with auditory learning style perform the lowest mathematical connection ability.

Objective assessment of operator performance during ultrasound-guided procedures.

PubMed

Tabriz, David M; Street, Mandie; Pilgram, Thomas K; Duncan, James R

2011-09-01

Simulation permits objective assessment of operator performance in a controlled and safe environment. Image-guided procedures often require accurate needle placement, and we designed a system to monitor how ultrasound guidance is used to monitor needle advancement toward a target. The results were correlated with other estimates of operator skill. The simulator consisted of a tissue phantom, ultrasound unit, and electromagnetic tracking system. Operators were asked to guide a needle toward a visible point target. Performance was video-recorded and synchronized with the electromagnetic tracking data. A series of algorithms based on motor control theory and human information processing were used to convert raw tracking data into different performance indices. Scoring algorithms converted the tracking data into efficiency, quality, task difficulty, and targeting scores that were aggregated to create performance indices. After initial feasibility testing, a standardized assessment was developed. Operators (N = 12) with a broad spectrum of skill and experience were enrolled and tested. Overall scores were based on performance during ten simulated procedures. Prior clinical experience was used to independently estimate operator skill. When summed, the performance indices correlated well with estimated skill. Operators with minimal or no prior experience scored markedly lower than experienced operators. The overall score tended to increase according to operator's clinical experience. Operator experience was linked to decreased variation in multiple aspects of performance. The aggregated results of multiple trials provided the best correlation between estimated skill and performance. A metric for the operator's ability to maintain the needle aimed at the target discriminated between operators with different levels of experience. This study used a highly focused task model, standardized assessment, and objective data analysis to assess performance during simulated ultrasound-guided needle placement. The performance indices were closely related to operator experience.

Beyond Measurement-Driven Instruction: Achieving Deep Learning Based on Constructivist Learning Theory, Integrated Assessment, and a Flipped Classroom Approach

ERIC Educational Resources Information Center

Bernauer, James A.; Fuller, Richard G.

2017-01-01

The authors focus on the critical role of assessment within a flipped classroom environment where instruction is based on constructivist learning theory and where desired student outcomes are at the higher levels of Bloom's Taxonomy. While assessment is typically thought of in terms of providing summative measures of performance or achievement, it…

Brazelton Neonatal Assessment for School Psychologists.

ERIC Educational Resources Information Center

Stoudt, Calvin L.

This speech addresses the "What,""Why," and "How" of Brazelton Neonatal Assessment Training for school psychologists. "What" concerns the Brazelton Neonatal Behavioral Assessment Scale, its administration, and what it assesses. Based on the best performance, the infant's score on this scale is scored in the…

Internet Training Resulted in Improved Trainee Performance in a Simulated Opioid-Poisoned Patient as Measured by Checklist.

PubMed

Kim, Hong; Heverling, Harry; Cordeiro, Michael; Vasquez, Vanessa; Stolbach, Andrew

2016-09-01

Opioid overdose is a leading cause of death in the USA. Internet-based teaching can improve medical knowledge among trainees, but there are limited data to show the effect of Internet-based teaching on clinical competence in medical training, including management of opioid poisoning. We used an ecological design to assess the effect of an Internet-based teaching module on the management of a simulated opioid-poisoned patient. We enrolled two consecutive classes of post-graduate year-1 residents from a single emergency medicine program. The first group (RA) was instructed to read a toxicology textbook chapter and the second group (IT) took a brief Internet training module. All participants subsequently managed a simulated opioid-poisoned patient. The participants' performance was evaluated with two types of checklist (simple and time-weighted), along with global assessment scores. Internet-trained participants performed better on both checklist scales. The difference between mean simple checklist scores by the IT and RA groups was 0.23 (95 % CI, 0.016-0.44). The difference between mean time-weighted checklist scores was 0.27 (95 % CI, 0.048-0.49). When measured by global assessment, there was no statistically significant difference between RA and IT participants. These data suggest that the Internet module taught basic principles of management of the opioid-poisoned patient. In this scenario, global assessment and checklist assessment may not measure the same proficiencies. These encouraging results are not sufficient to show that this Internet tool improves clinical performance. We should assess the impact of the Internet module on performance in a true clinical environment.

Effects of a Peer Assessment System Based on a Grid-Based Knowledge Classification Approach on Computer Skills Training

ERIC Educational Resources Information Center

Hsu, Ting-Chia

2016-01-01

In this study, a peer assessment system using the grid-based knowledge classification approach was developed to improve students' performance during computer skills training. To evaluate the effectiveness of the proposed approach, an experiment was conducted in a computer skills certification course. The participants were divided into three…

Validity in work-based assessment: expanding our horizons.

PubMed

Govaerts, Marjan; van der Vleuten, Cees P M

2013-12-01

Although work-based assessments (WBA) may come closest to assessing habitual performance, their use for summative purposes is not undisputed. Most criticism of WBA stems from approaches to validity consistent with the quantitative psychometric framework. However, there is increasing research evidence that indicates that the assumptions underlying the predictive, deterministic framework of psychometrics may no longer hold. In this discussion paper we argue that meaningfulness and appropriateness of current validity evidence can be called into question and that we need alternative strategies to assessment and validity inquiry that build on current theories of learning and performance in complex and dynamic workplace settings. Drawing from research in various professional fields we outline key issues within the mechanisms of learning, competence and performance in the context of complex social environments and illustrate their relevance to WBA. In reviewing recent socio-cultural learning theory and research on performance and performance interpretations in work settings, we demonstrate that learning, competence (as inferred from performance) as well as performance interpretations are to be seen as inherently contextualised, and can only be under-stood 'in situ'. Assessment in the context of work settings may, therefore, be more usefully viewed as a socially situated interpretive act. We propose constructivist-interpretivist approaches towards WBA in order to capture and understand contextualised learning and performance in work settings. Theoretical assumptions underlying interpretivist assessment approaches call for a validity theory that provides the theoretical framework and conceptual tools to guide the validation process in the qualitative assessment inquiry. Basic principles of rigour specific to qualitative research have been established, and they can and should be used to determine validity in interpretivist assessment approaches. If used properly, these strategies generate trustworthy evidence that is needed to develop the validity argument in WBA, allowing for in-depth and meaningful information about professional competence. © 2013 John Wiley & Sons Ltd.

Team-based assessment of professional behavior in medical students.

PubMed

Raee, Hojat; Amini, Mitra; Momen Nasab, Ameneh; Malek Pour, Abdolrasoul; Jafari, Mohammad Morad

2014-07-01

Self and peer assessment provides important information about the individual's performance and behavior in all aspects of their professional environment work. The aim of this study is to evaluate the professional behavior and performance in medical students in the form of team based assessment. In a cross-sectional study, 100 medical students in the 7(th) year of education were randomly selected and enrolled; for each student five questionnaires were filled out, including one self-assessment, two peer assessments and two residents assessment. The scoring system of the questionnaires was based on seven point Likert scale.  After filling out the questions in the questionnaire, numerical data and written comments provided to the students were collected, analyzed and discussed. Internal consistency (Cronbach's alpha) of the questionnaires was assessed. A p<0.05 was considered as significant level. Internal consistency was acceptable (Cronbach's alpha 0.83). Interviews revealed that the majority of students and assessors interviewed found the method acceptable. The range of scores was 1-6 (Mean±SD=4.39±0.57) for the residents' assessment, 2-6 (Mean±SD= 4.49±0.53) for peer assessment, and 3-7 (Mean±SD=5.04±0.32) for self-assessment. There was a significant difference between self assessment and other methods of assessment. This study demonstrates that a team-based assessment is an acceptable and feasible method for peer and self-assessment of medical students' learning in a clinical clerkship, and has some advantages over traditional assessment methods. Further studies are needed to focus on the strengths and weaknesses.

From nationwide standardized testing to school-based alternative embedded assessment in Israel: Students' performance in the matriculation 2000 project

NASA Astrophysics Data System (ADS)

Dori, Yehudit J.

2003-01-01

Matriculation 2000 was a 5-year project aimed at moving from the nationwide traditional examination system in Israel to a school-based alternative embedded assessment. Encompassing 22 high schools from various communities in the country, the Project aimed at fostering deep understanding, higher-order thinking skills, and students' engagement in learning through alternative teaching and embedded assessment methods. This article describes research conducted during the fifth year of the Project at 2 experimental and 2 control schools. The research objective was to investigate students' learning outcomes in chemistry and biology in the Matriculation 2000 Project. The assumption was that alternative embedded assessment has some effect on students' performance. The experimental students scored significantly higher than their control group peers on low-level assignments and more so on assignments that required higher-order thinking skills. The findings indicate that given adequate support and teachers' consent and collaboration, schools can transfer from nationwide or statewide standardized testing to school-based alter-native embedded assessment.

A Study of Wind Turbine Comprehensive Operational Assessment Model Based on EM-PCA Algorithm

NASA Astrophysics Data System (ADS)

Zhou, Minqiang; Xu, Bin; Zhan, Yangyan; Ren, Danyuan; Liu, Dexing

2018-01-01

To assess wind turbine performance accurately and provide theoretical basis for wind farm management, a hybrid assessment model based on Entropy Method and Principle Component Analysis (EM-PCA) was established, which took most factors of operational performance into consideration and reach to a comprehensive result. To verify the model, six wind turbines were chosen as the research objects, the ranking obtained by the method proposed in the paper were 4#>6#>1#>5#>2#>3#, which are completely in conformity with the theoretical ranking, which indicates that the reliability and effectiveness of the EM-PCA method are high. The method could give guidance for processing unit state comparison among different units and launching wind farm operational assessment.

Development of performance assessment instrument based contextual learning for measuring students laboratory skills

NASA Astrophysics Data System (ADS)

Susilaningsih, E.; Khotimah, K.; Nurhayati, S.

2018-04-01

The assessment of laboratory skill in general hasn’t specific guideline in assessment, while the individual assessment of students during a performance and skill in performing laboratory is still not been observed and measured properly. Alternative assessment that can be used to measure student laboratory skill is use performance assessment. The purpose of this study was to determine whether the performance assessment instrument that the result of research can be used to assess basic skills student laboratory. This research was conducted by the Research and Development. The result of the data analysis performance assessment instruments developed feasible to implement and validation result 62.5 with very good categories for observation sheets laboratory skills and all of the components with the very good category. The procedure is the preliminary stages of research and development stages. Preliminary stages are divided in two, namely the field studies and literature studies. The development stages are divided into several parts, namely 1) development of the type instrument, 2) validation by an expert, 3) a limited scale trial, 4) large-scale trials and 5) implementation of the product. The instrument included in the category of effective because 26 from 29 students have very high laboratory skill and high laboratory skill. The research of performance assessment instrument is standard and can be used to assess basic skill student laboratory.

Validity and applicability of a video-based animated tool to assess mobility in elderly Latin American populations.

PubMed

Guerra, Ricardo Oliveira; Oliveira, Bruna Silva; Alvarado, Beatriz Eugenia; Curcio, Carmen Lucia; Rejeski, W Jack; Marsh, Anthony P; Ip, Edward H; Barnard, Ryan T; Guralnik, Jack M; Zunzunegui, Maria Victoria

2014-10-01

To assess the reliability and the validity of Portuguese- and Spanish-translated versions of the video-based short-form Mobility Assessment Tool in assessing self-reported mobility, and to provide evidence for the applicability of these videos in elderly Latin American populations as a complement to physical performance measures. The sample consisted of 300 elderly participants (150 from Brazil, 150 from Colombia) recruited at neighborhood social centers. Mobility was assessed with the Mobility Assessment Tool, and compared with the Short Physical Performance Battery score and self-reported functional limitations. Reliability was calculated using intraclass correlation coefficients. Multiple linear regression analyses were used to assess associations among mobility assessment tools and health, and sociodemographic variables. A significant gradient of increasing Mobility Assessment Tool score with better physical function was observed for both self-reported and objective measures, and in each city. Associations between self-reported mobility and health were strong, and significant. Mobility Assessment Tool scores were lower in women at both sites. Intraclass correlation coefficients of the Mobility Assessment Tool were 0.94 (95% confidence interval 0.90-0.97) in Brazil and 0.81 (95% confidence interval 0.66-0.91) in Colombia. Mobility Assessment Tool scores were lower in Manizales than in Natal after adjustment by Short Physical Performance Battery, self-rated health and sex. These results provide evidence for high reliability and good validity of the Mobility Assessment Tool in its Spanish and Portuguese versions used in Latin American populations. In addition, the Mobility Assessment Tool can detect mobility differences related to environmental features that cannot be captured by objective performance measures. © 2013 Japan Geriatrics Society.

Assessing a traditional case-based application exercise and a student question creation exercise on student performance and perceptions.

PubMed

Tatachar, Amulya; Kominski, Carol

2017-07-01

To compare the impact of a traditional case-based application exercise with a student question creation exercise on a) student exam performance, b) student perceptions of enjoyment, competence, understanding, effort, interest in continuing participation, and interest in the subject. Subjects were 84 second-year pharmacy students in a pharmacotherapy course. The research focus was active learning involving the topic of chronic kidney disease-mineral bone disorder. Student teams were randomly assigned to either case-based or student question creation exercises using PeerWise. Student performance was assessed by a pre- and posttest and on block and final exams. After completion, an online survey assessed student perceptions of both exercises. Statistically significant differences were revealed in favor of the student question creation group on enjoyment and interest in the subject matter. No statistically differences were found between the traditional case-based group and the student question creation group on gain score from pre-test to posttest. The student question creation group performed slightly better than the case-based application group on two of the five questions on the block exam but none of these differences reached statistical significance. Students randomly assigned to groups that created and reviewed questions exhibited slightly improved summative exam performance and reported significantly more positive perceptions than students engaging in a more traditional case-based learning activity. Student question creation has demonstrated potential as a useful learning activity. Despite inherent difficulties in designing studies involving educational research in a controlled environment, students who have submitted, created, rated, and answered peers' questions have overall performed well. Copyright © 2017 Elsevier Inc. All rights reserved.

Web-Based Assessment of Visual and Visuospatial Symptoms in Parkinson's Disease

PubMed Central

Amick, Melissa M.; Miller, Ivy N.; Neargarder, Sandy; Cronin-Golomb, Alice

2012-01-01

Visual and visuospatial dysfunction is prevalent in Parkinson's disease (PD). To promote assessment of these often overlooked symptoms, we adapted the PD Vision Questionnaire for Internet administration. The questionnaire evaluates visual and visuospatial symptoms, impairments in activities of daily living (ADLs), and motor symptoms. PD participants of mild to moderate motor severity (n = 24) and healthy control participants (HC, n = 23) completed the questionnaire in paper and web-based formats. Reliability was assessed by comparing responses across formats. Construct validity was evaluated by reference to performance on measures of vision, visuospatial cognition, ADLs, and motor symptoms. The web-based format showed excellent reliability with respect to the paper format for both groups (all P′s < 0.001; HC completing the visual and visuospatial section only). Demonstrating the construct validity of the web-based questionnaire, self-rated ADL and visual and visuospatial functioning were significantly associated with performance on objective measures of these abilities (all P′s < 0.01). The findings indicate that web-based administration may be a reliable and valid method of assessing visual and visuospatial and ADL functioning in PD. PMID:22530162

Spiral and Project-Based Learning with Peer Assessment in a Computer Science Project Management Course

NASA Astrophysics Data System (ADS)

Jaime, Arturo; Blanco, José Miguel; Domínguez, César; Sánchez, Ana; Heras, Jónathan; Usandizaga, Imanol

2016-06-01

Different learning methods such as project-based learning, spiral learning and peer assessment have been implemented in science disciplines with different outcomes. This paper presents a proposal for a project management course in the context of a computer science degree. Our proposal combines three well-known methods: project-based learning, spiral learning and peer assessment. Namely, the course is articulated during a semester through the structured (progressive and incremental) development of a sequence of four projects, whose duration, scope and difficulty of management increase as the student gains theoretical and instrumental knowledge related to planning, monitoring and controlling projects. Moreover, the proposal is complemented using peer assessment. The proposal has already been implemented and validated for the last 3 years in two different universities. In the first year, project-based learning and spiral learning methods were combined. Such a combination was also employed in the other 2 years; but additionally, students had the opportunity to assess projects developed by university partners and by students of the other university. A total of 154 students have participated in the study. We obtain a gain in the quality of the subsequently projects derived from the spiral project-based learning. Moreover, this gain is significantly bigger when peer assessment is introduced. In addition, high-performance students take advantage of peer assessment from the first moment, whereas the improvement in poor-performance students is delayed.

The use of observational diaries in in-training evaluation: student perceptions.

PubMed

Govaerts, Marjan J B; van der Vleuten, Cees P M; Schuwirth, Lambert W T; Muijtjens, Arno M M

2005-08-01

In health science education clinical clerkships serve the twofold purpose of guiding student learning and assessment of performance. Evidently, both formative and summative assessment procedures are needed in clerkship assessment. In-training evaluation (ITE) has the potential to serve both assessment functions. Implementation of effective ITE, however, has been shown to be problematic, partly because integration of assessment functions may have negative consequences for teaching and learning. This study investigates student perceptions of the impact of an integrated assessment approach, seeking to refine criteria for effective ITE. In the curriculum of Maastricht Midwifery School (MMS), clerkship assessment is based on ITE serving both assessment functions. The ITE model is based on principles of extensive work sampling, and frequent documentation of performance. A focus group technique was used to explore student perceptions on the impact of the ITE approach on student learning and supervisor teaching behaviour, and on the usefulness of information for decision making. Results indicate that the assessment approach is effective in guidance of student learning. Furthermore, students consider the frequent performance documentation essential in clerkship grading. Acceptance and effectivity of ITE requires a learning environment which is safe and respectful. Transparency of assessment processes is the key to success. Suggestions for improvement focus on variation in evaluation formats, improvement of feedback (narrative, complete) and student involvement in assessment. ITE can fulfill both its formative and summative purposes when some crucial conditions are taken into account. Careful training of both supervisors and students in the use of ITE for student learning and performance measurement is essential.

Performance-based tests versus behavioral ratings in the assessment of executive functioning in preschoolers: associations with ADHD symptoms and reading achievement.

PubMed

Miranda, Ana; Colomer, Carla; Mercader, Jessica; Fernández, M Inmaculada; Presentación, M Jesús

2015-01-01

The early assessment of the executive processes using ecologically valid instruments is essential for identifying deficits and planning actions to deal with possible adverse consequences. The present study has two different objectives. The first objective is to analyze the relationship between preschoolers' performance on tests of Working Memory and Inhibition and parents' and teachers' ratings of these executive functions (EFs) using the Behavior Rating Inventory of Executive Function (BRIEF). The second objective consists of studying the predictive value of the different EF measures (performance-based test and rating scales) on Inattention and Hyperactivity/Impulsivity behaviors and on indicators of word reading performance. The participants in the study were 209 children in the last year of preschool, their teachers and their families. Performance-based tests of Working Memory and Inhibition were administered, as well as word reading measures (accuracy and speed). The parents and teachers filled out rating scales of the EF and typical behaviors of attention deficit hyperactivity disorder (ADHD) symptomatology. Moderate correlation values were found between the different EF assessments procedures, although the results varied depending on the different domains. Metacognition Index from the BRIEF presented stronger correlations with verbal working memory tests than with inhibition tests. Both the rating scales and the performance-based tests were significant predictors of Inattention and Hyperactivity/Impulsivity behaviors and the reading achievement measures. However, the BRIEF explained a greater percentage of variance in the case of the ADHD symptomatology, while the performance-based tests explained reading achievement to a greater degree. The implications of the findings for research and clinical practice are discussed.

Sustained effect of simulation-based ultrasound training on clinical performance: a randomized trial

PubMed Central

Tolsgaard, M G; Ringsted, C; Dreisler, E; Nørgaard, L N; Petersen, J H; Madsen, M E; Freiesleben, N L C; Sørensen, J L; Tabor, A

2015-01-01

Objective To study the effect of initial simulation-based transvaginal sonography (TVS) training compared with clinical training only, on the clinical performance of residents in obstetrics and gynecology (Ob-Gyn), assessed 2 months into their residency. Methods In a randomized study, new Ob-Gyn residents (n = 33) with no prior ultrasound experience were recruited from three teaching hospitals. Participants were allocated to either simulation-based training followed by clinical training (intervention group; n = 18) or clinical training only (control group; n = 15). The simulation-based training was performed using a virtual-reality TVS simulator until an expert performance level was attained, and was followed by training on a pelvic mannequin. After 2 months of clinical training, one TVS examination was recorded for assessment of each resident's clinical performance (n = 26). Two ultrasound experts blinded to group allocation rated the scans using the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale. Results During the 2 months of clinical training, participants in the intervention and control groups completed an average ± SD of 58 ± 41 and 63 ± 47 scans, respectively (P = 0.67). In the subsequent clinical performance test, the intervention group achieved higher OSAUS scores than did the control group (mean score, 59.1% vs 37.6%, respectively; P < 0.001). A greater proportion of the intervention group passed a pre-established pass/fail level than did controls (85.7% vs 8.3%, respectively; P < 0.001). Conclusion Simulation-based ultrasound training leads to substantial improvement in clinical performance that is sustained after 2 months of clinical training. © 2015 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of the International Society of Ultrasound in Obstetrics and Gynecology. PMID:25580809

Performance-based tests versus behavioral ratings in the assessment of executive functioning in preschoolers: associations with ADHD symptoms and reading achievement

PubMed Central

Miranda, Ana; Colomer, Carla; Mercader, Jessica; Fernández, M. Inmaculada; Presentación, M. Jesús

2015-01-01

The early assessment of the executive processes using ecologically valid instruments is essential for identifying deficits and planning actions to deal with possible adverse consequences. The present study has two different objectives. The first objective is to analyze the relationship between preschoolers’ performance on tests of Working Memory and Inhibition and parents’ and teachers’ ratings of these executive functions (EFs) using the Behavior Rating Inventory of Executive Function (BRIEF). The second objective consists of studying the predictive value of the different EF measures (performance-based test and rating scales) on Inattention and Hyperactivity/Impulsivity behaviors and on indicators of word reading performance. The participants in the study were 209 children in the last year of preschool, their teachers and their families. Performance-based tests of Working Memory and Inhibition were administered, as well as word reading measures (accuracy and speed). The parents and teachers filled out rating scales of the EF and typical behaviors of attention deficit hyperactivity disorder (ADHD) symptomatology. Moderate correlation values were found between the different EF assessments procedures, although the results varied depending on the different domains. Metacognition Index from the BRIEF presented stronger correlations with verbal working memory tests than with inhibition tests. Both the rating scales and the performance-based tests were significant predictors of Inattention and Hyperactivity/Impulsivity behaviors and the reading achievement measures. However, the BRIEF explained a greater percentage of variance in the case of the ADHD symptomatology, while the performance-based tests explained reading achievement to a greater degree. The implications of the findings for research and clinical practice are discussed. PMID:25972833

A qualitative and quantitative assessment for a bone marrow harvest simulator.

PubMed

Machado, Liliane S; Moraes, Ronei M

2009-01-01

Several approaches to perform assessment in training simulators based on virtual reality have been proposed. There are two kinds of assessment methods: offline and online. The main requirements related to online training assessment methodologies applied to virtual reality systems are the low computational complexity and the high accuracy. In the literature it can be found several approaches for general cases which can satisfy such requirements. An inconvenient about those approaches is related to an unsatisfactory solution for specific cases, as in some medical procedures, where there are quantitative and qualitative information available to perform the assessment. In this paper, we present an approach to online training assessment based on a Modified Naive Bayes which can manipulate qualitative and quantitative variables simultaneously. A special medical case was simulated in a bone marrow harvest simulator. The results obtained were satisfactory and evidenced the applicability of the method.

[The methodological assessment and qualitative evaluation of psychometric performance tests based on the example of modern tests that assess reading and spelling skills].

PubMed

Galuschka, Katharina; Rothe, Josefine; Schulte-Körne, Gerd

2015-09-01

This article looks at a means of objectively evaluating the quality of psychometric tests. This approach enables users to evaluate psychometric tests based on their methodological characteristics, in order to decide which instrument should be used. Reading and spelling assessment tools serve as examples. The paper also provides a review of German psychometric tests for the assessment of reading and spelling skills. This method facilitates the identification of psychometric tests.of high methodological quality which can be used for the assessment of reading and spelling skills. Reading performance should ideally be assessed with the following instruments: ELFE 1-6, LGVT 6-12, LESEN 6-7, LESEN 8-9, or WLLP-R. The tests to be used for the evaluation of spelling skills are DERET 1-2+, DERET 3-4+, WRT 1+, WRT 2+, WRT 3+, WRT 4+ or HSP 1-10.

[Performance assessment employing slide sets: A tool for the classification of senior microscopists from Colombia´s Malaria Control Program].

PubMed

Mendoza, Nohora Marcela; González, Nohora Elizabeth

2015-01-01

One of the most important activities for quality assurance of malaria diagnosis is performance assessment. In Colombia, performance assessment of malaria microscopists has been done through the external performance assessment and indirect external performance assessment programs. To assess the performance of malaria microscopists of public reference laboratories using slide sets, and to describe the methodology used for this purpose. This was a retrospective study to evaluate the concordance of senior microscopists regarding parasite detection, species identification and parasite count based on the results of the assessment of competences using two sets, one comprising 40 slides, and another one with 17 slides. The concordance for parasite detection was 96.9% (95% CI: 96.0-97.5) and 88.7% (95% CI: 86.6-90.5) for species identification. The average percentage of concordant slides in the group evaluated was 89.7% (95% CI: 87.5-91.6). Most of the senior microscopists in Colombia were classified in the two top categories in the performance assessment using slide sets. The most common difficulty encountered was the identification of parasite species. The use of this tool to assess individual performance of microscopists in the evaluation of samples with different degrees of difficulty allows for characterizing the members of the malaria diagnosis network and strengthening the abilities of those who require it.

Detection of Gender-Based Differential Item Functioning in a Mathematics Performance Assessment.

ERIC Educational Resources Information Center

Wang, Ning; Lane, Suzanne

This study used three different differential item functioning (DIF) procedures to examine the extent to which items in a mathematics performance assessment functioned differently for matched gender groups. In addition to examining the appropriateness of individual items in terms of DIF with respect to gender, an attempt was made to identify…

Fish Assemblage Indicators for the National Rivers and Streams Assessment: Performance of model-based vs. traditionally constructed multimetric indices

EPA Science Inventory

The development of multimetric indices (MMIs) for use in assessing the ecological condition of rivers and streams has advanced in recent years with the use of various types of modeling approaches to factor out the influence of natural variability and improve the performance. Ass...

Driving and Low Vision: Validity of Assessments for Predicting Performance of Drivers

ERIC Educational Resources Information Center

Strong, J. Graham; Jutai, Jeffrey W.; Russell-Minda, Elizabeth; Evans, Mal

2008-01-01

The authors conducted a systematic review to examine whether vision-related assessments can predict the driving performance of individuals who have low vision. The results indicate that measures of visual field, contrast sensitivity, cognitive and attention-based tests, and driver screening tools have variable utility for predicting real-world…

The Importance of Visual Experience, Gender, and Emotion in the Assessment of an Assistive Tactile Mouse.

PubMed

Brayda, Luca; Campus, Claudio; Memeo, Mariacarla; Lucagrossi, Laura

2015-01-01

Tactile maps are efficient tools to improve spatial understanding and mobility skills of visually impaired people. Their limited adaptability can be compensated with haptic devices which display graphical information, but their assessment is frequently limited to performance-based metrics only which can hide potential spatial abilities in O&M protocols. We assess a low-tech tactile mouse able to deliver three-dimensional content considering how performance, mental workload, behavior, and anxiety status vary with task difficulty and gender in congenitally blind, late blind, and sighted subjects. Results show that task difficulty coherently modulates the efficiency and difficulty to build mental maps, regardless of visual experience. Although exhibiting attitudes that were similar and gender-independent, the females had lower performance and higher cognitive load, especially when congenitally blind. All groups showed a significant decrease in anxiety after using the device. Tactile graphics with our device seems therefore to be applicable with different visual experiences, with no negative emotional consequences of mentally demanding spatial tasks. Going beyond performance-based assessment, our methodology can help with better targeting technological solutions in orientation and mobility protocols.

A suite of phantom-based test methods for assessing image quality of photoacoustic tomography systems

NASA Astrophysics Data System (ADS)

Vogt, William C.; Jia, Congxian; Wear, Keith A.; Garra, Brian S.; Pfefer, T. Joshua

2017-03-01

As Photoacoustic Tomography (PAT) matures and undergoes clinical translation, objective performance test methods are needed to facilitate device development, regulatory clearance and clinical quality assurance. For mature medical imaging modalities such as CT, MRI, and ultrasound, tissue-mimicking phantoms are frequently incorporated into consensus standards for performance testing. A well-validated set of phantom-based test methods is needed for evaluating performance characteristics of PAT systems. To this end, we have constructed phantoms using a custom tissue-mimicking material based on PVC plastisol with tunable, biologically-relevant optical and acoustic properties. Each phantom is designed to enable quantitative assessment of one or more image quality characteristics including 3D spatial resolution, spatial measurement accuracy, ultrasound/PAT co-registration, uniformity, penetration depth, geometric distortion, sensitivity, and linearity. Phantoms contained targets including high-intensity point source targets and dye-filled tubes. This suite of phantoms was used to measure the dependence of performance of a custom PAT system (equipped with four interchangeable linear array transducers of varying design) on design parameters (e.g., center frequency, bandwidth, element geometry). Phantoms also allowed comparison of image artifacts, including surface-generated clutter and bandlimited sensing artifacts. Results showed that transducer design parameters create strong variations in performance including a trade-off between resolution and penetration depth, which could be quantified with our method. This study demonstrates the utility of phantom-based image quality testing in device performance assessment, which may guide development of consensus standards for PAT systems.

Performance-Based Contracting Within a State Substance Abuse Treatment System: A Preliminary Exploration of Differences in Client Access and Client Outcomes

PubMed Central

Brucker, Debra L.; Stewart, Maureen

2013-01-01

To explore whether the implementation of performance-based contracting (PBC) within the State of Maine’s substance abuse treatment system resulted in improved performance, one descriptive and two empirical analyses were conducted. The first analysis examined utilization and payment structure. The second study was designed to examine whether timeliness of access to outpatient (OP) and intensive outpatient (IOP) substance abuse assessments and treatment, measures that only became available after the implementation of PBC, differed between PBC and non-PBC agencies in the year following implementation of PBC. Using treatment admission records from the state treatment data system (N=9,128), logistic regression models run using generalized equation estimation techniques found no significant difference between PBC agencies and other agencies on timeliness of access to assessments or treatment, for both OP and IOP services. The third analysis, conducted using discharge data from the years prior to and after the implementation of performance-based contracting (N=6,740) for those agencies that became a part of the performance-based contracting system, was designed to assess differences in level of participation, retention, and completion of treatment. Regression models suggest that performance on OP client engagement and retention measures was significantly poorer the year after the implementation of PBC, but that temporal rather than a PBC effects were more significant. No differences were found between years for IOP level of participation or completion of treatment measures. PMID:21249461

Preliminary assessment of the impact of incorporating a detailed algorithm for the effects of nuclear irradiation on combat crew performance into the Janus combat simulation

DOE Office of Scientific and Technical Information (OSTI.GOV)

Warshawsky, A.S.; Uzelac, M.J.; Pimper, J.E.

The Crew III algorithm for assessing time and dose dependent combat crew performance subsequent to nuclear irradiation was incorporated into the Janus combat simulation system. Battle outcomes using this algorithm were compared to outcomes based on the currently used time-independent cookie-cutter'' assessment methodology. The results illustrate quantifiable differences in battle outcome between the two assessment techniques. Results suggest that tactical nuclear weapons are more effective than currently assumed if performance degradation attributed to radiation doses between 150 to 3000 rad are taken into account. 6 refs., 9 figs.

Helicopter simulation validation using flight data

NASA Technical Reports Server (NTRS)

Key, D. L.; Hansen, R. S.; Cleveland, W. B.; Abbott, W. Y.

1982-01-01

A joint NASA/Army effort to perform a systematic ground-based piloted simulation validation assessment is described. The best available mathematical model for the subject helicopter (UH-60A Black Hawk) was programmed for real-time operation. Flight data were obtained to validate the math model, and to develop models for the pilot control strategy while performing mission-type tasks. The validated math model is to be combined with motion and visual systems to perform ground based simulation. Comparisons of the control strategy obtained in flight with that obtained on the simulator are to be used as the basis for assessing the fidelity of the results obtained in the simulator.

Developing integrated benchmarks for DOE performance measurement

DOE Office of Scientific and Technical Information (OSTI.GOV)

Barancik, J.I.; Kramer, C.F.; Thode, Jr. H.C.

1992-09-30

The objectives of this task were to describe and evaluate selected existing sources of information on occupational safety and health with emphasis on hazard and exposure assessment, abatement, training, reporting, and control identifying for exposure and outcome in preparation for developing DOE performance benchmarks. Existing resources and methodologies were assessed for their potential use as practical performance benchmarks. Strengths and limitations of current data resources were identified. Guidelines were outlined for developing new or improved performance factors, which then could become the basis for selecting performance benchmarks. Data bases for non-DOE comparison populations were identified so that DOE performance couldmore » be assessed relative to non-DOE occupational and industrial groups. Systems approaches were described which can be used to link hazards and exposure, event occurrence, and adverse outcome factors, as needed to generate valid, reliable, and predictive performance benchmarks. Data bases were identified which contain information relevant to one or more performance assessment categories . A list of 72 potential performance benchmarks was prepared to illustrate the kinds of information that can be produced through a benchmark development program. Current information resources which may be used to develop potential performance benchmarks are limited. There is need to develop an occupational safety and health information and data system in DOE, which is capable of incorporating demonstrated and documented performance benchmarks prior to, or concurrent with the development of hardware and software. A key to the success of this systems approach is rigorous development and demonstration of performance benchmark equivalents to users of such data before system hardware and software commitments are institutionalized.« less

Assessment of DEMN-based IM Formulations for Octol Replacement

DTIC Science & Technology

2012-08-01

experimentally for performance in this study. The performance was first assessed numerically using the thermochemical equilibrium code Cheetah , v5.0...Fine Grain Octol (FGO). The Cheetah estimates suggest that the proposed formulations will have lower detonation pressure than Octol level performance...Materials Technology Symposium. 3. Fried, L.E., Howard, W.M., Souers, P.C., and Vitello, P.A. Cheetah 5.0, Energetic Materials Center, Lawrence Livermore

The Effect of Interactive Whiteboard-Based Instruction on Mathematics Performance of English Learners

ERIC Educational Resources Information Center

Link, Tammy

2012-01-01

A quasi-experimental research study was conducted to investigate the performance of English learners (ELs) on mathematics assessments when using interactive whiteboard (IWB)-based instruction as compared to text-based instruction. A sample of 47 seventh-and eighth-grade EL students from ABC Middle School (name has been changed) was included in…

The SAFE-T assessment tool: derivation and validation of a web-based application for point-of-care evaluation of gastroenterology fellow performance in colonoscopy.

PubMed

Kumar, Navin L; Kugener, Guillaume; Perencevich, Molly L; Saltzman, John R

2018-01-01

Attending assessment is a critical part of endoscopic education for gastroenterology fellows. The aim of this study was to develop and validate a concise assessment tool to evaluate real-time fellow performance in colonoscopy administered via a web-based application. The Skill Assessment in Fellow Endoscopy Training (SAFE-T) tool was derived as a novel 5-question evaluation tool that captures both summative and formative feedback adapted into a web-based application. A prospective study of 15 gastroenterology fellows (5 fellows each from years 1 to 3 of training) was performed using the SAFE-T tool. An independent reviewer evaluated a subset of these procedures and completed the SAFE-T tool and Mayo Colonoscopy Skills Assessment Tool (MCSAT) for reliability testing. Twenty-six faculty completed 350 SAFE-T evaluations of the 15 fellows in the study. The mean SAFE-T overall score (year 1, 2.00; year 2, 3.84; year 3, 4.28) differentiated each sequential fellow year of training (P < .0001). The mean SAFE-T overall score decreased with increasing case complexity score, with straightforward cases compared with average cases (4.07 vs 3.50, P < .0001), and average cases compared with challenging cases (3.50 vs 3.08, P = .0134). In dual-observed procedures, the SAFE-T tool showed excellent inter-rater reliability with a kappa agreement statistic of 0.898 (P < .0001). Correlation of the SAFE-T overall score with the MCSAT overall hands-on and individual motor scores was excellent (each r > 0.90, P < .0001). We developed and validated the SAFE-T assessment tool, a concise and web-based means of assessing real-time gastroenterology fellow performance in colonoscopy. Copyright © 2018 American Society for Gastrointestinal Endoscopy. Published by Elsevier Inc. All rights reserved.

A Prototype Tool to Enable Farmers to Measure and Improve the Welfare Performance of the Farm Animal Enterprise: The Unified Field Index

PubMed Central

Colditz, Ian G.; Ferguson, Drewe M.; Collins, Teresa; Matthews, Lindsay; Hemsworth, Paul H.

2014-01-01

Simple Summary Benchmarking is a tool widely used in agricultural industries that harnesses the experience of farmers to generate knowledge of practices that lead to better on-farm productivity and performance. We propose, by analogy with production performance, a method for measuring the animal welfare performance of an enterprise and describe a tool for farmers to monitor and improve the animal welfare performance of their business. A general framework is outlined for assessing and monitoring risks to animal welfare based on measures of animals, the environment they are kept in and how they are managed. The tool would enable farmers to continually improve animal welfare. Abstract Schemes for the assessment of farm animal welfare and assurance of welfare standards have proliferated in recent years. An acknowledged short-coming has been the lack of impact of these schemes on the welfare standards achieved on farm due in part to sociological factors concerning their implementation. Here we propose the concept of welfare performance based on a broad set of performance attributes of an enterprise and describe a tool based on risk assessment and benchmarking methods for measuring and managing welfare performance. The tool termed the Unified Field Index is presented in a general form comprising three modules addressing animal, resource, and management factors. Domains within these modules accommodate the principle conceptual perspectives for welfare assessment: biological functioning; emotional states; and naturalness. Pan-enterprise analysis in any livestock sector could be used to benchmark welfare performance of individual enterprises and also provide statistics of welfare performance for the livestock sector. An advantage of this concept of welfare performance is its use of continuous scales of measurement rather than traditional pass/fail measures. Through the feedback provided via benchmarking, the tool should help farmers better engage in on-going improvement of farm practices that affect animal welfare. PMID:26480317

Individual differences in infant fearfulness and cognitive performance: a testing, performance, or competence effect?

PubMed

Rieser-Danner, Loretta A

2003-02-01

The author conducted 2 studies to examine the relations between infant fear and cognitive testing performance in 12-month-old infants. In Study 1, fear was assessed by using 2 standard temperament questionnaires and a laboratory-based, standardized stranger approach. Individual differences in cognitive development were assessed using the Object Permanence Scale of the Infant Psychological Development Scales (I. C. Uzgiris & J. M. Hunt, 1975). All 3 assessments of fear significantly predicted object permanence performance, with correlations ranging from -.32 to -.35. In Study 2, fear was assessed via a maternal report questionnaire, and habituation performance was assessed via a basic-level categorization task. Familiarity with the examiner and with the testing environment was manipulated to test for a familiarity influence on performance. Testing revealed individual differences in both fear and habituation. Results suggest that highly fearful infants required more trials to habituate and were less likely to meet the habituation criterion than infants who were less fearful. Methodological and conceptual implications of these results are discussed.

A probabilistic seismic risk assessment procedure for nuclear power plants: (II) Application

USGS Publications Warehouse

Huang, Y.-N.; Whittaker, A.S.; Luco, N.

2011-01-01

This paper presents the procedures and results of intensity- and time-based seismic risk assessments of a sample nuclear power plant (NPP) to demonstrate the risk-assessment methodology proposed in its companion paper. The intensity-based assessments include three sets of sensitivity studies to identify the impact of the following factors on the seismic vulnerability of the sample NPP, namely: (1) the description of fragility curves for primary and secondary components of NPPs, (2) the number of simulations of NPP response required for risk assessment, and (3) the correlation in responses between NPP components. The time-based assessment is performed as a series of intensity-based assessments. The studies illustrate the utility of the response-based fragility curves and the inclusion of the correlation in the responses of NPP components directly in the risk computation. ?? 2011 Published by Elsevier B.V.

Assessing Conceptual Understanding via Literacy-Infused, Inquiry-Based Science among Middle School English Learners and Economically-Challenged Students

ERIC Educational Resources Information Center

Lara-Alecio, Rafael; Irby, Beverly J.; Tong, Fuhui; Guerrero, Cindy; Koch, Janice; Sutton-Jones, Kara L.

2018-01-01

The overarching purpose of our study was to compare performances of treatment and control condition students who completed a literacy-infused, inquiry-based science intervention through sixth grade as measured by a big idea assessment tool which we refer to as the Big Ideas in Science Assessment (BISA). First, we determine the concurrent validity…

Competitive Debate as Competency-Based Learning: Civic Engagement and Next-Generation Assessment in the Era of the Common Core Learning Standards

ERIC Educational Resources Information Center

McIntosh, Jonathan; Milam, Myra

2016-01-01

As the adoption and execution of the Common Core State Standards (CCSS) have steadily increased, the debate community is presented with an opportunity to be more forward thinking and sustainable through the translation to curriculum planning and next-generation assessment as a movement towards Performance-Based Assessments. This paper focuses on…

Performance Equivalency between Computer-Based and Traditional Pen-and-Paper Assessment: A Case Study in Clinical Anatomy

ERIC Educational Resources Information Center

Guimarães, Bruno; Ribeiro, José; Cruz, Bernardo; Ferreira, André; Alves, Hélio; Cruz-Correia, Ricardo; Madeira, Maria Dulce; Ferreira, Maria Amélia

2018-01-01

The time, material, and staff-consuming nature of anatomy's traditional pen-and-paper assessment system, the increase in the number of students enrolling in medical schools and the ever-escalating workload of academic staff have made the use of computer-based assessment (CBA) an attractive proposition. To understand the impact of such shift in the…

Base Rates of Social Skills Acquisition/Performance Deficits, Strengths, and Problem Behaviors: An Analysis of the Social Skills Improvement System-Rating Scales

ERIC Educational Resources Information Center

Gresham, Frank M.; Elliott, Stephen N.; Kettler, Ryan J.

2010-01-01

Base rate information is important in clinical assessment because one cannot know how unusual or typical a phenomenon is without first knowing its base rate in the population. This study empirically determined the base rates of social skills acquisition and performance deficits, social skills strengths, and problem behaviors using a nationally…

Web-Based vs. Paper-Based Homework to Evaluate Students' Performance in Introductory Physics Courses and Students' Perceptions: Two Years Experience

ERIC Educational Resources Information Center

Demirci, Neset

2010-01-01

The main aim of this study was to assess and compare undergraduate students' homework performance using a web-based testing system with paper-based, hand-graded one in introductory physics courses. Students' perceptions about each method were then investigated. Every semester during the two-year period, one of the two identical sections of…

DISTRIBUTED GRID-CONNECTED PHOTOVOLTAIC POWER SYSTEM EMISSION OFFSET ASSESSMENT: STATISTICAL TEST OF SIMULATED- AND MEASURED-BASED DATA

EPA Science Inventory

This study assessed the pollutant emission offset potential of distributed grid-connected photovoltaic (PV) power systems. Computer-simulated performance results were utilized for 211 PV systems located across the U.S. The PV systems' monthly electrical energy outputs were based ...

Reliability and validity of play-based assessments of motor and cognitive skills for infants and young children: a systematic review.

PubMed

O'Grady, Michael G; Dusing, Stacey C

2015-01-01

Play is vital for development. Infants and children learn through play. Traditional standardized developmental tests measure whether a child performs individual skills within controlled environments. Play-based assessments can measure skill performance during natural, child-driven play. The purpose of this study was to systematically review reliability, validity, and responsiveness of all play-based assessments that quantify motor and cognitive skills in children from birth to 36 months of age. Studies were identified from a literature search using PubMed, ERIC, CINAHL, and PsycINFO databases and the reference lists of included papers. Included studies investigated reliability, validity, or responsiveness of play-based assessments that measured motor and cognitive skills for children to 36 months of age. Two reviewers independently screened 40 studies for eligibility and inclusion. The reviewers independently extracted reliability, validity, and responsiveness data. They examined measurement properties and methodological quality of the included studies. Four current play-based assessment tools were identified in 8 included studies. Each play-based assessment tool measured motor and cognitive skills in a different way during play. Interrater reliability correlations ranged from .86 to .98 for motor development and from .23 to .90 for cognitive development. Test-retest reliability correlations ranged from .88 to .95 for motor development and from .45 to .91 for cognitive development. Structural validity correlations ranged from .62 to .90 for motor development and from .42 to .93 for cognitive development. One study assessed responsiveness to change in motor development. Most studies had small and poorly described samples. Lack of transparency in data management and statistical analysis was common. Play-based assessments have potential to be reliable and valid tools to assess cognitive and motor skills, but higher-quality research is needed. Psychometric properties should be considered for each play-based assessment before it is used in clinical and research practice. © 2015 American Physical Therapy Association.

Evaluation of an interactive, case-based review session in teaching medical microbiology.

PubMed

Blewett, Earl L; Kisamore, Jennifer L

2009-08-27

Oklahoma State University-Center for Health Sciences (OSU-CHS) has replaced its microbiology wet laboratory with a variety of tutorials including a case-based interactive session called Microbial Jeopardy!. The question remains whether the time spent by students and faculty in the interactive case-based tutorial is worthwhile? This study was designed to address this question by analyzing both student performance data and assessing students' perceptions regarding the tutorial. Both quantitative and qualitative data were used in the current study. Part One of the study involved assessing student performance using archival records of seven case-based exam questions used in the 2004, 2005, 2006, and 2007 OSU-CHS Medical Microbiology course. Two sample t-tests for proportions were used to test for significant differences related to tutorial usage. Part Two used both quantitative and qualitative means to assess student's perceptions of the Microbial Jeopardy! session. First, a retrospective survey was administered to students who were enrolled in Medical Microbiology in 2006 or 2007. Second, responses to open-ended items from the 2008 course evaluations were reviewed for comments regarding the Microbial Jeopardy! session. Both student performance and student perception data support continued use of the tutorials. Quantitative and qualitative data converge to suggest that students like and learn from the interactive, case-based session. The case-based tutorial appears to improve student performance on case-based exam questions. Additionally, students perceived the tutorial as helpful in preparing for exam questions and reviewing the course material. The time commitment for use of the case-based tutorial appears to be justified.

Evaluation of an interactive, case-based review session in teaching medical microbiology

PubMed Central

Blewett, Earl L; Kisamore, Jennifer L

2009-01-01

Background Oklahoma State University-Center for Health Sciences (OSU-CHS) has replaced its microbiology wet laboratory with a variety of tutorials including a case-based interactive session called Microbial Jeopardy!. The question remains whether the time spent by students and faculty in the interactive case-based tutorial is worthwhile? This study was designed to address this question by analyzing both student performance data and assessing students' perceptions regarding the tutorial. Methods Both quantitative and qualitative data were used in the current study. Part One of the study involved assessing student performance using archival records of seven case-based exam questions used in the 2004, 2005, 2006, and 2007 OSU-CHS Medical Microbiology course. Two sample t-tests for proportions were used to test for significant differences related to tutorial usage. Part Two used both quantitative and qualitative means to assess student's perceptions of the Microbial Jeopardy! session. First, a retrospective survey was administered to students who were enrolled in Medical Microbiology in 2006 or 2007. Second, responses to open-ended items from the 2008 course evaluations were reviewed for comments regarding the Microbial Jeopardy! session. Results Both student performance and student perception data support continued use of the tutorials. Quantitative and qualitative data converge to suggest that students like and learn from the interactive, case-based session. Conclusion The case-based tutorial appears to improve student performance on case-based exam questions. Additionally, students perceived the tutorial as helpful in preparing for exam questions and reviewing the course material. The time commitment for use of the case-based tutorial appears to be justified. PMID:19712473

Photometer Performance Assessment in Kepler Science Data Processing

NASA Technical Reports Server (NTRS)

Li, Jie; Allen, Christopher; Bryson, Stephen T.; Caldwell, Douglas A.; Chandrasekaran, Hema; Clarke, Bruce D.; Gunter, Jay P.; Jenkins, Jon M.; Klaus, Todd C.; Quintana, Elisa V.;

The grout/glass performance assessment code system (GPACS) with verification and benchmarking

DOE Office of Scientific and Technical Information (OSTI.GOV)

Piepho, M.G.; Sutherland, W.H.; Rittmann, P.D.

1994-12-01

GPACS is a computer code system for calculating water flow (unsaturated or saturated), solute transport, and human doses due to the slow release of contaminants from a waste form (in particular grout or glass) through an engineered system and through a vadose zone to an aquifer, well and river. This dual-purpose document is intended to serve as a user`s guide and verification/benchmark document for the Grout/Glass Performance Assessment Code system (GPACS). GPACS can be used for low-level-waste (LLW) Glass Performance Assessment and many other applications including other low-level-waste performance assessments and risk assessments. Based on all the cses presented, GPACSmore » is adequate (verified) for calculating water flow and contaminant transport in unsaturated-zone sediments and for calculating human doses via the groundwater pathway.« less

A Constructive Response to the APU.

ERIC Educational Resources Information Center

Galton, Maurice

1979-01-01

Suggesting that, like the National Assessment of Educational Progress (NAEP) in the United States, Britain's Assessment of Performance Unit is likely to evoke opposition, the author proposes a teacher-based assessment system as an alternative. (SJL) Aspect of National Assessment (NAEP) dealt with in this document: Procedures (Evaluation).

Assessing family medicine trainees--what can we learn from the European neighbours?

PubMed

Flum, Elisabeth; Maagaard, Roar; Godycki-Cwirko, Maciek; Scarborough, Nigel; Scherpbier, Nynke; Ledig, Thomas; Roos, Marco; Steinhäuser, Jost

2015-01-01

Although demands on family physicians (FP) are to a large extent similar in the European Union, uniform assessment standards for family medicine (FM) specialty training and assessment do not exist. Aim of this pilot study was to elicit and compare the different modalities and assessment methods of FM specialty training in five European countries. A semi structured survey was undertaken based on a convenient sample in five European countries (Denmark, Germany, Poland, the Netherlands and the United Kingdom). The respondents were asked to respond to ten items about aspects of FM specialty training and assessment methods in their respective countries. If available, this data was completed with information from official websites of the countries involved. FM specialty training is performed heterogeneously in the surveyed countries. Training time periods range from three to five years, in some countries requiring a foundation program of up to two years. Most countries perform longitudinal assessment during FM specialty training using a combination of competence-based approach with additional formative and summative assessment. There is some evidence on the assessments methods used, however the assessment method used and costs of assessment differs remarkably between the participating countries. Longitudinal and competence-based assessment is the presently preferred approach for FM specialty training. Countries which use less multifaceted methods for assessment could learn from best practice. Potential changes have significant cost implications.

Implementation of Performance Assessment in STEM (Science, Technology, Engineering, Mathematics) Education to Detect Science Process Skill

NASA Astrophysics Data System (ADS)

Septiani, A.; Rustaman, N. Y.

2017-02-01

A descriptive study about the implementation of performance assessment in STEM based instruction was carried out to investigate the tenth grade of Vocational school students’ science process skills during the teaching learning processes. A number of tenth grade agriculture students was involved as research subjects selected through cluster random sampling technique (n=35). Performance assessment was planned on skills during the teaching learning process through observation and on product resulted from their engineering practice design. The procedure conducted in this study included thinking phase (identifying problem and sharing idea), designing phase, construction phase, and evaluation phase. Data was collected through the use of science process skills (SPS) test, observation sheet on student activity, as well as tasks and rubrics for performance assessment during the instruction. Research findings show that the implementation of performance assessment in STEM education in planting media could detect students science process skills better from the observation individually compared through SPS test. It was also found that the result of performance assessment was diverse when it was correlated to each indicator of SPS (strong and positive; weak and positive).

Performance-Based Education Project: A Component of the Institutional Outcomes Model. Course Prototype. U.S. History 121 (HIS121).

ERIC Educational Resources Information Center

John Wood Community Coll., Quincy, IL.

This document is an assessment of a performance-based education project that involved a United States history course, offered at a community college. Thirty-two student performance objectives are outlined and lesson plans designed to achieve each objective are presented. Each lesson plan consists of an instructional topic, prerequisites, interest…

Anxiety and performance of nursing students in regard to assessment via clinical simulations in the classroom versus filmed assessments.

PubMed

de Souza Teixeira, Carla Regina; Kusumota, Luciana; Alves Pereira, Marta Cristiane; Merizio Martins Braga, Fernanda Titareli; Pirani Gaioso, Vanessa; Mara Zamarioli, Cristina; Campos de Carvalho, Emilia

2014-01-01

To compare the level of anxiety and performance of nursing students when performing a clinical simulation through the traditional method of assessment with the presence of an evaluator and through a filmed assessment without the presence of an evaluator. Controlled trial with the participation of Brazilian public university 20 students who were randomly assigned to one of two groups: a) assessment through the traditional method with the presence of an evaluator; or b) filmed assessment. The level of anxiety was assessed using the Zung test and performance was measured based on the number of correct answers. Averages of 32 and 27 were obtained on the anxiety scale by the group assessed through the traditional method before and after the simulation, respectively, while the filmed group obtained averages of 33 and 26; the final scores correspond to mild anxiety. Even though there was a statistically significant reduction in the intra-groups scores before and after the simulation, there was no difference between the groups. As for the performance assessments in the clinical simulation, the groups obtained similar percentages of correct answers (83% in the traditional assessment and 84% in the filmed assessment) without statistically significant differences. Filming can be used and encouraged as a strategy to assess nursing undergraduate students.

Utility of Combining a Simulation-Based Method With a Lecture-Based Method for Fundoscopy Training in Neurology Residency.

PubMed

Gupta, Deepak K; Khandker, Namir; Stacy, Kristin; Tatsuoka, Curtis M; Preston, David C

2017-10-01

Fundoscopic examination is an essential component of the neurologic examination. Competence in its performance is mandated as a required clinical skill for neurology residents by the American Council of Graduate Medical Education. Government and private insurance agencies require its performance and documentation for moderate- and high-level neurologic evaluations. Traditionally, assessment and teaching of this key clinical examination technique have been difficult in neurology residency training. To evaluate the utility of a simulation-based method and the traditional lecture-based method for assessment and teaching of fundoscopy to neurology residents. This study was a prospective, single-blinded, education research study of 48 neurology residents recruited from July 1, 2015, through June 30, 2016, at a large neurology residency training program. Participants were equally divided into control and intervention groups after stratification by training year. Baseline and postintervention assessments were performed using questionnaire, survey, and fundoscopy simulators. After baseline assessment, both groups initially received lecture-based training, which covered fundamental knowledge on the components of fundoscopy and key neurologic findings observed on fundoscopic examination. The intervention group additionally received simulation-based training, which consisted of an instructor-led, hands-on workshop that covered practical skills of performing fundoscopic examination and identifying neurologically relevant findings on another fundoscopy simulator. The primary outcome measures were the postintervention changes in fundoscopy knowledge, skills, and total scores. A total of 30 men and 18 women were equally distributed between the 2 groups. The intervention group had significantly higher mean (SD) increases in skills (2.5 [2.3] vs 0.8 [1.8], P = .01) and total (9.3 [4.3] vs 5.3 [5.8], P = .02) scores compared with the control group. Knowledge scores (6.8 [3.3] vs 4.5 [4.9], P = .11) increased nonsignificantly in both groups. This study supports the use of a simulation-based method as a supplementary tool to the lecture-based method in the assessment and teaching of fundoscopic examination in neurology residency.

Incorporating Team-Based Learning Into a Physician Assistant Clinical Pharmacology Course.

PubMed

Nguyen, Timothy; Wong, Elaine; Pham, Antony

2016-03-01

To obtain student perceptions of team-based learning and compare the effectiveness of team-based learning and traditional lecture formats in a clinical pharmacology course for physician assistant (PA) students. Clinical pharmacology is a course offered to PA students in their first year of training at LIU Brooklyn, Brooklyn, NY. In spring 2014, half of the course was offered in a traditional lecture format and the remaining half was offered in a team-based learning format. The team-based learning format had 3 components: (1) prereading assignments, (2) individual readiness assessment tests, and (3) team readiness assessment tests. So that student perceptions of the integration of team-based learning activities into the course could be evaluated, presurveys and postsurveys were administered. The effectiveness of team-based learning was evaluated by comparing overall student performance with student performance in the preceding year. Thirty-three students were enrolled in the course and completed the presurveys and postsurveys. The survey results are presented in Table 1. Comparison of student performance on examinations with performance from the previous year showed similar outcomes. Incorporating a team-based learning pedagogical approach in the PA pharmacology course yielded similar examination results to those of traditional lecture formats. Presurvey and postsurvey questionnaires yielded various student perceptions of team-based learning.

School-based assessment of attention deficit-hyperactivity disorder.

PubMed

Atkins, M S; Pelham, W E

1991-04-01

Schools are uniquely situated to provide information relevant to the assessment and treatment of Attention Deficit-Hyperactivity Disorder (ADHD) across a variety of tasks, settings, and observers. The importance of the school setting to the assessment and treatment of ADHD has resulted in the development of numerous measures appropriate for such assessment, but few guidelines for their use. The purpose of this article is to describe school-based assessment procedures and their relationship to ADHD. These procedures include teacher rating measures, direct observations in classrooms and on playgrounds, peer rating and sociometric measures, and permanent product measures such as academic performance. It is recommended that school-based assessment of ADHD involve the concurrent use of multiple measures to assess adequately the wide variety of symptoms associated with this disorder. It is further recommended that assessment serve the purpose of determining treatment efficacy to avoid unwarranted preoccupation with diagnostic issues that are as yet unresolved. The components of a comprehensive school-based assessment of ADHD are described, and their relationship to school-based treatment is discussed.

A systematic review of surgical skills transfer after simulation-based training: laparoscopic cholecystectomy and endoscopy.

PubMed

Dawe, Susan R; Windsor, John A; Broeders, Joris A J L; Cregan, Patrick C; Hewett, Peter J; Maddern, Guy J

2014-02-01

A systematic review to determine whether skills acquired through simulation-based training transfer to the operating room for the procedures of laparoscopic cholecystectomy and endoscopy. Simulation-based training assumes that skills are directly transferable to the operation room, but only a few studies have investigated the effect of simulation-based training on surgical performance. A systematic search strategy that was used in 2006 was updated to retrieve relevant studies. Inclusion of articles was determined using a predetermined protocol, independent assessment by 2 reviewers, and a final consensus decision. Seventeen randomized controlled trials and 3 nonrandomized comparative studies were included in this review. In most cases, simulation-based training was in addition to patient-based training programs. Only 2 studies directly compared simulation-based training in isolation with patient-based training. For laparoscopic cholecystectomy (n = 10 studies) and endoscopy (n = 10 studies), participants who reached simulation-based skills proficiency before undergoing patient-based assessment performed with higher global assessment scores and fewer errors in the operating room than their counterparts who did not receive simulation training. Not all parameters measured were improved. Two of the endoscopic studies compared simulation-based training in isolation with patient-based training with different results: for sigmoidoscopy, patient-based training was more effective, whereas for colonoscopy, simulation-based training was equally effective. Skills acquired by simulation-based training seem to be transferable to the operative setting for laparoscopic cholecystectomy and endoscopy. Future research will strengthen these conclusions by evaluating predetermined competency levels on the same simulators and using objective validated global rating scales to measure operative performance.

Engineered Barrier System performance requirements systems study report. Revision 02

DOE Office of Scientific and Technical Information (OSTI.GOV)

Balady, M.A.

This study evaluates the current design concept for the Engineered Barrier System (EBS), in concert with the current understanding of the geologic setting to assess whether enhancements to the required performance of the EBS are necessary. The performance assessment calculations are performed by coupling the EBS with the geologic setting based on the models (some of which were updated for this study) and assumptions used for the 1995 Total System Performance Assessment (TSPA). The need for enhancements is determined by comparing the performance assessment results against the EBS related performance requirements. Subsystem quantitative performance requirements related to the EBS includemore » the requirement to allow no more than 1% of the waste packages (WPs) to fail before 1,000 years after permanent closure of the repository, as well as a requirement to control the release rate of radionuclides from the EBS. The EBS performance enhancements considered included additional engineered components as well as evaluating additional performance available from existing design features but for which no performance credit is currently being taken.« less

48 CFR 37.601 - General.

Code of Federal Regulations, 2010 CFR

2010-10-01

... a performance work statement or a statement of objectives (see 37.602). (b) Performance-based contracts for services shall include— (1) A performance work statement (PWS); (2) Measurable performance standards (i.e., in terms of quality, timeliness, quantity, etc.) and the method of assessing contractor...

48 CFR 37.601 - General.

Code of Federal Regulations, 2012 CFR

2012-10-01

... a performance work statement or a statement of objectives (see 37.602). (b) Performance-based contracts for services shall include— (1) A performance work statement (PWS); (2) Measurable performance standards (i.e., in terms of quality, timeliness, quantity, etc.) and the method of assessing contractor...

48 CFR 37.601 - General.

Code of Federal Regulations, 2014 CFR

2014-10-01

... a performance work statement or a statement of objectives (see 37.602). (b) Performance-based contracts for services shall include— (1) A performance work statement (PWS); (2) Measurable performance standards (i.e., in terms of quality, timeliness, quantity, etc.) and the method of assessing contractor...

Informant Report of Financial Capacity for Individuals With Chronic Acquired Brain Injury: An Assessment of Informant Accuracy.

PubMed

Sunderaraman, Preeti; Cosentino, Stephanie; Lindgren, Karen; James, Angela; Schultheis, Maria

2018-03-29

Primarily, to investigate the association between informant report and objective performance on specific financial capacity (FC) tasks by adults with chronic, moderate to severe acquired brain injury, and to examine the nature of misestimates by the informants. Cross-sectional design. A postacute, community-based rehabilitation center. Data were obtained from 22 chronic acquired brain injury (CABI) adults, mean age of 46.6 years (SD = 8.67), mean years of education of 13.45 years (SD = 2.15), with moderate to severe acquired brain injury (86% had traumatic brain injury), with a mean postinjury period of 17.14 years (SD = 9.5). Whereas the CABI adults completed the Financial Competence Assessment Inventory interview-a combination of self-report and performance-based assessment, 22 informants completed a specifically designed parallel version of the interview. Pearson correlations and 1-sample t tests based on the discrepancy scores between informant report and CABI group's performance were used. The CABI group's performance was not associated with its informant's perceptions. One-sample t tests revealed that informants both underestimated and overestimated CABI group's performance. Results indicate lack of correspondence between self- and informant ratings. Further investigation revealed that misestimations by informants occurred in contrary directions with CABI adults' performance being inaccurately rated. These findings raise critical issues related to assuming that the informant report can be used as a "gold standard" for collecting functional data related to financial management, and the idea that obtaining objective data on financial tasks may represent a more valid method of assessing financial competency in adults with brain injury.

Performance-based comparison of neonatal intubation training outcomes: simulator and live animal.

PubMed

Andreatta, Pamela B; Klotz, Jessica J; Dooley-Hash, Suzanne L; Hauptman, Joe G; Biddinger, Bea; House, Joseph B

2015-02-01

The purpose of this article was to establish psychometric validity evidence for competency assessment instruments and to evaluate the impact of 2 forms of training on the abilities of clinicians to perform neonatal intubation. To inform the development of assessment instruments, we conducted comprehensive task analyses including each performance domain associated with neonatal intubation. Expert review confirmed content validity. Construct validity was established using the instruments to differentiate between the intubation performance abilities of practitioners (N = 294) with variable experience (novice through expert). Training outcomes were evaluated using a quasi-experimental design to evaluate performance differences between 294 subjects randomly assigned to 1 of 2 training groups. The training intervention followed American Heart Association Pediatric Advanced Life Support and Neonatal Resuscitation Program protocols with hands-on practice using either (1) live feline or (2) simulated feline models. Performance assessment data were captured before and directly following the training. All data were analyzed using analysis of variance with repeated measures and statistical significance set at P < .05. Content validity, reliability, and consistency evidence were established for each assessment instrument. Construct validity for each assessment instrument was supported by significantly higher scores for subjects with greater levels of experience, as compared with those with less experience (P = .000). Overall, subjects performed significantly better in each assessment domain, following the training intervention (P = .000). After controlling for experience level, there were no significant differences among the cognitive, performance, and self-efficacy outcomes between clinicians trained with live animal model or simulator model. Analysis of retention scores showed that simulator trained subjects had significantly higher performance scores after 18 weeks (P = .01) and 52 weeks (P = .001) and cognitive scores after 52 weeks (P = .001). The results of this study demonstrate the feasibility of using valid, reliable assessment instruments to assess clinician competency and self-efficacy in the performance of neonatal intubation. We demonstrated the relative equivalency of live animal and simulation-based models as tools to support acquisition of neonatal intubation skills. Retention of performance abilities was greater for subjects trained using the simulator, likely because it afforded greater opportunity for repeated practice. Outcomes in each assessment area were influenced by the previous intubation experience of participants. This suggests that neonatal intubation training programs could be tailored to the level of provider experience to make efficient use of time and educational resources. Future research focusing on the uses of assessment in the applied clinical environment, as well as identification of optimal training cycles for performance retention, is merited.

Development, construct validity and test-retest reliability of a field-based wheelchair mobility performance test for wheelchair basketball.

PubMed

de Witte, Annemarie M H; Hoozemans, Marco J M; Berger, Monique A M; van der Slikke, Rienk M A; van der Woude, Lucas H V; Veeger, Dirkjan H E J

2018-01-01

The aim of this study was to develop and describe a wheelchair mobility performance test in wheelchair basketball and to assess its construct validity and reliability. To mimic mobility performance of wheelchair basketball matches in a standardised manner, a test was designed based on observation of wheelchair basketball matches and expert judgement. Forty-six players performed the test to determine its validity and 23 players performed the test twice for reliability. Independent-samples t-tests were used to assess whether the times needed to complete the test were different for classifications, playing standards and sex. Intraclass correlation coefficients (ICC) were calculated to quantify reliability of performance times. Males performed better than females (P < 0.001, effect size [ES] = -1.26) and international men performed better than national men (P < 0.001, ES = -1.62). Performance time of low (≤2.5) and high (≥3.0) classification players was borderline not significant with a moderate ES (P = 0.06, ES = 0.58). The reliability was excellent for overall performance time (ICC = 0.95). These results show that the test can be used as a standardised mobility performance test to validly and reliably assess the capacity in mobility performance of elite wheelchair basketball athletes. Furthermore, the described methodology of development is recommended for use in other sports to develop sport-specific tests.

Massive integration of diverse protein quality assessment methods to improve template based modeling in CASP11.

PubMed

Cao, Renzhi; Bhattacharya, Debswapna; Adhikari, Badri; Li, Jilong; Cheng, Jianlin

2016-09-01

Model evaluation and selection is an important step and a big challenge in template-based protein structure prediction. Individual model quality assessment methods designed for recognizing some specific properties of protein structures often fail to consistently select good models from a model pool because of their limitations. Therefore, combining multiple complimentary quality assessment methods is useful for improving model ranking and consequently tertiary structure prediction. Here, we report the performance and analysis of our human tertiary structure predictor (MULTICOM) based on the massive integration of 14 diverse complementary quality assessment methods that was successfully benchmarked in the 11th Critical Assessment of Techniques of Protein Structure prediction (CASP11). The predictions of MULTICOM for 39 template-based domains were rigorously assessed by six scoring metrics covering global topology of Cα trace, local all-atom fitness, side chain quality, and physical reasonableness of the model. The results show that the massive integration of complementary, diverse single-model and multi-model quality assessment methods can effectively leverage the strength of single-model methods in distinguishing quality variation among similar good models and the advantage of multi-model quality assessment methods of identifying reasonable average-quality models. The overall excellent performance of the MULTICOM predictor demonstrates that integrating a large number of model quality assessment methods in conjunction with model clustering is a useful approach to improve the accuracy, diversity, and consequently robustness of template-based protein structure prediction. Proteins 2016; 84(Suppl 1):247-259. © 2015 Wiley Periodicals, Inc. © 2015 Wiley Periodicals, Inc.

Team-based assessment of professional behavior in medical students

PubMed Central

RAEE, HOJAT; AMINI, MITRA; MOMEN NASAB, AMENEH; MALEK POUR, ABDOLRASOUL; JAFARI, MOHAMMAD MORAD

2014-01-01

Introduction: Self and peer assessment provides important information about the individual’s performance and behavior in all aspects of their professional environment work. The aim of this study is to evaluate the professional behavior and performance in medical students in the form of team based assessment. Methods: In a cross-sectional study, 100 medical students in the 7th year of education were randomly selected and enrolled; for each student five questionnaires were filled out, including one self-assessment, two peer assessments and two residents assessment. The scoring system of the questionnaires was based on seven point Likert scale.  After filling out the questions in the questionnaire, numerical data and written comments provided to the students were collected, analyzed and discussed. Internal consistency (Cronbach’s alpha) of the questionnaires was assessed. A p<0.05 was considered as significant level. Results: Internal consistency was acceptable (Cronbach’s alpha 0.83). Interviews revealed that the majority of students and assessors interviewed found the method acceptable. The range of scores was 1-6 (Mean±SD=4.39±0.57) for the residents' assessment, 2-6 (Mean±SD= 4.49±0.53) for peer assessment, and 3-7 (Mean±SD=5.04±0.32) for self-assessment. There was a significant difference between self assessment and other methods of assessment. Conclusions: This study demonstrates that a team-based assessment is an acceptable and feasible method for peer and self-assessment of medical students’ learning in a clinical clerkship, and has some advantages over traditional assessment methods. Further studies are needed to focus on the strengths and weaknesses. PMID:25512933

Task-based exposure assessment of nanoparticles in the workplace

NASA Astrophysics Data System (ADS)

Ham, Seunghon; Yoon, Chungsik; Lee, Euiseung; Lee, Kiyoung; Park, Donguk; Chung, Eunkyo; Kim, Pilje; Lee, Byoungcheun

2012-09-01

Although task-based sampling is, theoretically, a plausible approach to the assessment of nanoparticle exposure, few studies using this type of sampling have been published. This study characterized and compared task-based nanoparticle exposure profiles for engineered nanoparticle manufacturing workplaces (ENMW) and workplaces that generated welding fumes containing incidental nanoparticles. Two ENMW and two welding workplaces were selected for exposure assessments. Real-time devices were utilized to characterize the concentration profiles and size distributions of airborne nanoparticles. Filter-based sampling was performed to measure time-weighted average (TWA) concentrations, and off-line analysis was performed using an electron microscope. Workplace tasks were recorded by researchers to determine the concentration profiles associated with particular tasks/events. This study demonstrated that exposure profiles differ greatly in terms of concentrations and size distributions according to the task performed. The size distributions recorded during tasks were different from both those recorded during periods with no activity and from the background. The airborne concentration profiles of the nanoparticles varied according to not only the type of workplace but also the concentration metrics. The concentrations measured by surface area and the number concentrations measured by condensation particle counter, particulate matter 1.0, and TWA mass concentrations all showed a similar pattern, whereas the number concentrations measured by scanning mobility particle sizer indicated that the welding fume concentrations at one of the welding workplaces were unexpectedly higher than were those at workplaces that were engineering nanoparticles. This study suggests that a task-based exposure assessment can provide useful information regarding the exposure profiles of nanoparticles and can therefore be used as an exposure assessment tool.

Validation of the updated ArthroS simulator: face and construct validity of a passive haptic virtual reality simulator with novel performance metrics.

PubMed

Garfjeld Roberts, Patrick; Guyver, Paul; Baldwin, Mathew; Akhtar, Kash; Alvand, Abtin; Price, Andrew J; Rees, Jonathan L

2017-02-01

To assess the construct and face validity of ArthroS, a passive haptic VR simulator. A secondary aim was to evaluate the novel performance metrics produced by this simulator. Two groups of 30 participants, each divided into novice, intermediate or expert based on arthroscopic experience, completed three separate tasks on either the knee or shoulder module of the simulator. Performance was recorded using 12 automatically generated performance metrics and video footage of the arthroscopic procedures. The videos were blindly assessed using a validated global rating scale (GRS). Participants completed a survey about the simulator's realism and training utility. This new simulator demonstrated construct validity of its tasks when evaluated against a GRS (p ≤ 0.003 in all cases). Regarding it's automatically generated performance metrics, established outputs such as time taken (p ≤ 0.001) and instrument path length (p ≤ 0.007) also demonstrated good construct validity. However, two-thirds of the proposed 'novel metrics' the simulator reports could not distinguish participants based on arthroscopic experience. Face validity assessment rated the simulator as a realistic and useful tool for trainees, but the passive haptic feedback (a key feature of this simulator) is rated as less realistic. The ArthroS simulator has good task construct validity based on established objective outputs, but some of the novel performance metrics could not distinguish between surgical experience. The passive haptic feedback of the simulator also needs improvement. If simulators could offer automated and validated performance feedback, this would facilitate improvements in the delivery of training by allowing trainees to practise and self-assess.

Assessing the welfare of laboratory mice in their home environment using animal-based measures--a benchmarking tool.

PubMed

Spangenberg, Elin M F; Keeling, Linda J

2016-02-01

Welfare problems in laboratory mice can be a consequence of an ongoing experiment, or a characteristic of a particular genetic line, but in some cases, such as breeding animals, they are most likely to be a result of the design and management of the home cage. Assessment of the home cage environment is commonly performed using resource-based measures, like access to nesting material. However, animal-based measures (related to the health status and behaviour of the animals) can be used to assess the current welfare of animals regardless of the inputs applied (i.e. the resources or management). The aim of this study was to design a protocol for assessing the welfare of laboratory mice using only animal-based measures. The protocol, to be used as a benchmarking tool, assesses mouse welfare in the home cage and does not contain parameters related to experimental situations. It is based on parameters corresponding to the 12 welfare criteria established by the Welfare Quality® project. Selection of animal-based measures was performed by scanning existing published, web-based and informal protocols, and by choosing parameters that matched these criteria, were feasible in practice and, if possible, were already validated indicators of mouse welfare. The parameters should identify possible animal welfare problems and enable assessment directly in an animal room during cage cleaning procedures, without the need for extra equipment. Thermal comfort behaviours and positive emotional states are areas where more research is needed to find valid, reliable and feasible animal-based measures. © The Author(s) 2015.

Differential Performance by English Language Learners on an Inquiry-Based Science Assessment

NASA Astrophysics Data System (ADS)

Turkan, Sultan; Liu, Ou Lydia

2012-10-01

The performance of English language learners (ELLs) has been a concern given the rapidly changing demographics in US K-12 education. This study aimed to examine whether students' English language status has an impact on their inquiry science performance. Differential item functioning (DIF) analysis was conducted with regard to ELL status on an inquiry-based science assessment, using a multifaceted Rasch DIF model. A total of 1,396 seventh- and eighth-grade students took the science test, including 313 ELL students. The results showed that, overall, non-ELLs significantly outperformed ELLs. Of the four items that showed DIF, three favored non-ELLs while one favored ELLs. The item that favored ELLs provided a graphic representation of a science concept within a family context. There is some evidence that constructed-response items may help ELLs articulate scientific reasoning using their own words. Assessment developers and teachers should pay attention to the possible interaction between linguistic challenges and science content when designing assessment for and providing instruction to ELLs.

Performance metrics for the assessment of satellite data products: an ocean color case study

PubMed Central

Seegers, Bridget N.; Stumpf, Richard P.; Schaeffer, Blake A.; Loftin, Keith A.; Werdell, P. Jeremy

2018-01-01

Performance assessment of ocean color satellite data has generally relied on statistical metrics chosen for their common usage and the rationale for selecting certain metrics is infrequently explained. Commonly reported statistics based on mean squared errors, such as the coefficient of determination (r2), root mean square error, and regression slopes, are most appropriate for Gaussian distributions without outliers and, therefore, are often not ideal for ocean color algorithm performance assessment, which is often limited by sample availability. In contrast, metrics based on simple deviations, such as bias and mean absolute error, as well as pair-wise comparisons, often provide more robust and straightforward quantities for evaluating ocean color algorithms with non-Gaussian distributions and outliers. This study uses a SeaWiFS chlorophyll-a validation data set to demonstrate a framework for satellite data product assessment and recommends a multi-metric and user-dependent approach that can be applied within science, modeling, and resource management communities. PMID:29609296

Teachers' Views on Performance-Based Assessments.

ERIC Educational Resources Information Center

King, Bruce

1991-01-01

Provides personal perspectives from teachers about the prospects and problems illuminated by Stanford University's Teacher Assessment Project. Teacher remarks address the portfolio development process, assessment centers, implications for national board certification for teachers, personal thoughts and perspectives on teaching, and the future of…

Posturography using the Wii Balance Board™: A feasibility study with healthy adults and adults post-stroke.

PubMed

Llorens, Roberto; Latorre, Jorge; Noé, Enrique; Keshner, Emily A

2016-01-01

Posturography systems that incorporate force platforms are considered to assess balance and postural control with greater sensitivity and objectivity than conventional clinical tests. The Wii Balance Board (WBB) system has been shown to have similar performance characteristics as other force platforms, but with lower cost and size. To determine the validity and reliability of a freely available WBB-based posturography system that combined the WBB with several traditional balance assessments, and to assess the performance of a cohort of stroke individuals with respect to healthy individuals. Healthy subjects and individuals with stroke were recruited. Both groups were assessed using the WBB-based posturography system. Individuals with stroke were also assessed using a laboratory grade posturography system and a battery of clinical tests to determine the concurrent validity of the system. A group of subjects were assessed twice with the WBB-based system to determine its reliability. A total of 144 healthy individuals and 53 individuals with stroke participated in the study. Concurrent validity with another posturography system was moderate to high. Correlations with clinical scales were consistent with previous research. The reliability of the system was excellent in almost all measures. In addition, the system successfully characterized individuals with stroke with respect to the healthy population. The WBB-based posturography system exhibited excellent psychometric properties and sensitivity for identifying balance performance of individuals with stroke in comparison with healthy subjects, which supports feasibility of the system as a clinical tool. Copyright © 2015 Elsevier B.V. All rights reserved.

Construct validity of individual and summary performance metrics associated with a computer-based laparoscopic simulator.

PubMed

Rivard, Justin D; Vergis, Ashley S; Unger, Bertram J; Hardy, Krista M; Andrew, Chris G; Gillman, Lawrence M; Park, Jason

2014-06-01

Computer-based surgical simulators capture a multitude of metrics based on different aspects of performance, such as speed, accuracy, and movement efficiency. However, without rigorous assessment, it may be unclear whether all, some, or none of these metrics actually reflect technical skill, which can compromise educational efforts on these simulators. We assessed the construct validity of individual performance metrics on the LapVR simulator (Immersion Medical, San Jose, CA, USA) and used these data to create task-specific summary metrics. Medical students with no prior laparoscopic experience (novices, N = 12), junior surgical residents with some laparoscopic experience (intermediates, N = 12), and experienced surgeons (experts, N = 11) all completed three repetitions of four LapVR simulator tasks. The tasks included three basic skills (peg transfer, cutting, clipping) and one procedural skill (adhesiolysis). We selected 36 individual metrics on the four tasks that assessed six different aspects of performance, including speed, motion path length, respect for tissue, accuracy, task-specific errors, and successful task completion. Four of seven individual metrics assessed for peg transfer, six of ten metrics for cutting, four of nine metrics for clipping, and three of ten metrics for adhesiolysis discriminated between experience levels. Time and motion path length were significant on all four tasks. We used the validated individual metrics to create summary equations for each task, which successfully distinguished between the different experience levels. Educators should maintain some skepticism when reviewing the plethora of metrics captured by computer-based simulators, as some but not all are valid. We showed the construct validity of a limited number of individual metrics and developed summary metrics for the LapVR. The summary metrics provide a succinct way of assessing skill with a single metric for each task, but require further validation.

Stress testing hydrologic models using bottom-up climate change assessment

NASA Astrophysics Data System (ADS)

Stephens, C.; Johnson, F.; Marshall, L. A.

2017-12-01

Bottom-up climate change assessment is a promising approach for understanding the vulnerability of a system to potential future changes. The technique has been utilised successfully in risk-based assessments of future flood severity and infrastructure vulnerability. We find that it is also an ideal tool for assessing hydrologic model performance in a changing climate. In this study, we applied bottom-up climate change to compare the performance of two different hydrologic models (an event-based and a continuous model) under increasingly severe climate change scenarios. This allowed us to diagnose likely sources of future prediction error in the two models. The climate change scenarios were based on projections for southern Australia, which indicate drier average conditions with increased extreme rainfall intensities. We found that the key weakness in using the event-based model to simulate drier future scenarios was the model's inability to dynamically account for changing antecedent conditions. This led to increased variability in model performance relative to the continuous model, which automatically accounts for the wetness of a catchment through dynamic simulation of water storages. When considering more intense future rainfall events, representation of antecedent conditions became less important than assumptions around (non)linearity in catchment response. The linear continuous model we applied may underestimate flood risk in a future climate with greater extreme rainfall intensity. In contrast with the recommendations of previous studies, this indicates that continuous simulation is not necessarily the key to robust flood modelling under climate change. By applying bottom-up climate change assessment, we were able to understand systematic changes in relative model performance under changing conditions and deduce likely sources of prediction error in the two models.

Qualitative elements of early motor development that influence reaching of the erect posture. A prospective cohort study.

PubMed

Gajewska, Ewa; Sobieska, Magdalena

2015-05-01

The proposed assessment sheet aims to show in detail, which qualitative elements of motor performance are performed correctly in the 2nd month of life by children who in the 9th month reached the erect posture. Similar analysis was performed for the qualitative assessment in the 6th month. The prospective investigation of motor development involved a group of 109 children (40 girls and 69 boys). The study was based on the previously described quantity and quality assessment sheet of motor performance, validated for the 2nd and 6th month. Final investigation took place in the 9th month of life and was based on a neurological assessment. It could be shown that a completely correct assessment at the age of 2 months precludes future severe motor development disorders, especially cerebral palsy, although it does not rule out a slight delay. Prematurity and the analyzed risk factors, particularly IVH, impair the motor performance. The absence of axial symmetry, the shoulders protraction and improper position of the pelvis are the most important alarming features at the 2nd month. Distal elements observed in the prone position at the 6th month show a good prognosis for the motor performance in the 9th month. Any abnormalities, mainly related to the body axis and symmetry observed at 2 months of age should encourage one to put a child under observation. Copyright © 2015 Elsevier Inc. All rights reserved.

Co-Constructional Task Analysis: Moving beyond Adult-Based Models to Assess Young Children's Task Performance

ERIC Educational Resources Information Center

Lee, Scott Weng Fai

2013-01-01

The assessment of young children's thinking competence in task performances has typically followed the novice-to-expert regimen involving models of strategies that adults use when engaged in cognitive tasks such as problem-solving and decision-making. Socio-constructivists argue for a balanced pedagogical approach between the adult and child that…

Using First-Grade Teacher Ratings to Predict Third-Grade English Language Arts and Mathematics Achievement on a High-Stakes Statewide Assessment

ERIC Educational Resources Information Center

Gallant, Dorinda J.

2013-01-01

Early childhood professional organizations support teachers as the best assessors of students' academic, social, emotional, and physical development. This study investigates the predictive nature of teacher ratings of first-grade students' performance on a standards-based curriculum-embedded performance assessment within the context of a state…

The Use of a Performance Assessment for Identifying Gifted Lebanese Students: Is DISCOVER Effective?

ERIC Educational Resources Information Center

Sarouphim, Ketty M.

2009-01-01

The purpose of this study was to investigate the effectiveness of DISCOVER, a performance- based assessment in identifying gifted Lebanese students. The sample consisted of 248 students (121 boys, 127 girls) from Grades 3-5 at two private schools in Beirut, Lebanon. Students were administered DISCOVER and the Raven Standard Progressive Matrices…

A Powerful Conversation: Teachers and Artists Collaborate in Performance-Based Assessment

ERIC Educational Resources Information Center

Oreck, Barry A.

2005-01-01

After many years of hearing teachers express surprise at their students' performance in arts classes, ArtsConnection set out to create a process that would help define and focus attention on artistic abilities and characteristics and bring further value and legitimacy to artists' views of children. The result is the Talent Assessment Process in…

Comparison of answer-until-correct and full-credit assessments in a team-based learning course.

PubMed

Farland, Michelle Z; Barlow, Patrick B; Levi Lancaster, T; Franks, Andrea S

2015-03-25

To assess the impact of awarding partial credit to team assessments on team performance and on quality of team interactions using an answer-until-correct method compared to traditional methods of grading (multiple-choice, full-credit). Subjects were students from 3 different offerings of an ambulatory care elective course, taught using team-based learning. The control group (full-credit) consisted of those enrolled in the course when traditional methods of assessment were used (2 course offerings). The intervention group consisted of those enrolled in the course when answer-until-correct method was used for team assessments (1 course offering). Study outcomes included student performance on individual and team readiness assurance tests (iRATs and tRATs), individual and team final examinations, and student assessment of quality of team interactions using the Team Performance Scale. Eighty-four students enrolled in the courses were included in the analysis (full-credit, n=54; answer-until-correct, n=30). Students who used traditional methods of assessment performed better on iRATs (full-credit mean 88.7 (5.9), answer-until-correct mean 82.8 (10.7), p<0.001). Students who used answer-until-correct method of assessment performed better on the team final examination (full-credit mean 45.8 (1.5), answer-until-correct 47.8 (1.4), p<0.001). There was no significant difference in performance on tRATs and the individual final examination. Students who used the answer-until-correct method had higher quality of team interaction ratings (full-credit 97.1 (9.1), answer-until-correct 103.0 (7.8), p=0.004). Answer-until-correct assessment method compared to traditional, full-credit methods resulted in significantly lower scores for iRATs, similar scores on tRATs and individual final examinations, improved scores on team final examinations, and improved perceptions of the quality of team interactions.

Cost considerations in using simulations for medical training.

PubMed

Fletcher, J D; Wind, Alexander P

2013-10-01

This article reviews simulation used for medical training, techniques for assessing simulation-based training, and cost analyses that can be included in such assessments. Simulation in medical training appears to take four general forms: human actors who are taught to simulate illnesses and ailments in standardized ways; virtual patients who are generally presented via computer-controlled, multimedia displays; full-body manikins that simulate patients using electronic sensors, responders, and controls; and part-task anatomical simulations of various body parts and systems. Techniques for assessing costs include benefit-cost analysis, return on investment, and cost-effectiveness analysis. Techniques for assessing the effectiveness of simulation-based medical training include the use of transfer effectiveness ratios and incremental transfer effectiveness ratios to measure transfer of knowledge and skill provided by simulation to the performance of medical procedures. Assessment of costs and simulation effectiveness can be combined with measures of transfer using techniques such as isoperformance analysis to identify ways of minimizing costs without reducing performance effectiveness or maximizing performance without increasing costs. In sum, economic analysis must be considered in training assessments if training budgets are to compete successfully with other requirements for funding. Reprint & Copyright © 2013 Association of Military Surgeons of the U.S.

An Evaluation of the Performance Diagnostic Checklist-Human Services (PDC-HS) Across Domains.

PubMed

Wilder, David A; Lipschultz, Joshua; Gehrman, Chana

2018-06-01

The Performance Diagnostic Checklist-Human Services (PDC-HS) is an informant-based tool designed to assess the environmental variables that contribute to poor employee performance in human service settings. Although the PDC-HS has been shown to effectively identify variables contributing to problematic performance, interventions based on only two of the four PDC-HS domains have been evaluated to date. In addition, the extent to which PDC-HS-indicated interventions are more effective than nonindicated interventions for two domains remains unclear. In the current study, we administered the PDC-HS to supervisors to assess the variables contributing to infrequent teaching of verbal operants and use of a timer by therapists at a center-based autism treatment program. Each of the four PDC-HS domains was identified as contributing to poor performance for at least one therapist. We then evaluated PDC-HS-indicated interventions for each domain. In addition, to assess the predictive validity of the tool, we evaluated various nonindicated interventions prior to implementing a PDC-HS-indicated intervention for two of the four domains. Results suggest that the PDC-HS-indicated interventions were effective across all four domains and were more effective than the nonindicated interventions for the two domains for which they were evaluated. Results are discussed in terms of the utility of the PDC-HS to identify appropriate interventions to manage therapist performance in human service settings.

Performance-Based Assessment in Schools: A Comment on Hojnoski, Morrison, Brown, and Matthews (2006)

ERIC Educational Resources Information Center

Smith, Steven R.

2007-01-01

This article addresses a 2006 article by Hojnoski, Morrison, Brown, and Matthews on the use of performance-based measurement among school-based practitioners. Their results suggest that many of their survey respondents favor the use of this form of measurement. This line of research is important and addresses an important issue in current clinical…

Measurement properties of performance-based measures to assess physical function in hip and knee osteoarthritis: a systematic review.

PubMed

Dobson, F; Hinman, R S; Hall, M; Terwee, C B; Roos, E M; Bennell, K L

2012-12-01

To systematically review the measurement properties of performance-based measures to assess physical function in people with hip and/or knee osteoarthritis (OA). Electronic searches were performed in MEDLINE, CINAHL, Embase, and PsycINFO up to the end of June 2012. Two reviewers independently rated measurement properties using the consensus-based standards for the selection of health status measurement instrument (COSMIN). "Best evidence synthesis" was made using COSMIN outcomes and the quality of findings. Twenty-four out of 1792 publications were eligible for inclusion. Twenty-one performance-based measures were evaluated including 15 single-activity measures and six multi-activity measures. Measurement properties evaluated included internal consistency (three measures), reliability (16 measures), measurement error (14 measures), validity (nine measures), responsiveness (12 measures) and interpretability (three measures). A positive rating was given to only 16% of possible measurement ratings. Evidence for the majority of measurement properties of tests reported in the review has yet to be determined. On balance of the limited evidence, the 40 m self-paced test was the best rated walk test, the 30 s-chair stand test and timed up and go test were the best rated sit to stand tests, and the Stratford battery, Physical Activity Restrictions and Functional Assessment System were the best rated multi-activity measures. Further good quality research investigating measurement properties of performance measures, including responsiveness and interpretability in people with hip and/or knee OA, is needed. Consensus on which combination of measures will best assess physical function in people with hip/and or knee OA is urgently required. Crown Copyright © 2012. Published by Elsevier Ltd. All rights reserved.

An Assessment of the 2009 Zimbabwe Government's Decision to Enrol Pupils into Form One Using School-Based Assessment as an Alternative to Public Examinations

ERIC Educational Resources Information Center

Mhishi, Misheck; Mandoga, Edward; Tunjera, Nyarai; Bhukuvhani, Crispen Erinos

2012-01-01

School-Based Assessment (SBA) can help achieve a holistic, child-centred and qualitative account of a pupil's performance. The same method has been criticised for being subjective, informal and open to teacher bias hence the reluctance by the Zimbabwe Schools Examination Council (ZIMSEC) to embrace it for certification and selection into further…

A Consensus Model: Shifting assessment practices in dietetics tertiary education.

PubMed

Bacon, Rachel; Kellett, Jane; Dart, Janeane; Knight-Agarwal, Cathy; Mete, Rebecca; Ash, Susan; Palermo, Claire

2018-02-21

The aim of this research was to evaluate a Consensus Model for competency-based assessment. An evaluative case study was used to allow a holistic examination of a constructivist-interpretivist programmatic model of assessment. Using a modified Delphi process, the competence of all 29 students enrolled in their final year of a Master of Nutrition and Dietetics course was assessed by a panel (with expertise in competency-based assessment; industry and academic representation) from a course e-portfolio (that included the judgements of student performance made by worksite educators) and a panel interview. Data were triangulated with assessments from a capstone internship. Qualitative descriptive studies with worksite educators (focus groups n = 4, n = 5, n = 8) and students (personal interviews n = 29) explored stakeholder experiences analysed using thematic analysis. Panel consensus was achieved for all cases by the third-round and corroborated by internship outcomes. For 34% of students this differed to the 'interpretations' of their performance made by their worksite educator/s. Emerging qualitative themes from stakeholder data found the model: (i) supported sustainable assessment practices; (ii) shifted the power relationship between students and worksite educators and (iii) provided a fair method to assess competence. To maximise benefits, more refinement, resources and training are required. This research questions competency-based assessment practices based on discrete placement units and supports a constructivist-interpretivist programmatic approach where evidence across a whole course of study is considered by a panel of assessors. © 2018 Dietitians Association of Australia.

Radiological performance assessment for the E-Area Vaults Disposal Facility

DOE Office of Scientific and Technical Information (OSTI.GOV)

Cook, J.R.

This report is the first revision to ``Radiological Performance Assessment for the E-Area Vaults Disposal Facility, Revision 0'', which was issued in April 1994 and received conditional DOE approval in September 1994. The title of this report has been changed to conform to the current name of the facility. The revision incorporates improved groundwater modeling methodology, which includes a large data base of site specific geotechnical data, and special Analyses on disposal of cement-based wasteforms and naval wastes, issued after publication of Revision 0.

A Decentralized Wireless Solution to Monitor and Diagnose PV Solar Module Performance Based on Symmetrized-Shifted Gompertz Functions

PubMed Central

Molina-García, Angel; Campelo, José Carlos; Blanc, Sara; Serrano, Juan José; García-Sánchez, Tania; Bueso, María C.

2015-01-01

This paper proposes and assesses an integrated solution to monitor and diagnose photovoltaic (PV) solar modules based on a decentralized wireless sensor acquisition system. Both DC electrical variables and environmental data are collected at PV module level using low-cost and high-energy efficiency node sensors. Data is real-time processed locally and compared with expected PV module performances obtained by a PV module model based on symmetrized-shifted Gompertz functions (as previously developed and assessed by the authors). Sensor nodes send data to a centralized sink-computing module using a multi-hop wireless sensor network architecture. Such integration thus provides extensive analysis of PV installations, and avoids off-line tests or post-processing processes. In comparison with previous approaches, this solution is enhanced with a low-cost system and non-critical performance constraints, and it is suitable for extensive deployment in PV power plants. Moreover, it is easily implemented in existing PV installations, since no additional wiring is required. The system has been implemented and assessed in a Spanish PV power plant connected to the grid. Results and estimations of PV module performances are also included in the paper. PMID:26230694

A Decentralized Wireless Solution to Monitor and Diagnose PV Solar Module Performance Based on Symmetrized-Shifted Gompertz Functions.

PubMed

Molina-García, Angel; Campelo, José Carlos; Blanc, Sara; Serrano, Juan José; García-Sánchez, Tania; Bueso, María C

2015-07-29

This paper proposes and assesses an integrated solution to monitor and diagnose photovoltaic (PV) solar modules based on a decentralized wireless sensor acquisition system. Both DC electrical variables and environmental data are collected at PV module level using low-cost and high-energy efficiency node sensors. Data is real-time processed locally and compared with expected PV module performances obtained by a PV module model based on symmetrized-shifted Gompertz functions (as previously developed and assessed by the authors). Sensor nodes send data to a centralized sink-computing module using a multi-hop wireless sensor network architecture. Such integration thus provides extensive analysis of PV installations, and avoids off-line tests or post-processing processes. In comparison with previous approaches, this solution is enhanced with a low-cost system and non-critical performance constraints, and it is suitable for extensive deployment in PV power plants. Moreover, it is easily implemented in existing PV installations, since no additional wiring is required. The system has been implemented and assessed in a Spanish PV power plant connected to the grid. Results and estimations of PV module performances are also included in the paper.

An evaluation of the effectiveness of a risk-based monitoring approach implemented with clinical trials involving implantable cardiac medical devices.

PubMed

Diani, Christopher A; Rock, Angie; Moll, Phil

2017-12-01

Background Risk-based monitoring is a concept endorsed by the Food and Drug Administration to improve clinical trial data quality by focusing monitoring efforts on critical data elements and higher risk investigator sites. BIOTRONIK approached this by implementing a comprehensive strategy that assesses risk and data quality through a combination of operational controls and data surveillance. This publication demonstrates the effectiveness of a data-driven risk assessment methodology when used in conjunction with a tailored monitoring plan. Methods We developed a data-driven risk assessment system to rank 133 investigator sites comprising 3442 subjects and identify those sites that pose a potential risk to the integrity of data collected in implantable cardiac device clinical trials. This included identification of specific risk factors and a weighted scoring mechanism. We conducted trend analyses for risk assessment data collected over 1 year to assess the overall impact of our data surveillance process combined with other operational monitoring efforts. Results Trending analyses of key risk factors revealed an improvement in the quality of data collected during the observation period. The three risk factors follow-up compliance rate, unavailability of critical data, and noncompliance rate correspond closely with Food and Drug Administration's risk-based monitoring guidance document. Among these three risk factors, 100% (12/12) of quantiles analyzed showed an increase in data quality. Of these, 67% (8/12) of the improving trends in worst performing quantiles had p-values less than 0.05, and 17% (2/12) had p-values between 0.05 and 0.06. Among the poorest performing site quantiles, there was a statistically significant decrease in subject follow-up noncompliance rates, protocol noncompliance rates, and incidence of missing critical data. Conclusion One year after implementation of a comprehensive strategy for risk-based monitoring, including a data-driven risk assessment methodology to target on-site monitoring visits, statistically significant improvement was seen in a majority of measurable risk factors at the worst performing site quantiles. For the three risk factors which are most critical to the overall compliance of cardiac rhythm management medical device studies: follow-up compliance rate, unavailability of critical data, and noncompliance rate, we measured significant improvement in data quality. Although the worst performing site quantiles improved but not significantly in some risk factors such as subject attrition, the data-driven risk assessment highlighted key areas on which to continue focusing both on-site and centralized monitoring efforts. Data-driven surveillance of clinical trial performance provides actionable observations that can improve site performance. Clinical trials utilizing risk-based monitoring by leveraging a data-driven quality assessment combined with specific operational procedures may lead to an improvement in data quality and resource efficiencies.

Shift manager workload assessment - A case study

DOE Office of Scientific and Technical Information (OSTI.GOV)

Berntson, K.; Kozak, A.; Malcolm, J. S.

2006-07-01

In early 2003, Bruce Power restarted two of its previously laid up units in the Bruce A generating station, Units 3 and 4. However, due to challenges relating to the availability of personnel with active Shift Manager licenses, an alternate shift structure was proposed to ensure the safe operation of the station. This alternate structure resulted in a redistribution of responsibility, and a need to assess the resulting changes in workload. Atomic Energy of Canada Limited was contracted to perform a workload assessment based on the new shift structure, and to provide recommendations, if necessary, to ensure Shift Managers hadmore » sufficient resources available to perform their required duties. This paper discusses the performance of that assessment, and lessons learned as a result of the work performed during the Restart project. (authors)« less

Evaluation of Verbal Behavior in Older Adults

PubMed Central

Gross, Amy C.; Fuqua, R. Wayne; Merritt, Todd A.

2013-01-01

Approximately 5% of older adults have a dementia diagnosis, and language deterioration is commonly associated with this disorder (Kempler, 2005). Several instruments have been developed to diagnose dementia and assess language capabilities of elderly adults. However, none of these instruments take a functional approach to language assessment as described by Skinner (1957). The purpose of this study was to develop and evaluate a function-based assessment for language deficits of older adults. Thirty-one participants were categorized into a control group (n  =  15) and a dementia group (n  =  16) based on their score on the Dementia Rating Scale-2. Individuals with dementia performed significantly worse on the tact assessment than those without dementia. Participants from both groups performed better on measures of tacts than intraverbals or mands, even though topographically identical responses were required in these assessments. The data provide support for Skinner's conceptualization of functionally independent verbal operants. PMID:23814369

Assessing practice-based learning and improvement.

PubMed

Lynch, Deirdre C; Swing, Susan R; Horowitz, Sheldon D; Holt, Kathleen; Messer, Joseph V

2004-01-01

Practice-based learning and improvement (PBLI) is 1 of 6 general competencies expected of physicians who graduate from an accredited residency education program in the United States and is an anticipated requirement for those who wish to maintain certification by the member boards of the American Board of Medical Specialties. This article describes methods used to assess PBLI. Six electronic databases were searched using several search terms pertaining to PBLI. The review indicated that 4 assessment methods have been used to assess some or all steps of PBLI: portfolios, projects, patient record and chart review, and performance ratings. Each method is described, examples of application are provided, and validity, reliability, and feasibility characteristics are discussed. Portfolios may be the most useful approach to assess residents' PBLI abilities. Active participation in peer-driven performance improvement initiatives may be a valuable approach to confirm practicing physician involvement in PBLI.

Enrichment assessment of multiple virtual screening strategies for Toll-like receptor 8 agonists based on a maximal unbiased benchmarking data set.

PubMed

Pei, Fen; Jin, Hongwei; Zhou, Xin; Xia, Jie; Sun, Lidan; Liu, Zhenming; Zhang, Liangren

2015-11-01

Toll-like receptor 8 agonists, which activate adaptive immune responses by inducing robust production of T-helper 1-polarizing cytokines, are promising candidates for vaccine adjuvants. As the binding site of toll-like receptor 8 is large and highly flexible, virtual screening by individual method has inevitable limitations; thus, a comprehensive comparison of different methods may provide insights into seeking effective strategy for the discovery of novel toll-like receptor 8 agonists. In this study, the performance of knowledge-based pharmacophore, shape-based 3D screening, and combined strategies was assessed against a maximum unbiased benchmarking data set containing 13 actives and 1302 decoys specialized for toll-like receptor 8 agonists. Prior structure-activity relationship knowledge was involved in knowledge-based pharmacophore generation, and a set of antagonists was innovatively used to verify the selectivity of the selected knowledge-based pharmacophore. The benchmarking data set was generated from our recently developed 'mubd-decoymaker' protocol. The enrichment assessment demonstrated a considerable performance through our selected three-layer virtual screening strategy: knowledge-based pharmacophore (Phar1) screening, shape-based 3D similarity search (Q4_combo), and then a Gold docking screening. This virtual screening strategy could be further employed to perform large-scale database screening and to discover novel toll-like receptor 8 agonists. © 2015 John Wiley & Sons A/S.

Metrological analysis of a virtual flowmeter-based transducer for cryogenic helium

DOE Office of Scientific and Technical Information (OSTI.GOV)

Arpaia, P., E-mail: pasquale.arpaia@unina.it; Technology Department, European Organization for Nuclear Research; Girone, M., E-mail: mario.girone@cern.ch

2015-12-15

The metrological performance of a virtual flowmeter-based transducer for monitoring helium under cryogenic conditions is assessed. At this aim, an uncertainty model of the transducer, mainly based on a valve model, exploiting finite-element approach, and a virtual flowmeter model, based on the Sereg-Schlumberger method, are presented. The models are validated experimentally on a case study for helium monitoring in cryogenic systems at the European Organization for Nuclear Research (CERN). The impact of uncertainty sources on the transducer metrological performance is assessed by a sensitivity analysis, based on statistical experiment design and analysis of variance. In this way, the uncertainty sourcesmore » most influencing metrological performance of the transducer are singled out over the input range as a whole, at varying operating and setting conditions. This analysis turns out to be important for CERN cryogenics operation because the metrological design of the transducer is validated, and its components and working conditions with critical specifications for future improvements are identified.« less

A radiation-free mixed-reality training environment and assessment concept for C-arm-based surgery.

PubMed

Stefan, Philipp; Habert, Séverine; Winkler, Alexander; Lazarovici, Marc; Fürmetz, Julian; Eck, Ulrich; Navab, Nassir

2018-06-25

The discrepancy of continuously decreasing opportunities for clinical training and assessment and the increasing complexity of interventions in surgery has led to the development of different training and assessment options like anatomical models, computer-based simulators or cadaver trainings. However, trainees, following training, assessment and ultimately performing patient treatment, still face a steep learning curve. To address this problem for C-arm-based surgery, we introduce a realistic radiation-free simulation system that combines patient-based 3D printed anatomy and simulated X-ray imaging using a physical C-arm. To explore the fidelity and usefulness of the proposed mixed-reality system for training and assessment, we conducted a user study with six surgical experts performing a facet joint injection on the simulator. In a technical evaluation, we show that our system simulates X-ray images accurately with an RMSE of 1.85 mm compared to real X-ray imaging. The participants expressed agreement with the overall realism of the simulation, the usefulness of the system for assessment and strong agreement with the usefulness of such a mixed-reality system for training of novices and experts. In a quantitative analysis, we furthermore evaluated the suitability of the system for the assessment of surgical skills and gather preliminary evidence for validity. The proposed mixed-reality simulation system facilitates a transition to C-arm-based surgery and has the potential to complement or even replace large parts of cadaver training, to provide a safe assessment environment and to reduce the risk for errors when proceeding to patient treatment. We propose an assessment concept and outline the steps necessary to expand the system into a test instrument that provides reliable and justified assessments scores indicative of surgical proficiency with sufficient evidence for validity.

Evaluating Curriculum-Based Measurement from a Behavioral Assessment Perspective

ERIC Educational Resources Information Center

Ardoin, Scott P.; Roof, Claire M.; Klubnick, Cynthia; Carfolite, Jessica

2008-01-01

Curriculum-based measurement Reading (CBM-R) is an assessment procedure used to evaluate students' relative performance compared to peers and to evaluate their growth in reading. Within the response to intervention (RtI) model, CBM-R data are plotted in time series fashion as a means modeling individual students' response to varying levels of…

Role-Playing Game Based Assessment to Fractional Concept in Second Grade Mathematics

ERIC Educational Resources Information Center

Chiu, Fu-Yuan; Hsieh, Mei-Ling

2017-01-01

This study developed a set of Role-playing game (RPG), which was used to explore whether significant differences exist in academic performance and learning attitudes between RPG-based assessment and traditional lectures. This study also investigated the satisfaction of students toward the RPG scenario. Research participants included 100 second…

Installation Flightline Fence Finding of No Significant Impact (FONSI)

DTIC Science & Technology

2003-07-28

Georgia Street,Moody AFB, GA ,31699 8. PERFORMING ORGANIZATION REPORT NUMBER 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR...Environmental Protection Committee Finding of No Sign~ficant Impact Installation Flightline Fence Moody Air Force Base, GA Environmental Assessment...Water Resources/Wetlands - Compliance Resources Moody Air Force Base, GA Environmental Assessment

Towards a Model of School-Based Curriculum Development and Assessment Using the SOLO Taxonomy.

ERIC Educational Resources Information Center

Biggs, John

1989-01-01

One factor preventing the wider acceptance of school-based curriculum development and assessment is the problem of comparing performances of different students, in different schools. The SOLO taxonomy is used to describe the complexity of learning outcomes in a language that is generally applicable across the curriculum. (Author/MLW)

Improving Workplace-Based Assessment and Feedback by an E-Portfolio Enhanced with Learning Analytics

ERIC Educational Resources Information Center

van der Schaaf, Marieke; Donkers, Jeroen; Slof, Bert; Moonen-van Loon, Joyce; van Tartwijk, Jan; Driessen, Eric; Badii, Atta; Serban, Ovidiu; Ten Cate, Olle

2017-01-01

Electronic portfolios (E-portfolios) are crucial means for workplace-based assessment and feedback. Although E-portfolios provide a useful approach to view each learner's progress, so far options for personalized feedback and potential data about a learner's performances at the workplace often remain unexploited. This paper advocates that…

76 FR 21985 - Notice of Final Priorities, Requirements, Definitions, and Selection Criteria

Federal Register 2010, 2011, 2012, 2013, 2014

2011-04-19

... only after a research base has been established to support the use of the assessments for such purposes..., research-based assessment practices. Discussion: We agree that the selection criteria should address the... selection criterion, which addresses methods of scoring, to allow for self-scoring of student performance on...

Training and Generalization Effects of a Reading Comprehension Learning Strategy on Computer and Paper-Pencil Assessments

ERIC Educational Resources Information Center

Worrell, Jamie; Duffy, Mary Lou; Brady, Michael P.; Dukes, Charles; Gonzalez-DeHass, Alyssa

2016-01-01

Many schools use computer-based testing to measure students' progress for end-of-the-year and statewide assessments. There is little research to support whether computer-based testing accurately reflects student progress, particularly among students with learning, performance, and generalization difficulties. This article summarizes an…

How Stereotype Threat Affects Healthy Older Adults' Performance on Clinical Assessments of Cognitive Decline: The Key Role of Regulatory Fit.

PubMed

Barber, Sarah J; Mather, Mara; Gatz, Margaret

2015-11-01

Stereotype threat can impair older adults' performance on clinical assessments for cognitive decline. We examined why this occurs. Based upon the regulatory focus account of stereotype threat, we predicted that the effects of stereotype threat should depend upon the assessments' reward structure. Stereotype threat should be associated with poor performance when the assessment emphasizes gaining correct answers, but not when it emphasizes avoiding mistakes. Healthy older adults completed a series of mental status examinations. Half of the participants completed these examinations under stereotype threat about their cognitive abilities. Monetary incentives were also manipulated. For half of the participants correct responding led to gains. For the remaining participants incorrect responding/forgetting led to losses. Consistent with the regulatory focus account, stereotype threat was associated with poor performance when the mental status examinations had a gains-based structure, but not when they had a losses-based structure. Older adults respond to stereotype threat by becoming vigilant to avoid the losses that will make them their worst. Researchers and clinicians can capitalize on this motivational change to combat stereotype threat's negative effects. By using a loss-avoidance frame, stereotype threat's negative effects can be attenuated or even eliminated. © The Author 2015. Published by Oxford University Press on behalf of The Gerontological Society of America. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

The role of feedback in improving the effectiveness of workplace based assessments: a systematic review.

PubMed

Saedon, Habiba; Salleh, Shizalia; Balakrishnan, Arun; Imray, Christopher H E; Saedon, Mahmud

2012-05-02

With recent emphasis placed on workplace based assessment (WBA) as a method of formative performance assessment, there is limited evidence in the current literature regarding the role of feedback in improving the effectiveness of WBA. The aim of this systematic review was to elucidate the impact of feedback on the effectiveness of WBA in postgraduate medical training. Searches were conducted using the following bibliographic databases to identify original published studies related to WBA and the role of feedback: Medline (1950-December 2010), Embase (1980-December 2010) and Journals@Ovid (English language only, 1996-December 2010). Studies which attempted to evaluate the role of feedback in WBA involving postgraduate doctors were included. 15 identified studies met the inclusion criteria and minimum quality threshold. They were heterogeneous in methodological design. 7 studies focused on multi source feedback, 3 studies were based on mini-clinical evaluation exercise, 2 looked at procedural based assessment, one study looked at workplace based assessments in general and 2 studies looked at a combination of 3 to 6 workplace based assessments. 7 studies originated from the United Kingdom. Others were from Canada, the United States and New Zealand. Study populations were doctors in various grades of training from a wide range of specialties including general practice, general medicine, general surgery, dermatology, paediatrics and anaesthetics. All studies were prospective in design, and non-comparative descriptive or observational studies using a variety of methods including questionnaires, one to one interviews and focus groups. The evidence base contains few high quality conclusive studies and more studies are required to provide further evidence for the effect of feedback from workplace based assessment on subsequent performance. There is, however, good evidence that if well implemented, feedback from workplace based assessments, particularly multisource feedback, leads to a perceived positive effect on practice.

Discourse and Assessment.

ERIC Educational Resources Information Center

McNamara, Tim; Hill, Kathryn; May, Lynette

2002-01-01

Focuses on the increase in discourse-based studies of oral proficiency assessment techniques. Discusses research carried out on a number of factors in the assessment setting, including the role of interlocutor, candidate, and rater, and the impact of tasks, task performance conditions, and rating criteria. (Author/VWL)

48 CFR 4.602 - General.

Code of Federal Regulations, 2011 CFR

2011-10-01

..., and the general public; (2) A means of measuring and assessing the effect of Federal contracting on... means of measuring and assessing the effect of Federal contracting for promoting sustainable... measuring and assessing the effect of other policy and management initiatives (e.g., performance based...

48 CFR 4.602 - General.

Code of Federal Regulations, 2013 CFR

2013-10-01

..., and the general public; (2) A means of measuring and assessing the effect of Federal contracting on... means of measuring and assessing the effect of Federal contracting for promoting sustainable... measuring and assessing the effect of other policy and management initiatives (e.g., performance based...

Space science/space station attached payload pointing accommodation study: Technology assessment white paper

NASA Technical Reports Server (NTRS)

Lin, Richard Y.; Mann, Kenneth E.; Laskin, Robert A.; Sirlin, Samuel W.

1987-01-01

Technology assessment is performed for pointing systems that accommodate payloads of large mass and large dimensions. Related technology areas are also examined. These related areas include active thermal lines or power cables across gimbals, new materials for increased passive damping, tethered pointing, and inertially reacting pointing systems. Conclusions, issues and concerns, and recommendations regarding the status and development of large pointing systems for space applications are made based on the performed assessments.

Measuring the value of process improvement initiatives in a preoperative assessment center using time-driven activity-based costing.

PubMed

French, Katy E; Albright, Heidi W; Frenzel, John C; Incalcaterra, James R; Rubio, Augustin C; Jones, Jessica F; Feeley, Thomas W

2013-12-01

The value and impact of process improvement initiatives are difficult to quantify. We describe the use of time-driven activity-based costing (TDABC) in a clinical setting to quantify the value of process improvements in terms of cost, time and personnel resources. Difficulty in identifying and measuring the cost savings of process improvement initiatives in a Preoperative Assessment Center (PAC). Use TDABC to measure the value of process improvement initiatives that reduce the costs of performing a preoperative assessment while maintaining the quality of the assessment. Apply the principles of TDABC in a PAC to measure the value, from baseline, of two phases of performance improvement initiatives and determine the impact of each implementation in terms of cost, time and efficiency. Through two rounds of performance improvements, we quantified an overall reduction in time spent by patient and personnel of 33% that resulted in a 46% reduction in the costs of providing care in the center. The performance improvements resulted in a 17% decrease in the total number of full time equivalents (FTE's) needed to staff the center and a 19% increase in the numbers of patients assessed in the center. Quality of care, as assessed by the rate of cancellations on the day of surgery, was not adversely impacted by the process improvements. © 2013 Published by Elsevier Inc.

Implementing assessments of robot-assisted technical skill in urological education: a systematic review and synthesis of the validity evidence.

PubMed

Goldenberg, Mitchell G; Lee, Jason Y; Kwong, Jethro C C; Grantcharov, Teodor P; Costello, Anthony

2018-03-31

To systematically review and synthesise the validity evidence supporting intraoperative and simulation-based assessments of technical skill in urological robot-assisted surgery (RAS), and make evidence-based recommendations for the implementation of these assessments in urological training. A literature search of the Medline, PsycINFO and Embase databases was performed. Articles using technical skill and simulation-based assessments in RAS were abstracted. Only studies involving urology trainees or faculty were included in the final analysis. Multiple tools for the assessment of technical robotic skill have been published, with mixed sources of validity evidence to support their use. These evaluations have been used in both the ex vivo and in vivo settings. Performance evaluations range from global rating scales to psychometrics, and assessments are carried out through automation, expert analysts, and crowdsourcing. There have been rapid expansions in approaches to RAS technical skills assessment, both in simulated and clinical settings. Alternative approaches to assessment in RAS, such as crowdsourcing and psychometrics, remain under investigation. Evidence to support the use of these metrics in high-stakes decisions is likely insufficient at present. © 2018 The Authors BJU International © 2018 BJU International Published by John Wiley & Sons Ltd.

Congenital Heart Surgery Case Mix Across North American Centers and Impact on Performance Assessment.

PubMed

Pasquali, Sara K; Wallace, Amelia S; Gaynor, J William; Jacobs, Marshall L; O'Brien, Sean M; Hill, Kevin D; Gaies, Michael G; Romano, Jennifer C; Shahian, David M; Mayer, John E; Jacobs, Jeffrey P

2016-11-01

Performance assessment in congenital heart surgery is challenging due to the wide heterogeneity of disease. We describe current case mix across centers, evaluate methodology inclusive of all cardiac operations versus the more homogeneous subset of Society of Thoracic Surgeons benchmark operations, and describe implications regarding performance assessment. Centers (n = 119) participating in the Society of Thoracic Surgeons Congenital Heart Surgery Database (2010 through 2014) were included. Index operation type and frequency across centers were described. Center performance (risk-adjusted operative mortality) was evaluated and classified when including the benchmark versus all eligible operations. Overall, 207 types of operations were performed during the study period (112,140 total cases). Few operations were performed across all centers; only 25% were performed at least once by 75% or more of centers. There was 7.9-fold variation across centers in the proportion of total cases comprising high-complexity cases (STAT 5). In contrast, the benchmark operations made up 36% of cases, and all but 2 were performed by at least 90% of centers. When evaluating performance based on benchmark versus all operations, 15% of centers changed performance classification; 85% remained unchanged. Benchmark versus all operation methodology was associated with lower power, with 35% versus 78% of centers meeting sample size thresholds. There is wide variation in congenital heart surgery case mix across centers. Metrics based on benchmark versus all operations are associated with strengths (less heterogeneity) and weaknesses (lower power), and lead to differing performance classification for some centers. These findings have implications for ongoing efforts to optimize performance assessment, including choice of target population and appropriate interpretation of reported metrics. Copyright © 2016 The Society of Thoracic Surgeons. Published by Elsevier Inc. All rights reserved.

Accelerometry-Based Activity Recognition and Assessment in Rheumatic and Musculoskeletal Diseases.

PubMed

Billiet, Lieven; Swinnen, Thijs Willem; Westhovens, Rene; de Vlam, Kurt; Van Huffel, Sabine

2016-12-16

One of the important aspects to be considered in rheumatic and musculoskeletal diseases is the patient's activity capacity (or performance), defined as the ability to perform a task. Currently, it is assessed by physicians or health professionals mainly by means of a patient-reported questionnaire, sometimes combined with the therapist's judgment on performance-based tasks. This work introduces an approach to assess the activity capacity at home in a more objective, yet interpretable way. It offers a pilot study on 28 patients suffering from axial spondyloarthritis (axSpA) to demonstrate its efficacy. Firstly, a protocol is introduced to recognize a limited set of six transition activities in the home environment using a single accelerometer. To this end, a hierarchical classifier with the rejection of non-informative activity segments has been developed drawing on both direct pattern recognition and statistical signal features. Secondly, the recognized activities should be assessed, similarly to the scoring performed by patients themselves. This is achieved through the interval coded scoring (ICS) system, a novel method to extract an interpretable scoring system from data. The activity recognition reaches an average accuracy of 93.5%; assessment is currently 64.3% accurate. These results indicate the potential of the approach; a next step should be its validation in a larger patient study.

Mind the Gap: The Prospects of Missing Data.

PubMed

McConnell, Meghan; Sherbino, Jonathan; Chan, Teresa M

2016-12-01

The increasing use of workplace-based assessments (WBAs) in competency-based medical education has led to large data sets that assess resident performance longitudinally. With large data sets, problems that arise from missing data are increasingly likely. The purpose of this study is to examine (1) whether data are missing at random across various WBAs, and (2) the relationship between resident performance and the proportion of missing data. During 2012-2013, a total of 844 WBAs of CanMEDs Roles were completed for 9 second-year emergency medicine residents. To identify whether missing data were randomly distributed across various WBAs, the total number of missing data points was calculated for each Role. To examine whether the amount of missing data was related to resident performance, 5 faculty members rank-ordered the residents based on performance. A median rank score was calculated for each resident and was correlated with the proportion of missing data. More data were missing for Health Advocate and Professional WBAs relative to other competencies ( P  < .001). Furthermore, resident rankings were not related to the proportion of missing data points ( r  = 0.29, P  > .05). The results of the present study illustrate that some CanMEDS Roles are less likely to be assessed than others. At the same time, the amount of missing data did not correlate with resident performance, suggesting lower-performing residents are no more likely to have missing data than their higher-performing peers. This article discusses several approaches to dealing with missing data.

An eye-tracking controlled neuropsychological battery for cognitive assessment in neurological diseases.

PubMed

Poletti, Barbara; Carelli, Laura; Solca, Federica; Lafronza, Annalisa; Pedroli, Elisa; Faini, Andrea; Zago, Stefano; Ticozzi, Nicola; Ciammola, Andrea; Morelli, Claudia; Meriggi, Paolo; Cipresso, Pietro; Lulé, Dorothée; Ludolph, Albert C; Riva, Giuseppe; Silani, Vincenzo

2017-04-01

Traditional cognitive assessment in neurological conditions involving physical disability is often prevented by the presence of verbal-motor impairment; to date, an extensive motor-verbal-free neuropsychological battery is not available for such purposes. We adapted a set of neuropsychological tests, assessing language, attentional abilities, executive functions and social cognition, for eye-tracking (ET) control, and explored its feasibility in a sample of healthy participants. Thirty healthy subjects performed a neuropsychological assessment, using an ET-based neuropsychological battery, together with standard "paper and pencil" cognitive measures for frontal (Frontal Assessment Battery-FAB) and working memory abilities (Digit Sequencing Task) and for global cognitive efficiency (Montreal Cognitive Assessment-MoCA). Psychological measures of anxiety (State-Trait Anxiety Inventory-Y-STAI-Y) and depression (Beck Depression Inventory-BDI) were also collected, and a usability questionnaire was administered. Significant correlations were observed between the "paper and pencil" screening of working memory abilities and the ET-based neuropsychological measures. The ET-based battery also correlated with the MoCA, while poor correlations were observed with the FAB. Usability aspects were found to be influenced by both working memory abilities and psychological components. The ET-based neuropsychological battery developed could provide an extensive assessment of cognitive functions, allowing participants to perform tasks independently from the integrity of motor or verbal channels. Further studies will be aimed at investigating validity and usability components in neurological populations with motor-verbal impairments.

Quantifying the cognitive cost of laparo-endoscopic single-site surgeries: Gaze-based indices.

PubMed

Di Stasi, Leandro L; Díaz-Piedra, Carolina; Ruiz-Rabelo, Juan Francisco; Rieiro, Héctor; Sanchez Carrion, Jose M; Catena, Andrés

2017-11-01

Despite the growing interest concerning the laparo-endoscopic single-site surgery (LESS) procedure, LESS presents multiple difficulties and challenges that are likely to increase the surgeon's cognitive cost, in terms of both cognitive load and performance. Nevertheless, there is currently no objective index capable of assessing the surgeon cognitive cost while performing LESS. We assessed if gaze-based indices might offer unique and unbiased measures to quantify LESS complexity and its cognitive cost. We expect that the assessment of surgeon's cognitive cost to improve patient safety by measuring fitness-for-duty and reducing surgeons overload. Using a wearable eye tracker device, we measured gaze entropy and velocity of surgical trainees and attending surgeons during two surgical procedures (LESS vs. multiport laparoscopy surgery [MPS]). None of the participants had previous experience with LESS. They performed two exercises with different complexity levels (Low: Pattern Cut vs. High: Peg Transfer). We also collected performance and subjective data. LESS caused higher cognitive demand than MPS, as indicated by increased gaze entropy in both surgical trainees and attending surgeons (exploration pattern became more random). Furthermore, gaze velocity was higher (exploration pattern became more rapid) for the LESS procedure independently of the surgeon's expertise. Perceived task complexity and laparoscopic accuracy confirmed gaze-based results. Gaze-based indices have great potential as objective and non-intrusive measures to assess surgeons' cognitive cost and fitness-for-duty. Furthermore, gaze-based indices might play a relevant role in defining future guidelines on surgeons' examinations to mark their achievements during the entire training (e.g. analyzing surgical learning curves). Copyright © 2017 Elsevier Ltd. All rights reserved.

Development and validation of a Clinical Assessment Tool for Nursing Education (CAT-NE).

PubMed

Skúladóttir, Hafdís; Svavarsdóttir, Margrét Hrönn

2016-09-01

The aim of this study was to develop a valid assessment tool to guide clinical education and evaluate students' performance in clinical nursing education. The development of the Clinical Assessment Tool for Nursing Education (CAT-NE) was based on the theory of nursing as professional caring and the Bologna learning outcomes. Benson and Clark's four steps of instrument development and validation guided the development and assessment of the tool. A mixed-methods approach with individual structured cognitive interviewing and quantitative assessments was used to validate the tool. Supervisory teachers, a pedagogical consultant, clinical expert teachers, clinical teachers, and nursing students at the University of Akureyri in Iceland participated in the process. This assessment tool is valid to assess the clinical performance of nursing students; it consists of rubrics that list the criteria for the students' expected performance. According to the students and their clinical teachers, the assessment tool clarified learning objectives, enhanced the focus of the assessment process, and made evaluation more objective. Training clinical teachers on how to assess students' performances in clinical studies and use the tool enhanced the quality of clinical assessment in nursing education. Copyright © 2016 Elsevier Ltd. All rights reserved.

Workplace based assessment: a step to promote competency based postgraduate training.

PubMed

Singh, Tejinder; Modi, Jyoti Nath

2013-06-08

There has been an increasing emphasis on defining outcomes of medical education in terms of performance of trainees. This is a step beyond the description of outcomes in terms of competence that encompasses mostly potential abilities rather than the actual performance. The contextual adaptations and behavior judgments of the trainees are best assessed by a program of in-training assessment. Workplace based assessment (WPBA) is one of the modalities, which assesses the trainee in authentic settings. Though Postgraduate (PG) medical training in India is said to be competency-based, most institutions do not have any formative or in-training assessment program for the same. The two cardinal elements of WPBA are direct observation and conducted in work place in addition to provision of feedback to the trainee. The WPBA conforms to the highest (Level 4: Does) of Millers pyramid and also has the potential to assess at all four levels. Some of the tools used for WPBA are: Logbooks, Clinical Encounter Cards (CEC), mini-Clinical Evaluation Exercise (mini-CEX), Case based discussions, Direct Observation of Procedural Skills (DOPS), Multisource feedback (peers, co-workers, seniors, patients) etc. These can be documented in the form of a portfolio that provides a longitudinal view of experiences and progress of the trainee. The WPBA scores high on validity and educational impact by virtue of being based on direct observation in real situation and contextual feedback. The feasibility and acceptability is enhanced by making appropriate choices of tools, advance planning, building of mutual trust, and training of assessors. Given the established benefits of WPBA in shaping clinical learning, there is an imminent need for including this mode of assessment in our clinical training programs especially PG training.

Teaching science through literature

NASA Astrophysics Data System (ADS)

Barth, Daniel

2007-12-01

The hypothesis of this study was that a multidisciplinary, activity rich science curriculum based around science fiction literature, rather than a conventional text book would increase student engagement with the curriculum and improve student performance on standards-based test instruments. Science fiction literature was chosen upon the basis of previous educational research which indicated that science fiction literature was able to stimulate and maintain interest in science. The study was conducted on a middle school campus during the regular summer school session. Students were self-selected from the school's 6 th, 7th, and 8th grade populations. The students used the science fiction novel Maurice on the Moon as their only text. Lessons and activities closely followed the adventures of the characters in the book. The student's initial level of knowledge in Earth and space science was assessed by a pre test. After the four week program was concluded, the students took a post test made up of an identical set of questions. The test included 40 standards-based questions that were based upon concepts covered in the text of the novel and in the classroom lessons and activities. The test also included 10 general knowledge questions that were based upon Earth and space science standards that were not covered in the novel or the classroom lessons or activities. Student performance on the standards-based question set increased an average of 35% for all students in the study group. Every subgroup disaggregated by gender and ethnicity improved from 28-47%. There was no statistically significant change in the performance on the general knowledge question set for any subgroup. Student engagement with the material was assessed by three independent methods, including student self-reports, percentage of classroom work completed, and academic evaluation of student work by the instructor. These assessments of student engagement were correlated with changes in student performance on the standards-based assessment tests. A moderate correlation was found to exist between the level of student engagement with the material and improvement in performance from pre to post test.

Introduction: Hazard mapping

USGS Publications Warehouse

Baum, Rex L.; Miyagi, Toyohiko; Lee, Saro; Trofymchuk, Oleksandr M

2014-01-01

Twenty papers were accepted into the session on landslide hazard mapping for oral presentation. The papers presented susceptibility and hazard analysis based on approaches ranging from field-based assessments to statistically based models to assessments that combined hydromechanical and probabilistic components. Many of the studies have taken advantage of increasing availability of remotely sensed data and nearly all relied on Geographic Information Systems to organize and analyze spatial data. The studies used a range of methods for assessing performance and validating hazard and susceptibility models. A few of the studies presented in this session also included some element of landslide risk assessment. This collection of papers clearly demonstrates that a wide range of approaches can lead to useful assessments of landslide susceptibility and hazard.

Novel Miscanthus Germplasm-Based Value Chains: A Life Cycle Assessment

PubMed Central

Wagner, Moritz; Kiesel, Andreas; Hastings, Astley; Iqbal, Yasir; Lewandowski, Iris

2017-01-01

In recent years, considerable progress has been made in miscanthus research: improvement of management practices, breeding of new genotypes, especially for marginal conditions, and development of novel utilization options. The purpose of the current study was a holistic analysis of the environmental performance of such novel miscanthus-based value chains. In addition, the relevance of the analyzed environmental impact categories was assessed. A Life Cycle Assessment was conducted to analyse the environmental performance of the miscanthus-based value chains in 18 impact categories. In order to include the substitution of a reference product, a system expansion approach was used. In addition, a normalization step was applied. This allowed the relevance of these impact categories to be evaluated for each utilization pathway. The miscanthus was cultivated on six sites in Europe (Aberystwyth, Adana, Moscow, Potash, Stuttgart and Wageningen) and the biomass was utilized in the following six pathways: (1) small-scale combustion (heat)—chips; (2) small-scale combustion (heat)—pellets; (3) large-scale combustion (CHP)—biomass baled for transport and storage; (4) large-scale combustion (CHP)—pellets; (5) medium-scale biogas plant—ensiled miscanthus biomass; and (6) large-scale production of insulation material. Thus, in total, the environmental performance of 36 site × pathway combinations was assessed. The comparatively high normalized results of human toxicity, marine, and freshwater ecotoxicity, and freshwater eutrophication indicate the relevance of these impact categories in the assessment of miscanthus-based value chains. Differences between the six sites can almost entirely be attributed to variations in biomass yield. However, the environmental performance of the utilization pathways analyzed varied widely. The largest differences were shown for freshwater and marine ecotoxicity, and freshwater eutrophication. The production of insulation material had the lowest impact on the environment, with net benefits in all impact categories expect three (marine eutrophication, human toxicity, agricultural land occupation). This performance can be explained by the multiple use of the biomass, first as material and subsequently as an energy carrier, and by the substitution of an emission-intensive reference product. The results of this study emphasize the importance of assessing all environmental impacts when selecting appropriate utilization pathways. PMID:28642784

Assessing Student Reasoning in Upper-Division Electricity and Magnetism at Oregon State University

ERIC Educational Resources Information Center

Zwolak, Justyna P.; Manogue, Corinne A.

2015-01-01

Standardized assessment tests that allow researchers to compare the performance of students under various curricula are highly desirable. There are several research-based conceptual tests that serve as instruments to assess and identify students' difficulties in lower-division courses. At the upper-division level assessing students' difficulties…

76 FR 56820 - Detroit Edison Company, Fermi 2; Environmental Assessment and Finding of No Significant Impact

Federal Register 2010, 2011, 2012, 2013, 2014

2011-09-14

.... Ken Yale, of the Michigan Department of Natural Resources and Environment regarding the environmental...; Environmental Assessment and Finding of No Significant Impact The U.S. Nuclear Regulatory Commission (NRC) is..., the NRC performed an environmental assessment. Based on the results of the environmental assessment...

Web-Based Portfolio Assessment: Validation of an Open Source Platform

ERIC Educational Resources Information Center

Collins, Regina; Elliot, Norbert; Klobucar, Andrew; Deek, Fadi P.

2013-01-01

Assessment of educational outcomes through purchased tests is commonplace in the evaluation of individual student ability and of educational programs. Focusing on the assessment of writing performance in a longitudinal study of first-time, full-time students (n = 598), this research describes the design, use, and assessment of an open-source…

Investigating Computer-Based Formative Assessments in a Medical Terminology Course

ERIC Educational Resources Information Center

Wilbanks, Jammie T.

2012-01-01

Research has been conducted on the effectiveness of formative assessments and on effectively teaching medical terminology; however, research had not been conducted on the use of formative assessments in a medical terminology course. A quantitative study was performed which captured data from a pretest, self-assessment, four module exams, and a…

User manual and spreadsheet tool for implementing performance planning for rural planning organizations.

DOT National Transportation Integrated Search

2017-03-01

The performance-planning tool developed as part of this project is intended for use with the guidebook for establishing and using rural performance based transportation system assessment, monitoring, planning, and programming to support the rural pla...

Assessment of driving-related skills for older drivers : traffic tech.

DOT National Transportation Integrated Search

2010-04-01

Relating behind-the-wheel driving performance to performance : on office-based screening tools is challenging. It is : important to use tools that are predictive of poor driving : performance (sensitivity), but also to find tools that do not : have h...

Protocol for Reliability Assessment of Structural Health Monitoring Systems Incorporating Model-assisted Probability of Detection (MAPOD) Approach

DTIC Science & Technology

2011-09-01

a quality evaluation with limited data, a model -based assessment must be...that affect system performance, a multistage approach to system validation, a modeling and experimental methodology for efficiently addressing a ...affect system performance, a multistage approach to system validation, a modeling and experimental methodology for efficiently addressing a wide range

Perception and Practice: The Impact of Teachers' Scoring Experience on Performance-Based Instruction and Classroom Assessment.

ERIC Educational Resources Information Center

Goldberg, Gail Lynn; Roswell, Barbara Sherr

Teachers' reactions to the administration and scoring of the Maryland School Performance Assessment Program tests (MSPAP) were studied, focusing on their direct and indirect exposure to tasks and evaluative criteria through the experience of scoring the MSPAP. Since its inception in 1991, the MSPAP has been scored in-state by certified teachers…

Differential Item Functioning Comparisons on a Performance-Based Alternate Assessment for Students with Severe Cognitive Impairments, Autism and Orthopedic Impairments

ERIC Educational Resources Information Center

Laitusis, Cara Cahalan; Maneckshana, Behroz; Monfils, Lora; Ahlgrim-Delzell, Lynn

2009-01-01

The purpose of this study was to examine Differential Item Functioning (DIF) by disability groups on an on-demand performance assessment for students with severe cognitive impairments. Researchers examined the presence of DIF for two comparisons. One comparison involved students with severe cognitive impairments who served as the reference group…

The Relationship of Time Orientation with Perceived Academic Performance and Preparation for Assessment in Adolescents

ERIC Educational Resources Information Center

Bowles, Terry

2008-01-01

The purpose of this study was to operationalise a model of time orientation and investigate the variability of its factors based on preparation for assessment and perceived academic performance. Responses from 113 male adolescents (mean age = 16.46 years) and 115 female adolescents (mean age = 16.42 years) to items operationalising an expanded…

Storying Teacher Education Policy: Critical Counternarratives of Curricular, Pedagogical, and Activist Responses to State-Mandated Teacher Performance Assessments

ERIC Educational Resources Information Center

Henning, Nick; Dover, Alison; Dotson, Erica K.; Agarwal-Rangath, Ruchi

2018-01-01

The rise of high-stakes, standardized, teacher performance assessments (TPAs) is central to the industry being created out of the regulation, policing, and evaluation of university-based teacher education In addition to reinforcing a narrow and counter-critical framework, TPAs can shift responsibility for the evaluation of teacher candidates from…

Multigroup Confirmatory Factor Analysis of U.S. and Italian Children's Performance on the PASS Theory of Intelligence as Measured by the Cognitive Assessment System

ERIC Educational Resources Information Center

Naglieri, Jack A.; Taddei, Stefano; Williams, Kevin M.

2013-01-01

This study examined Italian and U.S. children's performance on the English and Italian versions, respectively, of the Cognitive Assessment System (CAS; Naglieri & Conway, 2009; Naglieri & Das, 1997), a test based on a neurocognitive theory of intelligence entitled PASS (Planning, Attention, Simultaneous, and Successive; Naglieri & Das,…

Monitoring the Impact of Testing and Evaluation Innovations Project: State Activity and Interest Concerning Performance-Based Assessment.

ERIC Educational Resources Information Center

Center for Research on Evaluation, Standards, and Student Testing, Los Angeles, CA.

Information was gathered about current state interest, activity, and concerns related to performance assessment for students. The Center for Research on Evaluation, Standards, and Student Testing of the University of California (Los Angeles) conducted telephone interviews with directors of testing in each of the 50 states in the spring of 1990.…

Public Management Reform and Organizational Performance: An Empirical Assessment of the U.K. Labour Government's Public Service Improvement Strategy

ERIC Educational Resources Information Center

Walker, Richard M.; Boyne, George A.

2006-01-01

We present the first empirical assessment of the U.K. Labour government's program of public management reform. This reform program is based on rational planning, devolution and delegation, flexibility and incentives, and enhanced choice. Measures of these variables are tested against external and internal indicators of organizational performance.…

Navigating the Language Demands of an Inquiry-Based Science Performance Assessment: Classroom Challenges and Opportunities for English Learners

ERIC Educational Resources Information Center

Lyon, Edward G.; Bunch, George C.; Shaw, Jerome M.

2012-01-01

Science performance assessments (SPAs) are designed to elicit a wider range of scientific knowledge and abilities than ordinarily measured by more traditional paper-and-pencil tests. To engage in SPAs and thus demonstrate abilities such as scientific inquiry, students must interact with various participants and communicate in a variety of ways.…

Teaching Performance Assessment: A Comparative Study of Implementation and Impact amongst California State University Campuses

ERIC Educational Resources Information Center

Guaglianone, Curtis L.; Payne, Maggie; Kinsey, Gary W.; Chiero, Robin

2009-01-01

This article is based on the perceptions of California State University administrators and provides a comparative study of the challenges and benefits resulting from the implementation of the teaching performance assessment requirement of SB 2042 standards 19-21 on the California State University (CSU) campuses. With 23 campuses and almost 450,000…

Measuring Primary Students' Graph Interpretation Skills via a Performance Assessment: A Case Study in Instrument Development

ERIC Educational Resources Information Center

Peterman, Karen; Cranston, Kayla A.; Pryor, Marie; Kermish-Allen, Ruth

2015-01-01

This case study was conducted within the context of a place-based education project that was implemented with primary school students in the USA. The authors and participating teachers created a performance assessment of standards-aligned tasks to examine 6-10-year-old students' graph interpretation skills as part of an exploratory research…

Validity, Reliability, and Equity Issues in an Observational Talent Assessment Process in the Performing Arts

ERIC Educational Resources Information Center

Oreck, Barry A.; Owen, Steven V.; Baum, Susan M.

2003-01-01

The lack of valid, research-based methods to identify potential artistic talent hampers the inclusion of the arts in programs for the gifted and talented. The Talent Assessment Process in Dance, Music, and Theater (D/M/T TAP) was designed to identify potential performing arts talent in diverse populations, including bilingual and special education…

Workplace-based assessment and students' approaches to learning: a qualitative inquiry.

PubMed

Al-Kadri, Hanan M; Al-Kadi, Mohammed T; Van Der Vleuten, Cees P M

2013-01-01

We have performed this research to assess the effect of work-place based assessment (WBA) practice on medical students' learning approaches. The research was conducted at the King Saud bin Abdulaziz University for Health Sciences, College of Medicine from 1 March to 31 July 2012. We conducted a qualitative, phenomenological research utilizing semi-structured individual interviews with medical students exposed to WBA. The audio-taped interviews were transcribed verbatim, analyzed, and themes were identified. We preformed investigators' triangulation, member checking with clinical supervisors and we triangulated the data with a similar research performed prior to the implementation of WBA. WBA results in variable learning approaches. Based on several affecting factors; clinical supervisors, faculty-given feedback, and assessment function, students may swing between surface, deep and effort and achievement learning approaches. Students' and supervisors' orientations on the process of WBA, utilization of peer feedback and formative rather than summative assessment facilitate successful implementation of WBA and lead to students' deeper approaches to learning. Interestingly, students and their supervisors have contradicting perceptions to WBA. A change in culture to unify students' and supervisors' perceptions of WBA, more accommodation of formative assessment, and feedback may result in students' deeper approach to learning.

RCTA capstone assessment

NASA Astrophysics Data System (ADS)

Lennon, Craig; Bodt, Barry; Childers, Marshal; Dean, Robert; Oh, Jean; DiBerardino, Chip; Keegan, Terence

2015-05-01

The Army Research Laboratory's Robotics Collaborative Technology Alliance (RCTA) is a program intended to change robots from tools that soldiers use into teammates with which soldiers can work. This requires the integration of fundamental and applied research in perception, artificial intelligence, and human-robot interaction. In October of 2014, the RCTA assessed progress towards integrating this research. This assessment was designed to evaluate the robot's performance when it used new capabilities to perform selected aspects of a mission. The assessed capabilities included the ability of the robot to: navigate semantically outdoors with respect to structures and landmarks, identify doors in the facades of buildings, and identify and track persons emerging from those doors. We present details of the mission-based vignettes that constituted the assessment, and evaluations of the robot's performance in these vignettes.

A case study by life cycle assessment

NASA Astrophysics Data System (ADS)

Li, Shuyun

2017-05-01

This article aims to assess the potential environmental impact of an electrical grinder during its life cycle. The Life Cycle Inventory Analysis was conducted based on the Simplified Life Cycle Assessment (SLCA) Drivers that calculated from the Valuation of Social Cost and Simplified Life Cycle Assessment Model (VSSM). The detailed results for LCI can be found under Appendix II. The Life Cycle Impact Assessment was performed based on Eco-indicator 99 method. The analysis results indicated that the major contributor to the environmental impact as it accounts for over 60% overall SLCA output. In which, 60% of the emission resulted from the logistic required for the maintenance activities. This was measured by conducting the hotspot analysis. After performing sensitivity analysis, it is evidenced that changing fuel type results in significant decrease environmental footprint. The environmental benefit can also be seen from the negative output values of the recycling activities. By conducting Life Cycle Assessment analysis, the potential environmental impact of the electrical grinder was investigated.

Safety analysis of interchanges

DOT National Transportation Integrated Search

2007-06-01

The objectives of this research are to synthesize the current state of knowledge concerning the safety assessment of new or modified interchanges; develop a spreadsheet-based computational tool for performing safety assessments of interchanges; and i...

Better Assessment Science Integrating Point and Non-point Sources (BASINS)

EPA Pesticide Factsheets

Better Assessment Science Integrating Point and Nonpoint Sources (BASINS) is a multipurpose environmental analysis system designed to help regional, state, and local agencies perform watershed- and water quality-based studies.

The assessment of eco-design with a comprehensive index incorporating environmental impact and economic profit

NASA Astrophysics Data System (ADS)

Yang, Shuo; Fu, Yun; Wang, Xiuteng; Xu, Bingsheng; Li, Zheng

2017-11-01

Eco-design is an advanced design approach which plays an important part in the national innovation project and serves as a key point for the successful transformation of the supply structure. However, the practical implementation of the pro-environmental designs and technologies always faces a dilemma situation, where some processes can effectively control their emissions to protect the environment at relatively high costs, while others pursue the individual interest in making profit by ignoring the possible adverse environmental impacts. Thus, the assessment on the eco-design process must be carried out based on the comprehensive consideration of the economic and environmental aspects. Presently, the assessment systems in China are unable to fully reflect the new environmental technologies regarding their innovative features or performance. Most of the assessment systems adopt scoring method based on the judgments of the experts, which are easy to use but somewhat subjective. The assessment method presented in this paper includes the environmental impact (EI) assessment based on LCA principal and willingness-to-pay theory, and economic profit (EP) assessment mainly based on market price. The results from the assessment are in the form of EI/EP, which evaluate the targeted process from a combined perspective of environmental and economic performance. A case study was carried out upon the utilization process of coal fly ash, which indicates the proposed method can compare different technical processes in an effective and objective manner, and provide explicit and insightful suggestions for decision making.

The assessment of professional competence: building blocks for theory development.

PubMed

van der Vleuten, C P M; Schuwirth, L W T; Scheele, F; Driessen, E W; Hodges, B

2010-12-01

This article presents lessons learnt from experiences with assessment of professional competence. Based on Miller's pyramid, a distinction is made between established assessment technology for assessing 'knows', 'knowing how' and 'showing how' and more recent developments in the assessment of (clinical) performance at the 'does' level. Some general lessons are derived from research of and experiences with the established assessment technology. Here, many paradoxes are revealed and empirical outcomes are often counterintuitive. Instruments for assessing the 'does' level are classified and described, and additional general lessons for this area of performance assessment are derived. These lessons can also be read as general principles of assessment (programmes) and may provide theoretical building blocks to underpin appropriate and state-of-the-art assessment practices. Copyright © 2010 Elsevier Ltd. All rights reserved.

Making the Grade in a Portfolio-Based System: Student Performance and the Student Perspective

PubMed Central

Nowacki, Amy S.

2013-01-01

Assessment is such an integral part of the educational system that we rarely reflect on its value and impact. Portfolios have gained in popularity, but much attention has emphasized the end-user and portfolio assessment. Here we focus on the portfolio creator (the student) and examine whether their educational needs are met with such an assessment method. This study aims to investigate how assessment practices influence classroom performance and the learning experience of the student in a graduate education setting. Studied were 33 medical students at the Cleveland Clinic Lerner College of Medicine of Case Western Reserve University, a program utilizing a portfolio-based system. The students may elect to simultaneously enroll in a Masters program; however, these programs employ traditional letter grades. Thus creating a unique opportunity to assess 25 portfolio only (P) students and 8 portfolio and grade (PG) students concurrently taking a course that counts for both programs. Classroom performance was measured via a comprehensive evaluation where the PG students scored modestly better (median total scores, 72% P vs. 76% PG). Additionally, a survey was conducted to gain insight into student’s perspective on how assessment method impacts the learning experience. The students in the PG group (those receiving a grade) reported increased stress but greater affirmation and self-assurance regarding their knowledge and skill mastery. Incorporation of such affirmation remains a challenge for portfolio-based systems and an area for investigation and improvement. PMID:23565103

Making the grade in a portfolio-based system: student performance and the student perspective.

PubMed

Nowacki, Amy S

2013-01-01

Assessment is such an integral part of the educational system that we rarely reflect on its value and impact. Portfolios have gained in popularity, but much attention has emphasized the end-user and portfolio assessment. Here we focus on the portfolio creator (the student) and examine whether their educational needs are met with such an assessment method. This study aims to investigate how assessment practices influence classroom performance and the learning experience of the student in a graduate education setting. Studied were 33 medical students at the Cleveland Clinic Lerner College of Medicine of Case Western Reserve University, a program utilizing a portfolio-based system. The students may elect to simultaneously enroll in a Masters program; however, these programs employ traditional letter grades. Thus creating a unique opportunity to assess 25 portfolio only (P) students and 8 portfolio and grade (PG) students concurrently taking a course that counts for both programs. Classroom performance was measured via a comprehensive evaluation where the PG students scored modestly better (median total scores, 72% P vs. 76% PG). Additionally, a survey was conducted to gain insight into student's perspective on how assessment method impacts the learning experience. The students in the PG group (those receiving a grade) reported increased stress but greater affirmation and self-assurance regarding their knowledge and skill mastery. Incorporation of such affirmation remains a challenge for portfolio-based systems and an area for investigation and improvement.

An Analysis of High School Students' Perceptions and Academic Performance in Laboratory Experiences

ERIC Educational Resources Information Center

Mirchin, Robert Douglas

2012-01-01

This research study is an investigation of student-laboratory (i.e., lab) learning based on students' perceptions of experiences using questionnaire data and evidence of their science-laboratory performance based on paper-and-pencil assessments using Maryland-mandated criteria, Montgomery County Public Schools (MCPS) criteria, and published…

A multilevel modeling approach to examining individual differences in skill acquisition for a computer-based task.

PubMed

Nair, Sankaran N; Czaja, Sara J; Sharit, Joseph

2007-06-01

This article explores the role of age, cognitive abilities, prior experience, and knowledge in skill acquisition for a computer-based simulated customer service task. Fifty-two participants aged 50-80 performed the task over 4 consecutive days following training. They also completed a battery that assessed prior computer experience and cognitive abilities. The data indicated that overall quality and efficiency of performance improved with practice. The predictors of initial level of performance and rate of change in performance varied according to the performance parameter assessed. Age and fluid intelligence predicted initial level and rate of improvement in overall quality, whereas crystallized intelligence and age predicted initial e-mail processing time, and crystallized intelligence predicted rate of change in e-mail processing time over days. We discuss the implications of these findings for the design of intervention strategies.

Effect of students' learning styles on classroom performance in problem-based learning.

PubMed

Alghasham, Abdullah A

2012-01-01

Since problem-based learning (PBL) sessions require a combination of active discussion, group interaction, and inductive and reflective thinking, students with different learning styles can be expected to perform differently in the PBL sessions. Using "Learning Style Inventory Questionnaire," students were divided into separate active and reflective learner groups. Tutors were asked to observe and assess the students' behavioral performance during the PBL sessions for a period of 5 weeks. A questionnaire of 24 items was developed to assess students' behavioral performance in PBL sessions. Active students tended to use multiple activities to obtain the needed information were more adjusted to the group norms and regulation and more skillful in using reasoning and problem-solving skills and in participation in discussion. On the other hand, reflective students used independent study more, listened actively and carefully to others and used previously acquired information in the discussion more frequently. Formative assessment quizzes did not indicate better performance of either group. There were no significant gender differences in PBL behavioral performance or quizzes' scores. Active and reflective learners differ in PBL class behavioral performance but not in the formative assessment. We recommend that students should be informed about their learning style and that they should learn strategies to compensate for any lacks in PBL sessions through self-study. Also, educational planners should ensure an adequate mix of students with different learning styles in the PBL groups to achieve PBL desired objectives.

Assessing Cognitive Ability and Simulator-Based Driving Performance in Poststroke Adults

PubMed Central

Falkmer, Torbjörn; Willstrand, Tania Dukic

2017-01-01

Driving is an important activity of daily living, which is increasingly relied upon as the population ages. It has been well-established that cognitive processes decline following a stroke and these processes may influence driving performance. There is much debate on the use of off-road neurological assessments and driving simulators as tools to predict driving performance; however, the majority of research uses unlicensed poststroke drivers, making the comparability of poststroke adults to that of a control group difficult. It stands to reason that in order to determine whether simulators and cognitive assessments can accurately assess driving performance, the baseline should be set by licenced drivers. Therefore, the aim of this study was to assess differences in cognitive ability and driving simulator performance in licensed community-dwelling poststroke drivers and controls. Two groups of licensed drivers (37 poststroke and 43 controls) were assessed using several cognitive tasks and using a driving simulator. The poststroke adults exhibited poorer cognitive ability; however, there were no differences in simulator performance between groups except that the poststroke drivers demonstrated less variability in driver headway. The application of these results as a prescreening toolbox for poststroke drivers is discussed. PMID:28559646

Preliminary Assessment of Detection Efficiency for the Geostationary Lightning Mapper Using Intercomparisons with Ground-Based Systems

NASA Technical Reports Server (NTRS)

Bateman, Monte; Mach, Douglas; Blakeslee, Richard J.; Koshak, William

2018-01-01

As part of the calibration/validation (cal/val) effort for the Geostationary Lightning Mapper (GLM) on GOES-16, we need to assess instrument performance (detection efficiency and accuracy). One major effort is to calculate the detection efficiency of GLM by comparing to multiple ground-based systems. These comparisons will be done pair-wise between GLM and each other source. A complication in this process is that the ground-based systems sense different properties of the lightning signal than does GLM (e.g., RF vs. optical). Also, each system has a different time and space resolution and accuracy. Preliminary results indicate that GLM is performing at or above its specification.

Performance Prediction of a MongoDB-Based Traceability System in Smart Factory Supply Chains

PubMed Central

Kang, Yong-Shin; Park, Il-Ha; Youm, Sekyoung

2016-01-01

In the future, with the advent of the smart factory era, manufacturing and logistics processes will become more complex, and the complexity and criticality of traceability will further increase. This research aims at developing a performance assessment method to verify scalability when implementing traceability systems based on key technologies for smart factories, such as Internet of Things (IoT) and BigData. To this end, based on existing research, we analyzed traceability requirements and an event schema for storing traceability data in MongoDB, a document-based Not Only SQL (NoSQL) database. Next, we analyzed the algorithm of the most representative traceability query and defined a query-level performance model, which is composed of response times for the components of the traceability query algorithm. Next, this performance model was solidified as a linear regression model because the response times increase linearly by a benchmark test. Finally, for a case analysis, we applied the performance model to a virtual automobile parts logistics. As a result of the case study, we verified the scalability of a MongoDB-based traceability system and predicted the point when data node servers should be expanded in this case. The traceability system performance assessment method proposed in this research can be used as a decision-making tool for hardware capacity planning during the initial stage of construction of traceability systems and during their operational phase. PMID:27983654

Performance Prediction of a MongoDB-Based Traceability System in Smart Factory Supply Chains.

PubMed

Kang, Yong-Shin; Park, Il-Ha; Youm, Sekyoung

2016-12-14

In the future, with the advent of the smart factory era, manufacturing and logistics processes will become more complex, and the complexity and criticality of traceability will further increase. This research aims at developing a performance assessment method to verify scalability when implementing traceability systems based on key technologies for smart factories, such as Internet of Things (IoT) and BigData. To this end, based on existing research, we analyzed traceability requirements and an event schema for storing traceability data in MongoDB, a document-based Not Only SQL (NoSQL) database. Next, we analyzed the algorithm of the most representative traceability query and defined a query-level performance model, which is composed of response times for the components of the traceability query algorithm. Next, this performance model was solidified as a linear regression model because the response times increase linearly by a benchmark test. Finally, for a case analysis, we applied the performance model to a virtual automobile parts logistics. As a result of the case study, we verified the scalability of a MongoDB-based traceability system and predicted the point when data node servers should be expanded in this case. The traceability system performance assessment method proposed in this research can be used as a decision-making tool for hardware capacity planning during the initial stage of construction of traceability systems and during their operational phase.

Quantitative assessments of municipal waste management systems: using different indicators to compare and rank programs in New York State.

PubMed

Greene, Krista L; Tonjes, David J

2014-04-01

The primary objective of waste management technologies and policies in the United States is to reduce the harmful environmental impacts of waste, particularly those relating to energy consumption and climate change. Performance indicators are frequently used to evaluate the environmental quality of municipal waste systems, as well as to compare and rank programs relative to each other in terms of environmental performance. However, there currently is no consensus on the best indicator for performing these environmental evaluations. The purpose of this study is to examine the common performance indicators used to assess the environmental benefits of municipal waste systems to determine if there is agreement between them regarding which system performs best environmentally. Focus is placed on how indicator selection influences comparisons between municipal waste management programs and subsequent system rankings. The waste systems of ten municipalities in the state of New York, USA, were evaluated using each common performance indicator and Spearman correlations were calculated to see if there was a significant association between system rank orderings. Analyses showed that rank orders of waste systems differ substantially when different indicators are used. Therefore, comparative system assessments based on indicators should be considered carefully, especially those intended to gauge environmental quality. Insight was also gained into specific factors which may lead to one system achieving higher rankings than another. However, despite the insufficiencies of indicators for comparative quality assessments, they do provide important information for waste managers and they can assist in evaluating internal programmatic performance and progress. To enhance these types of assessments, a framework for scoring indicators based on criteria that evaluate their utility and value for system evaluations was developed. This framework was used to construct an improved model for waste system performance assessments. Copyright © 2014 Elsevier Ltd. All rights reserved.

Relationships Between the Way Students Are Assessed in Science Classrooms and Science Achievement Across Canada

NASA Astrophysics Data System (ADS)

Chu, Man-Wai; Fung, Karen

2018-04-01

Canadian students experience many different assessments throughout their schooling (O'Connor 2011). There are many benefits to using a variety of assessment types, item formats, and science-based performance tasks in the classroom to measure the many dimensions of science education. Although using a variety of assessments is beneficial, it is unclear exactly what types, format, and tasks are used in Canadian science classrooms. Additionally, since assessments are often administered to help improve student learning, this study identified assessments that may improve student learning as measured using achievement scores on a standardized test. Secondary analyses of the students' and teachers' responses to the questionnaire items asked in the Pan-Canadian Assessment Program were performed. The results of the hierarchical linear modeling analyses indicated that both students and teachers identified teacher-developed classroom tests or quizzes as the most common types of assessments used. Although this ranking was similar across the country, statistically significant differences in terms of the assessments that are used in science classrooms among the provinces were also identified. The investigation of which assessment best predicted student achievement scores indicated that minds-on science performance-based tasks significantly explained 4.21% of the variance in student scores. However, mixed results were observed between the student and teacher responses towards tasks that required students to choose their own investigation and design their own experience or investigation. Additionally, teachers that indicated that they conducted more demonstrations of an experiment or investigation resulted in students with lower scores.

Reliable and valid assessment of Lichtenstein hernia repair skills.

PubMed

Carlsen, C G; Lindorff-Larsen, K; Funch-Jensen, P; Lund, L; Charles, P; Konge, L

2014-08-01

Lichtenstein hernia repair is a common surgical procedure and one of the first procedures performed by a surgical trainee. However, formal assessment tools developed for this procedure are few and sparsely validated. The aim of this study was to determine the reliability and validity of an assessment tool designed to measure surgical skills in Lichtenstein hernia repair. Key issues were identified through a focus group interview. On this basis, an assessment tool with eight items was designed. Ten surgeons and surgical trainees were video recorded while performing Lichtenstein hernia repair, (four experts, three intermediates, and three novices). The videos were blindly and individually assessed by three raters (surgical consultants) using the assessment tool. Based on these assessments, validity and reliability were explored. The internal consistency of the items was high (Cronbach's alpha = 0.97). The inter-rater reliability was very good with an intra-class correlation coefficient (ICC) = 0.93. Generalizability analysis showed a coefficient above 0.8 even with one rater. The coefficient improved to 0.92 if three raters were used. One-way analysis of variance found a significant difference between the three groups which indicates construct validity, p < 0.001. Lichtenstein hernia repair skills can be assessed blindly by a single rater in a reliable and valid fashion with the new procedure-specific assessment tool. We recommend this tool for future assessment of trainees performing Lichtenstein hernia repair to ensure that the objectives of competency-based surgical training are met.

Tablet-Based Math Assessment: What Can We Learn from Math Apps?

ERIC Educational Resources Information Center

Cayton-Hodges, Gabrielle A.; Feng, Gary; Pan, Xingyu

2015-01-01

In this report, we describe a survey of mathematics education apps in the Apple App Store, conducted as part of a research project to develop a tablet-based assessment prototype for elementary mathematics. This survey was performed with the goal of understanding the design principles and techniques used in mathematics apps designed for tablets. We…

Promoting Formative Assessment in a Connected Classroom Environment: Design and Implementation of Digital Resources

ERIC Educational Resources Information Center

Cusi, Annalisa; Morselli, Francesca; Sabena, Cristina

2017-01-01

This paper is based on a design-based research project investigating how to use digital resources to help activate formative assessment processes in the classroom. Performed as part of FaSMEd, a European Union project, our own project adopts a comprehensive theoretical framework, including the different functionalities of technology, formative…

A Longitudinal Assessment of an Initial Cohort in a Psychology Learning Community

ERIC Educational Resources Information Center

Buch, Kim; Spaulding, Sue

2008-01-01

Discipline-based learning communities have become a popular strategy for improving student performance and satisfaction. This article describes the goals and features of a university-based, first-year psychology learning community (PLC) implemented in Fall 2003. We also report the results of a longitudinal assessment of the impact of the PLC on…

Marking Essays on Screen: An Investigation into the Reliability of Marking Extended Subjective Texts

ERIC Educational Resources Information Center

Johnson, Martin; Nadas, Rita; Bell, John F.

2010-01-01

There is a growing body of research literature that considers how the mode of assessment, either computer-based or paper-based, might affect candidates' performances. Despite this, there is a fairly narrow literature that shifts the focus of attention to those making assessment judgements and which considers issues of assessor consistency when…

An Evaluation of Test Speededness in an Assessment for Third-Grade Gifted Students

ERIC Educational Resources Information Center

Hailey, Emily; Callahan, Carolyn M.; Azano, Amy; Moon, Tonya R.

2012-01-01

Reliability and validity are integral concepts in assessment design. Test speededness, the influence of time constraints on test taker performance, is often an overlooked threat to reliability and validity, especially in classroom-based testing. The purpose of this study is to evaluate the degree of test speededness of classroom-based assessments…

Systematic review of skills transfer after surgical simulation-based training.

PubMed

Dawe, S R; Pena, G N; Windsor, J A; Broeders, J A J L; Cregan, P C; Hewett, P J; Maddern, G J

2014-08-01

Simulation-based training assumes that skills are directly transferable to the patient-based setting, but few studies have correlated simulated performance with surgical performance. A systematic search strategy was undertaken to find studies published since the last systematic review, published in 2007. Inclusion of articles was determined using a predetermined protocol, independent assessment by two reviewers and a final consensus decision. Studies that reported on the use of surgical simulation-based training and assessed the transferability of the acquired skills to a patient-based setting were included. Twenty-seven randomized clinical trials and seven non-randomized comparative studies were included. Fourteen studies investigated laparoscopic procedures, 13 endoscopic procedures and seven other procedures. These studies provided strong evidence that participants who reached proficiency in simulation-based training performed better in the patient-based setting than their counterparts who did not have simulation-based training. Simulation-based training was equally as effective as patient-based training for colonoscopy, laparoscopic camera navigation and endoscopic sinus surgery in the patient-based setting. These studies strengthen the evidence that simulation-based training, as part of a structured programme and incorporating predetermined proficiency levels, results in skills transfer to the operative setting. © 2014 BJS Society Ltd. Published by John Wiley & Sons Ltd.

Surgical skill acquisition with self-directed practice using computer-based video training.

PubMed

Jowett, Nathan; LeBlanc, Vicki; Xeroulis, George; MacRae, Helen; Dubrowski, Adam

2007-02-01

Computer-based video training (CBVT) provides flexible opportunities for surgical trainees to learn fundamental technical skills, but may be ineffective in self-directed practice settings because of poor trainee self-assessment. This study examined whether CBVT is effective in a self-directed learning environment among novice trainees. Thirty novice trainees used CBVT to learn the 1-handed square knot while self-assessing their proficiency every 3 minutes. On reaching self-assessed skill proficiency, trainees were randomized to either cease practice or to complete additional practice. Performance was evaluated with computer and expert-based measures during practice and on pretests, posttests, and 1-week retention tests. Analyses revealed performance improvements for both groups (all P < .05), but no differences between the 2 groups (all P > .05) on all tests. CBVT for the 1-handed square knot is effective in a self-directed learning environment among novices. This lends support to the implementation of self-directed digital media-based learning within surgical curricula.

Validating workplace performance assessments in health sciences students: a case study from speech pathology.

PubMed

McAllister, Sue; Lincoln, Michelle; Ferguson, Allison; McAllister, Lindy

2013-01-01

Valid assessment of health science students' ability to perform in the real world of workplace practice is critical for promoting quality learning and ultimately certifying students as fit to enter the world of professional practice. Current practice in performance assessment in the health sciences field has been hampered by multiple issues regarding assessment content and process. Evidence for the validity of scores derived from assessment tools are usually evaluated against traditional validity categories with reliability evidence privileged over validity, resulting in the paradoxical effect of compromising the assessment validity and learning processes the assessments seek to promote. Furthermore, the dominant statistical approaches used to validate scores from these assessments fall under the umbrella of classical test theory approaches. This paper reports on the successful national development and validation of measures derived from an assessment of Australian speech pathology students' performance in the workplace. Validation of these measures considered each of Messick's interrelated validity evidence categories and included using evidence generated through Rasch analyses to support score interpretation and related action. This research demonstrated that it is possible to develop an assessment of real, complex, work based performance of speech pathology students, that generates valid measures without compromising the learning processes the assessment seeks to promote. The process described provides a model for other health professional education programs to trial.

A model of four hierarchical levels to train Chinese residents' teaching skills for "practice-based learning and improvement" competency.

PubMed

Yang, Ying-Ying; Yang, Ling-Yu; Hsu, Hui-Chi; Huang, Chia-Chang; Huang, Chin-Chou; Kirby, Ralph; Cheng, Hao Min; Chang, Ching-Chi; Chuang, Chiao-Lin; Liang, Jen-Feng; Lin, Chun-Chi; Lee, Wei-Shin; Ho, Shung-Tai; Lee, Fa-Yauh

2015-01-01

The current study focused on validating a protocol for training and auditing the resident's practice-based learning and improvement (PBLI) and quality improvement (QI) competencies for primary care. Twelve second-year (R2), 12 first-year (R1) and 12 postgraduate year-1 residents were enrolled into group A, B and C, respectively, as trainees. After three training protocols had been completed, a writing test, self-assessed questionnaire and mini-OSTE and end-of-rotation assessment were used in auditing the PBLI competency, performance and teaching ability of trainees. Baseline expert-assessed PBLI and QI knowledge application tool writing scores were low for the R1 and R2 residents. After three training protocols, PBLI and QI proficiencies, performance and teaching abilities were improved to similar levels cross the three training levels of residents based on the expert-assessed writing test-audited assessments and on the faculty and standardized clerk-assessed end-of-rotation-/mini-OSTE-audited assessments. The different four-level hierarchical protocols used to teach group A, B and C were equally beneficial and fitted their needs; namely the different levels of the trainees. Specifically, each level was able to augment their PBLI and QI proficiency. This educational intervention helps medical institutions to train residents as PBLI instructors.

Critique of Research Assessments

ERIC Educational Resources Information Center

Currie, Jan

2008-01-01

This article introduces the issue of articles on research assessments and rankings. It focuses on the British Research Assessment Exercises (RAEs) and New Zealand's Performance Based Research Fund (PBRF). It contrasts these with systems that are not tied to funding formulas and emphasise quality enhancement rather than quality assurance. It…

Assessment of school mathematics: Teachers' perceptions and practices

NASA Astrophysics Data System (ADS)

Pfannkuch, Maxine

2001-12-01

This is the first report of a proposed ten-year interval longitudinal study about teacher assessment practice in Auckland, New Zealand. Interviews with teachers of Year 3, 6, 8, 10, and 13 students are analysed. These interviews indicate that primary teachers are using a variety of assessment strategies in a mastery-based system. Their judgement of mathematical performance is dominated by the belief that all students must feel that they are achieving. The secondary teacher interviews indicate common use of alternative assessment strategies in non-examination classes. Judgement of student performance is benchmarked against national examinations. It is conjectured that an education system effect determines teachers' assessment practices.

Mental practice enhances surgical technical skills: a randomized controlled study.

PubMed

Arora, Sonal; Aggarwal, Rajesh; Sirimanna, Pramudith; Moran, Aidan; Grantcharov, Teodor; Kneebone, Roger; Sevdalis, Nick; Darzi, Ara

2011-02-01

To assess the effects of mental practice on surgical performance. Increasing concerns for patient safety have highlighted a need for alternative training strategies outside the operating room. Mental practice (MP), "the cognitive rehearsal of a task before performance," has been successful in sport and music to enhance skill. This study investigates whether MP enhances performance in laparoscopic surgery. After baseline skills testing, 20 novice surgeons underwent training on an evidence-based virtual reality curriculum. After randomization using the closed envelope technique, all participants performed 5 Virtual Reality (VR) laparoscopic cholecystectomies (LC). Mental practice participants performed 30 minutes of MP before each LC; control participants viewed an online lecture. Technical performance was assessed using video Objective Structured Assessment of Technical Skills-based global ratings scale (scored from 7 to 35). Mental imagery was assessed using a previously validated Mental Imagery Questionnaire. Eighteen participants completed the study. There were no intergroup differences in baseline technical ability. Learning curves were demonstrated for both MP and control groups. Mental practice was superior to control (global ratings) for the first LC (median 20 vs 15, P = 0.005), second LC (20.5 vs 13.5, P = 0.001), third LC (24 vs 15.5, P < 0.001), fourth LC (25.5 vs 15.5, P < 0.001) and the fifth LC (27.5 vs 19.5, P = 0.00). The imagery for the MP group was also significantly superior to the control group across all sessions (P < 0.05). Improved imagery significantly correlated with better quality of performance (ρ 0.51–0.62, Ps < 0.05). This is the first randomized controlled study to show that MP enhances the quality of performance based on VR laparoscopic cholecystectomy. This may be a time- and cost-effective strategy to augment traditional training in the OR thus potentially improving patient care.

Predicting Airport Screening Officers' Visual Search Competency With a Rapid Assessment.

PubMed

Mitroff, Stephen R; Ericson, Justin M; Sharpe, Benjamin

2018-03-01

Objective The study's objective was to assess a new personnel selection and assessment tool for aviation security screeners. A mobile app was modified to create a tool, and the question was whether it could predict professional screeners' on-job performance. Background A variety of professions (airport security, radiology, the military, etc.) rely on visual search performance-being able to detect targets. Given the importance of such professions, it is necessary to maximize performance, and one means to do so is to select individuals who excel at visual search. A critical question is whether it is possible to predict search competency within a professional search environment. Method Professional searchers from the USA Transportation Security Administration (TSA) completed a rapid assessment on a tablet-based X-ray simulator (XRAY Screener, derived from the mobile technology app Airport Scanner; Kedlin Company). The assessment contained 72 trials that were simulated X-ray images of bags. Participants searched for prohibited items and tapped on them with their finger. Results Performance on the assessment significantly related to on-job performance measures for the TSA officers such that those who were better XRAY Screener performers were both more accurate and faster at the actual airport checkpoint. Conclusion XRAY Screener successfully predicted on-job performance for professional aviation security officers. While questions remain about the underlying cognitive mechanisms, this quick assessment was found to significantly predict on-job success for a task that relies on visual search performance. Application It may be possible to quickly assess an individual's visual search competency, which could help organizations select new hires and assess their current workforce.

The influence of test mode and visuospatial ability on mathematics assessment performance

NASA Astrophysics Data System (ADS)

Logan, Tracy

2015-12-01

Mathematics assessment and testing are increasingly situated within digital environments with international tests moving to computer-based testing in the near future. This paper reports on a secondary data analysis which explored the influence the mode of assessment—computer-based (CBT) and pencil-and-paper based (PPT)—and visuospatial ability had on students' mathematics test performance. Data from 804 grade 6 Singaporean students were analysed using the knowledge discovery in data design. The results revealed statistically significant differences between performance on CBT and PPT test modes across content areas concerning whole number algebraic patterns and data and chance. However, there were no performance differences for content areas related to spatial arrangements geometric measurement or other number. There were also statistically significant differences in performance between those students who possess higher levels of visuospatial ability compared to those with lower levels across all six content areas. Implications include careful consideration for the comparability of CBT and PPT testing and the need for increased attention to the role of visuospatial reasoning in student's mathematics reasoning.

Video and accelerometer-based motion analysis for automated surgical skills assessment.

PubMed

Zia, Aneeq; Sharma, Yachna; Bettadapura, Vinay; Sarin, Eric L; Essa, Irfan

2018-03-01

Basic surgical skills of suturing and knot tying are an essential part of medical training. Having an automated system for surgical skills assessment could help save experts time and improve training efficiency. There have been some recent attempts at automated surgical skills assessment using either video analysis or acceleration data. In this paper, we present a novel approach for automated assessment of OSATS-like surgical skills and provide an analysis of different features on multi-modal data (video and accelerometer data). We conduct a large study for basic surgical skill assessment on a dataset that contained video and accelerometer data for suturing and knot-tying tasks. We introduce "entropy-based" features-approximate entropy and cross-approximate entropy, which quantify the amount of predictability and regularity of fluctuations in time series data. The proposed features are compared to existing methods of Sequential Motion Texture, Discrete Cosine Transform and Discrete Fourier Transform, for surgical skills assessment. We report average performance of different features across all applicable OSATS-like criteria for suturing and knot-tying tasks. Our analysis shows that the proposed entropy-based features outperform previous state-of-the-art methods using video data, achieving average classification accuracies of 95.1 and 92.2% for suturing and knot tying, respectively. For accelerometer data, our method performs better for suturing achieving 86.8% average accuracy. We also show that fusion of video and acceleration features can improve overall performance for skill assessment. Automated surgical skills assessment can be achieved with high accuracy using the proposed entropy features. Such a system can significantly improve the efficiency of surgical training in medical schools and teaching hospitals.

Reliability of a computer-based system for measuring visual performance skills.

PubMed

Erickson, Graham B; Citek, Karl; Cove, Michelle; Wilczek, Jennifer; Linster, Carolyn; Bjarnason, Brendon; Langemo, Nathan

2011-09-01

Athletes have demonstrated better visual abilities than nonathletes. A vision assessment for an athlete should include methods to evaluate the quality of visual performance skills in the most appropriate, accurate, and repeatable manner. This study determines the reliability of the visual performance measures assessed with a computer-based system, known as the Nike Sensory Station. One hundred twenty-five subjects (56 men, 69 women), age 18 to 30, completed Phase I of the study. Subjects attended 2 sessions, separated by at least 1 week, in which identical protocols were followed. Subjects completed the following assessments: Visual Clarity, Contrast Sensitivity, Depth Perception, Near-Far Quickness, Target Capture, Perception Span, Eye-Hand Coordination, Go/No Go, and Reaction Time. An additional 36 subjects (20 men, 16 women), age 22 to 35, completed Phase II of the study involving modifications to the equipment, instructions, and protocols from Phase I. Results show no significant change in performance over time on assessments of Visual Clarity, Contrast Sensitivity, Depth Perception, Target Capture, Perception Span, and Reaction Time. Performance did improve over time for Near-Far Quickness, Eye-Hand Coordination, and Go/No Go. The results of this study show that many of the Nike Sensory Station assessments show repeatability and no learning effect over time. The measures that did improve across sessions show an expected learning effect caused by the motor response characteristics being measured. Copyright © 2011 American Optometric Association. Published by Elsevier Inc. All rights reserved.

Comparison of virtual patient simulation with mannequin-based simulation for improving clinical performances in assessing and managing clinical deterioration: randomized controlled trial.

PubMed

Liaw, Sok Ying; Chan, Sally Wai-Chi; Chen, Fun-Gee; Hooi, Shing Chuan; Siau, Chiang

2014-09-17

Virtual patient simulation has grown substantially in health care education. A virtual patient simulation was developed as a refresher training course to reinforce nursing clinical performance in assessing and managing deteriorating patients. The objective of this study was to describe the development of the virtual patient simulation and evaluate its efficacy, by comparing with a conventional mannequin-based simulation, for improving the nursing students' performances in assessing and managing patients with clinical deterioration. A randomized controlled study was conducted with 57 third-year nursing students who were recruited through email. After a baseline evaluation of all participants' clinical performance in a simulated environment, the experimental group received a 2-hour fully automated virtual patient simulation while the control group received 2-hour facilitator-led mannequin-based simulation training. All participants were then re-tested one day (first posttest) and 2.5 months (second posttest) after the intervention. The participants from the experimental group completed a survey to evaluate their learning experiences with the newly developed virtual patient simulation. Compared to their baseline scores, both experimental and control groups demonstrated significant improvements (P<.001) in first and second post-test scores. While the experimental group had significantly lower (P<.05) second post-test scores compared with the first post-test scores, no significant difference (P=.94) was found between these two scores for the control group. The scores between groups did not differ significantly over time (P=.17). The virtual patient simulation was rated positively. A virtual patient simulation for a refreshing training course on assessing and managing clinical deterioration was developed. Although the randomized controlled study did not show that the virtual patient simulation was superior to mannequin-based simulation, both simulations have demonstrated to be effective refresher learning strategies for improving nursing students' clinical performance. Given the greater resource requirements of mannequin-based simulation, the virtual patient simulation provides a more promising alternative learning strategy to mitigate the decay of clinical performance over time.

Looking Under the Streetlight? A Framework for Differentiating Performance Measures by Level of Care in a Value-Based Payment Environment

PubMed Central

Van Such, Monica B.; Nesse, Robert E.; Dilling, James A.; Swensen, Stephen J.; Thompson, Kristine M.; Orlowski, Janis M.; Santrach, Paula J.

2017-01-01

The majority of quality measures used to assess providers and hospitals are based on easily obtained data, focused on a few dimensions of quality, and developed mainly for primary/community care and population health. While this approach supports efforts focused on addressing the triple aim of health care, many current quality report cards and assessments do not reflect the breadth or complexity of many referral center practices. In this article, the authors highlight the differences between population health efforts and referral care and address issues related to value measurement and performance assessment. They discuss why measures may need to differ across the three levels of care (primary/community care, secondary care, complex care) and illustrate the need for further risk adjustment to eliminate referral bias. With continued movement toward value-based purchasing, performance measures and reimbursement schemes need to reflect the increased level of intensity required to provide complex care. The authors propose a framework to operationalize value measurement and payment for specialty care, and they make specific recommendations to improve performance measurement for complex patients. Implementing such a framework to differentiate performance measures by level of care involves coordinated efforts to change both policy and operational platforms. An essential component of this framework is a new model that defines the characteristics of patients who require complex care and standardizes metrics that incorporate those definitions. PMID:28353502

Looking Under the Streetlight? A Framework for Differentiating Performance Measures by Level of Care in a Value-Based Payment Environment.

PubMed

Naessens, James M; Van Such, Monica B; Nesse, Robert E; Dilling, James A; Swensen, Stephen J; Thompson, Kristine M; Orlowski, Janis M; Santrach, Paula J

2017-07-01

The majority of quality measures used to assess providers and hospitals are based on easily obtained data, focused on a few dimensions of quality, and developed mainly for primary/community care and population health. While this approach supports efforts focused on addressing the triple aim of health care, many current quality report cards and assessments do not reflect the breadth or complexity of many referral center practices.In this article, the authors highlight the differences between population health efforts and referral care and address issues related to value measurement and performance assessment. They discuss why measures may need to differ across the three levels of care (primary/community care, secondary care, complex care) and illustrate the need for further risk adjustment to eliminate referral bias.With continued movement toward value-based purchasing, performance measures and reimbursement schemes need to reflect the increased level of intensity required to provide complex care. The authors propose a framework to operationalize value measurement and payment for specialty care, and they make specific recommendations to improve performance measurement for complex patients. Implementing such a framework to differentiate performance measures by level of care involves coordinated efforts to change both policy and operational platforms. An essential component of this framework is a new model that defines the characteristics of patients who require complex care and standardizes metrics that incorporate those definitions.

Solar industrial process heat systems: An assessment of standards for materials and components

NASA Astrophysics Data System (ADS)

Rossiter, W. J.; Shipp, W. E.

1981-09-01

A study was conducted to obtain information on the performance of materials and components in operational solar industrial process heat (PH) systems, and to provide recommendations for the development of standards including evaluative test procedures for materials and components. An assessment of the needs for standards for evaluating the long-term performance of materials and components of IPH systems was made. The assessment was based on the availability of existing standards, and information obtained from a field survey of operational systems, the literature, and discussions with individuals in the industry. Field inspections of 10 operational IPH systems were performed.

What Is the Relationship Between a Preclerkship Portfolio Review and Later Performance in Clerkships?

PubMed

O'Brien, Celia Laird; Thomas, John X; Green, Marianne M

2018-01-01

Medical educators struggle to find effective ways to assess essential competencies such as communication, professionalism, and teamwork. Portfolio-based assessment provides one method of addressing this problem by allowing faculty reviewers to judge performance, as based on a longitudinal record of student behavior. At the Feinberg School of Medicine, the portfolio system measures behavioral competence using multiple assessments collected over time. This study examines whether a preclerkship portfolio review is a valid method of identifying problematic student behavior affecting later performance in clerkships. The authors divided students into two groups based on a summative preclerkship portfolio review in 2014: students who had concerning behavior in one or more competencies and students progressing satisfactorily. They compared how students in these groups later performed on two clerkship outcomes as of October 2015: final grades in required clerkships, and performance on a clerkship clinical composite score. They used Mann-Whitney tests and multiple linear regression to examine the relationship between portfolio review results and clerkship outcomes. They used USMLE Step 1 to control for knowledge acquisition. Students with concerning behavior preclerkship received significantly lower clerkship grades than students progressing satisfactorily (P = .002). They also scored significantly lower on the clinical composite score (P < .001). Regression analysis indicated concerning behavior was associated with lower clinical composite scores, even after controlling for knowledge acquisition. The results show a preclerkship portfolio review can identify behaviors that impact clerkship performance. A comprehensive portfolio system is a valid way to measure behavioral competencies.

SMART empirical approaches for predicting field performance of PV modules from results of reliability tests

NASA Astrophysics Data System (ADS)

Hardikar, Kedar Y.; Liu, Bill J. J.; Bheemreddy, Venkata

2016-09-01

Gaining an understanding of degradation mechanisms and their characterization are critical in developing relevant accelerated tests to ensure PV module performance warranty over a typical lifetime of 25 years. As newer technologies are adapted for PV, including new PV cell technologies, new packaging materials, and newer product designs, the availability of field data over extended periods of time for product performance assessment cannot be expected within the typical timeframe for business decisions. In this work, to enable product design decisions and product performance assessment for PV modules utilizing newer technologies, Simulation and Mechanism based Accelerated Reliability Testing (SMART) methodology and empirical approaches to predict field performance from accelerated test results are presented. The method is demonstrated for field life assessment of flexible PV modules based on degradation mechanisms observed in two accelerated tests, namely, Damp Heat and Thermal Cycling. The method is based on design of accelerated testing scheme with the intent to develop relevant acceleration factor models. The acceleration factor model is validated by extensive reliability testing under different conditions going beyond the established certification standards. Once the acceleration factor model is validated for the test matrix a modeling scheme is developed to predict field performance from results of accelerated testing for particular failure modes of interest. Further refinement of the model can continue as more field data becomes available. While the demonstration of the method in this work is for thin film flexible PV modules, the framework and methodology can be adapted to other PV products.

Assessment of Template-Based Modeling of Protein Structure in CASP11

PubMed Central

Modi, Vivek; Xu, Qifang; Adhikari, Sam; Dunbrack, Roland L.

2016-01-01

We present the assessment of predictions submitted in the template-based modeling (TBM) category of CASP11 (Critical Assessment of Protein Structure Prediction). Model quality was judged on the basis of global and local measures of accuracy on all atoms including side chains. The top groups on 39 human-server targets based on model 1 predictions were LEER, Zhang, LEE, MULTICOM, and Zhang-Server. The top groups on 81 targets by server groups based on model 1 predictions were Zhang-Server, nns, BAKER-ROSETTASERVER, QUARK, and myprotein-me. In CASP11, the best models for most targets were equal to or better than the best template available in the Protein Data Bank, even for targets with poor templates. The overall performance in CASP11 is similar to the performance of predictors in CASP10 with slightly better performance on the hardest targets. For most targets, assessment measures exhibited bimodal probability density distributions. Multi-dimensional scaling of an RMSD matrix for each target typically revealed a single cluster with models similar to the target structure, with a mode in the GDT-TS density between 40 and 90, and a wide distribution of models highly divergent from each other and from the experimental structure, with density mode at a GDT-TS value of ~20. The models in this peak in the density were either compact models with entirely the wrong fold, or highly non-compact models. The results argue for a density-driven approach in future CASP TBM assessments that accounts for the bimodal nature of these distributions instead of Z-scores, which assume a unimodal, Gaussian distribution. PMID:27081927

Validity evidence for an OSCE to assess competency in systems-based practice and practice-based learning and improvement: a preliminary investigation.

PubMed

Varkey, Prathibha; Natt, Neena; Lesnick, Timothy; Downing, Steven; Yudkowsky, Rachel

2008-08-01

To determine the psychometric properties and validity of an OSCE to assess the competencies of Practice-Based Learning and Improvement (PBLI) and Systems-Based Practice (SBP) in graduate medical education. An eight-station OSCE was piloted at the end of a three-week Quality Improvement elective for nine preventive medicine and endocrinology fellows at Mayo Clinic. The stations assessed performance in quality measurement, root cause analysis, evidence-based medicine, insurance systems, team collaboration, prescription errors, Nolan's model, and negotiation. Fellows' performance in each of the stations was assessed by three faculty experts using checklists and a five-point global competency scale. A modified Angoff procedure was used to set standards. Evidence for the OSCE's validity, feasibility, and acceptability was gathered. Evidence for content and response process validity was judged as excellent by institutional content experts. Interrater reliability of scores ranged from 0.85 to 1 for most stations. Interstation correlation coefficients ranged from -0.62 to 0.99, reflecting case specificity. Implementation cost was approximately $255 per fellow. All faculty members agreed that the OSCE was realistic and capable of providing accurate assessments. The OSCE provides an opportunity to systematically sample the different subdomains of Quality Improvement. Furthermore, the OSCE provides an opportunity for the demonstration of skills rather than the testing of knowledge alone, thus making it a potentially powerful assessment tool for SBP and PBLI. The study OSCE was well suited to assess SBP and PBLI. The evidence gathered through this study lays the foundation for future validation work.

A systematic review of a functional assessment Tool: UCSD Performance-based skill assessment (UPSA).

PubMed

Becattini-Oliveira, Ana Claudia; Dutra, Douglas de Farias; Spenciere de Oliveira Campos, Bárbara; de Araujo, Verônica Carvalho; Charchat-Fichman, Helenice

2018-05-18

Performance based assessment instruments have been employed in functional capacity measurement of mental disorders. The aim of this systematic review was to identify the psychometric properties of the UCSD Performance-based Skill Assessment (UPSA). A search was conducted using the PRISMA protocol and 'UPSA' as key word term on electronic databases, with a date range for articles published from 2001-2017. Published studies involving community-dwelling adults were included. Pharmacological and/or clinical interventions involving clinical outcomes and/or institutionalized samples were excluded. Data related to construct validity, test-retest reliability and sensitivity/specificity were extracted, summarized and analyzed according to UPSA versions and psychiatric disorders. Fifty-eight studies including 8782 Community-dwelling adults met selection criteria. Data supporting the construct and known-groups validity were extracted from 41 studies involving Schizophrenia and schizoaffective disorders and 17 studies involving other metal illness. The UPSA was culturally adapted to 8 different languages and employed in 17 countries. Few studies reported sensitivity and specificity and the cut-off points could not be generalized. Moderate to strong evidence of construct validity and test-retest reliability was found. Few studies proposed cut-off points. The UPSA showed good psychometric properties in different versions including those culturally adapted. Copyright © 2018 Elsevier B.V. All rights reserved.

A New Network Modeling Tool for the Ground-based Nuclear Explosion Monitoring Community

NASA Astrophysics Data System (ADS)

Merchant, B. J.; Chael, E. P.; Young, C. J.

2013-12-01

Network simulations have long been used to assess the performance of monitoring networks to detect events for such purposes as planning station deployments and network resilience to outages. The standard tool has been the SAIC-developed NetSim package. With correct parameters, NetSim can produce useful simulations; however, the package has several shortcomings: an older language (FORTRAN), an emphasis on seismic monitoring with limited support for other technologies, limited documentation, and a limited parameter set. Thus, we are developing NetMOD (Network Monitoring for Optimal Detection), a Java-based tool designed to assess the performance of ground-based networks. NetMOD's advantages include: coded in a modern language that is multi-platform, utilizes modern computing performance (e.g. multi-core processors), incorporates monitoring technologies other than seismic, and includes a well-validated default parameter set for the IMS stations. NetMOD is designed to be extendable through a plugin infrastructure, so new phenomenological models can be added. Development of the Seismic Detection Plugin is being pursued first. Seismic location and infrasound and hydroacoustic detection plugins will follow. By making NetMOD an open-release package, it can hopefully provide a common tool that the monitoring community can use to produce assessments of monitoring networks and to verify assessments made by others.

Flight simulator fidelity assessment in a rotorcraft lateral translation maneuver

NASA Technical Reports Server (NTRS)

Hess, R. A.; Malsbury, T.; Atencio, A., Jr.

1992-01-01

A model-based methodology for assessing flight simulator fidelity in closed-loop fashion is exercised in analyzing a rotorcraft low-altitude maneuver for which flight test and simulation results were available. The addition of a handling qualities sensitivity function to a previously developed model-based assessment criteria allows an analytical comparison of both performance and handling qualities between simulation and flight test. Model predictions regarding the existence of simulator fidelity problems are corroborated by experiment. The modeling approach is used to assess analytically the effects of modifying simulator characteristics on simulator fidelity.

High-Accuracy Decoupling Estimation of the Systematic Coordinate Errors of an INS and Intensified High Dynamic Star Tracker Based on the Constrained Least Squares Method

PubMed Central

Jiang, Jie; Yu, Wenbo; Zhang, Guangjun

2017-01-01

Navigation accuracy is one of the key performance indicators of an inertial navigation system (INS). Requirements for an accuracy assessment of an INS in a real work environment are exceedingly urgent because of enormous differences between real work and laboratory test environments. An attitude accuracy assessment of an INS based on the intensified high dynamic star tracker (IHDST) is particularly suitable for a real complex dynamic environment. However, the coupled systematic coordinate errors of an INS and the IHDST severely decrease the attitude assessment accuracy of an INS. Given that, a high-accuracy decoupling estimation method of the above systematic coordinate errors based on the constrained least squares (CLS) method is proposed in this paper. The reference frame of the IHDST is firstly converted to be consistent with that of the INS because their reference frames are completely different. Thereafter, the decoupling estimation model of the systematic coordinate errors is established and the CLS-based optimization method is utilized to estimate errors accurately. After compensating for error, the attitude accuracy of an INS can be assessed based on IHDST accurately. Both simulated experiments and real flight experiments of aircraft are conducted, and the experimental results demonstrate that the proposed method is effective and shows excellent performance for the attitude accuracy assessment of an INS in a real work environment. PMID:28991179

Quantifying biological integrity by taxonomic completeness: its utility in regional and global assessments.

PubMed

Hawkins, Charles P

2006-08-01

Water resources managers and conservation biologists need reliable, quantitative, and directly comparable methods for assessing the biological integrity of the world's aquatic ecosystems. Large-scale assessments are constrained by the lack of consistency in the indicators used to assess biological integrity and our current inability to translate between indicators. In theory, assessments based on estimates of taxonomic completeness, i.e., the proportion of expected taxa that were observed (observed/expected, O/E) are directly comparable to one another and should therefore allow regionally and globally consistent summaries of the biological integrity of freshwater ecosystems. However, we know little about the true comparability of O/E assessments derived from different data sets or how well O/E assessments perform relative to other indicators in use. I compared the performance (precision, bias, and sensitivity to stressors) of O/E assessments based on five different data sets with the performance of the indicators previously applied to these data (three multimetric indices, a biotic index, and a hybrid method used by the state of Maine). Analyses were based on data collected from U.S. stream ecosystems in North Carolina, the Mid-Atlantic Highlands, Maine, and Ohio. O/E assessments resulted in very similar estimates of mean regional conditions compared with most other indicators once these indicators' values were standardized relative to reference-site means. However, other indicators tended to be biased estimators of O/E, a consequence of differences in their response to natural environmental gradients and sensitivity to stressors. These results imply that, in some cases, it may be possible to compare assessments derived from different indicators by standardizing their values (a statistical approach to data harmonization). In situations where it is difficult to standardize or otherwise harmonize two or more indicators, O/E values can easily be derived from existing raw sample data. With some caveats, O/E should provide more directly comparable assessments of biological integrity across regions than is possible by harmonizing values of a mix of indicators.

Spatial rule-based assessment of habitat potential to predict impact of land use changes on biodiversity at municipal scale.

PubMed

Scolozzi, Rocco; Geneletti, Davide

2011-03-01

In human dominated landscapes, ecosystems are under increasing pressures caused by urbanization and infrastructure development. In Alpine valleys remnant natural areas are increasingly affected by habitat fragmentation and loss. In these contexts, there is a growing risk of local extinction for wildlife populations; hence assessing the consequences on biodiversity of proposed land use changes is extremely important. The article presents a methodology to assess the impacts of land use changes on target species at a local scale. The approach relies on the application of ecological profiles of target species for habitat potential (HP) assessment, using high resolution GIS-data within a multiple level framework. The HP, in this framework, is based on a species-specific assessment of the suitability of a site, as well of surrounding areas. This assessment is performed through spatial rules, structured as sets of queries on landscape objects. We show that by considering spatial dependencies in habitat assessment it is possible to perform better quantification of impacts of local-level land use changes on habitats.

Application of High-Throughput In Vitro Assays for Risk-Based Chemical Safety Decisions of Environmental and Industrial Chemicals (SOT presentation)

EPA Science Inventory

Multiple drivers shape the types of human-health assessments performed on chemicals by U.S. EPA resulting in chemical assessments are “fit-for-purpose” ranging from prioritization for further testing to full risk assessments. Layered on top of the diverse assessment needs are the...

Assessing the Effectiveness of New Hampshire Elementary Schools: An Effective Schools Approach

ERIC Educational Resources Information Center

Fortner, Tara

2017-01-01

Despite consistently strong performances among NH 4th graders on the NAEP assessments, large disparities have been observed among NH elementary students on the NECAP assessments based on race and SES. The current study assessed the effectiveness of NH elementary schools, as defined by the effective schools research. Of the 209 elementary schools…

EFF/NRS Data Collection Project, 2000-2001: An Interim Report on the Development of the EFF Assessment Framework.

ERIC Educational Resources Information Center

SRI International, Arlington, VA.

The Equipped for the Future (EFF) framework describes adult performance along these four dimensions: structure of the knowledge-base; fluency of performance; independence of performance; and range of conditions for performance. When completed it will include a developmental sequence of descriptions of learner performance for each EFF standard;…

Chesapeake Bay Low Freshwater Inflow Study. Phase II. MAP FOLIO. Biota Assessment.

DTIC Science & Technology

1982-05-01

conditions. These were: 1) Base Average -- average freshwater inflow conditions. by increased water consumption projected for the year 2020. 3) Base Drought...RESOLUTION TEST CHART NATIONAL BUREAU OF STANDARDS. 1963- A TAI m - ii J May 1982 Chesapeake Bay Low Freshwater Inflow Study Phase II Biota Assessment Map...A PERIOD ZOVERED change was found to CIESAPEAKE BAY LOW FRESHWATER INFLOW STUDY FINAL BIOTA ASSESSMENT PHASE II: FINAL REPORT MAP FOLIO s PERFORMING

TOPEX Radar Altimeter Engineering Assessment Report. Update: Launch to January 1, 1997

NASA Technical Reports Server (NTRS)

Hancock, D. W., III; Hayne, G. S.; Brooks, R. L.; Lee, J. E.; Lockwood, D. W.

1997-01-01

The initial TOPEX Mission Radar Altimeter Engineering Assessment Report, in February 1994, presented performance results for the NASA Radar Altimeter on the TOPEX/POSEIDON spacecraft, from its launch in August 1992 to February 1994. There have been supplemental Engineering Assessment Reports, issued in March 1995 and again in May 1996, which updated the performance results through the end of calendar years 1994 and 1995, respectively. This supplement updates the altimeter performance to the end of calendar year 1996, and describes significant events that occurred during 1996. As the performance data base has expanded, and as analysis tools and techniques continue to evolve, the longer-term trends of the altimeter data have become more apparent. The updated findings are presented here.

Defining the performance gap: Conducting a self-assessment

NASA Technical Reports Server (NTRS)

Braymer, Susan A.; Stoner, David L.; Powell, William C.

1992-01-01

This paper presents two different approaches to performing self-assessments of continuous improvement activities. Case Study 1 describes the activities performed by JSC to assess the implementation of continuous improvement efforts at the NASA Center. The JSC approach included surveys administered to randomly selected NASA personnel and personal interviews with NASA and contractor management personnel. Case Study 2 describes the continuous improvement survey performed by the JSC Safety, Reliability, and Quality Assurance (SR&QA) organization. This survey consisted of a short questionnaire (50 questions) administered to all NASA and contractor SR&QA personnel. The questionnaire is based on the eight categories of the President's Award for Quality and Productivity Improvement. It is designed to objectively determine placement on the TQ benchmark and identify a roadmap for improvement.

Information and problem report usage in system saftey engineering division

NASA Technical Reports Server (NTRS)

Morrissey, Stephen J.

1990-01-01

Five basic problems or question areas are examined. They are as follows: (1) Evaluate adequacy of current problem/performance data base; (2) Evaluate methods of performing trend analysis; (3) Methods and sources of data for probabilistic risk assessment; and (4) How is risk assessment documentation upgraded and/or updated. The fifth problem was to provide recommendations for each of the above four areas.

Operator Performance Measures for Assessing Voice Communication Effectiveness

DTIC Science & Technology

1989-07-01

performance and work- load assessment techniques have been based.I Broadbent (1958) described a limited capacity filter model of human information...INFORMATION PROCESSING 20 3.1.1. Auditory Attention 20 3.1.2. Auditory Memory 24 3.2. MODELS OF INFORMATION PROCESSING 24 3.2.1. Capacity Theories 25...Learning 0 Attention * Language Specialization • Decision Making• Problem Solving Auditory Information Processing Models of Processing Ooemtor

A balanced perspective: using nonfinancial measures to assess financial performance.

PubMed

Watkins, Ann L

2003-11-01

Assessments of hospitals' financial performance have traditionally been based exclusively on analysis of a concise set of key financial ratios. One study, however, demonstrates that analysis of a hospital's financial condition can be significantly enhanced with the addition of several nonfinancial measures, including case-mix adjusted admissions, case-mix adjusted admissions per full-time equivalent, and case-mix adjusted admissions per beds in service.

Curriculum Based Functional Vocational Assessment Guide. Correlation of the Life Centered Career Education Curriculum with Specific Occupational Programs, Standards of Work Performance, and Campus Assessment Instruments.

ERIC Educational Resources Information Center

Quinones, William A.

This document is a guide developed for the Dallas Independent School District's (DISD's) Multiple Careers Magnet Center, which provides special education students with training in standards of work performance and specific occupational skills for successful transition to community life and employment. The document also provides information for…