Evaluate E-loyalty of sales website: a Fuzzy mathematics method

NASA Astrophysics Data System (ADS)

Yi, Ying; Liu, Zhen-Yu; Xiong, Ying-Zi

The study about online consumer loyalty is limited, but how to evaluate the customers' E-loyalty to a sales website is always a noticeable question. By using some methods of fuzzy mathematics, we provide a more accurate way to evaluate E-loyalty of sales website. Moreover, this method can differentiate level and degree of each factor that influences E-loyalty.

The Evaluation of HRD: A Critical Study with Applications

ERIC Educational Resources Information Center

Tome, Eduardo

2009-01-01

Purpose: The purpose of this paper is to analyze critically the most important methods that are used in the evaluation of human resource development (HRD). Design/methodology/approach: The approach is to ask two questions: What are the methods available to define the impact of HRD in the economy? How can we evaluate the evaluations that have been…

Development and Application of a Rubric for Evaluating Students' Performance on Newton's Laws of Motion

ERIC Educational Resources Information Center

Kocakulah, Mustafa Sabri

2010-01-01

This study aims to develop and apply a rubric to evaluate the solutions of pre-service primary science teachers to questions about Newton's Laws of Motion. Two groups were taught the topic using the same teaching methods and administered four questions before and after teaching. Furthermore, 76 students in the experiment group were instructed…

Social Question and Answer Services versus Library Virtual Reference: Evaluation and Comparison from the Users' Perspective

ERIC Educational Resources Information Center

Zhang, Yin; Deng, Shengli

2014-01-01

Introduction: In recent years, the introduction of social question and answer services and other Internet tools have expanded the ways in which people have their questions answered. There has been speculation and debate over whether such services and other Internet tools are replacing library virtual reference services. Method: Most previous…

An Evaluation of Research Replication with Q Method and Its Utility in Market Segmentation.

ERIC Educational Resources Information Center

Adams, R. C.

Precipitated by questions of using Q methodology in television market segmentation and of the replicability of such research, this paper reports on both a reexamination of 1968 research by Joseph M. Foley and an attempt to replicate Foley's study. By undertaking a reanalysis of the Foley data, the question of replication in Q method is addressed.…

The role of student’s critical asking question in developing student’s critical thinking skills

NASA Astrophysics Data System (ADS)

Santoso, T.; Yuanita, L.; Erman, E.

2018-01-01

Questioning means thinking, and thinking is manifested in the form of questions. Research that studies the relationship between questioning and students’ critical thinking skills is little, if any. The aim of this study is to examine how student’s questions skill correlates to student’s critical thinking skills in learning of chemistry. The research design used was one group pretest-posttest design. The participants involved were 94 students, all of whom attended their last semesters, Chemistry Education of Tadulako University. A pre-test was administered to check participants’ ability to ask critical questions and critical thinking skills in learning chemistry. Then, the students were taught by using questioning technique. After accomplishing the lesson, a post-test was given to evaluate their progress. Obtained data were analyzed by using Pair-Samples T.Test and correlation methods. The result shows that the level of the questions plays an important role in critical thinking skills is the question levels of predictive, analysis, evaluation and inference.

Assessment of medical communication skills by computer: assessment method and student experiences.

PubMed

Hulsman, R L; Mollema, E D; Hoos, A M; de Haes, J C J M; Donnison-Speijer, J D

2004-08-01

A computer-assisted assessment (CAA) program for communication skills designated ACT was developed using the objective structured video examination (OSVE) format. This method features assessment of cognitive scripts underlying communication behaviour, a broad range of communication problems covered in 1 assessment, highly standardised assessment and rating procedures, and large group assessments without complex organisation. The Academic Medical Centre (AMC) at the University of Amsterdam, the Netherlands. Aims To describe the development of the AMC Communication Test (ACT); to describe our experiences with the examination and rating procedures; to present test score descriptives, and to present the students' opinions of ACT. The ACT presents films on history taking, breaking bad news and shared decision making. Each film is accompanied by 3 types of short essay questions derived from our assessment model: "knows", "knows why/when" and "knows how". Evaluation questions about ACT were integrated into the assessment. Participants A total of 210 third year medical undergraduates were assessed. This study reports on the 110 (53%) students who completed all evaluation questions. Marking 210 examinations took about 17 days. The test score matched a normal distribution and showed a good level of discrimination of the students. About 75% passed the examination. Some support for the validity of our assessment model was found in the students' differential performance on the 3 types of questions. The ACT was well received. Student evaluations confirmed our efforts to develop realistic films that related well to the communication training programme. The ACT is a useful assessment method which complements interpersonal assessment methods for the evaluation of the medical communication skills of undergraduates.

Answering medical questions at the point of care: a cross-sectional study comparing rapid decisions based on PubMed and Epistemonikos searches with evidence-based recommendations developed with the GRADE approach.

PubMed

Izcovich, Ariel; Criniti, Juan Martín; Popoff, Federico; Ragusa, Martín Alberto; Gigler, Cristel; Gonzalez Malla, Carlos; Clavijo, Manuela; Manzotti, Matias; Diaz, Martín; Catalano, Hugo Norberto; Neumann, Ignacio; Guyatt, Gordon

2017-08-07

Using the best current evidence to inform clinical decisions remains a challenge for clinicians. Given the scarcity of trustworthy clinical practice guidelines providing recommendations to answer clinicians' daily questions, clinical decision support systems (ie, assistance in question identification and answering) emerge as an attractive alternative. The trustworthiness of the recommendations achieved by such systems is unknown. To evaluate the trustworthiness of a question identification and answering system that delivers timely recommendations. Cross-sectional study. We compared the responses to 100 clinical questions related to inpatient management provided by two rapid response methods with 'Gold Standard' recommendations. One of the rapid methods was based on PubMed and the other on Epistemonikos database. We defined our 'Gold Standard' as trustworthy published evidence-based recommendations or, when unavailable, recommendations developed locally by a panel of six clinicians following the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. Recommendations provided by the rapid strategies were classified as potentially misleading or reasonable. We also determined if the potentially misleading recommendations could have been avoided with the appropriate implementation of searching and evidence summary tools. We were able to answer all of the 100 questions with both rapid methods. Of the 200 recommendations obtained, 6.5% (95% CI 3% to 9.9%) were classified as potentially misleading and 93.5% (95% CI 90% to 96.9%) as reasonable. 6 of the 13 potentially misleading recommendations could have been avoided by the appropriate usage of the Epistemonikos matrix tool or by constructing summary of findings tables. No significant differences were observed between the evaluated rapid response methods. A question answering service based on the GRADE approach proved feasible to implement and provided appropriate guidance for most identified questions. Our approach could help stakeholders in charge of managing resources and defining policies for patient care to improve evidence-based decision-making in an efficient and feasible manner. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

INTEGRATION OF SPATIAL DATA: METHODS EVALUATION WITH REGARD TO DATA ISSUES AND ASSESSMENT QUESTIONS

EPA Science Inventory

EPA's Regional Vulnerability Assessment (REVA) Program is developing and demonstrating approaches to assess current and future environmental vulnerabilities at a regional scale. An initial effort within this research program has been to develop and evaluate methods to synthesize ...

Accurate evaluation and analysis of functional genomics data and methods

PubMed Central

Greene, Casey S.; Troyanskaya, Olga G.

2016-01-01

The development of technology capable of inexpensively performing large-scale measurements of biological systems has generated a wealth of data. Integrative analysis of these data holds the promise of uncovering gene function, regulation, and, in the longer run, understanding complex disease. However, their analysis has proved very challenging, as it is difficult to quickly and effectively assess the relevance and accuracy of these data for individual biological questions. Here, we identify biases that present challenges for the assessment of functional genomics data and methods. We then discuss evaluation methods that, taken together, begin to address these issues. We also argue that the funding of systematic data-driven experiments and of high-quality curation efforts will further improve evaluation metrics so that they more-accurately assess functional genomics data and methods. Such metrics will allow researchers in the field of functional genomics to continue to answer important biological questions in a data-driven manner. PMID:22268703

Minimizing effects of methodological decisions on interpretation and prediction in species distribution studies: An example with background selection

USGS Publications Warehouse

Jarnevich, Catherine S.; Talbert, Marian; Morisette, Jeffrey T.; Aldridge, Cameron L.; Brown, Cynthia; Kumar, Sunil; Manier, Daniel; Talbert, Colin; Holcombe, Tracy R.

2017-01-01

Evaluating the conditions where a species can persist is an important question in ecology both to understand tolerances of organisms and to predict distributions across landscapes. Presence data combined with background or pseudo-absence locations are commonly used with species distribution modeling to develop these relationships. However, there is not a standard method to generate background or pseudo-absence locations, and method choice affects model outcomes. We evaluated combinations of both model algorithms (simple and complex generalized linear models, multivariate adaptive regression splines, Maxent, boosted regression trees, and random forest) and background methods (random, minimum convex polygon, and continuous and binary kernel density estimator (KDE)) to assess the sensitivity of model outcomes to choices made. We evaluated six questions related to model results, including five beyond the common comparison of model accuracy assessment metrics (biological interpretability of response curves, cross-validation robustness, independent data accuracy and robustness, and prediction consistency). For our case study with cheatgrass in the western US, random forest was least sensitive to background choice and the binary KDE method was least sensitive to model algorithm choice. While this outcome may not hold for other locations or species, the methods we used can be implemented to help determine appropriate methodologies for particular research questions.

Using evaluation to adapt health information outreach to the complex environments of community-based organizations.

PubMed

Olney, Cynthia A

2005-10-01

After arguing that most community-based organizations (CBOs) function as complex adaptive systems, this white paper describes the evaluation goals, questions, indicators, and methods most important at different stages of community-based health information outreach. This paper presents the basic characteristics of complex adaptive systems and argues that the typical CBO can be considered this type of system. It then presents evaluation as a tool for helping outreach teams adapt their outreach efforts to the CBO environment and thus maximize success. Finally, it describes the goals, questions, indicators, and methods most important or helpful at each stage of evaluation (community assessment, needs assessment and planning, process evaluation, and outcomes assessment). Literature from complex adaptive systems as applied to health care, business, and evaluation settings is presented. Evaluation models and applications, particularly those based on participatory approaches, are presented as methods for maximizing the effectiveness of evaluation in dynamic CBO environments. If one accepts that CBOs function as complex adaptive systems-characterized by dynamic relationships among many agents, influences, and forces-then effective evaluation at the stages of community assessment, needs assessment and planning, process evaluation, and outcomes assessment is critical to outreach success.

Using a mixed-methods design to examine nurse practitioner integration in British Columbia.

PubMed

Sangster-Gormley, Esther; Griffith, Janessa; Schreiber, Rita; Borycki, Elizabeth

2015-07-01

To discuss and provide examples of how mixed-methods research was used to evaluate the integration of nurse practitioners (NPs) into a Canadian province. Legislation enabling NPs to practise in British Columbia (BC) was enacted in 2005. This research evaluated the integration of NPs and their effect on the BC healthcare system. Data were collected using surveys, focus groups, participant interviews and case studies over three years. Data sources and methods were triangulated to determine how the findings addressed the research questions. The challenges and benefits of using the multiphase design are highlighted in the paper. The multiphase mixed-methods research design was selected because of its applicability to evaluation research. The design proved to be robust and flexible in answering research questions. As sub-studies within the multiphase design are often published separately, it can be difficult for researchers to find examples. This paper highlights ways that a multiphase mixed-methods design can be conducted for researchers unfamiliar with the process.

Simulation for Nurse Anesthesia Program Selection: Redesigned

ERIC Educational Resources Information Center

Roebuck, John Arthur

2017-01-01

Purpose: This project is meant to answer the research question: What applicant character traits do Nurse Anesthesia Program Directors and Faculty identify as favorable predictors for successful completion of a nurse anesthesia program, and what evaluation methods are best to evaluate these traits in prospective students? Methods: A prospective…

EVALUATION OF POSSIBLE BIASES IN THE U.S. EPA'S METHOD 101A-MERCURY EMISSIONS FROM STATIONARY SOURCES

EPA Science Inventory

The U.S. Environmental Protection Agency (EPA), Research Triangle Park, North Carolina, has a program to evaluate and standardize source testing methods for hazardous pollutants in support of current and future air quality regulations. ccasionally, questions arise concerning an e...

Criteria for Evaluating the Performance of Compilers

DTIC Science & Technology

1974-10-01

cannot be made to fit, then an auxiliary mechanism outside the parser might be used . Finally, changing the choice of parsing tech - nique to a...was not useful in providing a basic for compiler evaluation. The study of the first question eztablished criteria and methodb for assigning four...program. The study of the second question estab- lished criteria for defining a "compiler Gibson mix", and established methods for using this "mix" to

Helping physics teacher-candidates develop questioning skills through innovative technology use

NASA Astrophysics Data System (ADS)

Milner-Bolotin, Marina

2015-12-01

Peer Instruction has been used successfully in undergraduate classrooms for decades. Its success depends largely on the quality of multiple-choice questions. Yet it is still rare in secondary schools because of teachers' lack of experience in designing, evaluating, and implementing conceptual questions. Research-based multiple-choice conceptual questions are also underutilized in physics teacher education. This study explores the implementation of Peer Instruction enhanced by PeerWise collaborative online system, in a physics methods course in a physics teacher education program.

Development and pilot test of a process to identify research needs from a systematic review.

PubMed

Saldanha, Ian J; Wilson, Lisa M; Bennett, Wendy L; Nicholson, Wanda K; Robinson, Karen A

2013-05-01

To ensure appropriate allocation of research funds, we need methods for identifying high-priority research needs. We developed and pilot tested a process to identify needs for primary clinical research using a systematic review in gestational diabetes mellitus. We conducted eight steps: abstract research gaps from a systematic review using the Population, Intervention, Comparison, Outcomes, and Settings (PICOS) framework; solicit feedback from the review authors; translate gaps into researchable questions using the PICOS framework; solicit feedback from multidisciplinary stakeholders at our institution; establish consensus among multidisciplinary external stakeholders on the importance of the research questions using the Delphi method; prioritize outcomes; develop conceptual models to highlight research needs; and evaluate the process. We identified 19 research questions. During the Delphi method, external stakeholders established consensus for 16 of these 19 questions (15 with "high" and 1 with "medium" clinical benefit/importance). We pilot tested an eight-step process to identify clinically important research needs. Before wider application of this process, it should be tested using systematic reviews of other diseases. Further evaluation should include assessment of the usefulness of the research needs generated using this process for primary researchers and funders. Copyright © 2013 Elsevier Inc. All rights reserved.

Evaluation Program on the Implementation of Industrial Apprenticeship (Prakerin) in Electrical Engineering

NASA Astrophysics Data System (ADS)

Maulana, I.; Sumarto; Nurafiati, P.; Puspita, R. H.

2018-02-01

This research aims to find out the evaluation program of the Industrial apprenticeship (Prakerin) in electrical engineering. This research includes on four variables of CIPP. (1). Context (a). programme planning (b). design. (2). Input (a). readiness of students (b). performance of vocational education teachers (c). Facilities and infrastructure, (3). process (a). performance students (b). performance mentors, (4). Product (a). readiness of student work. This research is a type of program evaluation research with Stake model approach. Data collection methods used are questionnaires with closed questions and frequently asked questions.

Traditional learning and problem-based learning: self-perception of preparedness for internship.

PubMed

Millan, Laís Pereira Bueno; Semer, Beatriz; Rodrigues, José Mauro da Silva; Gianini, Reinaldo José

2012-01-01

This study aims to evaluate Pontificia Universidade Católica de São Paulo (PUC-SP) medical students' perception of their preparedness to attend the internship course by comparing students who entered the internship in 2009, who were taught according to the traditional learning method, and those who entered the internship in 2010, who were taught according to the new method, i.e. problem-based learning (PBL). 50 traditional learning method students answered a standard Lickert scale questionnaire upon entering internship in 2009. In 2010, the process was repeated with PBL students. The questionnaire was based upon the Preparation for Hospital Practice Questionnaire. This questionnaire was evaluated by professors from three medical schools in Brazil regarding its applicability. The original questions were classified according to the importance these professors attributed to them, and less important questions were removed. Scores obtained from the Student's t-test were considered significant with p < 0.05. A significant statistical difference was observed in 16 questions, and the traditional learning method students reported higher average scores. When questions were divided into dimensions, a significant statistical difference appeared in the dimensions " social aspects of health", "medical skills", and "ethical concepts"; traditional learning method students again reported higher scores (p < 0.001 for all dimensions). Higher scores were also reported when the average of the answers to the whole questionnaire was calculated. Traditional learning method students consider themselves to be better prepared for internship activities than PBL students, according to the following three comparative means: by analyzing the answers to each question, by grouping these answers into dimensions, and by calculating the means of answers to the whole questionnaire.

DO TIE LABORATORY BASED METHODS REALLY REFLECT FIELD CONDITIONS

EPA Science Inventory

Sediment Toxicity Identification and Evaluation (TIE) methods have been developed for both interstitial waters and whole sediments. These relatively simple laboratory methods are designed to identify specific toxicants or classes of toxicants in sediments; however, the question ...

Evaluating the Diagnostic Validity of a Facet-Based Formative Assessment System

ERIC Educational Resources Information Center

DeBarger, Angela Haydel; DiBello, Louis; Minstrell, Jim; Feng, Mingyu; Stout, William; Pellegrino, James; Haertel, Geneva; Harris, Christopher; Ructinger, Liliana

2011-01-01

This paper describes methods for an alignment study and psychometric analyses of a formative assessment system, Diagnoser Tools for physics. Diagnoser Tools begin with facet clusters as the interpretive framework for designing questions and instructional activities. Thus each question in the diagnostic assessments includes distractors that…

Sequence Matters but How Exactly? A Method for Evaluating Activity Sequences from Data

ERIC Educational Resources Information Center

Doroudi, Shayan; Holstein, Kenneth; Aleven, Vincent; Brunskill, Emma

2016-01-01

How should a wide variety of educational activities be sequenced to maximize student learning? Although some experimental studies have addressed this question, educational data mining methods may be able to evaluate a wider range of possibilities and better handle many simultaneous sequencing constraints. We introduce Sequencing Constraint…

Benefits of Multiple Methods for Evaluating HIV Counseling and Testing Sites in Pennsylvania.

ERIC Educational Resources Information Center

Encandela, John A.; Gehl, Mary Beth; Silvestre, Anthony; Schelzel, George

1999-01-01

Examines results from two methods used to evaluate publicly funded human immunodeficiency virus (HIV) counseling and testing in Pennsylvania. Results of written mail surveys of all sites and interviews from a random sample of 30 sites were similar in terms of questions posed and complementary in other ways. (SLD)

Does Research on Evaluation Matter? Findings from a Survey of American Evaluation Association Members and Prominent Evaluation Theorists and Scholars

ERIC Educational Resources Information Center

Coryn, Chris L. S.; Ozeki, Satoshi; Wilson, Lyssa N.; Greenman, Gregory D., II; Schröter, Daniela C.; Hobson, Kristin A.; Azzam, Tarek; Vo, Anne T.

2016-01-01

Research on evaluation theories, methods, and practices has increased considerably in the past decade. Even so, little is known about whether published findings from research on evaluation are read by evaluators and whether such findings influence evaluators' thinking about evaluation or their evaluation practice. To address these questions, and…

A guideline for the validation of likelihood ratio methods used for forensic evidence evaluation.

PubMed

Meuwly, Didier; Ramos, Daniel; Haraksim, Rudolf

2017-07-01

This Guideline proposes a protocol for the validation of forensic evaluation methods at the source level, using the Likelihood Ratio framework as defined within the Bayes' inference model. In the context of the inference of identity of source, the Likelihood Ratio is used to evaluate the strength of the evidence for a trace specimen, e.g. a fingermark, and a reference specimen, e.g. a fingerprint, to originate from common or different sources. Some theoretical aspects of probabilities necessary for this Guideline were discussed prior to its elaboration, which started after a workshop of forensic researchers and practitioners involved in this topic. In the workshop, the following questions were addressed: "which aspects of a forensic evaluation scenario need to be validated?", "what is the role of the LR as part of a decision process?" and "how to deal with uncertainty in the LR calculation?". The questions: "what to validate?" focuses on the validation methods and criteria and "how to validate?" deals with the implementation of the validation protocol. Answers to these questions were deemed necessary with several objectives. First, concepts typical for validation standards [1], such as performance characteristics, performance metrics and validation criteria, will be adapted or applied by analogy to the LR framework. Second, a validation strategy will be defined. Third, validation methods will be described. Finally, a validation protocol and an example of validation report will be proposed, which can be applied to the forensic fields developing and validating LR methods for the evaluation of the strength of evidence at source level under the following propositions. Copyright © 2016. Published by Elsevier B.V.

Evaluating Digital Health Interventions: Key Questions and Approaches.

PubMed

Murray, Elizabeth; Hekler, Eric B; Andersson, Gerhard; Collins, Linda M; Doherty, Aiden; Hollis, Chris; Rivera, Daniel E; West, Robert; Wyatt, Jeremy C

2016-11-01

Digital health interventions have enormous potential as scalable tools to improve health and healthcare delivery by improving effectiveness, efficiency, accessibility, safety, and personalization. Achieving these improvements requires a cumulative knowledge base to inform development and deployment of digital health interventions. However, evaluations of digital health interventions present special challenges. This paper aims to examine these challenges and outline an evaluation strategy in terms of the research questions needed to appraise such interventions. As they are at the intersection of biomedical, behavioral, computing, and engineering research, methods drawn from all of these disciplines are required. Relevant research questions include defining the problem and the likely benefit of the digital health intervention, which in turn requires establishing the likely reach and uptake of the intervention, the causal model describing how the intervention will achieve its intended benefit, key components, and how they interact with one another, and estimating overall benefit in terms of effectiveness, cost effectiveness, and harms. Although RCTs are important for evaluation of effectiveness and cost effectiveness, they are best undertaken only when: (1) the intervention and its delivery package are stable; (2) these can be implemented with high fidelity; and (3) there is a reasonable likelihood that the overall benefits will be clinically meaningful (improved outcomes or equivalent outcomes at lower cost). Broadening the portfolio of research questions and evaluation methods will help with developing the necessary knowledge base to inform decisions on policy, practice, and research. Copyright © 2016 American Journal of Preventive Medicine. Published by Elsevier Inc. All rights reserved.

Core-Oriented Lexical Translation Evaluation (COLTE)

ERIC Educational Resources Information Center

Jahangiri, A. M.

1998-01-01

Objective measurement in translation evaluation is a relatively untouched area, especially at the level of the lexis. No thoroughly worked-out method has been put forward for the scientific evaluation of a translation. Yet few translators would question the potential value of such a device. Certainly translation evaluation is not limited to the…

DO TIE LABORATORY BASED ASSESSMENT METHODS REALLY PREDICT FIELD EFFECTS?

EPA Science Inventory

Sediment Toxicity Identification and Evaluation (TIE) methods have been developed for both porewaters and whole sediments. These relatively simple laboratory methods are designed to identify specific toxicants or classes of toxicants in sediments; however, the question of whethe...

Evaluating the Good Ontology Design Guideline (GoodOD) with the Ontology Quality Requirements and Evaluation Method and Metrics (OQuaRE)

PubMed Central

Duque-Ramos, Astrid; Boeker, Martin; Jansen, Ludger; Schulz, Stefan; Iniesta, Miguela; Fernández-Breis, Jesualdo Tomás

2014-01-01

Objective To (1) evaluate the GoodOD guideline for ontology development by applying the OQuaRE evaluation method and metrics to the ontology artefacts that were produced by students in a randomized controlled trial, and (2) informally compare the OQuaRE evaluation method with gold standard and competency questions based evaluation methods, respectively. Background In the last decades many methods for ontology construction and ontology evaluation have been proposed. However, none of them has become a standard and there is no empirical evidence of comparative evaluation of such methods. This paper brings together GoodOD and OQuaRE. GoodOD is a guideline for developing robust ontologies. It was previously evaluated in a randomized controlled trial employing metrics based on gold standard ontologies and competency questions as outcome parameters. OQuaRE is a method for ontology quality evaluation which adapts the SQuaRE standard for software product quality to ontologies and has been successfully used for evaluating the quality of ontologies. Methods In this paper, we evaluate the effect of training in ontology construction based on the GoodOD guideline within the OQuaRE quality evaluation framework and compare the results with those obtained for the previous studies based on the same data. Results Our results show a significant effect of the GoodOD training over developed ontologies by topics: (a) a highly significant effect was detected in three topics from the analysis of the ontologies of untrained and trained students; (b) both positive and negative training effects with respect to the gold standard were found for five topics. Conclusion The GoodOD guideline had a significant effect over the quality of the ontologies developed. Our results show that GoodOD ontologies can be effectively evaluated using OQuaRE and that OQuaRE is able to provide additional useful information about the quality of the GoodOD ontologies. PMID:25148262

Relations between Inductive Reasoning and Deductive Reasoning

ERIC Educational Resources Information Center

Heit, Evan; Rotello, Caren M.

2010-01-01

One of the most important open questions in reasoning research is how inductive reasoning and deductive reasoning are related. In an effort to address this question, we applied methods and concepts from memory research. We used 2 experiments to examine the effects of logical validity and premise-conclusion similarity on evaluation of arguments.…

A Consideration of the Case for Using Life Story Interviews in the Evaluation of Widening Access Interventions

ERIC Educational Resources Information Center

Raven, Neil

2016-01-01

Whilst published data sources exist for evaluating interventions aimed at widening higher education access, there is value for practitioners in conducting their own research. However, recognition of the contribution afforded by generating new data raises questions over which research methods to utilise. One method relatively new to widening…

Evaluating the reliability, validity, acceptability, and practicality of SMS text messaging as a tool to collect research data: results from the Feeding Your Baby project

PubMed Central

Donnan, Peter T; Symon, Andrew G; Kellett, Gillian; Monteith-Hodge, Ewa; Rauchhaus, Petra; Wyatt, Jeremy C

2012-01-01

Objective To test the reliability, validity, acceptability, and practicality of short message service (SMS) messaging for collection of research data. Materials and methods The studies were carried out in a cohort of recently delivered women in Tayside, Scotland, UK, who were asked about their current infant feeding method and future feeding plans. Reliability was assessed by comparison of their responses to two SMS messages sent 1 day apart. Validity was assessed by comparison of their responses to text questions and the same question administered by phone 1 day later, by comparison with the same data collected from other sources, and by correlation with other related measures. Acceptability was evaluated using quantitative and qualitative questions, and practicality by analysis of a researcher log. Results Reliability of the factual SMS message gave perfect agreement. Reliabilities for the numerical question were reasonable, with κ between 0.76 (95% CI 0.56 to 0.96) and 0.80 (95% CI 0.59 to 1.00). Validity for data compared with that collected by phone within 24 h (κ =0.92 (95% CI 0.84 to 1.00)) and with health visitor data (κ =0.85 (95% CI 0.73 to 0.97)) was excellent. Correlation validity between the text responses and other related demographic and clinical measures was as expected. Participants found the method a convenient and acceptable way of providing data. For researchers, SMS text messaging provided an easy and functional method of gathering a large volume of data. Conclusion In this sample and for these questions, SMS was a reliable and valid method for capturing research data. PMID:22539081

Development and evaluation of the Dignity Talk question framework for palliative patients and their families: A mixed-methods study.

PubMed

Guo, Qiaohong; Chochinov, Harvey Max; McClement, Susan; Thompson, Genevieve; Hack, Tom

2018-01-01

Effective patient-family communication can reduce patients' psychosocial distress and relieve family members' current suffering and their subsequent grief. However, terminally ill patients and their family members often experience great difficulty in communicating their true feelings, concerns, and needs to each other. To develop a novel means of facilitating meaningful conversations for palliative patients and family members, coined Dignity Talk, explore anticipated benefits and challenges of using Dignity Talk, and solicit suggestions for protocol improvement. A convergent parallel mixed-methods design. Dignity Talk, a self-administered question list, was designed to prompt end-of-life conversations, adapted from the Dignity Therapy question framework. Participants were surveyed to evaluate the Dignity Talk question framework. Data were analyzed using qualitative and quantitative methods. A total of 20 palliative patients, 20 family members, and 34 healthcare providers were recruited from two inpatient palliative care units in Winnipeg, Canada. Most Dignity Talk questions were endorsed by the majority of patients and families (>70%). Dignity Talk was revised to be convenient and flexible to use, broadly accessible, clearly stated, and sensitively worded. Participants felt Dignity Talk would be valuable in promoting conversations, enhancing family connections and relationships, enhancing patient sense of value and dignity, promoting effective interaction, and attending to unfinished business. Participants suggested that patients and family members be given latitude to respond only to questions that are meaningful to them and within their emotional capacity to broach. Dignity Talk may provide a gentle means of facilitating important end-of-life conversations.

Development and evaluation of the Dignity Talk question framework for palliative patients and their families: A mixed-methods study

PubMed Central

Guo, Qiaohong; Chochinov, Harvey Max; McClement, Susan; Thompson, Genevieve; Hack, Tom

2017-01-01

Background: Effective patient–family communication can reduce patients’ psychosocial distress and relieve family members’ current suffering and their subsequent grief. However, terminally ill patients and their family members often experience great difficulty in communicating their true feelings, concerns, and needs to each other. Aim: To develop a novel means of facilitating meaningful conversations for palliative patients and family members, coined Dignity Talk, explore anticipated benefits and challenges of using Dignity Talk, and solicit suggestions for protocol improvement. Design: A convergent parallel mixed-methods design. Dignity Talk, a self-administered question list, was designed to prompt end-of-life conversations, adapted from the Dignity Therapy question framework. Participants were surveyed to evaluate the Dignity Talk question framework. Data were analyzed using qualitative and quantitative methods. Setting/participants: A total of 20 palliative patients, 20 family members, and 34 healthcare providers were recruited from two inpatient palliative care units in Winnipeg, Canada. Results: Most Dignity Talk questions were endorsed by the majority of patients and families (>70%). Dignity Talk was revised to be convenient and flexible to use, broadly accessible, clearly stated, and sensitively worded. Participants felt Dignity Talk would be valuable in promoting conversations, enhancing family connections and relationships, enhancing patient sense of value and dignity, promoting effective interaction, and attending to unfinished business. Participants suggested that patients and family members be given latitude to respond only to questions that are meaningful to them and within their emotional capacity to broach. Conclusion: Dignity Talk may provide a gentle means of facilitating important end-of-life conversations. PMID:29130367

What and How Are We Evaluating? Meta-Evaluation Study of the NASA Innovations in Climate Education (NICE) Portfolio

NASA Astrophysics Data System (ADS)

Martin, A. M.; Barnes, M. H.; Chambers, L. H.; Pippin, M. R.

2011-12-01

As part of NASA's Minority University Research and Education Program (MUREP), the NASA Innovations in Climate Education (NICE) project at Langley Research Center has funded 71 climate education initiatives since 2008. The funded initiatives span across the nation and contribute to the development of a climate-literate public and the preparation of a climate-related STEM workforce through research experiences, professional development opportunities, development of data access and modeling tools, and educational opportunities in both K-12 and higher education. Each of the funded projects proposes and carries out its own evaluation plan, in collaboration with external or internal evaluation experts. Using this portfolio as an exemplar case, NICE has undertaken a systematic meta-evaluation of these plans, focused primarily on evaluation questions, approaches, and methods. This meta-evaluation study seeks to understand the range of evaluations represented in the NICE portfolio, including descriptive information (what evaluations, questions, designs, approaches, and methods are applied?) and questions of value (do these evaluations meet the needs of projects and their staff, and of NASA/NICE?). In the current climate, as federal funders of climate change and STEM education projects seek to better understand and incorporate evaluation into their decisions, evaluators and project leaders are also seeking to build robust understanding of program effectiveness. Meta-evaluations like this provide some baseline understanding of the current status quo and the kinds of evaluations carried out within such funding portfolios. These explorations are needed to understand the common ground between evaluative best practices, limited resources, and agencies' desires, capacity, and requirements. When NASA asks for evaluation of funded projects, what happens? Which questions are asked and answered, using which tools? To what extent do the evaluations meet the needs of projects and program officers? How do they contribute to best practices in climate science education? These questions are important to ask about STEM and climate literacy work more generally; the NICE portfolio provides a broad test case for thinking strategically, critically, and progressively about evaluation in our community. Our findings can inform the STEM education, communication, and public outreach communities, and prompt us to consider a broad range of informative evaluation options. During this presentation, we will consider the breadth, depth and utility of evaluations conducted through a NASA climate education funding opportunity. We will examine the relationship between what we want to know about education programs, what we want to achieve with our interventions, and what we ask in our evaluations.

What and How Are We Evaluating? Meta-Evaluation Study of the NASA Innovations in Climate Education (NICE) Portfolio

NASA Astrophysics Data System (ADS)

Martin, A. M.; Barnes, M. H.; Chambers, L. H.; Pippin, M. R.

2013-12-01

As part of NASA's Minority University Research and Education Program (MUREP), the NASA Innovations in Climate Education (NICE) project at Langley Research Center has funded 71 climate education initiatives since 2008. The funded initiatives span across the nation and contribute to the development of a climate-literate public and the preparation of a climate-related STEM workforce through research experiences, professional development opportunities, development of data access and modeling tools, and educational opportunities in both K-12 and higher education. Each of the funded projects proposes and carries out its own evaluation plan, in collaboration with external or internal evaluation experts. Using this portfolio as an exemplar case, NICE has undertaken a systematic meta-evaluation of these plans, focused primarily on evaluation questions, approaches, and methods. This meta-evaluation study seeks to understand the range of evaluations represented in the NICE portfolio, including descriptive information (what evaluations, questions, designs, approaches, and methods are applied?) and questions of value (do these evaluations meet the needs of projects and their staff, and of NASA/NICE?). In the current climate, as federal funders of climate change and STEM education projects seek to better understand and incorporate evaluation into their decisions, evaluators and project leaders are also seeking to build robust understanding of program effectiveness. Meta-evaluations like this provide some baseline understanding of the current status quo and the kinds of evaluations carried out within such funding portfolios. These explorations are needed to understand the common ground between evaluative best practices, limited resources, and agencies' desires, capacity, and requirements. When NASA asks for evaluation of funded projects, what happens? Which questions are asked and answered, using which tools? To what extent do the evaluations meet the needs of projects and program officers? How do they contribute to best practices in climate science education? These questions are important to ask about STEM and climate literacy work more generally; the NICE portfolio provides a broad test case for thinking strategically, critically, and progressively about evaluation in our community. Our findings can inform the STEM education, communication, and public outreach communities, and prompt us to consider a broad range of informative evaluation options. During this presentation, we will consider the breadth, depth and utility of evaluations conducted through a NASA climate education funding opportunity. We will examine the relationship between what we want to know about education programs, what we want to achieve with our interventions, and what we ask in our evaluations.

The Evaluation and Research of Multi-Project Programs: Program Component Analysis.

ERIC Educational Resources Information Center

Baker, Eva L.

1977-01-01

It is difficult to base evaluations on concepts irrelevant to state policy making. Evaluation of a multiproject program requires both time and differentiation of method. Data from the California Early Childhood Program illustrate process variables for program component analysis, and research questions for intraprogram comparison. (CP)

Evaluating the Good Ontology Design Guideline (GoodOD) with the ontology quality requirements and evaluation method and metrics (OQuaRE).

PubMed

Duque-Ramos, Astrid; Boeker, Martin; Jansen, Ludger; Schulz, Stefan; Iniesta, Miguela; Fernández-Breis, Jesualdo Tomás

2014-01-01

To (1) evaluate the GoodOD guideline for ontology development by applying the OQuaRE evaluation method and metrics to the ontology artefacts that were produced by students in a randomized controlled trial, and (2) informally compare the OQuaRE evaluation method with gold standard and competency questions based evaluation methods, respectively. In the last decades many methods for ontology construction and ontology evaluation have been proposed. However, none of them has become a standard and there is no empirical evidence of comparative evaluation of such methods. This paper brings together GoodOD and OQuaRE. GoodOD is a guideline for developing robust ontologies. It was previously evaluated in a randomized controlled trial employing metrics based on gold standard ontologies and competency questions as outcome parameters. OQuaRE is a method for ontology quality evaluation which adapts the SQuaRE standard for software product quality to ontologies and has been successfully used for evaluating the quality of ontologies. In this paper, we evaluate the effect of training in ontology construction based on the GoodOD guideline within the OQuaRE quality evaluation framework and compare the results with those obtained for the previous studies based on the same data. Our results show a significant effect of the GoodOD training over developed ontologies by topics: (a) a highly significant effect was detected in three topics from the analysis of the ontologies of untrained and trained students; (b) both positive and negative training effects with respect to the gold standard were found for five topics. The GoodOD guideline had a significant effect over the quality of the ontologies developed. Our results show that GoodOD ontologies can be effectively evaluated using OQuaRE and that OQuaRE is able to provide additional useful information about the quality of the GoodOD ontologies.

Formative student-authored question bank: perceptions, question quality and association with summative performance

PubMed Central

Walsh, Jason L; Harris, Benjamin H L; Denny, Paul; Smith, Phil

2018-01-01

Purpose of the study There are few studies on the value of authoring questions as a study method, the quality of the questions produced by students and student perceptions of student-authored question banks. Here we evaluate PeerWise, a widely used and free online resource that allows students to author, answer and discuss multiple-choice questions. Study design We introduced two undergraduate medical student cohorts to PeerWise (n=603). We looked at their patterns of PeerWise usage; identified associations between student engagement and summative exam performance; and used focus groups to assess student perceptions of the value of PeerWise for learning. We undertook item analysis to assess question difficulty and quality. Results Over two academic years, the two cohorts wrote 4671 questions, answered questions 606 658 times and posted 7735 comments. Question writing frequency correlated most strongly with summative performance (Spearman’s rank: 0.24, p=<0.001). Student focus groups found that: (1) students valued curriculum specificity; and (2) students were concerned about student-authored question quality. Only two questions of the 300 ’most-answered' questions analysed had an unacceptable discriminatory value (point-biserial correlation <0.2). Conclusions Item analysis suggested acceptable question quality despite student concerns. Quantitative and qualitative methods indicated that PeerWise is a valuable study tool. PMID:28866607

Scientific Process Flowchart Assessment (SPFA): A Method for Evaluating Changes in Understanding and Visualization of the Scientific Process in a Multidisciplinary Student Population

ERIC Educational Resources Information Center

Wilson, Kristy J.; Rigakos, Bessie

2016-01-01

The scientific process is nonlinear, unpredictable, and ongoing. Assessing the nature of science is difficult with methods that rely on Likert-scale or multiple-choice questions. This study evaluated conceptions about the scientific process using student-created visual representations that we term "flowcharts." The methodology,…

Evaluation of Three Microcomputer Teaching Modules. SUMIT Courseware Development Project.

ERIC Educational Resources Information Center

Soldan, Ted

The purpose of this series of experiments was to examine two questions related to the effectiveness of computer assisted instruction (CAI). Can microcomputer modules teach effectively, and do they enhance learning when used as a supplement to traditional teaching methods? Part 1 of this report addresses the former question and part 2 addresses the…

A Qualitative Evaluation of Instructors' Exam Questions at a Primary Education Department in Terms of Certain Variables

ERIC Educational Resources Information Center

Demir, Mehmet Kaan; Eryaman, Mustafa Yunus

2012-01-01

The purpose of this qualitative research study is to analyze instructors' exam questions at a Primary Education Department in terms of the exam's period, the comprehensibility of the instructions, cognitive level, and the appropriateness to the critical thinking. This qualitative study is based on document analysis method. 100 randomly selected…

Commentary: Writing and evaluating qualitative research reports

USDA-ARS?s Scientific Manuscript database

An overview of qualitative methods is provided, particularly for reviewers and authors who may be less familiar with qualitative research. A question and answer format is used to address considerations for writing and evaluating qualitative research. When producing qualitative research, individuals ...

On the comprehensibility and perceived privacy protection of indirect questioning techniques.

PubMed

Hoffmann, Adrian; Waubert de Puiseau, Berenike; Schmidt, Alexander F; Musch, Jochen

2017-08-01

On surveys that assess sensitive personal attributes, indirect questioning aims at increasing respondents' willingness to answer truthfully by protecting confidentiality. However, the assumption that subjects understand questioning procedures fully and trust them to protect their privacy is rarely tested. In a scenario-based design, we compared four indirect questioning procedures in terms of their comprehensibility and perceived privacy protection. All indirect questioning techniques were found to be less comprehensible by respondents than a conventional direct question used for comparison. Less-educated respondents experienced more difficulties when confronted with any indirect questioning technique. Regardless of education, the crosswise model was found to be the most comprehensible among the four indirect methods. Indirect questioning in general was perceived to increase privacy protection in comparison to a direct question. Unexpectedly, comprehension and perceived privacy protection did not correlate. We recommend assessing these factors separately in future evaluations of indirect questioning.

M-OSCE as a method to measure dental hygiene students' critical thinking: a pilot study.

PubMed

McComas, Martha J; Wright, Rebecca A; Mann, Nancy K; Cooper, Mary D; Jacks, Mary E

2013-04-01

Educators in all academic disciplines have been encouraged to utilize assessment strategies to evaluate students' critical thinking. The purpose of this study was to assess the viability of the modified objective structured clinical examination (m-OSCE) to evaluate critical thinking in dental hygiene education. This evaluation utilized a convenience sample of senior dental hygiene students. Students participated in the m-OSCE in which portions of a patient case were revealed at four stations. The exam consisted of multiple-choice questions intended to measure students' ability to utilize critical thinking skills. Additionally, there was one fill-in-the-blank question and a treatment plan that was completed at the fifth station. The results of this study revealed that the m-OSCE did not reliably measure dental hygiene students' critical thinking. Statistical analysis found no satisfactory reliability within the multiple-choice questions and moderately reliable results within the treatment planning portion of the examination. In addition, the item analysis found gaps in students' abilities to transfer clinical evidence/data to basic biomedical knowledge as demonstrated through the multiple-choice questioning results. This outcome warrants further investigation of the utility of the m-OSCE, with a focus on modifications to the evaluation questions, grading rubric, and patient case.

Evaluation of Adult Literacy Education in the United States: A Review of Methodological Issues

ERIC Educational Resources Information Center

Shi, Yan; Tsang, Mun C.

2008-01-01

This is a critical review of methodological issues in the evaluation of adult literacy education programs in the United States. It addresses the key research questions: What are the appropriate methods for evaluating these programs under given circumstances. It identifies 15 evaluation studies that are representative of a range of adult literacy…

Assessing and evaluating multidisciplinary translational teams: a mixed methods approach.

PubMed

Wooten, Kevin C; Rose, Robert M; Ostir, Glenn V; Calhoun, William J; Ameredes, Bill T; Brasier, Allan R

2014-03-01

A case report illustrates how multidisciplinary translational teams can be assessed using outcome, process, and developmental types of evaluation using a mixed-methods approach. Types of evaluation appropriate for teams are considered in relation to relevant research questions and assessment methods. Logic models are applied to scientific projects and team development to inform choices between methods within a mixed-methods design. Use of an expert panel is reviewed, culminating in consensus ratings of 11 multidisciplinary teams and a final evaluation within a team-type taxonomy. Based on team maturation and scientific progress, teams were designated as (a) early in development, (b) traditional, (c) process focused, or (d) exemplary. Lessons learned from data reduction, use of mixed methods, and use of expert panels are explored.

Future Issues and Perspectives in the Evaluation of Social Development.

ERIC Educational Resources Information Center

Marsden, David; Oakley, Peter

1991-01-01

An instrumental/technocratic approach to evaluation of social development relies on primarily quantitative methods. An interpretive approach resists claims to legitimacy and authority of "experts" and questions existing interpretations. The latter approach is characterized by cultural relativism and subjectivity. (SK)

Evaluating the reliability, validity, acceptability, and practicality of SMS text messaging as a tool to collect research data: results from the Feeding Your Baby project.

PubMed

Whitford, Heather M; Donnan, Peter T; Symon, Andrew G; Kellett, Gillian; Monteith-Hodge, Ewa; Rauchhaus, Petra; Wyatt, Jeremy C

2012-01-01

To test the reliability, validity, acceptability, and practicality of short message service (SMS) messaging for collection of research data. The studies were carried out in a cohort of recently delivered women in Tayside, Scotland, UK, who were asked about their current infant feeding method and future feeding plans. Reliability was assessed by comparison of their responses to two SMS messages sent 1 day apart. Validity was assessed by comparison of their responses to text questions and the same question administered by phone 1 day later, by comparison with the same data collected from other sources, and by correlation with other related measures. Acceptability was evaluated using quantitative and qualitative questions, and practicality by analysis of a researcher log. Reliability of the factual SMS message gave perfect agreement. Reliabilities for the numerical question were reasonable, with κ between 0.76 (95% CI 0.56 to 0.96) and 0.80 (95% CI 0.59 to 1.00). Validity for data compared with that collected by phone within 24 h (κ =0.92 (95% CI 0.84 to 1.00)) and with health visitor data (κ =0.85 (95% CI 0.73 to 0.97)) was excellent. Correlation validity between the text responses and other related demographic and clinical measures was as expected. Participants found the method a convenient and acceptable way of providing data. For researchers, SMS text messaging provided an easy and functional method of gathering a large volume of data. In this sample and for these questions, SMS was a reliable and valid method for capturing research data.

Comparison between Long-Menu and Open-Ended Questions in computerized medical assessments. A randomized controlled trial

PubMed Central

Rotthoff, Thomas; Baehring, Thomas; Dicken, Hans-Dieter; Fahron, Urte; Richter, Bernd; Fischer, Martin R; Scherbaum, Werner A

2006-01-01

Background Long-menu questions (LMQs) are viewed as an alternative method for answering open-ended questions (OEQs) in computerized assessment. So far this question type and its influence on examination scores have not been studied sufficiently. However, the increasing use of computerized assessments will also lead to an increasing use of this question type. Using a summative online key feature (KF) examination we evaluated whether LMQs can be compared with OEQs in regard to the level of difficulty, performance and response times. We also evaluated the content for its suitability for LMQs. Methods We randomized 146 fourth year medical students into two groups. For the purpose of this study we created 7 peer-reviewed KF-cases with a total of 25 questions. All questions had the same content in both groups, but nine questions had a different answer type. Group A answered 9 questions with an LM type, group B with an OE type. In addition to the LM answer, group A could give an OE answer if the appropriate answer was not included in the list. Results The average number of correct answers for LMQs and OEQs showed no significant difference (p = 0.93). Among all 630 LM answers only one correct term (0.32%) was not included in the list of answers. The response time for LMQs did not significantly differ from that of OEQs (p = 0.65). Conclusion LMQs and OEQs do not differ significantly. Compared to standard multiple-choice questions (MCQs), the response time for LMQs and OEQs is longer. This is probably due to the fact that they require active problem solving skills and more practice. LMQs correspond more suitable to Short answer questions (SAQ) then to OEQ and should only be used when the answers can be clearly phrased, using only a few, precise synonyms. LMQs can decrease cueing effects and significantly simplify the scoring in computerized assessment. PMID:17032439

[Primary care screening of problems in the elderly and a proposal for a screening protocol with a multidimensional approach].

PubMed

Lino, Valéria Teresa Saraiva; Portela, Margareth Crisóstomo; Camacho, Luiz Antonio Bastos; Rodrigues, Nadia Cristina Pinheiro; Andrade, Monica Kramer de Noronha; O'Dwyer, Gisele

2016-07-21

The objectives were to examine psychometric properties of a screening test for the elderly and to propose a protocol for use in primary care. The method consisted of four stages: (1) inter-evaluator reliability for performance tests and self-assessment questions for eight functions; (2) sensitivity and specificity of questions on depression and social support; (3) meeting of experts to select instrumental activities of daily living (IADL); and (4) elaboration of the protocol. Screening lasted 16 minutes. Inter-evaluator reliability was excellent for performance tests but poor for questions. Depression and social support showed satisfactory sensitivity and specificity (0.74/0.77 and 0.77/0.96). Four IADL were selected by more than 55% of the experts. Following the results, a screening protocol was elaborated that prioritized the use of performance tests, maintaining questions on mood, social support, and IADL. The study suggests better reproducibility of performance tests when compared to questions. For mood and social support, the questions may provide a first screening stage. The proposed protocol allows rapid screening of problems.

Influence of forest and rangeland management on anadromous fish habitat in Western North America: economic considerations.

Treesearch

William R. tech. ed. Meehan

1985-01-01

Although many effects of forest and rangeland management on anadromous fisheries are difficult to measure, economic methods for the evaluation of costs and benefits can be helpful. Such methods can be used to address questions of equity as well as efficiency. Evaluations of equity can show who bears the costs and who captures the benefits of management actions, but...

Information and problem report usage in system saftey engineering division

NASA Technical Reports Server (NTRS)

Morrissey, Stephen J.

1990-01-01

Five basic problems or question areas are examined. They are as follows: (1) Evaluate adequacy of current problem/performance data base; (2) Evaluate methods of performing trend analysis; (3) Methods and sources of data for probabilistic risk assessment; and (4) How is risk assessment documentation upgraded and/or updated. The fifth problem was to provide recommendations for each of the above four areas.

How Secure Is Education in Information Technology? A Method for Evaluating Security Education in IT

ERIC Educational Resources Information Center

Grover, Mark; Reinicke, Bryan; Cummings, Jeff

2016-01-01

As the popularity of Information Technology programs has expanded at many universities, there are a number of questions to be answered from a curriculum standpoint. As many of these programs are either interdisciplinary, or at least exist outside of the usual Computer Science and Information Systems programs, questions of what is appropriate for…

Formative student-authored question bank: perceptions, question quality and association with summative performance.

PubMed

Walsh, Jason L; Harris, Benjamin H L; Denny, Paul; Smith, Phil

2018-02-01

There are few studies on the value of authoring questions as a study method, the quality of the questions produced by students and student perceptions of student-authored question banks. Here we evaluate PeerWise, a widely used and free online resource that allows students to author, answer and discuss multiple-choice questions. We introduced two undergraduate medical student cohorts to PeerWise (n=603). We looked at their patterns of PeerWise usage; identified associations between student engagement and summative exam performance; and used focus groups to assess student perceptions of the value of PeerWise for learning. We undertook item analysis to assess question difficulty and quality. Over two academic years, the two cohorts wrote 4671 questions, answered questions 606 658 times and posted 7735 comments. Question writing frequency correlated most strongly with summative performance (Spearman's rank: 0.24, p=<0.001). Student focus groups found that: (1) students valued curriculum specificity; and (2) students were concerned about student-authored question quality. Only two questions of the 300 'most-answered' questions analysed had an unacceptable discriminatory value (point-biserial correlation <0.2). Item analysis suggested acceptable question quality despite student concerns. Quantitative and qualitative methods indicated that PeerWise is a valuable study tool. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Clinical evaluation of patients with patellofemoral disorders.

PubMed

Post, W R

1999-01-01

Accurate clinical evaluation of patients with patellofemoral disorders is the cornerstone of effective treatment. This article defines how a careful history and physical examination can direct strategies for nonoperative and operative management. A critical analysis of traditional methods of evaluation and a streamlined rational approach to clinical evaluation is presented. Key questions and important physical findings that affect treatment decisions are emphasized.

Can Value Added Add Value to Teacher Evaluation?

ERIC Educational Resources Information Center

Darling-Hammond, Linda

2015-01-01

The five thoughtful papers included in this issue of "Educational Researcher" ("ER") raise new questions about the use of value-added methods (VAMs) to estimate teachers' contributions to students' learning as part of personnel evaluation. The papers address both technical and implementation concerns, considering potential…

Evaluating Research Administration: Methods and Utility

ERIC Educational Resources Information Center

Marina, Sarah; Davis-Hamilton, Zoya; Charmanski, Kara E.

2015-01-01

Three studies were jointly conducted by the Office of Research Administration and Office of Proposal Development at Tufts University to evaluate the services within each respective office. The studies featured assessments that used, respectively, (1) quantitative metrics; (2) a quantitative satisfaction survey with limited qualitative questions;…

On the Unification of Psychology, Methodology, and Pedagogy.

ERIC Educational Resources Information Center

Wettersten, John

1987-01-01

The psychological and methodological bases of the Agassi teaching method are described to provide a context for evaluating the theory. A brief history of Selzian psychology and Popper's methodology is given. The Agassi method, which stresses learning through questioning, is detailed. (JL)

Management of intrathecal baclofen therapy for severe acquired brain injury: consensus and recommendations for good clinical practice.

PubMed

De Tanti, Antonio; Scarponi, Federico; Bertoni, Michele; Gasperini, Giulio; Lanzillo, Bernardo; Molteni, Franco; Posteraro, Federico; Vitale, Dino Francesco; Zanpolini, Mauro

2017-08-01

Although widespread in the treatment of generalised spasticity due to severe acquired brain injury, clinical use of intrathecal baclofen administered through an implanted catheter is not yet supported by full scientific evidence. The aim of the study is to provide recommendations for good clinical practice regarding intrathecal baclofen therapy. We used a modified RAND Delphi method to develop consensus-based medical guidelines, involving clinicians who use intrathecal baclofen therapy throughout Italy. The clinicians were asked 38 questions grouped in six areas (patient selection, contraindications for implant, tests prior to implant, method of implant and management of therapy, efficacy evaluation and goal setting, and management of complications). To establish consensus, 75% agreement was required in answers to every question. Consensus was reached on the second round of the Delphi process on 27/38 questions (71%), specifically those regarding identification of objectives, efficacy evaluation, and method of implant and management of therapy, whereas management of complications and contraindications for implant remained critical areas. Despite the limits of our method, a set of recommendations was drawn up for clinical practice in this sector. The study also revealed residual critical areas and indicated future lines of research necessary to reach evidence-based consensus.

Conducting systematic reviews of economic evaluations.

PubMed

Gomersall, Judith Streak; Jadotte, Yuri Tertilus; Xue, Yifan; Lockwood, Suzi; Riddle, Dru; Preda, Alin

2015-09-01

In 2012, a working group was established to review and enhance the Joanna Briggs Institute (JBI) guidance for conducting systematic review of evidence from economic evaluations addressing a question(s) about health intervention cost-effectiveness. The objective is to present the outcomes of the working group. The group conducted three activities to inform the new guidance: review of literature on the utility/futility of systematic reviews of economic evaluations and consideration of its implications for updating the existing methodology; assessment of the critical appraisal tool in the existing guidance against criteria that promotes validity in economic evaluation research and two other commonly used tools; and a workshop. The debate in the literature on the limitations/value of systematic review of economic evidence cautions that systematic reviews of economic evaluation evidence are unlikely to generate one size fits all answers to questions about the cost-effectiveness of interventions and their comparators. Informed by this finding, the working group adjusted the framing of the objectives definition in the existing JBI methodology. The shift is away from defining the objective as to determine one cost-effectiveness measure toward summarizing study estimates of cost-effectiveness and informed by consideration of the included study characteristics (patient, setting, intervention component, etc.), identifying conditions conducive to lowering costs and maximizing health benefits. The existing critical appraisal tool was included in the new guidance. The new guidance includes the recommendation that a tool designed specifically for the purpose of appraising model-based studies be used together with the generic appraisal tool for economic evaluations assessment to evaluate model-based evaluations. The guidance produced by the group offers reviewers guidance for each step of the systematic review process, which are the same steps followed in JBI reviews of other types of evidence. The updated JBI guidance will be useful for researchers wanting to synthesize evidence about economic questions, either as stand-alone reviews or part of comprehensive or mixed method evidence reviews. Although the updated methodology produced by the work of the working group has improved the JBI guidance for systematic reviews of economic evaluations, there are areas where further work is required. These include adjusting the critical appraisal tool to separate out questions addressing intervention cost and effectiveness measurement; providing more explicit guidance for assessing generalizability of findings; and offering a more robust method for evidence synthesis that facilitates achieving the more ambitious review objectives.

System and algorithm for evaluation of human auditory analyzer state

NASA Astrophysics Data System (ADS)

Bachynskiy, Mykhaylo V.; Azarkhov, Oleksandr Yu.; Shtofel, Dmytro Kh.; Horbatiuk, Svitlana M.; Ławicki, Tomasz; Kalizhanova, Aliya; Smailova, Saule; Askarova, Nursanat

2017-08-01

The paper discusses questions of human auditory state evaluation with technical means. It considers the disadvantages of existing clinical audiometry methods and systems. It is proposed to use method for evaluating of auditory analyzer state by means of pulsometry to get the medical study more objective and efficient. It provides for use of two optoelectronic sensors located on the carotid artery and ear lobe, Using this method the biotechnical system for evaluation and stimulation of human auditory analyzer stare wad developed. Its hardware and software were substantiated. Different modes of simulation in the designed system were tested and the influence of the procedure on a patient was studied.

Assessing and Evaluating Multidisciplinary Translational Teams: A Mixed Methods Approach

PubMed Central

Wooten, Kevin C.; Rose, Robert M.; Ostir, Glenn V.; Calhoun, William J.; Ameredes, Bill T.; Brasier, Allan R.

2014-01-01

A case report illustrates how multidisciplinary translational teams can be assessed using outcome, process, and developmental types of evaluation using a mixed methods approach. Types of evaluation appropriate for teams are considered in relation to relevant research questions and assessment methods. Logic models are applied to scientific projects and team development to inform choices between methods within a mixed methods design. Use of an expert panel is reviewed, culminating in consensus ratings of 11 multidisciplinary teams and a final evaluation within a team type taxonomy. Based on team maturation and scientific progress, teams were designated as: a) early in development, b) traditional, c) process focused, or d) exemplary. Lessons learned from data reduction, use of mixed methods, and use of expert panels are explored. PMID:24064432

Evaluating the Accessibility of Web-Based Instruction for Students with Disabilities.

ERIC Educational Resources Information Center

Hinn, D. Michelle

This paper presents the methods and results of a year-long evaluation study, conducted for the purpose of determining disability accessibility barriers and potential solutions for those barriers found in four World Wide Web-based learning environments. The primary questions used to frame the evaluation study were: (1) Are there any features of the…

A Question-Answer System for Mobile Devices in Lecture-Based Instruction: A Qualitative Analysis of Student Engagement and Learning

ERIC Educational Resources Information Center

Hatun Atas, Amine; Delialioglu, Ömer

2018-01-01

The aim of this study was to explore the opinions, perceptions and evaluations of students about their experiences with a question-answer system used on mobile devices in a lecture-based course. Basic qualitative research method was employed in this study to understand how students made sense of their experiences during the instruction. The…

Falsification Testing of Instrumental Variables Methods for Comparative Effectiveness Research.

PubMed

Pizer, Steven D

2016-04-01

To demonstrate how falsification tests can be used to evaluate instrumental variables methods applicable to a wide variety of comparative effectiveness research questions. Brief conceptual review of instrumental variables and falsification testing principles and techniques accompanied by an empirical application. Sample STATA code related to the empirical application is provided in the Appendix. Comparative long-term risks of sulfonylureas and thiazolidinediones for management of type 2 diabetes. Outcomes include mortality and hospitalization for an ambulatory care-sensitive condition. Prescribing pattern variations are used as instrumental variables. Falsification testing is an easily computed and powerful way to evaluate the validity of the key assumption underlying instrumental variables analysis. If falsification tests are used, instrumental variables techniques can help answer a multitude of important clinical questions. © Health Research and Educational Trust.

Realism and resources: Towards more explanatory economic evaluation

PubMed Central

Anderson, Rob; Hardwick, Rebecca

2016-01-01

To be successfully and sustainably adopted, policy-makers, service managers and practitioners want public programmes to be affordable and cost-effective, as well as effective. While the realist evaluation question is often summarised as what works for whom, under what circumstances, we believe the approach can be as salient to answering questions about resource use, costs and cost-effectiveness – the traditional domain of economic evaluation methods. This paper first describes the key similarities and differences between economic evaluation and realist evaluation. It summarises what health economists see as the challenges of evaluating complex interventions, and their suggested solutions. We then use examples of programme theory from a recent realist review of shared care for chronic conditions to illustrate two ways in which realist evaluations might better capture the resource requirements and resource consequences of programmes, and thereby produce explanations of how they are linked to outcomes (i.e. explanations of cost-effectiveness). PMID:27478402

A passage retrieval method based on probabilistic information retrieval model and UMLS concepts in biomedical question answering.

PubMed

Sarrouti, Mourad; Ouatik El Alaoui, Said

2017-04-01

Passage retrieval, the identification of top-ranked passages that may contain the answer for a given biomedical question, is a crucial component for any biomedical question answering (QA) system. Passage retrieval in open-domain QA is a longstanding challenge widely studied over the last decades. However, it still requires further efforts in biomedical QA. In this paper, we present a new biomedical passage retrieval method based on Stanford CoreNLP sentence/passage length, probabilistic information retrieval (IR) model and UMLS concepts. In the proposed method, we first use our document retrieval system based on PubMed search engine and UMLS similarity to retrieve relevant documents to a given biomedical question. We then take the abstracts from the retrieved documents and use Stanford CoreNLP for sentence splitter to make a set of sentences, i.e., candidate passages. Using stemmed words and UMLS concepts as features for the BM25 model, we finally compute the similarity scores between the biomedical question and each of the candidate passages and keep the N top-ranked ones. Experimental evaluations performed on large standard datasets, provided by the BioASQ challenge, show that the proposed method achieves good performances compared with the current state-of-the-art methods. The proposed method significantly outperforms the current state-of-the-art methods by an average of 6.84% in terms of mean average precision (MAP). We have proposed an efficient passage retrieval method which can be used to retrieve relevant passages in biomedical QA systems with high mean average precision. Copyright © 2017 Elsevier Inc. All rights reserved.

The relationship between facilitators' questions and the level of reflection in postsimulation debriefing.

PubMed

Husebø, Sissel Eikeland; Dieckmann, Peter; Rystedt, Hans; Søreide, Eldar; Friberg, Febe

2013-06-01

Simulation-based education is a learner-active method that may enhance teamwork skills such as leadership and communication. The importance of postsimulation debriefing to promote reflection is well accepted, but many questions concerning whether and how faculty promote reflection remain largely unanswered in the research literature. The aim of this study was therefore to explore the depth of reflection expressed in questions by facilitators and responses from nursing students during postsimulation debriefings. Eighty-one nursing students and 4 facilitators participated. The data were collected in February and March 2008, the analysis being conducted on 24 video-recorded debriefings from simulated resuscitation teamwork involving nursing students only. Using Gibbs' reflective cycle, we graded the facilitators' questions and nursing students' responses into stages of reflection and then correlated these. Facilitators asked most evaluative and fewest emotional questions, whereas nursing students answered most evaluative and analytic responses and fewest emotional responses. The greatest difference between facilitators and nursing students was in the analytic stage. Only 23 (20%) of 117 questions asked by the facilitators were analytic, whereas 45 (35%) of 130 students' responses were rated as analytic. Nevertheless, the facilitators' descriptive questions also elicited student responses in other stages such as evaluative and analytic responses. We found that postsimulation debriefings provide students with the opportunity to reflect on their simulation experience. Still, if the debriefing is going to pave the way for student reflection, it is necessary to work further on structuring the debriefing to facilitate deeper reflection. Furthermore, it is important that facilitators consider what kind of questions they ask to promote reflection. We think future research on debriefing should focus on developing an analytical framework for grading reflective questions. Such research will inform and support facilitators in devising strategies for the promotion of learning through reflection in postsimulation debriefings.

Psychosocial consequences of cancer cachexia: the development of an item bank.

PubMed

Häne, Hanspeter; Oberholzer, Rolf; Walker, Jochen; Hopkinson, Jane B; de Wolf-Linder, Susanne; Strasser, Florian

2013-12-01

Cancer cachexia syndrome (CCS) is often accompanied by psychosocial consequences (PSC). To alleviate PSC, a systematic assessment method is required. Currently, few assessment tools are available (e.g., Functional Assessment of Anorexia/Cachexia Therapy). There is no systematic assessment tool that captures the PSC of CCS. To develop a pilot item bank to assess the PSC of CCS. A total of 132 questions, generated from patient answers in a previous study, were reduced to 121 items by content analysis and evaluation by multidisciplinary experts (doctor, nutritionists, and nurses). In our two-step, cross-sectional study, patients, judged by staff to have PSC of CCS, were included, and the questions were randomly allocated to the patients. Questions were evaluated for understandability and triggering emotions, and patients were asked to provide a response using a four-point Likert scale. Subsequently, problematic questions were revised, reformulated, and retested. A total of 20 patients with a variety of tumor types participated. Of the 121 questions, 31 had to be reformulated after Step 1 and were retested in Step 2, after which seven were again evaluated as not being perfectly comprehensible. In Step 1, 22 questions were found to trigger emotions, but no item required remodeling. Item performance using the Likert scale revealed no consistent floor or ceiling effects. Our final pilot question bank comprised 117 questions. The final item bank contains questions that are understood and accepted by the patients. This item bank now needs to be developed into a measurement tool that groups items into domains and can be used in future research studies. Copyright © 2013 U.S. Cancer Pain Relief Committee. Published by Elsevier Inc. All rights reserved.

Using the ALA's "Evaluating Library Instruction" (1996).

ERIC Educational Resources Information Center

Roselle, Ann

1997-01-01

Using a multi-method approach--student and faculty surveys and a comparison of bibliographies from senior nursing students--to evaluate the Information Literacy Skills Program at the University of Botswana, this study found that information about the impact of instruction can be discovered through more open-ended questions; the ALA's…

Colorado Even Start. 1998-1999 Progress Report.

ERIC Educational Resources Information Center

Anderson, Beckie

Even Start programs integrate early childhood education, adult literacy or basic education, parenting education and support, and parent and child time to help break the cycle of poverty and illiteracy. This report describes the Even Start program in Colorado, and includes evaluation questions and methods. The report presents evaluation findings…

Teaching, Learning and Evaluation Techniques in the Engineering Courses.

ERIC Educational Resources Information Center

Vermaas, Luiz Lenarth G.; Crepaldi, Paulo Cesar; Fowler, Fabio Roberto

This article presents some techniques of professional formation from the Petra Model that can be applied in Engineering Programs. It shows its philosophy, teaching methods for listening, making abstracts, studying, researching, team working and problem solving. Some questions regarding planning and evaluation, based in the model are, as well,…

Parents Questioning Immunization: Evaluation of an Intervention

ERIC Educational Resources Information Center

Gust, Deborah A.; Kennedy, Allison; Weber, Deanne; Evans, Geoff; Kong, Yuan; Salmon, Daniel

2009-01-01

Objectives: To compare attitudes of parents who filed or considered filing an exemption to school immunization requirements and/or would not have their child immunized if it were not required by law (cases) to controls. To develop and evaluate a brochure intervention for parents considering an exemption. Methods: Interviews, focus groups, mailed…

Double Sampling with Multiple Imputation to Answer Large Sample Meta-Research Questions: Introduction and Illustration by Evaluating Adherence to Two Simple CONSORT Guidelines

PubMed Central

Capers, Patrice L.; Brown, Andrew W.; Dawson, John A.; Allison, David B.

2015-01-01

Background: Meta-research can involve manual retrieval and evaluation of research, which is resource intensive. Creation of high throughput methods (e.g., search heuristics, crowdsourcing) has improved feasibility of large meta-research questions, but possibly at the cost of accuracy. Objective: To evaluate the use of double sampling combined with multiple imputation (DS + MI) to address meta-research questions, using as an example adherence of PubMed entries to two simple consolidated standards of reporting trials guidelines for titles and abstracts. Methods: For the DS large sample, we retrieved all PubMed entries satisfying the filters: RCT, human, abstract available, and English language (n = 322, 107). For the DS subsample, we randomly sampled 500 entries from the large sample. The large sample was evaluated with a lower rigor, higher throughput (RLOTHI) method using search heuristics, while the subsample was evaluated using a higher rigor, lower throughput (RHITLO) human rating method. Multiple imputation of the missing-completely at-random RHITLO data for the large sample was informed by: RHITLO data from the subsample; RLOTHI data from the large sample; whether a study was an RCT; and country and year of publication. Results: The RHITLO and RLOTHI methods in the subsample largely agreed (phi coefficients: title = 1.00, abstract = 0.92). Compliance with abstract and title criteria has increased over time, with non-US countries improving more rapidly. DS + MI logistic regression estimates were more precise than subsample estimates (e.g., 95% CI for change in title and abstract compliance by year: subsample RHITLO 1.050–1.174 vs. DS + MI 1.082–1.151). As evidence of improved accuracy, DS + MI coefficient estimates were closer to RHITLO than the large sample RLOTHI. Conclusion: Our results support our hypothesis that DS + MI would result in improved precision and accuracy. This method is flexible and may provide a practical way to examine large corpora of literature. PMID:25988135

Development of a Nationally Coordinated Evaluation Plan for the Ghana National Strategy for Key Populations

PubMed Central

Reynolds, Heidi W; Atuahene, Kyeremeh; Sutherland, Elizabeth; Amenyah, Richard; Kwao, Isaiah Doe; Larbi, Emmanuel Tettey

2015-01-01

Objective Just as HIV prevention programs need to be tailored to the local epidemic, so should evaluations be country-owned and country-led to ensure use of those results in decision making and policy. The objective of this paper is to describe the process undertaken in Ghana to develop a national evaluation plan for the Ghana national strategy for key populations. Methods This was a participatory process that involved meetings between the Ghana AIDS Commission (GAC), other partners in Ghana working to prevent HIV among key populations, and MEASURE Evaluation. The process included three two-day, highly structured yet participatory meetings over the course of 12 months during which participants shared information about on-going and planned data and identified research questions and methods. Results An evaluation plan was prepared to inform stakeholders about which data collection activities need to be prioritized for funding, who would implement the study, the timing of data collection, the research question the data will help answer, and the analysis methods. The plan discusses various methods that can be used including the recommendation for the study design using multiple data sources. It has an evaluation conceptual model, proposed analyses, proposed definition of independent variables, estimated costs for filling data gaps, roles and responsibilities of stakeholders to carry out the plan, and considerations for ethics, data sharing and authorship. Conclusion The experience demonstrates that it is possible to design an evaluation responsive to national strategies and priorities with country leadership, regardless of stakeholders' experiences with evaluations. This process may be replicable elsewhere, where stakeholders want to plan and implement an evaluation of a large-scale program at the national or subnational level that is responsive to national priorities and part of a comprehensive monitoring and evaluation system. PMID:26120495

Subtle Emotional Process in Early Childhood Pedagogy: Evaluating the Contribution of the Tavistock Observation Method

ERIC Educational Resources Information Center

Elfer, Peter

2017-01-01

Nursery experience is now common for young children and their families. Questions of quality have focussed mainly on safety and early learning. The roles of subtle emotional processes in daily pedagogic interactions have received surprisingly little attention. This paper discusses the Tavistock Observation Method (TOM), a naturalistic method of…

A Design Science Research Methodology for Developing a Computer-Aided Assessment Approach Using Method Marking Concept

ERIC Educational Resources Information Center

Genemo, Hussein; Miah, Shah Jahan; McAndrew, Alasdair

2016-01-01

Assessment has been defined as an authentic method that plays an important role in evaluating students' learning attitude in acquiring lifelong knowledge. Traditional methods of assessment including the Computer-Aided Assessment (CAA) for mathematics show limited ability to assess students' full work unless multi-step questions are sub-divided…

Kirkpatrick and Beyond: A Review of Models of Training Evaluation. IES Report.

ERIC Educational Resources Information Center

Tamkin, P.; Yarnall, J.; Kerrin, M.

Many organizations are not satisfied that their methods of evaluating training are rigorous or extensive enough to answer questions of value to them. Complaints about Kirkpatrick's popular four-step model (1959) of training evaluation are that each level is assumed to be associated with the previous and next levels and that the model is too simple…

Math and Science Academy: Year 4 Evaluation Report. CSE Technical Report 648

ERIC Educational Resources Information Center

Osmundson, Ellen; Herman, Joan

2005-01-01

This evaluation report summarizes Year 4 of the Math and Science Academy (MSA), an initiative of the Northern New Mexico Council on Excellence in Education (NNMCEE). The report begins with an overview of the project and its objectives, and then outlines the research questions and methods used to carry out the evaluation. Findings from the Year 4…

Comments on Professor Bloom's Paper Entitled "Toward a Theory of Testing Which Includes Measurement-Evaluation-Assessment."

ERIC Educational Resources Information Center

Glass, Gene V.

The objective of evaluation in education is distinct from the objectives of measurement and assessment or testing. Accordingly, a synthesis of the three procedures into a single theory is not viable. Evaluation is a method of gathering empirical evidence for decision making by answering questions about the worth of educational materials,…

The evaluation of a peer-led question-writing task.

PubMed

Nwosu, Amara; Mason, Stephen; Roberts, Anita; Hugel, Heino

2013-06-01

  Novel studies have previously highlighted the educational benefits of peer-led learning and peer marking of examinations. Limited data exist about the educational value of students writing their own exam questions and sharing these with other students.   To evaluate the potential for medical students to learn about palliative care through the process of writing examination questions. methods:  Fourth-year medical students on a palliative medicine rotation were invited to write a short-answer exam question in a similar format to the official examination run by the medical school. The questions were checked for accuracy and applicability by the coordinator, and were then distributed to students. The answers to the student-generated questions were discussed in a feedback forum at the end of the rotation.   Twenty students took part in the exercise. All agreed the exercise was beneficial to their learning. Eighteen (90%) students did not feel the task was too much additional work. Eight (40%) students felt more confident in passing the official medical school exams. Students enjoyed the exercise, with 19 (95%) indicating that they were keen to repeat this in the future.   Peer-led examination question writing could complement the delivery of palliative care teaching for medical students. © 2013 John Wiley & Sons Ltd.

Dealing with Actors and Compliance in Intervention Operations in a Non-permissive Hybrid Environment

DTIC Science & Technology

2009-12-01

Patton, (2002). Qualitative research & evaluation methods (Vol. Third). California, London, New Delhi: Sage publications, Inc. 59 See these themes...non-compliant actors … … the study follows a question based approach with a general research question in the centre of interest:12 Which...1 Version 1.0 December 2009 Study authors: Col Dieter Muhr (AUT) Hon. Assoc. Prof. Dr. Andrea Riemer, Ph.D. (AUT) Objective

Approche du Processus d'Auto-Formation Inspiree de la Methode des "Histoires de vie" dans le cadre de la Formation continuee des institutrices maternelles en Communaute francaise de Belgique (Approach to the Process of Self-Instruction Inspired by the "Life History" Method in the Framework of the Continuing Education of Preschool Teachers in the French-Speaking Community in Belgium).

ERIC Educational Resources Information Center

Brandt, Madeleine

1994-01-01

Describes a study of the use of a critical events questionnaire and "educational biography" as means of training preschool teachers to question their role as teachers; to question their relationship with children; and to liberate their consciousness. Discusses the quantitative and qualitative results of the self-evaluation processes. (AC)

Cognitive Change Questionnaire as a method for cognitive impairment screening

PubMed Central

Damin, Antonio Eduardo; Nitrini, Ricardo; Brucki, Sonia Maria Dozzi

2015-01-01

The Cognitive Change Questionnaire (CCQ) was created as an effective measure of cognitive change that is easy to use and suitable for application in Brazil. Objective To evaluate whether the CCQ can accurately distinguish normal subjects from individuals with Mild Cognitive Impairment (MCI) and/or early stage dementia and to develop a briefer questionnaire, based on the original 22-item CCQ (CCQ22), that contains fewer questions. Methods A total of 123 individuals were evaluated: 42 healthy controls, 40 patients with MCI and 41 with mild dementia. The evaluation was performed using cognitive tests based on individual performance and on questionnaires administered to informants. The CCQ22 was created based on a selection of questions that experts deemed useful in screening for early stage dementia. Results The CCQ22 showed good accuracy for distinguishing between the groups. Statistical models selected the eight questions with the greatest power to discriminate between the groups. The AUC ROC corresponding to the final version of the 8-item CCQ (CCQ8), demonstrated good accuracy in differentiating between groups, good correlation with the final diagnosis (r=0.861) and adequate internal consistency (Cronbach's α=0.876). Conclusion The CCQ8 can be used to accurately differentiate between normal subjects and individuals with cognitive impairment, constituting a brief and appropriate instrument for cognitive screening. PMID:29213967

The effectiveness of the 1996 Wood Magic Science Fair as an experiential field trip

NASA Astrophysics Data System (ADS)

Pearson, Robert Wayne

In response to the need for evaluating the effectiveness of the 1996 Wood Magic Science Fair (WMSF) at Mississippi State University's Forest and Wildlife Research Center/Forest Products Laboratory, two tests were developed. One test was designed for third-grade students and one for fourth-grade students who would attend the WMSF. Both tests have multiple choice answers and contained thirteen questions each. Five of the questions addressed general issues of the forest products industry that would be mentioned in an opening video but not stressed in the oral presentations of the WMSF. The students would have no active involvement in these presentations. These general issue questions represent passive involvement teaching. The eight remaining questions addressed specific information that would be stressed in the WMSF oral presentations and would allow students to actively participate in the presentations. The participation of the students in these presentations represents participatory teaching. The tests were given to the students (third and fourth grades) both before they attended the fair as a pre-test and after their attendance as a post-test with the only difference being that the multiple choice answers were arranged in a different order. Classroom teachers administered the tests to the students. The test results were evaluated for each class individually. Each question was evaluated and the results recorded in a table. The general information questions were grouped together as were the specific information questions. This grouping allowed a comparison to be made between passive and participatory learning. The results from the tests will help those making the science fair presentations evaluate their materials and methods. Statistical analysis of the results would indicate which questions had a significant change in the number of correct answers between the pre-test and post-test. This information will help the Fair planners and presenters in developing future programs.

Integrating species distributional, conservation planning, and individual based population models: A case study in critical habitat evaluation for the Northern Spotted Owl

EPA Science Inventory

Background / Question / Methods As part of the ongoing northern spotted owl recovery planning effort, we evaluated a series of alternative potential critical habitat scenarios using a species-distribution model (MaxEnt), a conservation-planning model (Zonation), and an individua...

Designing Evaluations. 2012 Revision. Applied Research and Methods. GAO-12-208G

ERIC Educational Resources Information Center

US Government Accountability Office, 2012

2012-01-01

GAO assists congressional decision makers in their deliberations by furnishing them with analytical information on issues and options. Many diverse methodologies are needed to develop sound and timely answers to the questions the Congress asks. To provide GAO evaluators with basic information about the more commonly used methodologies, GAO's…

Teaching and Evaluating Point of Care Learning with an Internet-Based Clinical-Question Portfolio

ERIC Educational Resources Information Center

Green, Michael L.; Reddy, Siddharta G.; Holmboe, Eric

2009-01-01

Introduction: Diplomates in the American Board of Internal Medicine (ABIM) Maintenance of Certification (MOC) program satisfy the self-evaluation of medical knowledge requirement by completing open-book multiple-choice exams. However, this method remains unlikely to affect practice change and often covers content areas not relevant to diplomates'…

Introduction of Sustainable Development in Engineers' Curricula: Problematic and Evaluation Methods

ERIC Educational Resources Information Center

Lourdel, N.; Gondran, N.; Laforest, V.; Brodhag, C.

2005-01-01

Purpose: Owing to its complexity, sustainable development cannot be simply integrated as a supplementary course within the engineers' curricula. The first point of this paper aims to focalise on how to reflect pedagogically. After dealing with these questions, a tool that can evaluate the student's understanding of sustainable development concepts…

Evaluating an Objective Structured Clinical Examination (OSCE) Adapted for Social Work

ERIC Educational Resources Information Center

Bogo, Marion; Regehr, Cheryl; Katz, Ellen; Logie, Carmen; Tufford, Lea; Litvack, Andrea

2012-01-01

Objectives: To evaluate an objective structured clinical examination (OSCE) adapted for social work in a lab course and examine the degree to which it predicts competence in the practicum. Method: 125 Masters students participated in a one-scenario OSCE and wrote responses to standardized reflection questions. OSCE performance and reflections were…

Group members' questions shape participation in health counselling and health education.

PubMed

Logren, Aija; Ruusuvuori, Johanna; Laitinen, Jaana

2017-10-01

This study examines how group members' questions shape member participation in health counselling and health education groups. The study applies conversation analytic principles as a method. The data consist of video-recorded health education lessons in secondary school and health counselling sessions for adults with a high risk of Type 2 diabetes. Group members' questions accomplish a temporary change in participatory roles. They are used to 1) request counselling, 2) do counselling or 3) challenge previous talk. They are usually treated as relevant and legitimate actions by the participants, but are occasionally interpreted as transitions outside the current action or topic. Group members' questions result in a shift from leader-driven to member-driven discussion. Thus they constitute a pivot point for detecting changes in participation in group interventions. Observing the occurrence of group members' questions helps group leaders to adjust their own actions accordingly and thus facilitate or guide group participation. Comparison of the type and frequency of members' questions is a way to detect different trajectories for delivering group interventions and can thus be used to develop methods for process evaluation of interventions. Copyright © 2017 Elsevier B.V. All rights reserved.

Use of questioning during lectures in a dental hygiene didactic course.

PubMed

Hessheimer, Heather M; Rogo, Ellen J; Howlett, Bernadette

2011-08-01

The purposes of this quasi-experimental, one-group crossover study were to determine the effect of questioning during dental hygiene lectures on low-level and high-level learning and to evaluate student perceptions of questioning. Twenty-three dental hygiene students participated in two control lectures using traditional lecturing methods. The students served as their own controls by next participating in two experimental lectures with questions asked throughout the lecture at both low and high cognitive levels. Student performance was measured with an examination containing low- and high-level questions. The interaction between the group and the level of questions was analyzed using ANOVA, and no statistically significant difference was found. Based on a Likert scale (1 to 6), average ratings for student perceptions were as follows: enjoyment of use, 4.5; understanding the lecture material, 4.74; and questioning effectiveness, 4.35. Student perceptions of questioning were positive; however, this strategy was found to be no more effective than the traditional lecture in promoting retention of information.

Intelligence's likelihood and evolutionary time frame

NASA Astrophysics Data System (ADS)

Bogonovich, Marc

2011-04-01

This paper outlines hypotheses relevant to the evolution of intelligent life and encephalization in the Phanerozoic. If general principles are inferable from patterns of Earth life, implications could be drawn for astrobiology. Many of the outlined hypotheses, relevant data, and associated evolutionary and ecological theory are not frequently cited in astrobiological journals. Thus opportunity exists to evaluate reviewed hypotheses with an astrobiological perspective. A quantitative method is presented for testing one of the reviewed hypotheses (hypothesis i; the diffusion hypothesis). Questions are presented throughout, which illustrate that the question of intelligent life's likelihood can be expressed as multiple, broadly ranging, more tractable questions.

Estimates of Dietary Sodium Consumption in Patients With Chronic Heart Failure.

PubMed

Colin-Ramirez, Eloisa; Arcand, JoAnne; Ezekowitz, Justin A

2015-12-01

Estimating dietary sodium intake is a key component of dietary assessment in the clinical setting of HF to effectively implement appropriate dietary interventions for sodium reduction and monitor adherence to the dietary treatment. In a research setting, assessment of sodium intake is crucial to an essential methodology to evaluate outcomes after a dietary or behavioral intervention. Current available sodium intake assessment methods include 24-hour urine collection, spot urine collections, multiple day food records, food recalls, and food frequency questionnaires. However, these methods have inherent limitations that make assessment of sodium intake challenging, and the utility of traditional methods may be questionable for estimating sodium intake in patients with HF. Thus, there are remaining questions about how to best assess dietary sodium intake in this patient population, and there is a need to identify a reliable method to assess and monitor sodium intake in the research and clinical setting of HF. This paper provides a comprehensive review of the current methods for sodium intake assessment, addresses the challenges for its accurate evaluation, and highlights the relevance of applying the highest-quality measurement methods in the research setting to minimize the risk of biased data. Copyright © 2015 Elsevier Inc. All rights reserved.

A Hybrid Approach to Clinical Question Answering

DTIC Science & Technology

2014-11-01

participation in TREC, we submitted a single run using a hybrid Natural Language Processing ( NLP )-driven approach to accomplish the given task. Evaluation re...for the CDS track uses a variety of NLP - based techniques to address the clinical questions provided. We present a description of our approach, and...discuss our experimental setup, results and eval- uation in the subsequent sections. 2 Description of Our Approach Our hybrid NLP -driven method presents a

Evaluation of clinical methods for peroneal muscle testing.

PubMed

Sarig-Bahat, Hilla; Krasovsky, Andrei; Sprecher, Elliot

2013-03-01

Manual muscle testing of the peroneal muscles is well accepted as a testing method in musculoskeletal physiotherapy for the assessment of the foot and ankle. The peroneus longus and brevis are primary evertors and secondary plantar flexors of the ankle joint. However, some international textbooks describe them as dorsi flexors, when instructing peroneal muscle testing. The identified variability raised a question whether these educational texts are reflected in the clinical field. The purposes of this study were to investigate what are the methods commonly used in the clinical field for peroneal muscle testing and to evaluate their compatibility with functional anatomy. A cross-sectional study was conducted, using an electronic questionnaire sent to 143 Israeli physiotherapists in the musculoskeletal field. The survey questioned on the anatomical location of manual resistance and the combination of motions resisted. Ninety-seven responses were received. The majority (69%) of respondents related correctly to the peronei as evertors, but asserted that resistance should be located over the dorsal aspect of the fifth metatarsus, thereby disregarding the peroneus longus. Moreover, 38% of the respondents described the peronei as dorsi flexors, rather than plantar flexors. Only 2% selected the correct method of resisting plantarflexion and eversion at the base of the first metatarsus. We consider this technique to be the most compatible with the anatomy of the peroneus longus and brevis. The Fisher-Freeman-Halton test indicated that there was a significant relationship between responses on the questions (P = 0.0253, 95% CI 0.0249-0.0257), thus justifying further correspondence analysis. The correspondence analysis found no clustering of the answers that were compatible with anatomical evidence and were applied in the correct technique, but did demonstrate a common error, resisting dorsiflexion rather than plantarflexion, which was in agreement with the described frequencies. Inconsistencies were identified between the instruction method commonly provided for peroneal muscle testing in textbook and the functional anatomy of these muscles. Results reflect the lack of accuracy in applying functional anatomy to peroneal testing. This may be due to limited use of peroneal muscle testing or to inadequate investigation of the existing evaluation methods and their validity. Accordingly, teaching materials and clinical methods used for this test should be re-evaluated. Further research should investigate the value of peroneal muscle testing in clinical ankle evaluation. Copyright © 2012 John Wiley & Sons, Ltd.

Use of Multi-Response Format Test in the Assessment of Medical Students’ Critical Thinking Ability

PubMed Central

Mafinejad, Mahboobeh Khabaz; Monajemi, Alireza; Jalili, Mohammad; Soltani, Akbar; Rasouli, Javad

2017-01-01

Introduction To evaluate students critical thinking skills effectively, change in assessment practices is must. The assessment of a student’s ability to think critically is a constant challenge, and yet there is considerable debate on the best assessment method. There is evidence that the intrinsic nature of open and closed-ended response questions is to measure separate cognitive abilities. Aim To assess critical thinking ability of medical students by using multi-response format of assessment. Materials and Methods A cross-sectional study was conducted on a group of 159 undergraduate third-year medical students. All the participants completed the California Critical Thinking Skills Test (CCTST) consisting of 34 multiple-choice questions to measure general critical thinking skills and a researcher-developed test that combines open and closed-ended questions. A researcher-developed 48-question exam, consisting of 8 short-answers and 5 essay questions, 19 Multiple-Choice Questions (MCQ), and 16 True-False (TF) questions, was used to measure critical thinking skills. Correlation analyses were performed using Pearson’s coefficient to explore the association between the total scores of tests and subtests. Results One hundred and fifty-nine students participated in this study. The sample comprised 81 females (51%) and 78 males (49%) with an age range of 20±2.8 years (mean 21.2 years). The response rate was 64.1%. A significant positive correlation was found between types of questions and critical thinking scores, of which the correlations of MCQ (r=0.82) and essay questions (r=0.77) were strongest. The significant positive correlations between multi-response format test and CCTST’s subscales were seen in analysis, evaluation, inference and inductive reasoning. Unlike CCTST subscales, multi-response format test have weak correlation with CCTST total score (r=0.45, p=0.06). Conclusion This study highlights the importance of considering multi-response format test in the assessment of critical thinking abilities of medical students by using both open and closed-ended response questions. PMID:29207742

Assessing the Effectiveness of "Wise Guys": A Mixed-Methods Approach

ERIC Educational Resources Information Center

Herrman, Judith W.; Gordon, Mellissa; Rahmer, Brian; Moore, Christopher C.; Habermann, Barbara; Haigh, Katherine M.

2017-01-01

Previous research raised questions on the validity of survey studies with the teen population. As one response, our team implemented a mixed-methods study to evaluate an evidence-based, interactive curriculum, "Wise Guys," which is designed to promote healthy relationships and sexual behavior in young men ages 4-17. The current study…

Soil carbon changes: comparing flux monitoring and mass balance in a box lysimeter experiment.

Treesearch

S.M. Nay; B.T. Bormann

2000-01-01

Direct measures of soil-surface respiration are needed to evaluate belowground biological processes, forest productivity, and ecosystem responses to global change. Although infra-red gas analyzer {IRGA) methods track reference CO2 flows in lab studies, questions remain for extrapolating IRGA methods to field conditions. We constructed 10 box...

EVALUATING PROGRAMMED TEST INTERPRETATION USING EMOTIONAL AROUSAL AS A CRITERION.

ERIC Educational Resources Information Center

FORSTER, JERALD R.

THE QUESTION AS TO WHETHER OR NOT THE EMOTIONAL ASPECTS OF TEST INTERPRETATION MAKE IT INAPPROPRIATE FOR PROGRAMED METHODS WAS EXAMINED BY TWO METHODS OF TEST RESULT COMMUNICATION--PROGRAMED MATERIALS AND VERBAL COMMUNICATION BY A COUNSELOR. DEPENDENT VARIABLES WERE MEASURES OF EMOTIONAL AROUSAL RECORDED BY SKIN CONDUCTANCE UNITS AND THE GAIN IN…

QA-driven Guidelines Generation for Bacteriotherapy

PubMed Central

Pasche, Emilie; Teodoro, Douglas; Gobeill, Julien; Ruch, Patrick; Lovis, Christian

2009-01-01

PURPOSE We propose a question-answering (QA) driven generation approach for automatic acquisition of structured rules that can be used in a knowledge authoring tool for antibiotic prescription guidelines management. METHODS: The rule generation is seen as a question-answering problem, where the parameters of the questions are known items of the rule (e.g. an infectious disease, caused by a given bacterium) and answers (e.g. some antibiotics) are obtained by a question-answering engine. RESULTS: When looking for a drug given a pathogen and a disease, top-precision of 0.55 is obtained by the combination of the Boolean engine (PubMed) and the relevance-driven engine (easyIR), which means that for more than half of our evaluation benchmark at least one of the recommended antibiotics was automatically acquired by the rule generation method. CONCLUSION: These results suggest that such an automatic text mining approach could provide a useful tool for guidelines management, by improving knowledge update and discovery. PMID:20351908

Comparison of Methods for Adjusting Incorrect Assignments of Items to Subtests: Oblique Multiple Group Method versus Confirmatory Common Factor Method

ERIC Educational Resources Information Center

Stuive, Ilse; Kiers, Henk A. L.; Timmerman, Marieke E.

2009-01-01

A common question in test evaluation is whether an a priori assignment of items to subtests is supported by empirical data. If the analysis results indicate the assignment of items to subtests under study is not supported by data, the assignment is often adjusted. In this study the authors compare two methods on the quality of their suggestions to…

The systematic review as a research process in music therapy.

PubMed

Hanson-Abromeit, Deanna; Sena Moore, Kimberly

2014-01-01

Music therapists are challenged to present evidence on the efficacy of music therapy treatment and incorporate the best available research evidence to make informed healthcare and treatment decisions. Higher standards of evidence can come from a variety of sources including systematic reviews. To define and describe a range of research review methods using examples from music therapy and related literature, with emphasis on the systematic review. In addition, the authors provide a detailed overview of methodological processes for conducting and reporting systematic reviews in music therapy. The systematic review process is described in five steps. Step 1 identifies the research plan and operationalized research question(s). Step 2 illustrates the identification and organization of the existing literature related to the question(s). Step 3 details coding of data extracted from the literature. Step 4 explains the synthesis of coded findings and analysis to answer the research question(s). Step 5 describes the strength of evidence evaluation and results presentation for practice recommendations. Music therapists are encouraged to develop and conduct systematic reviews. This methodology contributes to review outcome credibility and can determine how information is interpreted and used by clinicians, clients or patients, and policy makers. A systematic review is a methodologically rigorous research method used to organize and evaluate extant literature related to a clinical problem. Systematic reviews can assist music therapists in managing the ever-increasing literature, making well-informed evidence based practice and research decisions, and translating existing music-based and nonmusic based literature to clinical practice and research development. © the American Music Therapy Association 2014. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

The impact of podcasting on the learning and satisfaction of undergraduate nursing students.

PubMed

Vogt, Marjorie; Schaffner, Barbara; Ribar, Alicia; Chavez, Ruth

2010-01-01

Nursing education has evolved from traditional pedagogical models to current methods using technology for knowledge acquisition. Podcasting is one example of teaching methodology used in higher education. Studies demonstrate positive student satisfaction with podcasting, but there is limited data related to knowledge acquisition. The purpose of this study was to examine the impact of podcasting on nursing student learning and satisfaction. Two classes of junior baccalaureate nursing students had scores on selected exam questions compared. The 2007 class (n=63) had select content presented in traditional lecture format where the 2008 class (n=57) had the same content presented via podcast. Both methods used the same faculty and exam questions. The 2008 class completed a satisfaction survey. Results indicated no significant difference in correct responses on exam questions. Exam scores were better with the first podcast, equal for the second, and worse with the third podcast. Students were overall satisfied with the podcasting experience and commented positively on its portability and flexibility. This study contributes to the knowledge base of podcasting effectiveness and raises the question of evaluation of new teaching methodologies. Must an increase in learning occur for new methods to be considered effective, or is positive student satisfaction adequate to encourage the adoption of new technology methods?

[Qualitative Research in Health Services Research - Discussion Paper, Part 3: Quality of Qualitative Research].

PubMed

Stamer, M; Güthlin, C; Holmberg, C; Karbach, U; Patzelt, C; Meyer, T

2015-12-01

The third and final discussion paper of the German Network of Health Services Research's (DNVF) "Qualitative Methods Working Group" demonstrates methods for the evaluation and quality of qualitative research in health services research. In this paper we discuss approaches described in evaluating qualitative studies, including: an orientation to the general principles of empirical research, an approach-specific course of action, as well as procedures based on the research-process and criteria-oriented approaches. Divided into general and specific aspects to be considered in a qualitative study quality evaluation, the central focus of the discussion paper undertakes an extensive examination of the process and criteria-oriented approaches. The general aspects include the participation of relevant groups in the research process as well as ethical aspects of the research and data protection issues. The more specific aspects in evaluating the quality of qualitative research include considerations about the research interest, research questions, and the selection of data collection methods and types of analyses. The formulated questions are intended to guide reviewers and researchers to evaluate and to develop qualitative research projects appropriately. The intention of this discussion paper is to ensure a transparent research culture, and to reflect on and discuss the methodological and research approach of qualitative studies in health services research. With this paper we aim to initiate a discussion on high quality evaluation of qualitative health services research. © Georg Thieme Verlag KG Stuttgart · New York.

Scientific Evaluation and Review of Claims in Health Care (SEaRCH): A Streamlined, Systematic, Phased Approach for Determining "What Works" in Healthcare.

PubMed

Jonas, Wayne B; Crawford, Cindy; Hilton, Lara; Elfenbaum, Pamela

2017-01-01

Answering the question of "what works" in healthcare can be complex and requires the careful design and sequential application of systematic methodologies. Over the last decade, the Samueli Institute has, along with multiple partners, developed a streamlined, systematic, phased approach to this process called the Scientific Evaluation and Review of Claims in Health Care (SEaRCH™). The SEaRCH process provides an approach for rigorously, efficiently, and transparently making evidence-based decisions about healthcare claims in research and practice with minimal bias. SEaRCH uses three methods combined in a coordinated fashion to help determine what works in healthcare. The first, the Claims Assessment Profile (CAP), seeks to clarify the healthcare claim and question, and its ability to be evaluated in the context of its delivery. The second method, the Rapid Evidence Assessment of the Literature (REAL © ), is a streamlined, systematic review process conducted to determine the quantity, quality, and strength of evidence and risk/benefit for the treatment. The third method involves the structured use of expert panels (EPs). There are several types of EPs, depending on the purpose and need. Together, these three methods-CAP, REAL, and EP-can be integrated into a strategic approach to help answer the question "what works in healthcare?" and what it means in a comprehensive way. SEaRCH is a systematic, rigorous approach for evaluating healthcare claims of therapies, practices, programs, or products in an efficient and stepwise fashion. It provides an iterative, protocol-driven process that is customized to the intervention, consumer, and context. Multiple communities, including those involved in health service and policy, can benefit from this organized framework, assuring that evidence-based principles determine which healthcare practices with the greatest promise are used for improving the public's health and wellness.

The acoustical design of vehicles-a challenge for qualitative evaluation

NASA Astrophysics Data System (ADS)

Schulte-Fortkamp, Brigitte; Genuit, Klaus; Fiebig, Andre

2005-09-01

Whenever the acoustical design of vehicles is explored, the crucial question about the appropriate method of evaluation arises. Research shows that not only acoustic but also non-acoustic parameters have a major influence on the way sounds are evaluated. Therefore, new methods of evaluation have to be implemented. Methods are needed which give the opportunity to test the quality of the given ambience and to register the effects and evaluations in their functional interdependence as well as the influence of personal and contextual factors. Moreover, new methods have to give insight into processes of evaluation and their contextual parameters. In other words, the task of evaluating acoustical ambiences consists of designating a set of social, psychological, and cultural conditions which are important to determine particular individual and collective behavior, attitudes, and also emotions relative to the given ambience. However, no specific recommendations exist yet which comprise particular descriptions of how to assess those specific sound effects. That is why there is a need to develop alternative methods of evaluation with whose help effects of acoustical ambiences can be better predicted. A method of evaluation will be presented which incorporates a new sensitive approach for the evaluation of vehicle sounds.

Formalizing Evidence Type Definitions for Drug-Drug Interaction Studies to Improve Evidence Base Curation.

PubMed

Utecht, Joseph; Brochhausen, Mathias; Judkins, John; Schneider, Jodi; Boyce, Richard D

2017-01-01

In this research we aim to demonstrate that an ontology-based system can categorize potential drug-drug interaction (PDDI) evidence items into complex types based on a small set of simple questions. Such a method could increase the transparency and reliability of PDDI evidence evaluation, while also reducing the variations in content and seriousness ratings present in PDDI knowledge bases. We extended the DIDEO ontology with 44 formal evidence type definitions. We then manually annotated the evidence types of 30 evidence items. We tested an RDF/OWL representation of answers to a small number of simple questions about each of these 30 evidence items and showed that automatic inference can determine the detailed evidence types based on this small number of simpler questions. These results show proof-of-concept for a decision support infrastructure that frees the evidence evaluator from mastering relatively complex written evidence type definitions.

Multi-topic assignment for exploratory navigation of consumer health information in NetWellness using formal concept analysis

PubMed Central

2014-01-01

Background Finding quality consumer health information online can effectively bring important public health benefits to the general population. It can empower people with timely and current knowledge for managing their health and promoting wellbeing. Despite a popular belief that search engines such as Google can solve all information access problems, recent studies show that using search engines and simple search terms is not sufficient. Our objective is to provide an approach to organizing consumer health information for navigational exploration, complementing keyword-based direct search. Multi-topic assignment to health information, such as online questions, is a fundamental step for navigational exploration. Methods We introduce a new multi-topic assignment method combining semantic annotation using UMLS concepts (CUIs) and Formal Concept Analysis (FCA). Each question was tagged with CUIs identified by MetaMap. The CUIs were filtered with term-frequency and a new term-strength index to construct a CUI-question context. The CUI-question context and a topic-subject context were used for multi-topic assignment, resulting in a topic-question context. The topic-question context was then directly used for constructing a prototype navigational exploration interface. Results Experimental evaluation was performed on the task of automatic multi-topic assignment of 99 predefined topics for about 60,000 consumer health questions from NetWellness. Using example-based metrics, suitable for multi-topic assignment problems, our method achieved a precision of 0.849, recall of 0.774, and F1 measure of 0.782, using a reference standard of 278 questions with manually assigned topics. Compared to NetWellness’ original topic assignment, a 36.5% increase in recall is achieved with virtually no sacrifice in precision. Conclusion Enhancing the recall of multi-topic assignment without sacrificing precision is a prerequisite for achieving the benefits of navigational exploration. Our new multi-topic assignment method, combining term-strength, FCA, and information retrieval techniques, significantly improved recall and performed well according to example-based metrics. PMID:25086916

Can Simply Answering Research Questions Change Behaviour? Systematic Review and Meta Analyses of Brief Alcohol Intervention Trials

PubMed Central

McCambridge, Jim; Kypri, Kypros

2011-01-01

Background Participant reports of their own behaviour are critical for the provision and evaluation of behavioural interventions. Recent developments in brief alcohol intervention trials provide an opportunity to evaluate longstanding concerns that answering questions on behaviour as part of research assessments may inadvertently influence it and produce bias. The study objective was to evaluate the size and nature of effects observed in randomized manipulations of the effects of answering questions on drinking behaviour in brief intervention trials. Methodology/Principal Findings Multiple methods were used to identify primary studies. Between-group differences in total weekly alcohol consumption, quantity per drinking day and AUDIT scores were evaluated in random effects meta-analyses. Ten trials were included in this review, of which two did not provide findings for quantitative study, in which three outcomes were evaluated. Between-group differences were of the magnitude of 13.7 (−0.17 to 27.6) grams of alcohol per week (approximately 1.5 U.K. units or 1 standard U.S. drink) and 1 point (0.1 to 1.9) in AUDIT score. There was no difference in quantity per drinking day. Conclusions/Significance Answering questions on drinking in brief intervention trials appears to alter subsequent self-reported behaviour. This potentially generates bias by exposing non-intervention control groups to an integral component of the intervention. The effects of brief alcohol interventions may thus have been consistently under-estimated. These findings are relevant to evaluations of any interventions to alter behaviours which involve participant self-report. PMID:21998626

Teaching and evaluating point of care learning with an Internet-based clinical-question portfolio.

PubMed

Green, Michael L; Reddy, Siddharta G; Holmboe, Eric

2009-01-01

Diplomates in the American Board of Internal Medicine (ABIM) Maintenance of Certification (MOC) program satisfy the self-evaluation of medical knowledge requirement by completing open-book multiple-choice exams. However, this method remains unlikely to affect practice change and often covers content areas not relevant to diplomates' practices. We developed and evaluated an Internet-based point of care (POC) learning portfolio to serve as an alternative. Participants enter information about their clinical questions, including characteristics, information pursuit, application, and practice change. After documenting 20 questions, they reflect upon a summary report and write commitment-to-change statements about their learning strategies. They can link to help screens and medical information resources. We report on the beta test evaluation of the module, completed by 23 internists and 4 internal medicine residents. Participants found the instructions clear and navigated the module without difficulty. The majority preferred the POC portfolio to multiple-choice examinations, citing greater relevance to their practice, guidance in expanding their palette of information resources, opportunity to reflect on their learning needs, and "credit" for self-directed learning related to their patients. Participants entered a total of 543 clinical questions, of which 250 (46%) resulted in a planned practice change. After completing the module, 14 of 27 (52%) participants committed to at least 1 change in their POC learning strategies. Internists found the portfolio valuable, preferred it to multiple-choice examinations, often changed their practice after pursuing clinical questions, and productively reflected on their learning strategies. The ABIM will offer this portfolio as an elective option in MOC.

Teaching cross-cultural communication skills online: a multi-method evaluation.

PubMed

Lee, Amy L; Mader, Emily M; Morley, Christopher P

2015-04-01

Cultural competency education is an important and required part of undergraduate medical education. The objective of this study was to evaluate whether an online cross-cultural communication module could increase student use of cross-cultural communication questions that assess the patient's definition of the problem, the way the problem affects their life, their concerns about the problem, and what the treatment should be (PACT). We used multi-method assessment of students assigned to family medicine clerkship blocks that were randomized to receive online cultural competency and PACT training added to their standard curriculum or to a control group receiving the standard curriculum only. Outcomes included comparison, via analysis of variance, of number of PACT questions used during an observed Standardized Patient Exercise, end-of-year OSCE scores, and qualitative analysis of student narratives. Students (n=119) who participated in the online module (n=60) demonstrated increased use of cross-cultural communication PACT questions compared to the control group (n=59) and generally had positive themes emerge from their reflective writing. The module had the biggest impact on students who later went on to match in high communication specialties. Online teaching of cross-cultural communication skills can be effective at changing medical student behavior.

Neuroleptic-induced movement disorders in a naturalistic schizophrenia population: diagnostic value of actometric movement patterns

PubMed Central

Janno, Sven; Holi, Matti M; Tuisku, Katinka; Wahlbeck, Kristian

2008-01-01

Background Neuroleptic-induced movement disorders (NIMDs) have overlapping co-morbidity. Earlier studies have described typical clinical movement patterns for individual NIMDs. This study aimed to identify specific movement patterns for each individual NIMD using actometry. Methods A naturalistic population of 99 schizophrenia inpatients using conventional antipsychotics and clozapine was evaluated. Subjects with NIMDs were categorized using the criteria for NIMD found in the Diagnostic and Statistical Manual for Mental Disorders – Fourth Edition (DSM-IV). Two blinded raters evaluated the actometric-controlled rest activity data for activity periods, rhythmical activity, frequencies, and highest acceleration peaks. A simple subjective question was formulated to test patient-based evaluation of NIMD. Results The patterns of neuroleptic-induced akathisia (NIA) and pseudoakathisia (PsA) were identifiable in actometry with excellent inter-rater reliability. The answers to the subjective question about troubles with movements distinguished NIA patients from other patients rather well. Also actometry had rather good screening performances in distinguishing akathisia from other NIMD. Actometry was not able to reliably detect patterns of neuroleptic-induced parkinsonism and tardive dyskinesia. Conclusion The present study showed that pooled NIA and PsA patients had a different pattern in lower limb descriptive actometry than other patients in a non-selected sample. Careful questioning of patients is a useful method of diagnosing NIA in a clinical setting. PMID:18419829

VET Program Completion Rates: An Evaluation of the Current Method. Occasional Paper

ERIC Educational Resources Information Center

National Centre for Vocational Education Research (NCVER), 2016

2016-01-01

This work asks one simple question: "how reliable is the method used by the National Centre for Vocational Education Research (NCVER) to estimate projected rates of VET program completion?" In other words, how well do early projections align with actual completion rates some years later? Completion rates are simple to calculate with a…

Do We See through Their Eyes? Testing a Bilingual Questionnaire in Education Research Using Cognitive Interviews

ERIC Educational Resources Information Center

Sopromadze, Natia; Moorosi, Pontso

2017-01-01

The paper aims to demonstrate the value of cognitive interviewing (CI) as a survey pretesting method in comparative education research. Although rarely used by education researchers, CI has been successfully applied in different disciplines to evaluate and improve question performance. The method assumes that observing people's thought processes…

Effect of low temperatures on osserous and cartilaginous tissues

NASA Technical Reports Server (NTRS)

Pankov, Y. Y.; Babiychuk, G. A.; Malyshkina, S. V.; Zhigun, A. I.

1983-01-01

The use of extreme cold to treat tumoral afflictions of the extremities is discussed. Cryogenic methods and instruments are discussed, and the levels of accumulated knowledge in this area (as well as the areas still in question) are evaluated. The overall promise for cryogenic methods of treatment is acknowledged, and areas which need further development are noted.

Pupil Researchers Generation X: Educating Pupils as Active Participants--An Investigation into Gathering Sensitive Information from Early Adolescents

ERIC Educational Resources Information Center

Symonds, Jenny E.

2008-01-01

Developmentally appropriate research techniques were uncovered by involving ten Year 7 pupils as researchers in a four-hour workshop that investigated the effectiveness of multiple methods in gathering sensitive information from early adolescents. The pupils learned about, tried and evaluated the methods of generating interview questions, peer and…

Determining the Number of Factors in P-Technique Factor Analysis

ERIC Educational Resources Information Center

Lo, Lawrence L.; Molenaar, Peter C. M.; Rovine, Michael

2017-01-01

Determining the number of factors is a critical first step in exploratory factor analysis. Although various criteria and methods for determining the number of factors have been evaluated in the usual between-subjects R-technique factor analysis, there is still question of how these methods perform in within-subjects P-technique factor analysis. A…

Prevention and self-management interventions are top priorities for osteoarthritis systematic reviews.

PubMed

Jaramillo, Alejandra; Welch, Vivian A; Ueffing, Erin; Gruen, Russell L; Bragge, Peter; Lyddiatt, Anne; Tugwell, Peter

2013-05-01

To identify high-priority research questions for osteoarthritis systematic reviews with consideration of health equity and the social determinants of health (SDH). We consulted with experts and conducted a literature search to identify a priority-setting method that could be adapted to address the health equity and SDH. We selected the Global Evidence Mapping priority-setting method, and through consultations and consensus, we adapted the method to meet our objectives. This involves developing an evidence map of the existing systematic reviews on osteoarthritis; conducting one face-to-face workshop with patients and another one with clinicians, researchers, and patients; and conducting an online survey of patients to rank the top 10 research questions. We piloted the adapted method with the Cochrane Musculoskeletal Review Group to set research priorities for osteoarthritis. Our focus was on systematic reviews: we identified 34 high-priority research questions for osteoarthritis systematic reviews. Prevention and self-management interventions, mainly diet and exercise, are top priorities for osteoarthritis systematic reviews. Evaluation against our predefined objectives showed that this method did prioritize SDH (50% of the research questions considered SDH). There were marked gaps: no high-priority topics were identified for access to care until patients had advanced disease-lifestyle changes once the disease was diagnosed. This method was felt feasible if conducted annually. We confirmed the utility of an adapted priority-setting method that is feasible and considers SDH. Further testing of this method is needed to assess whether considerations of health equity are prioritized and involve disadvantaged groups of the population. Copyright © 2013 Elsevier Inc. All rights reserved.

Evaluating the evaluation of cancer driver genes

PubMed Central

Tokheim, Collin J.; Papadopoulos, Nickolas; Kinzler, Kenneth W.; Vogelstein, Bert; Karchin, Rachel

2016-01-01

Sequencing has identified millions of somatic mutations in human cancers, but distinguishing cancer driver genes remains a major challenge. Numerous methods have been developed to identify driver genes, but evaluation of the performance of these methods is hindered by the lack of a gold standard, that is, bona fide driver gene mutations. Here, we establish an evaluation framework that can be applied to driver gene prediction methods. We used this framework to compare the performance of eight such methods. One of these methods, described here, incorporated a machine-learning–based ratiometric approach. We show that the driver genes predicted by each of the eight methods vary widely. Moreover, the P values reported by several of the methods were inconsistent with the uniform values expected, thus calling into question the assumptions that were used to generate them. Finally, we evaluated the potential effects of unexplained variability in mutation rates on false-positive driver gene predictions. Our analysis points to the strengths and weaknesses of each of the currently available methods and offers guidance for improving them in the future. PMID:27911828

Integrating Prevention into Obstetrics/Gynecology.

ERIC Educational Resources Information Center

Carey, J. Christopher

2000-01-01

Discusses formats to teach preventive medicine in obstetrics and gynecology (including learning objectives, lectures/seminars, and rounds/office practice) and evaluation methods (oral examinations, computerized question banks, objective structured clinical examinations). Offers examples from specific programs at American medical schools, including…

Advective and diffusive dermal processes for estimating terrestrial amphibian pesticide exposure

EPA Science Inventory

Background/Question/Methods Dermal exposure presents a potentially significant but understudied route for pesticide uptake in terrestrial amphibians. Historically, evaluation of pesticide risk to both amphibians and reptiles has been achieved by comparing ingestion and inhalat...

Evaluating Periodontal Treatment to Prevent Cardiovascular Disease: Challenges and Possible Solutions.

PubMed

Merchant, Anwar T; Virani, Salim S

2017-01-01

Periodontal disease is correlated with cardiovascular disease (CVD) in observational studies, but a causal connection has not been established. The empirical evidence linking periodontal disease and CVD consists of a large body of observational and mechanistic studies, but a limited number of clinical trials evaluating the effects of periodontal treatment on surrogate CVD endpoints. No randomized controlled trial has been conducted to evaluate the effect of periodontal treatment on CVD risk. In this review, we have summarized these data, described possible biological mechanisms linking periodontal disease and CVD, discussed barriers to conducting a randomized controlled trial to evaluate this hypothesis, and provided an alternative analytical approach using causal inference methods to answer the question. The public health implications of addressing this question can be significant because periodontal disease is under-treated, and highly prevalent among adults at risk of CVD. Even a small beneficial effect of periodontal treatment on CVD risk can be important.

Enhanced NDE systems

NASA Technical Reports Server (NTRS)

1993-01-01

The goal of this contractual effort was to evaluate the Langley narrow-band ultrasonic debond detection method for a factory use configuration. Successful accomplishment requires establishing the robustness of the method, and enhancing it if necessary. It is also desirable to strive for simplicity of implementation, such as attachment to in-place scanning devices planned for nondestructive evaluation (NDE) measurements. The contract was established with three phases for the ultrasonic work: (1) establish the method and robustness of the ultrasonic method; (2) follow up on any questions which arise with respect to the method or its implementation and produce a Phase A design; and (3) fabricate and test the Phase A design. This is a report on Phase 1.

Water-energy-food nexus: concepts, questions and methodologies

NASA Astrophysics Data System (ADS)

Li, Y.; Chen, X.; Ding, W.; Zhang, C.; Fu, G.

2017-12-01

The term of water-energy -food nexus has gained increasing attention in the research and policy making communities as the security of water, energy and food becomes severe under changing environment. Ignorance of their closely interlinkages accompanied by their availability and service may result in unforeseeable, adverse consequences. This paper comprehensively reviews the state-of-the-art in the field of water-energy-food, with a focus on concepts, research questions and methodologies. First, two types of nexus definition are compared and discussed to understand the essence of nexus research issues. Then, three kinds of nexus research questions are presented, including internal relationship analysis, external impact analysis, and evaluation of the nexus system. Five nexus modelling approaches are discussed in terms of their advantages, disadvantages and application, with an aim to identify research gaps in current nexus methods. Finally, future research areas and challenges are discussed, including system boundary, data uncertainty and modelling, underlying mechanism of nexus issues and system performance evaluation. This study helps bring research efforts together to address the challenging questions in the nexus and develop the consensus on building resilient water, energy and food systems.

Evaluating the impact of clinical librarians on clinical questions during inpatient rounds

PubMed Central

Brian, Riley; Orlov, Nicola; Werner, Debra; Martin, Shannon K.; Arora, Vineet M.; Alkureishi, Maria

2018-01-01

Objective The investigation sought to determine the effects of a clinical librarian (CL) on inpatient team clinical questioning quality and quantity, learner self-reported literature searching skills, and use of evidence-based medicine (EBM). Methods Clinical questioning was observed over 50 days of inpatient pediatric and internal medicine attending rounds. A CL was present for 25 days and absent for 25 days. Questioning was compared between groups. Question quality was assessed by a blinded evaluator, who used a rubric adapted from the Fresno Test of Competence in Evidence-Based Medicine. Team members were surveyed to assess perceived impacts of the CL on rounds. Results Rounds with a CL (CLR) were associated with significantly increased median number of questions asked (5 questions CLR vs. 3 NCLR; p<0.01) and answered (3 CLR vs. 2 NCLR; p<0.01) compared to rounds without a CL (NCLR). CLR were also associated with increased mean time spent asking (1.39 minutes CLR vs. 0.52 NCLR; p<0.01) and answering (2.15 minutes CLR vs. 1.05 NCLR; p=0.02) questions. Rounding time per patient was not significantly different between CLR and NCLR. Questions during CLR were 2 times higher in adapted Fresno Test quality than during NCLR (p<0.01). Select participants described how the CL’s presence improved their EBM skills and care decisions. Conclusions Inpatient CLR were associated with more and improved clinical questioning and subjectively perceived to improve clinicians’ EBM skills. CLs may directly affect patient care; further study is required to assess this. CLs on inpatient rounds may be an effective means for clinicians to learn and use EBM skills. PMID:29632440

Aquatic organism passage at road-stream crossings—synthesis and guidelines for effectiveness monitoring

USGS Publications Warehouse

Hoffman, Robert L.; Dunham, Jason B.; Hansen, Bruce P.

2012-01-01

Restoration and maintenance of passage for aquatic organisms at road-stream crossings represents a major management priority, involving an investment of hundreds of millions of dollars (for example, U.S. Government Accounting Office, 2001). In recent years, passage at hundreds of crossings has been restored, primarily by replacing barrier road culverts with bridges or stream simulation culverts designed to pass all species and all life stages of aquatic life and simulate natural hydro-geomorphic processes (U.S. Forest Service, 2008). The current situation has motivated two general questions: 1. Are current design standards for stream simulation culverts adequately re-establishing passage for aquatic biota? and 2. How do we monitor and evaluate effectiveness of passage restoration? To address the latter question, a national workshop was held in March 2010, in Portland, Oregon. The workshop included experts on aquatic organism passage from across the nation (see table of participants, APPENDIX) who addressed four classes of methods for monitoring effectiveness of aquatic organism passage—individual movement, occupancy, demography, and genetics. This report has been written, in part, for field biologists who will be undertaking and evaluating the effectiveness of aquatic organism passage restoration projects at road-stream crossings. The report outlines basic methods for evaluating road-stream crossing passage impairment and restoration and discusses under what circumstances and conditions each method will be useful; what questions each method can potentially answer; how to design and implement an evaluation study; and points out the fundamental reality that most evaluation projects will require special funding and partnerships among researchers and resource managers. The report is organized into the following sections, which can be read independently: 1. Historical context: In this section, we provide a brief history of events leading up to the present situation involving aquatic organism passage as a useful context for the issues covered herein. 2. Importance of connectivity for aquatic organisms: In this section, we provide background information regarding the movement characteristics of aquatic organisms and their vulnerability to passage impairment, and the importance of connectivity for a broad diversity of aquatic vertebrates and invertebrates. This section should be useful for practitioners in selecting what species to monitor in relation to aquatic organism passage. 3. Methods for evaluating aquatic organism passage: In this section, we present a range of perspectives on alternatives for assessing and monitoring aquatic organism passage impairment and the effectiveness of passage restoration actions, including the following methods: Individual Movement, Occupancy Models, Abundance (Demography), and Molecular Genetic Markers. 4. Relevance, strengths, and limitations of the four methods: In this section, we discuss the utility of each of the methods as a tool for assessing and quantifying passage impairment and restoration effectiveness. 5. Guidelines for selecting a method: In this section, we review some fundamental criteria and guidelines to consider when selecting a method for monitoring in the context of answering three important questions that should be addressed when developing a plan for evaluating aquatic organism passage. 6. Study and monitoring design considerations: In this section, we discuss four key design elements that need to be considered when developing a monitoring design for assessing passage impairment and restoration. The basic objectives of the report are to: 1. Review the movement characteristics of five groups of aquatic organisms that inhabit streams and to assess their general vulnerability to passage impairment at road-stream crossings; 2. Review four methods for monitoring aquatic organism passage impairment and the effectiveness of actions to restore passage at road-stream crossing structures; 3. Assess the relevance, strengths, and limitations of each method as a monitoring tool; 4. Identify and discuss guidelines that will be useful for selecting a monitoring method; and 5. Discuss what we have identified as the four key elements that need to be considered when developing a monitoring design for assessing passage impairment and restoration at road-stream crossings.

High Priority Research Needs for Gestational Diabetes Mellitus

PubMed Central

Robinson, Karen A.; Saldanha, Ian J.; Wilson, Lisa M.; Nicholson, Wanda K.

2012-01-01

Abstract Objective Identification of unanswered research questions about the management of gestational diabetes mellitus (GDM) is necessary to focus future research endeavors. We developed a process for elucidating the highest priority research questions on GDM. Methods Using a systematic review on GDM as a starting point, we developed an eight-step process: (1) identification of research gaps, (2) feedback from the review's authors, (3) translation of gaps into researchable questions using population, intervention, comparators, outcomes, setting (PICOS) framework, (4) local institutions' stakeholders' refinement of research questions, (5) national stakeholders' use of Delphi method to develop consensus on the importance of research questions, (6) prioritization of outcomes, (7) conceptual framework, and (8) evaluation. Results We identified 15 high priority research questions for GDM. The research questions focused on medication management of GDM (e.g., various oral agents vs. insulin), delivery management for women with GDM (e.g., induction vs. expectant management), and identification of risk factors for, prevention of, and screening for type 2 diabetes in women with prior GDM. Stakeholders rated the development of chronic diseases in offspring, cesarean delivery, and birth trauma as high priority outcomes to measure in future studies. Conclusions We developed an eight-step process using a multidisciplinary group of stakeholders to identify 15 research questions of high clinical importance. Researchers, policymakers, and funders can use this list to direct research efforts and resources to the highest priority areas to improve care for women with GDM. PMID:22747422

Evaluation Criteria for Interactive E-Books for Open and Distance Learning

ERIC Educational Resources Information Center

Bozkurt, Aras; Bozkaya, Mujgan

2015-01-01

The aim of this mixed method study is to identify evaluation criteria for interactive e-books. To find answers for the research questions of the study, both quantitative and qualitative data were collected through a four-round Delphi study with a panel consisting of 30 experts. After that, a total of 20 interactive e-books were examined with…

Adding to the Pool of Methods for Program Evaluation: A Comparison of Latent Class Analysis and Propensity Score Analysis

ERIC Educational Resources Information Center

Taylor, Lauren Christine

2013-01-01

Considering the amount of funding that is distributed to educational research each year, leaders and policymakers have a vested interest in finding scientifically based evidence that answers causal questions regarding program effectiveness. The importance of program evaluation has long been recognized in many fields of research; however, the most…

Outstanding Questions In First Amendment Law Related To Food Labeling Disclosure Requirements For Health.

PubMed

Pomeranz, Jennifer L

2015-11-01

The federal and state governments are increasingly focusing on food labeling as a method to support good health. Many such laws are opposed by the food industry and may be challenged in court, raising the question of what is legally feasible. This article analyzes outstanding questions in First Amendment law related to commercial disclosure requirements and conducts legal analysis and policy evaluation for three current policies. These include the Food and Drug Administration's draft regulation requiring an added sugar disclosure on the Nutrition Facts panel, California's proposed sugar-sweetened beverage safety warning label bill, and Vermont's law requiring labels of genetically engineered food to disclose this information. I recommend several methods for policy makers to enact food labeling laws within First Amendment parameters, including imposing factual commercial disclosure requirements, disclosing the government entity issuing a warning, collecting evidence, and identifying legitimate governmental interests. Project HOPE—The People-to-People Health Foundation, Inc.

CMU OAQA at TREC 2015 LiveQA: Discovering the Right Answer with Clues

DTIC Science & Technology

2015-11-20

QA) system that was evaluated in the TREC 2015 LiveQA Challenge. This system answers real-user questions freshly submitted to the Yahoo ! Answers...questions on the Yahoo ! Answers site 1, which have not yet received a human answer. As per the requirements for this track, participants must deploy their... Yahoo ! Answers. We also designed and im- plemented a new data model and novel relevance ranking methods for LiveQA. During the official run, our QA web

The Nursing Leadership Institute program evaluation: a critique

PubMed Central

Havaei, Farinaz; MacPhee, Maura

2015-01-01

A theory-driven program evaluation was conducted for a nursing leadership program, as a collaborative project between university faculty, the nurses’ union, the provincial Ministry of Health, and its chief nursing officers. A collaborative logic model process was used to engage stakeholders, and mixed methods approaches were used to answer evaluation questions. Despite demonstrated, successful outcomes, the leadership program was not supported with continued funding. This paper examines what happened during the evaluation process: What factors failed to sustain this program? PMID:29355180

Intrapartum ultrasound: A useful method for evaluating labor progress and predicting operative vaginal delivery

PubMed Central

Ahn, Ki Hoon

2014-01-01

The last step of a successful pregnancy is the safe delivery of the fetus. An important question is if the delivery should vaginal or operative. In addition to the use of conventional antenatal ultrasound, the use of intrapartum ultrasound to evaluate fetal head station, position, cervical ripening, and placental separation is promising. This review evaluates and summarizes the usefulness of intrapartum ultrasound for the evaluation of labor progress and predicting successful operative vaginal delivery. PMID:25469329

Critical thinking in physics education

NASA Astrophysics Data System (ADS)

Sadidi, Farahnaz

2016-07-01

We agree that training the next generation of leaders of the society, who have the ability to think critically and form a better judgment is an important goal. It is a long-standing concern of Educators and a long-term desire of teachers to establish a method in order to teach to think critically. To this end, many questions arise on three central aspects: the definition, the evaluation and the design of the course: What is Critical Thinking? How can we define Critical Thinking? How can we evaluate Critical Thinking? Therefore, we want to implement Critical Thinking in physics education. How can we teach for Critical Thinking in physics? What should the course syllabus and materials be? We present examples from classical physics and give perspectives for astro-particle physics. The main aim of this paper is to answer the questions and provide teachers with the opportunity to change their classroom to an active one, in which students are encouraged to ask questions and learn to reach a good judgment. Key words: Critical Thinking, evaluation, judgment, design of the course.

Flight Force Measurements on a Spacecraft to Launch Vehicle Interface

NASA Astrophysics Data System (ADS)

Kaufman, Daniel S.; Gordon, Scott A.

2012-07-01

For several years we had wanted to measure interface forces between a launch vehicle and the Payload. Finally in July 2006 a proposal was made and funded to evaluate the use of flight force measurements (FFM) to improve the loads process of a Spacecraft in its design and test cycle. A NASA/Industry team was formed, the core Team consisted of 20 people. The proposal identified two questions that this assessment would attempt to address by obtaining the flight forces. These questions were: 1) Is flight correlation and reconstruction with acceleration methods sufficient? 2) How much can the loads and therefore the design and qualification be reduced by having force measurements? The objective was to predict the six interface driving forces between the Spacecraft and the Launch Vehicle throughout the boost phase. Then these forces would be compared with reconstructed loads analyses for evaluation in an attempt to answer them. The paper will present the development of a strain based force measurement system and also an acceleration method, actual flight results, post flight evaluations and lessons learned.

Development and validation of a questionnaire to evaluate infection control in oral radiology

PubMed Central

Pinelli, Camila; da Silva Tagliaferro, Elaine P; Corrente, José E; Ambrosano, Glaucia M B

2017-01-01

Objectives: To create and validate a questionnaire to evaluate infection control in oral radiology. Methods: The questionnaire was developed after review of the literature, which included published articles and the biosafety protocols available from healthcare agencies. The initial version of the questionnaire was composed of 14 multiple choice questions and was divided into 3 domains on handwashing, disinfection/protection of surfaces and disinfectant used. Content validity was assessed by two expert committees, which reviewed the content and scope of the questionnaire and the relevance of each item, respectively. Reliability was evaluated using test–retest and internal consistency methods with 115 undergraduate dentistry students. Construct validity was assessed using the known-groups technique and factor analysis. The known-groups technique involved 641 undergraduate dentistry students, 20 PhD students and 15 oral radiology professors. In the factor analysis, 3 radiology technicians also participated in addition to the 641 undergraduates, 20 PhD students and 15 oral radiology professors. Results: The content validity results were found to be satisfactory to excellent for the ordinal variables (intraclass correlation coefficient = 0.722–1.000) and good to great for the yes/no questions (kappa = 0.662–0.913) in terms of reliability and good internal consistency (Cronbach's alpha = 0.88). After a factor analysis, some questions were excluded, and the questions were grouped into new domains. Significant differences were observed between answers from different groups. The final version of the questionnaire was composed of nine domains. Conclusions: The questionnaire created was found to exhibit good psychometric properties for assessing infection control in oral radiology. PMID:28112553

A software technology evaluation program

NASA Technical Reports Server (NTRS)

Novaes-Card, David N.

1985-01-01

A set of quantitative approaches is presented for evaluating software development methods and tools. The basic idea is to generate a set of goals which are refined into quantifiable questions which specify metrics to be collected on the software development and maintenance process and product. These metrics can be used to characterize, evaluate, predict, and motivate. They can be used in an active as well as passive way by learning form analyzing the data and improving the methods and tools based upon what is learned from that analysis. Several examples were given representing each of the different approaches to evaluation. The cost of the approaches varied inversely with the level of confidence in the interpretation of the results.

Business Process-Based Resource Importance Determination

NASA Astrophysics Data System (ADS)

Fenz, Stefan; Ekelhart, Andreas; Neubauer, Thomas

Information security risk management (ISRM) heavily depends on realistic impact values representing the resources’ importance in the overall organizational context. Although a variety of ISRM approaches have been proposed, well-founded methods that provide an answer to the following question are still missing: How can business processes be used to determine resources’ importance in the overall organizational context? We answer this question by measuring the actual importance level of resources based on business processes. Therefore, this paper presents our novel business process-based resource importance determination method which provides ISRM with an efficient and powerful tool for deriving realistic resource importance figures solely from existing business processes. The conducted evaluation has shown that the calculation results of the developed method comply to the results gained in traditional workshop-based assessments.

Impacts of Socratic questioning on moral reasoning of nursing students.

PubMed

Torabizadeh, Camellia; Homayuni, Leyla; Moattari, Marzieh

2018-03-01

Nurses are often faced with complex situations that made them to make ethical decisions; and to make such decisions, they need to possess the power of moral reasoning. Studies in Iran show that the majority of nursing students lack proper ethical development. Socratic teaching is a student-centered method which is strongly opposed to the lecturing method. This study was conducted to evaluate the impacts of Socratic questioning on the moral reasoning of the nursing students. In a quasi-experimental study, Crisham's Nursing Dilemma Test was used to evaluate the results of three groups before, immediately after, and 2 months after intervention. The data were analyzed using the SPSS statistical software (v 15). Participants and research context: Through random allocation, 103 nursing students were divided into three groups. In experiment group 1 (37 students), intervention consisted of Socratic questioning-based sessions on ethics and how to deal with moral dilemmas; experiment group 2 (33 students) attended a 4-h workshop; and the control group (33 students) was not subject to any interventions. Signed informed consent forms: This research was approved by the Ethics Committee of the University. All the participants signed written informed consents. There were significant differences between experiment group 1 and experiment group 2's pre-test and post-test scores on moral reasoning (p ≤ 0.001, p ≤ 0.001), nursing principled thinking (p ≤ 0.001, p ≤ 0.001), and practical considerations (p ≤ 0.001, p ≤ 0.031). Both the teaching approaches improved the subjects' moral reasoning; however, Socratic questioning proved more effective than lecturing. Compared to other similar studies in Iran and other countries, the students had inadequate moral reasoning competence. This study confirms the need for the development of an efficient course on ethics in the nursing curriculum. Also, it appears that Socratic questioning is an effective method to teach nursing ethics and develop nursing students' competence of moral reasoning.

An Empirical Comparison of Five Linear Equating Methods for the NEAT Design

ERIC Educational Resources Information Center

Suh, Youngsuk; Mroch, Andrew A.; Kane, Michael T.; Ripkey, Douglas R.

2009-01-01

In this study, a data base containing the responses of 40,000 candidates to 90 multiple-choice questions was used to mimic data sets for 50-item tests under the "nonequivalent groups with anchor test" (NEAT) design. Using these smaller data sets, we evaluated the performance of five linear equating methods for the NEAT design with five levels of…

[Acceptance and understandability of various methods of health valuations for the chronically ill: willingness to pay, visual analogue scale and rating scale].

PubMed

Meder, M; Farin, E

2009-11-01

Health valuations are one way of measuring patient preferences with respect to the results of their treatment. The study examines three different methods of health valuations--willingness to pay (WTP), visual analogue scale (VAS), and a rating question for evaluating the subjective significance. The goal is to test the understandability and acceptance of these methods for implementation in questionnaires. In various rehabilitation centres, a total of six focus groups were conducted with 5-9 patients each with a mean age of 57.1 years. The illnesses considered were chronic-ischaemic heart disease, chronic back pain, and breast cancer. Patients filled out a questionnaire that was then discussed in the group. In addition to the quantitative evaluation of the data in the questionnaire, a qualitative analysis of the contents of the group discussion protocols was made. We have results from a total of 42 patients. 14.6% of the patients had "great difficulties" understanding the WTP or rated it as "completely incomprehensible"; this value was 7.3% for VAS and 0% for the rating scale. With respect to acceptance, 31.0% of the patients indicated that they were "not really" or "not at all" willing to answer such a WTP question in a questionnaire; this was 6.6% for the VAS, and again 0% for the rating scale. The qualitative analysis provided an indication as to why some patients view the WTP question in particular in a negative light. Many difficulties in understanding it were related to the formulation of the question and the structure of the questionnaire. However, the patients' statements also made it apparent that the hypothetical nature of the WTP questionnaire was not always recognised. The most frequent reason for the lack of acceptance of the WTP was the patients' fear of negative financial consequences of their responses. With respect to understandability and acceptance, VAS questions appear to be better suited for reflecting patient preferences than WTP questions. The rating scale for assessing the personally estimated importance was understandable, but had poor distribution properties and was considered by patients to be superfluous.

Combining Open-domain and Biomedical Knowledge for Topic Recognition in Consumer Health Questions.

PubMed

Mrabet, Yassine; Kilicoglu, Halil; Roberts, Kirk; Demner-Fushman, Dina

2016-01-01

Determining the main topics in consumer health questions is a crucial step in their processing as it allows narrowing the search space to a specific semantic context. In this paper we propose a topic recognition approach based on biomedical and open-domain knowledge bases. In the first step of our method, we recognize named entities in consumer health questions using an unsupervised method that relies on a biomedical knowledge base, UMLS, and an open-domain knowledge base, DBpedia. In the next step, we cast topic recognition as a binary classification problem of deciding whether a named entity is the question topic or not. We evaluated our approach on a dataset from the National Library of Medicine (NLM), introduced in this paper, and another from the Genetic and Rare Disease Information Center (GARD). The combination of knowledge bases outperformed the results obtained by individual knowledge bases by up to 16.5% F1 and achieved state-of-the-art performance. Our results demonstrate that combining open-domain knowledge bases with biomedical knowledge bases can lead to a substantial improvement in understanding user-generated health content.

A Very Different Non-Stressful Comprehensive Final Exam that Achieve Our Goals for Student Evaluation and Learning

NASA Astrophysics Data System (ADS)

Bhavsar, Suketu

2015-08-01

I will introduce the radical concept of a final exam where the questions are given beforehand, a method I first encountered as a graduate student at Princeton University from an outstanding and well known astrophysicist and exceptional teacher, Lyman Spitzer.Every Instructor aspires for students to master all the material covered. A comprehensive final can assess the breadth and depth of their learning. Students are required to review early material in light of later topics, create connections and integrate understanding, thus retaining knowledge for the long term. Comprehensive finals can therefore be a significant basis for student learning and evaluation, but are especially daunting for non-STEM majors in required GE synthesis STEM classes. The exam format proposed here calmed student fears and encouraged thorough review.Ten days before the exam students received 20-30 challenging, well-crafted, numbered questions that interconnected and spanned the entire range of topics. The key is crafting questions that lead to deeply understanding the subject matter and mastering skills to solve problems. At the final, each student was required to pick a number out of a hat and answer that numbered question in a 5-minute presentation. They also had to critically comment on 10 other presentations of their peers. They are graded equally on both.The exam sets up definite goals for a student. Equally important, it enhances collaborative learning and peer mentoring. The conceptual questions and problems that students are required to answer can be studied together in study groups. The final presentation is theirs and they are not only encouraged but required to be constructively critical of their peer presentations.I will provide examples of some of the conceptual and problem solving questions I used. These were crafted to interconnect and span the entire range of topics. This method requires students to be prepared for all of the multitude of crafted question encouraging interaction and communication while studying. Knowing the questions beforehand provides a guide to their studying as well as allays their fears about what could be asked. The students also receive guidance to what constitutes a good answer, namely accuracy, thoroughness and the quality of the presentation.

The financial crisis and the well-being of Americans

PubMed Central

Deaton, Angus

2011-01-01

The Great Recession was associated with large changes in income, wealth, and unemployment, changes that affected many lives. Since January 2008, the Gallup Organization has been collecting daily data on 1,000 Americans each day, with a range of self-reported well-being (SWB) questions. I use these data to examine how the recession affected the emotional and evaluative lives of the population, as well as of subgroups within it. In the fall of 2008, around the time of the collapse of Lehman Brothers, and lasting into the spring of 2009, at the bottom of the stock market, Americans reported sharp declines in their life evaluation, sharp increases in worry and stress, and declines in positive affect. By the end of 2010, in spite of continuing high unemployment, these measures had largely recovered, though worry remained higher and life evaluation lower than in January 2008. The SWB measures do a much better job of monitoring short-run levels of anxiety as the crisis unfolded than they do of reflecting the evolution of the economy over a year or two. Even large macroeconomic shocks to income and unemployment can be expected to produce only small and hard to detect effects on SWB measures. SWB, particularly evaluation of life as a whole, is sensitive to question order effects. Asking political questions before the life evaluation question reduces reported life evaluation by an amount that dwarfs the effects of even the worst of the crisis; these order effects persist deep into the interview, and condition the reporting of hedonic experience and of satisfaction with standard of living. Methods for controlling these effects need to be developed and tested if national measures are to be comparable over space and time. PMID:22389532

The financial crisis and the well-being of Americans.

PubMed

Deaton, Angus

2012-01-01

The Great Recession was associated with large changes in income, wealth, and unemployment, changes that affected many lives. Since January 2008, the Gallup Organization has been collecting daily data on 1,000 Americans each day, with a range of self-reported well-being (SWB) questions. I use these data to examine how the recession affected the emotional and evaluative lives of the population, as well as of subgroups within it. In the fall of 2008, around the time of the collapse of Lehman Brothers, and lasting into the spring of 2009, at the bottom of the stock market, Americans reported sharp declines in their life evaluation, sharp increases in worry and stress, and declines in positive affect. By the end of 2010, in spite of continuing high unemployment, these measures had largely recovered, though worry remained higher and life evaluation lower than in January 2008. The SWB measures do a much better job of monitoring short-run levels of anxiety as the crisis unfolded than they do of reflecting the evolution of the economy over a year or two. Even large macroeconomic shocks to income and unemployment can be expected to produce only small and hard to detect effects on SWB measures. SWB, particularly evaluation of life as a whole, is sensitive to question order effects. Asking political questions before the life evaluation question reduces reported life evaluation by an amount that dwarfs the effects of even the worst of the crisis; these order effects persist deep into the interview, and condition the reporting of hedonic experience and of satisfaction with standard of living. Methods for controlling these effects need to be developed and tested if national measures are to be comparable over space and time.

Sensitivity analysis for simulating pesticide impacts on honey bee colonies

EPA Science Inventory

Background/Question/Methods Regulatory agencies assess risks to honey bees from pesticides through a tiered process that includes predictive modeling with empirical toxicity and chemical data of pesticides as a line of evidence. We evaluate the Varroapop colony model, proposed by...

Effect of Michigan multi-axle trucks on pavement distress and profile: volume 2.

DOT National Transportation Integrated Search

2009-02-01

With the adoption of the new mechanistic-empirical pavement design method and the employment of axle load spectra, the question of evaluating the pavement damage resulting from different axle and truck configurations has become more relevant. In part...

Open Education: How to Get There If You Want to Go

ERIC Educational Resources Information Center

Flurry, Ruth C.

1973-01-01

Cautions against rush to import British informal-style classrooms asserting that open education can't be brought about by mandate, questionable motivations, or by a quick, band-aid approach. More long-range planning and better evaluation methods are urged. (ST)

Effect of Michigan multi-axle trucks on pavement distress and profile: volume 3.

DOT National Transportation Integrated Search

2009-02-01

With the adoption of the new mechanistic-empirical pavement design method and the employment of axle load : spectra, the question of evaluating the pavement damage resulting from different axle and truck configurations has : become more relevant. In ...

Effect of Michigan multi-axle trucks on pavement distress and profile : volume 1.

DOT National Transportation Integrated Search

2009-02-01

With the adoption of the new mechanistic-empirical pavement design method and the employment of axle load : spectra, the question of evaluating the pavement damage resulting from different axle and truck configurations has : become more relevant. In ...

Pre-Service Teachers' Evaluations of Practices in Teaching English to Young Learners in Terms of 21st Century Teacher Qualifications

ERIC Educational Resources Information Center

Çakir, Abdulvahit; Güngör, Müzeyyen Nazli

2017-01-01

This study investigates the 3rd year and 4th year pre-service teachers' evaluations of the practices in teaching English to young learners in terms of 21st century teacher qualifications. Employing a mixed methods design, this study includes the qualitative data that come from semi-structured interviewed questions administered to pre-service…

Practical Use of Computationally Frugal Model Analysis Methods

DOE PAGES

Hill, Mary C.; Kavetski, Dmitri; Clark, Martyn; ...

2015-03-21

Computationally frugal methods of model analysis can provide substantial benefits when developing models of groundwater and other environmental systems. Model analysis includes ways to evaluate model adequacy and to perform sensitivity and uncertainty analysis. Frugal methods typically require 10s of parallelizable model runs; their convenience allows for other uses of the computational effort. We suggest that model analysis be posed as a set of questions used to organize methods that range from frugal to expensive (requiring 10,000 model runs or more). This encourages focus on method utility, even when methods have starkly different theoretical backgrounds. We note that many frugalmore » methods are more useful when unrealistic process-model nonlinearities are reduced. Inexpensive diagnostics are identified for determining when frugal methods are advantageous. Examples from the literature are used to demonstrate local methods and the diagnostics. We suggest that the greater use of computationally frugal model analysis methods would allow questions such as those posed in this work to be addressed more routinely, allowing the environmental sciences community to obtain greater scientific insight from the many ongoing and future modeling efforts« less

Quality of data in multiethnic health surveys.

PubMed Central

Pasick, R. J.; Stewart, S. L.; Bird, J. A.; D'Onofrio, C. N.

2001-01-01

OBJECTIVE: There has been insufficient research on the influence of ethno-cultural and language differences in public health surveys. Using data from three independent studies, the authors examine methods to assess data quality and to identify causes of problematic survey questions. METHODS: Qualitative and quantitative methods were used in this exploratory study, including secondary analyses of data from three baseline surveys (conducted in English, Spanish, Cantonese, Mandarin, and Vietnamese). Collection of additional data included interviews with investigators and interviewers; observations of item development; focus groups; think-aloud interviews; a test-retest assessment survey; and a pilot test of alternatively worded questions. RESULTS: The authors identify underlying causes for the 12 most problematic variables in three multiethnic surveys and describe them in terms of ethnic differences in reliability, validity, and cognitive processes (interpretation, memory retrieval, judgment formation, and response editing), and differences with regard to cultural appropriateness and translation problems. CONCLUSIONS: Multiple complex elements affect measurement in a multiethnic survey, many of which are neither readily observed nor understood through standard tests of data quality. Multiethnic survey questions are best evaluated using a variety of quantitative and qualitative methods that reveal different types and causes of problems. PMID:11889288

Fort Collins Science Center: Policy Analysis and Science Assistance

USGS Publications Warehouse

Lamb, Berton L.

2004-01-01

PASA's mission is to integrate biological, social, and economic research so that resource managers can use the resulting information to make informed decisions and resolve resource management conflicts. PASA scientists pursue and conduct scientific analyses that help agencies and Native American tribes to (1) identify impending policy controversies and areas where social and natural science research is needed to address future policy questions; (2) develop methods and approaches to assist researchers in preparing scientific evidence; (3) assess habitat alteration in a manner consistent with policy needs; and (4) evaluate policy options. Branch scientists also evaluate policy options (e.g., effects of different land treatments, fish and wildlife management practices, or visitor/recreation management practices) in response to specific questions faced by policymakers and managers.

Applied information system-based in enhancing students' understanding towards higher order thinking (HOTS)

NASA Astrophysics Data System (ADS)

Hua, Ang Kean; Ping, Owi Wei

2017-05-01

The application of information and communications technology (ICT) had become more important in our daily life, especially in educational field. Teachers are encouraged to use information system-based in teaching Mathematical courses. Higher Order Thinking Skills (HOTS) approach is unable to explain using chalk and talk methods. It needs students to analyze, evaluate, and create by their own natural abilities. The aim of this research study was to evaluate the effectiveness of the application information system-based in enhance the students understanding about HOTS question. Mixed-methods or quantitative and qualitative approach was applied in collecting data, which involve only the standard five students and the teachers in Sabak Bernam, Selangor. Pra-postests was held before and after using information system-based in teaching to evaluate the students' understanding. The result from post-test indicates significant improvement which proves that the use of information system based able to enhance students' understanding about HOTS question and solve it. There were several factor influenced the students such as students' attitude, teachers attraction, school facilities, and computer approach. Teachers play an important role in attracting students to learn. Therefore, the school should provide a conducive learning environment and good facilities for students to learn so that they are able to access more information and always exposed to new knowledge. As conclusion, information system-based are able to enhance students understanding the need of HOTS questions and solve it.

Physicians’ attitudes toward pharmacogenetic testing before and after pharmacogenetic education

PubMed Central

Luzum, Jasmine A; Luzum, Matthew J

2016-01-01

Aim: Our aim was to evaluate physicians’ attitudes toward pharmacogenetic testing before and after pharmacogenetic education. Methods: In total, 12 physicians (˜40% response rate) completed a survey with eight questions on 10-point scales on their attitudes toward pharmacogenetic testing before and after a 1-h grand rounds presentation on pharmacogenetics. Differences in question scores overall, among training levels (resident/fellow/attending), and specific drugs (clopidogrel/simvastatin/warfarin) were assessed using Wilcoxon signed-rank and exact Kruskal–Wallis tests. Results & conclusion: The scores for all eight questions increased, with statistically significant (p < 0.05) increases for four out of eight questions. The scores were similar among training levels, but the postscores for clopidogrel were significantly higher than for simvastatin and warfarin. In conclusion, brief pharmacogenetic education can significantly affect physicians’ attitudes toward pharmacogenetic testing. PMID:29749904

Evaluating Differential Effects Using Regression Interactions and Regression Mixture Models

ERIC Educational Resources Information Center

Van Horn, M. Lee; Jaki, Thomas; Masyn, Katherine; Howe, George; Feaster, Daniel J.; Lamont, Andrea E.; George, Melissa R. W.; Kim, Minjung

2015-01-01

Research increasingly emphasizes understanding differential effects. This article focuses on understanding regression mixture models, which are relatively new statistical methods for assessing differential effects by comparing results to using an interactive term in linear regression. The research questions which each model answers, their…

Assessing Creative Problem-Solving with Automated Text Grading

ERIC Educational Resources Information Center

Wang, Hao-Chuan; Chang, Chun-Yen; Li, Tsai-Yen

2008-01-01

The work aims to improve the assessment of creative problem-solving in science education by employing language technologies and computational-statistical machine learning methods to grade students' natural language responses automatically. To evaluate constructs like creative problem-solving with validity, open-ended questions that elicit…

Questionnaire design and the recall of pharmacological treatments: a systematic review.

PubMed

Gama, Helena; Correia, Sofia; Lunet, Nuno

2009-03-01

We aimed to review systematically the published evidence regarding the effect of questionnaire design on the recall of pharmacological treatments. The electronic databases Pubmed, EMBASE, and Cochrane Library were searched from inception to October 2007, using the following search terms: drug utilization, pharmaceutical preparations, pharmacoepidemiology, validation studies, methods, epidemiologic methods, interviews, data collection, and questionnaires. Drug utilization studies comparing different types of questionnaire or methods of questionnaire administration were included. Backward and forward citation tracking were also conducted. Eight studies were included in the systematic review, comparing questions asking for specific drugs or indications with open-ended questions (n = 5), evaluating the use of memory aids (n = 1), or studying the influence of response order on recall (n = 2). The studies were heterogeneous, namely regarding the populations evaluated (e.g., pregnant women, hypertensive patients, general population), mode of questionnaire administration (e.g., personal or telephone interview, self-administered), recall period (e.g., current use, 1 week, previous episode of a disease), or drugs evaluated (e.g., analgesics, antimalarials, all medicines). Despite the lack of standardization in presentation of results, the prevalence of drug use may vary between 5 and 40% when drug names and indications or pictures are used as memory aids, or as a result of primacy effects in self-administered questionnaires. The yielding of the questionnaires depended on the pharmacological groups evaluated. Scientific work regarding methods for drug utilization data collection is scarce. The available evidence highlights the importance of knowing the questionnaire characteristics for a proper interpretation of results from drug utilization studies. (c) 2009 John Wiley & Sons, Ltd.

An educational game for teaching clinical practice guidelines to Internal Medicine residents: development, feasibility and acceptability

PubMed Central

Akl, Elie A; Mustafa, Reem; Slomka, Thomas; Alawneh, Alia; Vedavalli, Abhishek; Schünemann, Holger J

2008-01-01

Background Adherence to Clinical Practice Guidelines (CPGs) remains suboptimal among internal medicine trainees. Educational games are of growing interest and have the potential to improve adherence to CPGs. The objectives of this study were to develop an educational game to teach CPGs in Internal Medicine residency programs and to evaluate its feasibility and acceptability. Methods We developed the Guide-O-Game© in the format of a TV game show with questions based on recommendations of CPGs. The development of the Guide-O-Game© consisted of the creation of a multimedia interactive tool, the development of recommendation-based questions, and the definition of the game's rules. We evaluated its feasibility through pilot testing and its acceptability through a qualitative process. Results The multimedia interactive tool uses a Macromedia Flash web application and consists of a manager interface and a user interface. The user interface allows the choice of two game styles. We created so far 16 sets of questions relating to 9 CPGs. The pilot testing proved that the game was feasible. The qualitative evaluation showed that residents considered the game to be acceptable. Conclusion We developed an educational game to teach CPGs to Internal Medicine residents that is both feasible and acceptable. Future work should evaluate its impact on educational outcomes. PMID:19017400

Exploiting the systematic review protocol for classification of medical abstracts.

PubMed

Frunza, Oana; Inkpen, Diana; Matwin, Stan; Klement, William; O'Blenis, Peter

2011-01-01

To determine whether the automatic classification of documents can be useful in systematic reviews on medical topics, and specifically if the performance of the automatic classification can be enhanced by using the particular protocol of questions employed by the human reviewers to create multiple classifiers. The test collection is the data used in large-scale systematic review on the topic of the dissemination strategy of health care services for elderly people. From a group of 47,274 abstracts marked by human reviewers to be included in or excluded from further screening, we randomly selected 20,000 as a training set, with the remaining 27,274 becoming a separate test set. As a machine learning algorithm we used complement naïve Bayes. We tested both a global classification method, where a single classifier is trained on instances of abstracts and their classification (i.e., included or excluded), and a novel per-question classification method that trains multiple classifiers for each abstract, exploiting the specific protocol (questions) of the systematic review. For the per-question method we tested four ways of combining the results of the classifiers trained for the individual questions. As evaluation measures, we calculated precision and recall for several settings of the two methods. It is most important not to exclude any relevant documents (i.e., to attain high recall for the class of interest) but also desirable to exclude most of the non-relevant documents (i.e., to attain high precision on the class of interest) in order to reduce human workload. For the global method, the highest recall was 67.8% and the highest precision was 37.9%. For the per-question method, the highest recall was 99.2%, and the highest precision was 63%. The human-machine workflow proposed in this paper achieved a recall value of 99.6%, and a precision value of 17.8%. The per-question method that combines classifiers following the specific protocol of the review leads to better results than the global method in terms of recall. Because neither method is efficient enough to classify abstracts reliably by itself, the technology should be applied in a semi-automatic way, with a human expert still involved. When the workflow includes one human expert and the trained automatic classifier, recall improves to an acceptable level, showing that automatic classification techniques can reduce the human workload in the process of building a systematic review. Copyright © 2010 Elsevier B.V. All rights reserved.

An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition.

PubMed

Tsatsaronis, George; Balikas, Georgios; Malakasiotis, Prodromos; Partalas, Ioannis; Zschunke, Matthias; Alvers, Michael R; Weissenborn, Dirk; Krithara, Anastasia; Petridis, Sergios; Polychronopoulos, Dimitris; Almirantis, Yannis; Pavlopoulos, John; Baskiotis, Nicolas; Gallinari, Patrick; Artiéres, Thierry; Ngomo, Axel-Cyrille Ngonga; Heino, Norman; Gaussier, Eric; Barrio-Alvers, Liliana; Schroeder, Michael; Androutsopoulos, Ion; Paliouras, Georgios

2015-04-30

This article provides an overview of the first BIOASQ challenge, a competition on large-scale biomedical semantic indexing and question answering (QA), which took place between March and September 2013. BIOASQ assesses the ability of systems to semantically index very large numbers of biomedical scientific articles, and to return concise and user-understandable answers to given natural language questions by combining information from biomedical articles and ontologies. The 2013 BIOASQ competition comprised two tasks, Task 1a and Task 1b. In Task 1a participants were asked to automatically annotate new PUBMED documents with MESH headings. Twelve teams participated in Task 1a, with a total of 46 system runs submitted, and one of the teams performing consistently better than the MTI indexer used by NLM to suggest MESH headings to curators. Task 1b used benchmark datasets containing 29 development and 282 test English questions, along with gold standard (reference) answers, prepared by a team of biomedical experts from around Europe and participants had to automatically produce answers. Three teams participated in Task 1b, with 11 system runs. The BIOASQ infrastructure, including benchmark datasets, evaluation mechanisms, and the results of the participants and baseline methods, is publicly available. A publicly available evaluation infrastructure for biomedical semantic indexing and QA has been developed, which includes benchmark datasets, and can be used to evaluate systems that: assign MESH headings to published articles or to English questions; retrieve relevant RDF triples from ontologies, relevant articles and snippets from PUBMED Central; produce "exact" and paragraph-sized "ideal" answers (summaries). The results of the systems that participated in the 2013 BIOASQ competition are promising. In Task 1a one of the systems performed consistently better from the NLM's MTI indexer. In Task 1b the systems received high scores in the manual evaluation of the "ideal" answers; hence, they produced high quality summaries as answers. Overall, BIOASQ helped obtain a unified view of how techniques from text classification, semantic indexing, document and passage retrieval, question answering, and text summarization can be combined to allow biomedical experts to obtain concise, user-understandable answers to questions reflecting their real information needs.

Multi-topic assignment for exploratory navigation of consumer health information in NetWellness using formal concept analysis.

PubMed

Cui, Licong; Xu, Rong; Luo, Zhihui; Wentz, Susan; Scarberry, Kyle; Zhang, Guo-Qiang

2014-08-03

Finding quality consumer health information online can effectively bring important public health benefits to the general population. It can empower people with timely and current knowledge for managing their health and promoting wellbeing. Despite a popular belief that search engines such as Google can solve all information access problems, recent studies show that using search engines and simple search terms is not sufficient. Our objective is to provide an approach to organizing consumer health information for navigational exploration, complementing keyword-based direct search. Multi-topic assignment to health information, such as online questions, is a fundamental step for navigational exploration. We introduce a new multi-topic assignment method combining semantic annotation using UMLS concepts (CUIs) and Formal Concept Analysis (FCA). Each question was tagged with CUIs identified by MetaMap. The CUIs were filtered with term-frequency and a new term-strength index to construct a CUI-question context. The CUI-question context and a topic-subject context were used for multi-topic assignment, resulting in a topic-question context. The topic-question context was then directly used for constructing a prototype navigational exploration interface. Experimental evaluation was performed on the task of automatic multi-topic assignment of 99 predefined topics for about 60,000 consumer health questions from NetWellness. Using example-based metrics, suitable for multi-topic assignment problems, our method achieved a precision of 0.849, recall of 0.774, and F₁ measure of 0.782, using a reference standard of 278 questions with manually assigned topics. Compared to NetWellness' original topic assignment, a 36.5% increase in recall is achieved with virtually no sacrifice in precision. Enhancing the recall of multi-topic assignment without sacrificing precision is a prerequisite for achieving the benefits of navigational exploration. Our new multi-topic assignment method, combining term-strength, FCA, and information retrieval techniques, significantly improved recall and performed well according to example-based metrics.

Working towards consensus on methods used to elicit participant-reported safety data in uncomplicated malaria clinical drug studies: a Delphi technique study.

PubMed

Mandimika, Nyaradzo; Barnes, Karen I; Chandler, Clare I R; Pace, Cheryl; Allen, Elizabeth N

2017-01-28

Eliciting adverse event (AE) and non-study medication data reports from clinical research participants is integral to evaluating drug safety. However, using different methods to question participants yields inconsistent results, compromising the interpretation, comparison and pooling of data across studies. This is particularly important given the widespread use of anti-malarials in vulnerable populations, and their increasing use in healthy, but at-risk individuals, as preventive treatment or to reduce malaria transmission. Experienced and knowledgeable anti-malarial drug clinical researchers were invited to participate in a Delphi technique study, to facilitate consensus on what are considered optimal (relevant, important and feasible) methods, tools, and approaches for detecting participant-reported AE and non-study medication data in uncomplicated malaria treatment studies. Of 72 invited, 25, 16 and 10 panellists responded to the first, second and third rounds of the Delphi, respectively. Overall, 68% (68/100) of all questioning items presented for rating achieved consensus. When asking general questions about health, panellists agreed on the utility of a question/concept about any change in health, taking care to ensure that such questions/concepts do not imply causality. Eighty-nine percent (39/44) of specific signs and symptoms questions were rated as optimal. For non-study medications, a general question and most structured questioning items were considered an optimal approach. The use of mobile phones, patient diaries, rating scales as well as openly engaging with participants to discuss concerns were also considered optimal complementary data-elicitation tools. This study succeeded in reaching consensus within a section of the anti-malarial drug clinical research community about using a general question concept, and structured questions for eliciting data about AEs and non-study medication reports. The concepts and items considered in this Delphi to be relevant, important and feasible should be further investigated for potential inclusion in a harmonized approach to collect participant-elicited anti-malarial drug safety data. This, in turn, should improve understanding of anti-malarial drug safety.

On the Effective Evaluation of TCP

NASA Technical Reports Server (NTRS)

Allman, Mark; Falk, Aaron

2000-01-01

Understanding the performance of the Internet's Transmission Control Protocol (TCP) is important because it is the dominant protocol used in the Internet today. Various testing methods exist to evaluate TCP performance, however all have pitfalls an that need to be understood prior to obtaining useful results. Simulating TCP is difficult because of the wide range of variables, environments, and implementations available. Testing TCP modifications in the global Internet may not be the answer either: testing new protocols on real networks endangers other people's traffic and, if not done correctly, may also yield inaccurate or misleading results. In order for TCP research to be independently evaluated in the Internet research community there is a set of questions that researchers should try to answer. This paper attempts to list some of those questions and make recommendations as to how TCP testing can be structured to be provide useful answers.

Ion thruster design and analysis

NASA Technical Reports Server (NTRS)

Kami, S.; Schnelker, D. E.

1976-01-01

Questions concerning the mechanical design of a thruster are considered, taking into account differences in the design of an 8-cm and a 30-cm model. The components of a thruster include the thruster shell assembly, the ion extraction electrode assembly, the cathode isolator vaporizer assembly, the neutralizer isolator vaporizer assembly, ground screen and mask, and the main isolator vaporizer assembly. Attention is given to the materials used in thruster fabrication, the advanced manufacturing methods used, details of thruster performance, an evaluation of thruster life, structural and thermal design considerations, and questions of reliability and quality assurance.

Online Social Networks and Smoking Cessation: A Scientific Research Agenda

PubMed Central

Graham, Amanda L; Byron, M. Justin; Niaura, Raymond S; Abrams, David B

2011-01-01

Background Smoking remains one of the most pressing public health problems in the United States and internationally. The concurrent evolution of the Internet, social network science, and online communities offers a potential target for high-yield interventions capable of shifting population-level smoking rates and substantially improving public health. Objective Our objective was to convene leading practitioners in relevant disciplines to develop the core of a strategic research agenda on online social networks and their use for smoking cessation, with implications for other health behaviors. Methods We conducted a 100-person, 2-day, multidisciplinary workshop in Washington, DC, USA. Participants worked in small groups to formulate research questions that could move the field forward. Discussions and resulting questions were synthesized by the workshop planning committee. Results We considered 34 questions in four categories (advancing theory, understanding fundamental mechanisms, intervention approaches, and evaluation) to be the most pressing. Conclusions Online social networks might facilitate smoking cessation in several ways. Identifying new theories, translating these into functional interventions, and evaluating the results will require a concerted transdisciplinary effort. This report presents a series of research questions to assist researchers, developers, and funders in the process of efficiently moving this field forward. PMID:22182518

Towards natural language question generation for the validation of ontologies and mappings.

PubMed

Ben Abacha, Asma; Dos Reis, Julio Cesar; Mrabet, Yassine; Pruski, Cédric; Da Silveira, Marcos

2016-08-08

The increasing number of open-access ontologies and their key role in several applications such as decision-support systems highlight the importance of their validation. Human expertise is crucial for the validation of ontologies from a domain point-of-view. However, the growing number of ontologies and their fast evolution over time make manual validation challenging. We propose a novel semi-automatic approach based on the generation of natural language (NL) questions to support the validation of ontologies and their evolution. The proposed approach includes the automatic generation, factorization and ordering of NL questions from medical ontologies. The final validation and correction is performed by submitting these questions to domain experts and automatically analyzing their feedback. We also propose a second approach for the validation of mappings impacted by ontology changes. The method exploits the context of the changes to propose correction alternatives presented as Multiple Choice Questions. This research provides a question optimization strategy to maximize the validation of ontology entities with a reduced number of questions. We evaluate our approach for the validation of three medical ontologies. We also evaluate the feasibility and efficiency of our mappings validation approach in the context of ontology evolution. These experiments are performed with different versions of SNOMED-CT and ICD9. The obtained experimental results suggest the feasibility and adequacy of our approach to support the validation of interconnected and evolving ontologies. Results also suggest that taking into account RDFS and OWL entailment helps reducing the number of questions and validation time. The application of our approach to validate mapping evolution also shows the difficulty of adapting mapping evolution over time and highlights the importance of semi-automatic validation.

Cumulative Risk and Impact Modeling on Environmental Chemical and Social Stressors.

PubMed

Huang, Hongtai; Wang, Aolin; Morello-Frosch, Rachel; Lam, Juleen; Sirota, Marina; Padula, Amy; Woodruff, Tracey J

2018-03-01

The goal of this review is to identify cumulative modeling methods used to evaluate combined effects of exposures to environmental chemicals and social stressors. The specific review question is: What are the existing quantitative methods used to examine the cumulative impacts of exposures to environmental chemical and social stressors on health? There has been an increase in literature that evaluates combined effects of exposures to environmental chemicals and social stressors on health using regression models; very few studies applied other data mining and machine learning techniques to this problem. The majority of studies we identified used regression models to evaluate combined effects of multiple environmental and social stressors. With proper study design and appropriate modeling assumptions, additional data mining methods may be useful to examine combined effects of environmental and social stressors.

Process evaluation of a walking programme delivered through the workplace in the South Pacific island Vanuatu.

PubMed

Siefken, Katja; Schofield, Grant; Schulenkorf, Nico

2015-06-01

The South Pacific region is experiencing significant rates of chronic diseases. Well-evaluated health promotion programmes are needed as a central piece of a strategic solution. Just as important as the evaluation itself is how that evaluation outcome can be communicated for future programme use by local programme planners. The objective of this study is to evaluate a physical activity (PA) programme that was designed for Pacific women in urban Vanuatu, and subsequently to develop new techniques to display data that support the understanding and communication of programme success and challenges. Data collection methods included quantitative Likert scale questions and qualitative open-ended questions. A new analysis technique visualises open-ended process evaluation data. We present themes using word sizes proportional to the frequency of the themes identified through thematic analysis. The Likert scale technique revealed little meaningful information; almost all participants rated most elements of the programme highly. This may be related to Pacific people being frequently inclined to assent with external ideas. Open-ended questions provided more significant insights. For example, we found a stronger change in eating habits (68.9%) than in exercise behaviour (28.2%). We present an evaluation of the first pedometer-based PA intervention in the Pacific and respond to the paucity of process evaluations that have been carried out in the context of low- and middle-income countries. Moreover, the new thematic data visualisation (TDV) approach may aid in understanding complex and cluttered data in a constructive and coordinated way; we present a new approach in health promotion research. © The Author(s) 2014.

Developing and using a rubric for evaluating evidence-based medicine point-of-care tools

PubMed Central

Foster, Margaret J

2011-01-01

Objective: The research sought to establish a rubric for evaluating evidence-based medicine (EBM) point-of-care tools in a health sciences library. Methods: The authors searched the literature for EBM tool evaluations and found that most previous reviews were designed to evaluate the ability of an EBM tool to answer a clinical question. The researchers' goal was to develop and complete rubrics for assessing these tools based on criteria for a general evaluation of tools (reviewing content, search options, quality control, and grading) and criteria for an evaluation of clinical summaries (searching tools for treatments of common diagnoses and evaluating summaries for quality control). Results: Differences between EBM tools' options, content coverage, and usability were minimal. However, the products' methods for locating and grading evidence varied widely in transparency and process. Conclusions: As EBM tools are constantly updating and evolving, evaluation of these tools needs to be conducted frequently. Standards for evaluating EBM tools need to be established, with one method being the use of objective rubrics. In addition, EBM tools need to provide more information about authorship, reviewers, methods for evidence collection, and grading system employed. PMID:21753917

Experimental evaluation of ontology-based HIV/AIDS frequently asked question retrieval system.

PubMed

Ayalew, Yirsaw; Moeng, Barbara; Mosweunyane, Gontlafetse

2018-05-01

This study presents the results of experimental evaluations of an ontology-based frequently asked question retrieval system in the domain of HIV and AIDS. The main purpose of the system is to provide answers to questions on HIV/AIDS using ontology. To evaluate the effectiveness of the frequently asked question retrieval system, we conducted two experiments. The first experiment focused on the evaluation of the quality of the ontology we developed using the OQuaRE evaluation framework which is based on software quality metrics and metrics designed for ontology quality evaluation. The second experiment focused on evaluating the effectiveness of the ontology in retrieving relevant answers. For this we used an open-source information retrieval platform, Terrier, with retrieval models BM25 and PL2. For the measurement of performance, we used the measures mean average precision, mean reciprocal rank, and precision at 5. The results suggest that frequently asked question retrieval with ontology is more effective than frequently asked question retrieval without ontology in the domain of HIV/AIDS.

Developing and using a rubric for evaluating evidence-based medicine point-of-care tools.

PubMed

Shurtz, Suzanne; Foster, Margaret J

2011-07-01

The research sought to establish a rubric for evaluating evidence-based medicine (EBM) point-of-care tools in a health sciences library. The authors searched the literature for EBM tool evaluations and found that most previous reviews were designed to evaluate the ability of an EBM tool to answer a clinical question. The researchers' goal was to develop and complete rubrics for assessing these tools based on criteria for a general evaluation of tools (reviewing content, search options, quality control, and grading) and criteria for an evaluation of clinical summaries (searching tools for treatments of common diagnoses and evaluating summaries for quality control). Differences between EBM tools' options, content coverage, and usability were minimal. However, the products' methods for locating and grading evidence varied widely in transparency and process. As EBM tools are constantly updating and evolving, evaluation of these tools needs to be conducted frequently. Standards for evaluating EBM tools need to be established, with one method being the use of objective rubrics. In addition, EBM tools need to provide more information about authorship, reviewers, methods for evidence collection, and grading system employed.

Medical student appraisal: searching on smartphones.

PubMed

Khalifian, S; Markman, T; Sampognaro, P; Mitchell, S; Weeks, S; Dattilo, J

2013-01-01

The rapidly growing industry for mobile medical applications provides numerous smartphone resources designed for healthcare professionals. However, not all applications are equally useful in addressing the questions of early medical trainees. Three popular, free, mobile healthcare applications were evaluated along with a Google(TM) web search on both Apple(TM) and Android(TM) devices. Six medical students at a large academic hospital evaluated each application for a one-week period while on various clinical rotations. Google(TM) was the most frequently used search method and presented multimedia resources but was inefficient for obtaining clinical management information. Epocrates(TM) Pill ID feature was praised for its clinical utility. Medscape(TM) had the highest satisfaction of search and excelled through interactive educational features. Micromedex(TM) offered both FDA and off-label dosing for drugs. Google(TM) was the preferred search method for questions related to basic disease processes and multimedia resources, but was inadequate for clinical management. Caution should also be exercised when using Google(TM) in front of patients. Medscape(TM) was the most appealing application due to a broad scope of content and educational features relevant to medical trainees. Students should also be cognizant of how mobile technology may be perceived by their evaluators to avoid false impressions.

Comparison of historical documents for writership

NASA Astrophysics Data System (ADS)

Ball, Gregory R.; Pu, Danjun; Stritmatter, Roger; Srihari, Sargur N.

2010-01-01

Over the last century forensic document science has developed progressively more sophisticated pattern recognition methodologies for ascertaining the authorship of disputed documents. These include advances not only in computer assisted stylometrics, but forensic handwriting analysis. We present a writer verification method and an evaluation of an actual historical document written by an unknown writer. The questioned document is compared against two known handwriting samples of Herman Melville, a 19th century American author who has been hypothesized to be the writer of this document. The comparison led to a high confidence result that the questioned document was written by the same writer as the known documents. Such methodology can be applied to many such questioned documents in historical writing, both in literary and legal fields.

Selecting Question-Specific Genes to Reduce Incongruence in Phylogenomics: A Case Study of Jawed Vertebrate Backbone Phylogeny.

PubMed

Chen, Meng-Yun; Liang, Dan; Zhang, Peng

2015-11-01

Incongruence between different phylogenomic analyses is the main challenge faced by phylogeneticists in the genomic era. To reduce incongruence, phylogenomic studies normally adopt some data filtering approaches, such as reducing missing data or using slowly evolving genes, to improve the signal quality of data. Here, we assembled a phylogenomic data set of 58 jawed vertebrate taxa and 4682 genes to investigate the backbone phylogeny of jawed vertebrates under both concatenation and coalescent-based frameworks. To evaluate the efficiency of extracting phylogenetic signals among different data filtering methods, we chose six highly intractable internodes within the backbone phylogeny of jawed vertebrates as our test questions. We found that our phylogenomic data set exhibits substantial conflicting signal among genes for these questions. Our analyses showed that non-specific data sets that are generated without bias toward specific questions are not sufficient to produce consistent results when there are several difficult nodes within a phylogeny. Moreover, phylogenetic accuracy based on non-specific data is considerably influenced by the size of data and the choice of tree inference methods. To address such incongruences, we selected genes that resolve a given internode but not the entire phylogeny. Notably, not only can this strategy yield correct relationships for the question, but it also reduces inconsistency associated with data sizes and inference methods. Our study highlights the importance of gene selection in phylogenomic analyses, suggesting that simply using a large amount of data cannot guarantee correct results. Constructing question-specific data sets may be more powerful for resolving problematic nodes. © The Author(s) 2015. Published by Oxford University Press, on behalf of the Society of Systematic Biologists. All rights reserved. For Permissions, please email: journals.permissions@oup.com.

Multiple and mixed methods in formative evaluation: Is more better? Reflections from a South African study.

PubMed

Odendaal, Willem; Atkins, Salla; Lewin, Simon

2016-12-15

Formative programme evaluations assess intervention implementation processes, and are seen widely as a way of unlocking the 'black box' of any programme in order to explore and understand why a programme functions as it does. However, few critical assessments of the methods used in such evaluations are available, and there are especially few that reflect on how well the evaluation achieved its objectives. This paper describes a formative evaluation of a community-based lay health worker programme for TB and HIV/AIDS clients across three low-income communities in South Africa. It assesses each of the methods used in relation to the evaluation objectives, and offers suggestions on ways of optimising the use of multiple, mixed-methods within formative evaluations of complex health system interventions. The evaluation's qualitative methods comprised interviews, focus groups, observations and diary keeping. Quantitative methods included a time-and-motion study of the lay health workers' scope of practice and a client survey. The authors conceptualised and conducted the evaluation, and through iterative discussions, assessed the methods used and their results. Overall, the evaluation highlighted programme issues and insights beyond the reach of traditional single methods evaluations. The strengths of the multiple, mixed-methods in this evaluation included a detailed description and nuanced understanding of the programme and its implementation, and triangulation of the perspectives and experiences of clients, lay health workers, and programme managers. However, the use of multiple methods needs to be carefully planned and implemented as this approach can overstretch the logistic and analytic resources of an evaluation. For complex interventions, formative evaluation designs including multiple qualitative and quantitative methods hold distinct advantages over single method evaluations. However, their value is not in the number of methods used, but in how each method matches the evaluation questions and the scientific integrity with which the methods are selected and implemented.

The evaluation of (social-)psychological comfort in clothing, a possible approach

NASA Astrophysics Data System (ADS)

Matté, L. L.; Broega, A. C.

2017-10-01

This paper presents the first results of a PhD research on psychological comfort of clothing. In order to understand and conceptualize the psychological aspects of clothing comfort, a variation of the Delphi Method was used to seek opinions from experts. This method was chosen because of its consensus-building features. The results were obtained from a qualitative text analysis, conducted over the experts’ responses to the first round of questions. The analytic process shed some light on the formation of the psychological comfort concept as well as the potential attributes to be evaluated when assessing this comfort dimension.

Use of Multi-Response Format Test in the Assessment of Medical Students' Critical Thinking Ability.

PubMed

Mafinejad, Mahboobeh Khabaz; Arabshahi, Seyyed Kamran Soltani; Monajemi, Alireza; Jalili, Mohammad; Soltani, Akbar; Rasouli, Javad

2017-09-01

To evaluate students critical thinking skills effectively, change in assessment practices is must. The assessment of a student's ability to think critically is a constant challenge, and yet there is considerable debate on the best assessment method. There is evidence that the intrinsic nature of open and closed-ended response questions is to measure separate cognitive abilities. To assess critical thinking ability of medical students by using multi-response format of assessment. A cross-sectional study was conducted on a group of 159 undergraduate third-year medical students. All the participants completed the California Critical Thinking Skills Test (CCTST) consisting of 34 multiple-choice questions to measure general critical thinking skills and a researcher-developed test that combines open and closed-ended questions. A researcher-developed 48-question exam, consisting of 8 short-answers and 5 essay questions, 19 Multiple-Choice Questions (MCQ), and 16 True-False (TF) questions, was used to measure critical thinking skills. Correlation analyses were performed using Pearson's coefficient to explore the association between the total scores of tests and subtests. One hundred and fifty-nine students participated in this study. The sample comprised 81 females (51%) and 78 males (49%) with an age range of 20±2.8 years (mean 21.2 years). The response rate was 64.1%. A significant positive correlation was found between types of questions and critical thinking scores, of which the correlations of MCQ (r=0.82) and essay questions (r=0.77) were strongest. The significant positive correlations between multi-response format test and CCTST's subscales were seen in analysis, evaluation, inference and inductive reasoning. Unlike CCTST subscales, multi-response format test have weak correlation with CCTST total score (r=0.45, p=0.06). This study highlights the importance of considering multi-response format test in the assessment of critical thinking abilities of medical students by using both open and closed-ended response questions.

Men ask more questions than women at a scientific conference

PubMed Central

Sutherland, William J.; Johnston, Alison

2017-01-01

Gender inequity in science and academia, especially in senior positions, is a recognised problem. The reasons are poorly understood, but include the persistence of historical gender ratios, discrimination and other factors, including gender-based behavioural differences. We studied participation in a professional context by observing question-asking behaviour at a large international conference with a clear equality code of conduct that prohibited any form of discrimination. Accounting for audience gender ratio, male attendees asked 1.8 questions for each question asked by a female attendee. Amongst only younger researchers, male attendees also asked 1.8 questions per female question, suggesting the pattern cannot be attributed to the temporary problem of demographic inertia. We link our findings to the ‘chilly’ climate for women in STEM, including wider experiences of discrimination likely encountered by women throughout their education and careers. We call for a broader and coordinated approach to understanding and addressing the barriers to women and other under-represented groups. We encourage the scientific community to recognise the context in which these gender differences occur, and evaluate and develop methods to support full participation from all attendees. PMID:29036191

Men ask more questions than women at a scientific conference.

PubMed

Hinsley, Amy; Sutherland, William J; Johnston, Alison

2017-01-01

Gender inequity in science and academia, especially in senior positions, is a recognised problem. The reasons are poorly understood, but include the persistence of historical gender ratios, discrimination and other factors, including gender-based behavioural differences. We studied participation in a professional context by observing question-asking behaviour at a large international conference with a clear equality code of conduct that prohibited any form of discrimination. Accounting for audience gender ratio, male attendees asked 1.8 questions for each question asked by a female attendee. Amongst only younger researchers, male attendees also asked 1.8 questions per female question, suggesting the pattern cannot be attributed to the temporary problem of demographic inertia. We link our findings to the 'chilly' climate for women in STEM, including wider experiences of discrimination likely encountered by women throughout their education and careers. We call for a broader and coordinated approach to understanding and addressing the barriers to women and other under-represented groups. We encourage the scientific community to recognise the context in which these gender differences occur, and evaluate and develop methods to support full participation from all attendees.

Use of the script concordance approach to evaluate clinical reasoning in food-ruminant practitioners.

PubMed

Dufour, Simon; Latour, Sylvie; Chicoine, Yvan; Fecteau, Gilles; Forget, Sylvain; Moreau, Jean; Trépanier, André

2012-01-01

A script concordance test (SCT) was developed measuring clinical reasoning of food-ruminant practitioners for whom potential clinical competence difficulties were identified by their provincial professional organization. The SCT was designed to be used as part of a broader evaluation procedure. A scoring key was developed based on answers from a reference panel of 12 experts and using the modified aggregate method commonly used for SCTs. A convenient sample of 29 food-ruminant practitioners was constituted to assess the reliability and precision of the SCT and to determine a fair threshold value for success. Cronbach's α coefficients were computed to evaluate internal reliability. To evaluate SCT precision, a test-retest methodology was used and measures of agreement beyond chance were computed at question and test levels. After optimization, the 36-question SCT yielded acceptable internal reliability (Cronbach's α=0.70). Precision of the SCT at question level was excellent with 33 questions (92%) yielding moderate to almost perfect agreement between administrations. At test level, fair agreement (concordance correlation coefficient=0.32) was observed between administrations. A slight SCT score improvement (M=+2.8 points) on the second administration was in part responsible for some of the disagreement and was potentially a result of an adaptation to the SCT format. Scores distribution was used to determine a fair threshold value for success, while considering the underlying objectives of the examination. The data suggest that the developed SCT can be used as a reliable and precise measurement of clinical reasoning of food-ruminant practitioners.

A three-dimensional electronic report of a venous echo color Doppler of the lower limbs: MEVeC®

PubMed Central

Galeandro, Aldo Innocente; Scicchitano, Pietro; Zito, Annapaola; Galeandro, Cristina; Gesualdo, Michele; Ciciarello, Francesco; Cecere, Annagrazia; Marzullo, Andrea; Contursi, Vincenzo; Annicchiarico, Annamaria; Ciccone, Marco Matteo

2014-01-01

Background The reports of ultrasound evaluation of lower limb veins are difficult to understand by general practitioners (GPs) and physicians who are not specialized. We developed software for a three-dimensional (3D) electronic report of venous hemodynamic mapping (MEVeC®) in order to represent lower limb venous vasculature in a 3D way. The aim of the study is to compare the novel 3D report with the standard report. Methods Thirty subjects (medical students and GPs) evaluated a standard report and a novel 3D report of the lower limb veins of a prespecified patient. The cases were randomly and blindly taken from an archive of 100 cases. GPs and students answered a questionnaire made up of 13 questions that were structured in order to investigate the readability and comprehension of the two reports. A score ranging from 0 to 10 (0= not understandable; 10= full comprehension) was attributed to each report for each question according to the readability of the venous scheme proposed. Results The scores from each question of the questionnaire were compared. The 3D report (MEVeC®) obtained higher scores than those from the evaluation of the standard report (P<0.0001). Each question revealed the superiority of the 3D report (MEVeC®) as compared with the standard report of the ultrasound evaluation of lower limbs. When dividing the scores according to percentiles, the 3D report (MEVeC®) still continued to show more readability than the standard report in a statistically significant way (P<0.0001). Conclusion The new 3D report (MEVeC®) concerning ultrasound evaluation of lower limb veins is more reproducible than the standard report when evaluated by medical physicians not specialized in the evaluation of the vein tree of lower limbs. PMID:25214793

Evaluating Sleep Disturbance: A Review of Methods

NASA Technical Reports Server (NTRS)

Smith, Roy M.; Oyung, R.; Gregory, K.; Miller, D.; Rosekind, M.; Rosekind, Mark R. (Technical Monitor)

1996-01-01

There are three general approaches to evaluating sleep disturbance in regards to noise: subjective, behavioral, and physiological. Subjective methods range from standardized questionnaires and scales to self-report measures designed for specific research questions. There are two behavioral methods that provide useful sleep disturbance data. One behavioral method is actigraphy, a motion detector that provides an empirical estimate of sleep quantity and quality. An actigraph, worn on the non-dominant wrist, provides a 24-hr estimate of the rest/activity cycle. The other method involves a behavioral response, either to a specific probe or stimuli or subject initiated (e.g., indicating wakefulness). The classic, gold standard for evaluating sleep disturbance is continuous physiological monitoring of brain, eye, and muscle activity. This allows detailed distinctions of the states and stages of sleep, awakenings, and sleep continuity. Physiological delta can be obtained in controlled laboratory settings and in natural environments. Current ambulatory physiological recording equipment allows evaluation in home and work settings. These approaches will be described and the relative strengths and limitations of each method will be discussed.

A conversation-based process tracing method for use with naturalistic decisions: an evaluation study.

PubMed

Williamson, J; Ranyard, R; Cuthbert, L

2000-05-01

This study is an evaluation of a process tracing method developed for naturalistic decisions, in this case a consumer choice task. The method is based on Huber et al.'s (1997) Active Information Search (AIS) technique, but develops it by providing spoken rather than written answers to respondents' questions, and by including think aloud instructions. The technique is used within a conversation-based situation, rather than the respondent thinking aloud 'into an empty space', as is conventionally the case in think aloud techniques. The method results in a concurrent verbal protocol as respondents make their decisions, and a retrospective report in the form of a post-decision summary. The method was found to be virtually non-reactive in relation to think aloud, although the variable of Preliminary Attribute Elicitation showed some evidence of reactivity. This was a methodological evaluation, and as such the data reported are essentially descriptive. Nevertheless, the data obtained indicate that the method is capable of producing information about decision processes which could have theoretical importance in terms of evaluating models of decision-making.

Evaluation of angiogram visualization methods for fast and reliable aneurysm diagnosis

NASA Astrophysics Data System (ADS)

Lesar, Žiga; Bohak, Ciril; Marolt, Matija

2015-03-01

In this paper we present the results of an evaluation of different visualization methods for angiogram volumetric data-ray casting, marching cubes, and multi-level partition of unity implicits. There are several options available with ray-casting: isosurface extraction, maximum intensity projection and alpha compositing, each producing fundamentally different results. Different visualization methods are suitable for different needs, so this choice is crucial in diagnosis and decision making processes. We also evaluate visual effects such as ambient occlusion, screen space ambient occlusion, and depth of field. Some visualization methods include transparency, so we address the question of relevancy of this additional visual information. We employ transfer functions to map data values to color and transparency, allowing us to view or hide particular tissues. All the methods presented in this paper were developed using OpenCL, striving for real-time rendering and quality interaction. An evaluation has been conducted to assess the suitability of the visualization methods. Results show superiority of isosurface extraction with ambient occlusion effects. Visual effects may positively or negatively affect perception of depth, motion, and relative positions in space.

Recertification in allergy and immunology: an historical review with special emphasis on the 1983 examination.

PubMed

Slavin, R G; Des Prez, L; Mansmann, H C; Meskauskas, J A; Pierson, W

1985-03-01

Recertification offers a method of evaluating a diplomate's cognitive knowledge of allergy and immunology. In 1983 candidates for the American Board of Allergy and Immunology recertification examination were offered the entire certifying examination but were informed that they would, for recertification purposes, be held responsible only for a subset of questions judged to be particularly clinically relevant. All 40 candidates elected to take the entire certifying examination. Differences between the performance of certifying and recertifying candidates on the recertifying questions were small. Except for the five-choice questions, the differences in performance between the two groups on the remaining questions were also small in an absolute sense. Recertification performance was not related to the time of original certification. Ninety-eight percent of the candidates completed a questionnaire after the examination. Ninety percent stated that they would encourage their colleagues to participate in the recertification process.

Commentary: Writing and Evaluating Qualitative Research Reports

PubMed Central

Thompson, Deborah; Aroian, Karen J.; McQuaid, Elizabeth L.; Deatrick, Janet A.

2016-01-01

Objective To provide an overview of qualitative methods, particularly for reviewers and authors who may be less familiar with qualitative research. Methods A question and answer format is used to address considerations for writing and evaluating qualitative research. Results and Conclusions When producing qualitative research, individuals are encouraged to address the qualitative research considerations raised and to explicitly identify the systematic strategies used to ensure rigor in study design and methods, analysis, and presentation of findings. Increasing capacity for review and publication of qualitative research within pediatric psychology will advance the field’s ability to gain a better understanding of the specific needs of pediatric populations, tailor interventions more effectively, and promote optimal health. PMID:27118271

Evaluation of the PCR method for identification of Bifidobacterium species.

PubMed

Youn, S Y; Seo, J M; Ji, G E

2008-01-01

Bifidobacterium species are known for their beneficial effects on health and their wide use as probiotics. Although various polymerase chain reaction (PCR) methods for the identification of Bifidobacterium species have been published, the reliability of these methods remains open to question. In this study, we evaluated 37 previously reported PCR primer sets designed to amplify 16S rDNA, 23S rDNA, intergenic spacer regions, or repetitive DNA sequences of various Bifidobacterium species. Ten of 37 experimental primer sets showed specificity for B. adolescentis, B. angulatum, B. pseudocatenulatum, B. breve, B. bifidum, B. longum, B. longum biovar infantis and B. dentium. The results suggest that published Bifidobacterium primer sets should be re-evaluated for both reproducibility and specificity for the identification of Bifidobacterium species using PCR. Improvement of existing PCR methods will be needed to facilitate identification of other Bifidobacterium strains, such as B. animalis, B. catenulatum, B. thermophilum and B. subtile.

Qualitative Evaluation Methods in Ethics Education: A Systematic Review and Analysis of Best Practices.

PubMed

Watts, Logan L; Todd, E Michelle; Mulhearn, Tyler J; Medeiros, Kelsey E; Mumford, Michael D; Connelly, Shane

2017-01-01

Although qualitative research offers some unique advantages over quantitative research, qualitative methods are rarely employed in the evaluation of ethics education programs and are often criticized for a lack of rigor. This systematic review investigated the use of qualitative methods in studies of ethics education. Following a review of the literature in which 24 studies were identified, each study was coded based on 16 best practices characteristics in qualitative research. General thematic analysis and grounded theory were found to be the dominant approaches used. Researchers are effectively executing a number of best practices, such as using direct data sources, structured data collection instruments, non-leading questioning, and expert raters. However, other best practices were rarely present in the courses reviewed, such as collecting data using multiple sources, methods, raters, and timepoints, evaluating reliability, and employing triangulation analyses to assess convergence. Recommendations are presented for improving future qualitative research studies in ethics education.

Insights into Students' Conceptual Understanding Using Textual Analysis: A Case Study in Signal Processing

ERIC Educational Resources Information Center

Goncher, Andrea M.; Jayalath, Dhammika; Boles, Wageeh

2016-01-01

Concept inventory tests are one method to evaluate conceptual understanding and identify possible misconceptions. The multiple-choice question format, offering a choice between a correct selection and common misconceptions, can provide an assessment of students' conceptual understanding in various dimensions. Misconceptions of some engineering…

A Salamander Tale: Effective Exhibits and Attitude Change

ERIC Educational Resources Information Center

Rollins, Jeffrey; Watson, Sunnie Lee

2017-01-01

Little information exists regarding intention behind the design and development of Extension outreach and educational exhibits. An evaluation of response to the exhibit "A Salamander Tale" indicates that the methods used to develop the exhibit resulted in an effective way to present information to an adult audience. Survey questions were…

Some Pragmatic Tips for Dealing with Clinical Uncertainty

ERIC Educational Resources Information Center

Ratner, Nan Bernstein

2011-01-01

Purpose: This article proposes some recommendations to enable clinicians to balance certainty and uncertainty when evaluating the currency and effectiveness of their treatment approaches. Method: I offer the following advice: (a) Question the authority of the information previously learned in one's career; (b) be cognizant of what we do not yet…

Evaluating Parental Autism Disclosure Strategies

ERIC Educational Resources Information Center

Austin, Jillian E.; Galijot, Ratka; Davies, W. Hobart

2018-01-01

The relative effects of different autism disclosure methods on the perceptions of a mother-child dyad were investigated. Using three conditions, disclosure card, disclosure bracelet, and no disclosure, U.S. community parents (N = 383) were asked 18 questions about their perceptions of the dyad. An ANOVA revealed significant protection from stigma…

Colorado Even Start. 1997-1998 Progress Report.

ERIC Educational Resources Information Center

Anderson, Beckie

Even Start programs integrate early childhood education, adult literacy or basic education, parenting education and support, and parent and child time together to help break the cycle of poverty and illiteracy. This report describes the Even Start program in Colorado, including evaluation questions and methods. The report also presents evaluation…

Community Participation in Schools in Developing Countries: Characteristics, Methods and Outcomes

ERIC Educational Resources Information Center

Russell, Kenneth A.

2009-01-01

This study examines how communities participate in schools across diverse contexts in developing countries and the results attributed to community participation. It reviews evaluations of participatory approaches to education in developing countries to answer two basic questions: 1) How do communities participate in school in developing countries?…

Speech Perception in Individuals with Auditory Neuropathy

ERIC Educational Resources Information Center

Zeng, Fan-Gang; Liu, Sheng

2006-01-01

Purpose: Speech perception in participants with auditory neuropathy (AN) was systematically studied to answer the following 2 questions: Does noise present a particular problem for people with AN: Can clear speech and cochlear implants alleviate this problem? Method: The researchers evaluated the advantage in intelligibility of clear speech over…

Theater and Dialogue to Increase Youth's Intentions to Advocate for LGBTQQ People

ERIC Educational Resources Information Center

Wernick, Laura J.; Kulick, Alex; Dessel, Adrienne B.; Graham, Louis F.

2016-01-01

Objective: This study evaluates the effectiveness of an intervention using theater and dialogue to raise awareness about homophobia and transphobia and increase intentions to participate in macro-level change efforts around lesbian, gay, bisexual, transgender, queer, and questioning (LGBTQQ) issues. Methods: Using a pretest-posttest design, this…

Phonics Instruction for Disabled Learners: Applying Theory to Method. Technical Report # 7.

ERIC Educational Resources Information Center

Fayne, Harriet R.

To design effective remedial phonics instruction, it is necessary to examine both learner characteristics and task requirements. The paper integrates research related to information processing and psycholinguistics to formulate questions which can be used to evaluate techniques and materials used with a learning disabled population. Information…

Using National Coastal Assessment Data to Model Estuarine Water Quality at Large Spatial Scales

EPA Science Inventory

Background/Question/MethodThe water quality of the Nation’s estuaries is attracting increasing scrutiny in light of burgeoning coastal population growth and enhanced delivery of nutrients via riverine flux. The USEPA has evaluated water quality in US estuaries in the Nation...

An Analysis of the Effect of Mobile Learning on Lebanese Higher Education

ERIC Educational Resources Information Center

Jabbour, Khayrazad Kari

2014-01-01

This research explores the effect of mobile technology in Lebanese higher education classrooms. Three components were utilized to evaluate the impact: student attitudes, student achievements, and educational process. This study used both quantitative and qualitative methods to examine the research questions. The main sources for data collection…

Visions and reality: the idea of competence-oriented assessment for German medical students is not yet realised in licensing examinations

PubMed Central

Huber-Lang, Markus; Palmer, Annette; Grab, Claudia; Boeckers, Anja; Boeckers, Tobias Maria; Oechsner, Wolfgang

2017-01-01

Objective: Competence orientation, often based on the CanMEDS model, has become an important goal for modern curricula in medical education. The National Competence Based Catalogue of Learning Objectives for Undergraduate Medical Education (NKLM) has been adopted in Germany. However, it is currently unknown whether the vision of competence orientation has also reached the licensing examination procedures. Methods: Therefore, a prospective, descriptive, single-centre, exemplary study design was applied to evaluate 4051 questions/tasks (from 28 examiners at 7 two-day licensing oral-practical exams) for undergraduate medical students at the University of Ulm. The oral and practical questions/tasks as well as the real bedside assessment were assigned to specific competence roles (NKLM section I), categories (NKLM section II) and taxonomy levels of learning domains. Results: Numerous questions/tasks were set per candidate (day 1/2: 70±24/86±19 questions) in the licensing oral-practical exam. Competence roles beyond the “medical expert” were scarcely considered. Furthermore, practical and communication skills at the bedside were hardly addressed (less than 3/15 min). Strikingly, there was a significant predominance of questions with a low-level taxonomy. Conclusions: The data indicate a misalignment of competence-oriented frameworks and the “real world” licensing practical-oral medical exam, which needs improvement in both evaluation and education processes. PMID:28584873

Impact of parasitic thermal effects on thermoelectric property measurements by Harman method.

PubMed

Kwon, Beomjin; Baek, Seung-Hyub; Kim, Seong Keun; Kim, Jin-Sang

2014-04-01

Harman method is a rapid and simple technique to measure thermoelectric properties. However, its validity has been often questioned due to the over-simplified assumptions that this method relies on. Here, we quantitatively investigate the influence of the previously ignored parasitic thermal effects on the Harman method and develop a method to determine an intrinsic ZT. We expand the original Harman relation with three extra terms: heat losses via both the lead wires and radiation, and Joule heating within the sample. Based on the expanded Harman relation, we use differential measurement of the sample geometry to measure the intrinsic ZT. To separately evaluate the parasitic terms, the measured ZTs with systematically varied sample geometries and the lead wire types are fitted to the expanded relation. A huge discrepancy (∼28%) of the measured ZTs depending on the measurement configuration is observed. We are able to separately evaluate those parasitic terms. This work will help to evaluate the intrinsic thermoelectric property with Harman method by eliminating ambiguities coming from extrinsic effects.

Student perceptions and learning outcomes of computer-assisted versus traditional instruction in physiology.

PubMed

Richardson, D

1997-12-01

This study compared student perceptions and learning outcomes of computer-assisted instruction against those of traditional didactic lectures. Components of Quantitative Circulatory Physiology (Biological Simulators) and Mechanical Properties of Active Muscle (Trinity Software) were used to teach regulation of tissue blood flow and muscle mechanics, respectively, in the course Medical Physiology. These topics were each taught, in part, by 1) standard didactic lectures, 2) computer-assisted lectures, and 3) computer laboratory assignment. Subjective evaluation was derived from a questionnaire assessing student opinions of the effectiveness of each method. Objective evaluation consisted of comparing scores on examination questions generated from each method. On a 1-10 scale, effectiveness ratings were higher (P < 0.0001) for the didactic lectures (7.7) compared with either computer-assisted lecture (3.8) or computer laboratory (4.2) methods. A follow-up discussion with representatives from the class indicated that students did not perceive computer instruction as being time effective. However, examination scores from computer laboratory questions (94.3%) were significantly higher compared with ones from either computer-assisted (89.9%; P < 0.025) or didactic (86.6%; P < 0.001) lectures. Thus computer laboratory instruction enhanced learning outcomes in medical physiology despite student perceptions to the contrary.

Economic evaluation of diagnostic methods used in dentistry. A systematic review.

PubMed

Christell, Helena; Birch, Stephen; Horner, Keith; Lindh, Christina; Rohlin, Madeleine

2014-11-01

To review the literature of economic evaluations regarding diagnostic methods used in dentistry. Four databases (MEDLINE, Web of Science, The Cochrane library, the NHS Economic Evaluation Database) were searched for studies, complemented by hand search, until February 2013. Two authors independently screened all titles or abstracts and then applied inclusion and exclusion criteria to select full-text publications published in English, which reported an economic evaluation comparing at least two alternative methods. Studies of diagnostic methods were assessed by four reviewers using a protocol based on the QUADAS tool regarding diagnostic methods and a check-list for economic evaluations. The results of the data extraction were summarized in a structured table and as a narrative description. From 476 identified full-text publications, 160 were considered to be economic evaluations. Only 12 studies (7%) were on diagnostic methods, whilst 78 studies (49%) were on prevention and 70 (40%) on treatment. Among studies on diagnostic methods, there was between-study heterogeneity methodologically, regarding the diagnostic method analysed and type of economic evaluation addressed. Generally, the choice of economic evaluation method was not justified and the perspective of the study not stated. Costing of diagnostic methods varied. A small body of literature addresses economic evaluation of diagnostic methods in dentistry. Thus, there is a need for studies from various perspectives with well defined research questions and measures of the cost and effectiveness. Economic resources in healthcare are finite. For diagnostic methods, an understanding of efficacy provides only part of the information needed for evidence-based practice. This study highlighted a paucity of economic evaluations of diagnostic methods used in dentistry, indicating that much of what we practise lacks sufficient evidence.

Use of recorded interactive seminars in orthodontic distance education.

PubMed

Miller, Kenneth T; Hannum, Wallace M; Morley, Tarrl; Proffit, William R

2007-09-01

Our objective was evaluate the effectiveness and acceptability of 3 methods of instructor interaction during distance learning with prerecorded seminars in orthodontic residencies and continuing education. After residents at 3 schools (Sydney, Australia; Winnipeg, Manitoba, Canada; and Manchester, United Kingdom) viewed a recorded interactive seminar, they discussed its content with the seminar leader at a distance via video conferencing, audio-only interaction by telephone, and Internet chat with Net Meeting software (Microsoft, Bellevue, Wash). The residents then completed evaluations containing both closed- and open-ended questions. In addition, attendees at the Iranian Orthodontic Congress also viewed a recorded seminar, had questions answered via an interpreter in a video conference, and completed summary evaluations. Video conferencing received the highest ratings and was never cited as the least favorite method of interaction. Telephone interaction was a close second in mean scores, and Internet chat was a distant third. All residents stated that they would like to be taught through distance education again. However, the Iranian orthodontists were less enthusiastic. Distance learning based on observation of recorded seminars and follow-up interaction is an acceptable method of instruction that can allow residents and practicing orthodontists access to various materials and experts, and perhaps help to ease the strains of current faculty shortages. More data are needed to determine whether video conferencing is worth the additional cost and complexity over audio-only interaction.

Questionnaire design: carry-over effects of overall acceptance question placement and pre-evaluation instructions on overall acceptance scores in central location tests.

PubMed

Bastian, Mauresa; Eggett, Dennis L; Jefferies, Laura K

2015-02-01

Question placement and usage of pre-evaluation instructions (PEI) in questionnaires for food sensory analysis may bias consumers' scores via carry-over effects. Data from consumer sensory panels previously conducted at a central location, spanning 11 years and covering a broad range of food product categories, were compiled. Overall acceptance (OA) question placement was studied with categories designated as first (the first evaluation question following demographic questions), after nongustation questions (immediately following questions that do not require panelists to taste the product), and later (following all other hedonic and just-about-right [JAR] questions, but occasionally before ranking, open-ended comments, and/or intent to purchase questions). Each panel was categorized as having or not having PEI in the questionnaire; PEI are instructions that appear immediately before the first evaluation question and show panelists all attributes they will evaluate before receiving test samples. Postpanel surveys were administered regarding the self-reported effect of PEI on panelists' evaluation experience. OA scores were analyzed and compared (1) between OA question placement categories and (2) between panels with and without PEI. For most product categories, OA scores tended to be lower when asked later in the questionnaire, suggesting evidence of a carry-over effect. Usage of PEI increased OA scores by 0.10 of a 9-point hedonic scale point, which is not practically significant. Postpanel survey data showed that presence of PEI typically improved the panelists' experience. Using PEI does not appear to introduce a meaningful carry-over effect. © 2015 Institute of Food Technologists®

Mesh fixation in endoscopic inguinal hernia repair: evaluation of methodology based on a systematic review of randomised clinical trials.

PubMed

Lederhuber, Hans; Stiede, Franziska; Axer, Stephan; Dahlstrand, Ursula

2017-11-01

The issue of mesh fixation in endoscopic inguinal hernia repair is frequently debated and still no conclusive data exist on differences between methods regarding long-term outcome and postoperative complications. The quantity of trials and the simultaneous lack of high-quality evidence raise the question how future trials should be planned. PubMed, EMBASE and the Cochrane Library were searched, using the filters "randomised clinical trials" and "humans". Trials that compared one method of mesh fixation with another fixation method or with non-fixation in endoscopic inguinal hernia repair were eligible. To be included, the trial was required to have assessed at least one of the following primary outcome parameters: recurrence; surgical site infection; chronic pain; or quality-of-life. Fourteen trials assessing 2161 patients and 2562 hernia repairs were included. Only two trials were rated as low risk for bias. Eight trials evaluated recurrence or surgical site infection; none of these could show significant differences between methods of fixation. Two of 11 trials assessing chronic pain described significant differences between methods of fixation. One of two trials evaluating quality-of-life showed significant differences between fixation methods in certain functions. High-quality evidence for differences between the assessed mesh fixation techniques is still lacking. From a socioeconomic and ethical point of view, it is necessary that future trials will be properly designed. As small- and medium-sized single-centre trials have proven unable to find answers, register studies or multi-centre studies with an evident focus on methodology and study design are needed in order to answer questions about mesh fixation in inguinal hernia repair.

Defining the methodological challenges and opportunities for an effective science of sociotechnical systems and safety.

PubMed

Waterson, Patrick; Robertson, Michelle M; Cooke, Nancy J; Militello, Laura; Roth, Emilie; Stanton, Neville A

2015-01-01

An important part of the application of sociotechnical systems theory (STS) is the development of methods, tools and techniques to assess human factors and ergonomics workplace requirements. We focus in this paper on describing and evaluating current STS methods for workplace safety, as well as outlining a set of six case studies covering the application of these methods to a range of safety contexts. We also describe an evaluation of the methods in terms of ratings of their ability to address a set of theoretical and practical questions (e.g. the degree to which methods capture static/dynamic aspects of tasks and interactions between system levels). The outcomes from the evaluation highlight a set of gaps relating to the coverage and applicability of current methods for STS and safety (e.g. coverage of external influences on system functioning; method usability). The final sections of the paper describe a set of future challenges, as well as some practical suggestions for tackling these. We provide an up-to-date review of STS methods, a set of case studies illustrating their use and an evaluation of their strengths and weaknesses. The paper concludes with a 'roadmap' for future work.

A comparison of two methods of teaching. Computer managed instruction and keypad questions versus traditional classroom lecture.

PubMed

Halloran, L

1995-01-01

Computers increasingly are being integrated into nursing education. One method of integration is through computer managed instruction (CMI). Recently, technology has become available that allows the integration of keypad questions into CMI. This brings a new type of interactivity between students and teachers into the classroom. The purpose of this study was to evaluate differences in achievement between a control group taught by traditional classroom lecture (TCL) and an experimental group taught using CMI and keypad questions. Both control and experimental groups consisted of convenience samples of junior nursing students in a baccalaureate program taking a medical/surgical nursing course. Achievement was measured by three instructor-developed multiple choice examinations. Findings demonstrated that although the experimental group demonstrated increasingly higher test scores as the semester progressed, no statistical difference was found in achievement between the two groups. One reason for this may be phenomenon of vampire video. Initially, the method of presentation overshadowed the content. As students became desensitized to the method, they were able to focus and absorb more content. This study suggests that CMI and keypads are a viable teaching option for nursing education. It is equal to TCL in student achievement and provides a new level of interaction in the classroom setting.

20 CFR 702.409 - Evaluation of medical questions; results disputed.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 20 Employees' Benefits 3 2010-04-01 2010-04-01 false Evaluation of medical questions; results disputed. 702.409 Section 702.409 Employees' Benefits EMPLOYMENT STANDARDS ADMINISTRATION, DEPARTMENT OF... PROCEDURE Medical Care and Supervision § 702.409 Evaluation of medical questions; results disputed. Any...

20 CFR 702.409 - Evaluation of medical questions; results disputed.

Code of Federal Regulations, 2014 CFR

2014-04-01

... 20 Employees' Benefits 4 2014-04-01 2014-04-01 false Evaluation of medical questions; results disputed. 702.409 Section 702.409 Employees' Benefits OFFICE OF WORKERS' COMPENSATION PROGRAMS, DEPARTMENT... PROCEDURE Medical Care and Supervision § 702.409 Evaluation of medical questions; results disputed. Any...

20 CFR 702.409 - Evaluation of medical questions; results disputed.

Code of Federal Regulations, 2012 CFR

2012-04-01

... 20 Employees' Benefits 4 2012-04-01 2012-04-01 false Evaluation of medical questions; results disputed. 702.409 Section 702.409 Employees' Benefits OFFICE OF WORKERS' COMPENSATION PROGRAMS, DEPARTMENT... PROCEDURE Medical Care and Supervision § 702.409 Evaluation of medical questions; results disputed. Any...

20 CFR 702.409 - Evaluation of medical questions; results disputed.

Code of Federal Regulations, 2011 CFR

2011-04-01

... 20 Employees' Benefits 3 2011-04-01 2011-04-01 false Evaluation of medical questions; results disputed. 702.409 Section 702.409 Employees' Benefits OFFICE OF WORKERS' COMPENSATION PROGRAMS, DEPARTMENT... PROCEDURE Medical Care and Supervision § 702.409 Evaluation of medical questions; results disputed. Any...

20 CFR 702.408 - Evaluation of medical questions; impartial specialists.

Code of Federal Regulations, 2010 CFR

2010-04-01

... 20 Employees' Benefits 3 2010-04-01 2010-04-01 false Evaluation of medical questions; impartial specialists. 702.408 Section 702.408 Employees' Benefits EMPLOYMENT STANDARDS ADMINISTRATION, DEPARTMENT OF... PROCEDURE Medical Care and Supervision § 702.408 Evaluation of medical questions; impartial specialists. In...

Applicability and feasibility of systematic review for performing evidence-based risk assessment in food and feed safety.

PubMed

Aiassa, E; Higgins, J P T; Frampton, G K; Greiner, M; Afonso, A; Amzal, B; Deeks, J; Dorne, J-L; Glanville, J; Lövei, G L; Nienstedt, K; O'connor, A M; Pullin, A S; Rajić, A; Verloo, D

2015-01-01

Food and feed safety risk assessment uses multi-parameter models to evaluate the likelihood of adverse events associated with exposure to hazards in human health, plant health, animal health, animal welfare, and the environment. Systematic review and meta-analysis are established methods for answering questions in health care, and can be implemented to minimize biases in food and feed safety risk assessment. However, no methodological frameworks exist for refining risk assessment multi-parameter models into questions suitable for systematic review, and use of meta-analysis to estimate all parameters required by a risk model may not be always feasible. This paper describes novel approaches for determining question suitability and for prioritizing questions for systematic review in this area. Risk assessment questions that aim to estimate a parameter are likely to be suitable for systematic review. Such questions can be structured by their "key elements" [e.g., for intervention questions, the population(s), intervention(s), comparator(s), and outcome(s)]. Prioritization of questions to be addressed by systematic review relies on the likely impact and related uncertainty of individual parameters in the risk model. This approach to planning and prioritizing systematic review seems to have useful implications for producing evidence-based food and feed safety risk assessment.

Best methods for evaluating educational impact: a comparison of the efficacy of commonly used measures of library instruction

PubMed Central

Schilling, Katherine; Applegate, Rachel

2012-01-01

Objectives and Background: Libraries are increasingly called upon to demonstrate student learning outcomes and the tangible benefits of library educational programs. This study reviewed and compared the efficacy of traditionally used measures for assessing library instruction, examining the benefits and drawbacks of assessment measures and exploring the extent to which knowledge, attitudes, and behaviors actually paralleled demonstrated skill levels. Methods: An overview of recent literature on the evaluation of information literacy education addressed these questions: (1) What evaluation measures are commonly used for evaluating library instruction? (2) What are the pros and cons of popular evaluation measures? (3) What are the relationships between measures of skills versus measures of attitudes and behavior? Research outcomes were used to identify relationships between measures of attitudes, behaviors, and skills, which are typically gathered via attitudinal surveys, written skills tests, or graded exercises. Results and Conclusions: Results provide useful information about the efficacy of instructional evaluation methods, including showing significant disparities between attitudes, skills, and information usage behaviors. This information can be used by librarians to implement the most appropriate evaluation methods for measuring important variables that accurately demonstrate students' attitudes, behaviors, or skills. PMID:23133325

Use of Simulated Patients to Evaluate Combined Oral Contraceptive Dispensing Practices of Community Pharmacists

PubMed Central

Obreli-Neto, Paulo Roque; Pereira, Leonardo Régis Leira; Guidoni, Camilo Molino; Baldoni, André de Oliveira; Marusic, Srecko; de Lyra-Júnior, Divaldo Pereira; de Almeida, Kelsen Luis; Pazete, Ana Claudia Montolezi; do Nascimento, Janaina Dutra; Kos, Mitja; Girotto, Edmarlon; Cuman, Roberto Kenji Nakamura

2013-01-01

Background Combined oral contraceptive (COC) use is the most commonly used reversible method of birth control. The incorrect use of COCs is frequent and one of the most common causes of unintended pregnancies. Community pharmacists (CPs) are in a strategic position to improve COC use because they are the last health professional to interact with patients before drug use. Objective To evaluate the COC dispensing practices of CPs in a developing country. Method A cross-sectional study was conducted in community pharmacies of Assis and Ourinhos microregions, Brazil, between June 1, 2012, and October 30, 2012. Four simulated patients (SPs) (with counseled audio recording) visited community pharmacies with a prescription for Ciclo 21® (a COC containing ethinyl estradiol 30 mcg + levonorgestrel 15 mcg). The audio recording of every SP visit was listened to independently by 3 researchers to evaluate the COC dispensing practice. The percentage of CPs who performed a screening for safe use of COCs (i.e., taking of patients’ medical and family history, and measuring of blood pressure) and provided counseling, as well as the quality of the screening and counseling, were evaluated. Results Of the 185 CPs contacted, 41 (22.2%) agreed to participate in the study and finished the study protocol. Only 3 CPs asked the SP a question (1 question asked by each professional), and all of the questions were closed-ended, viz., “do you smoke?” (n = 2) and “what is your age?” (n = 1). None of the CPs measured the patient’s blood pressure. Six CPs provided counseling when dispensing COCs (drug dosing, 5 CPs; possible adverse effects, 2 CPs), and one CP provided counseling regarding both aspects. Conclusion The CPs evaluated did not dispense COC appropriately and could influence in the occurrence of negatives therapeutic outcomes such as adverse effects and treatment failure. PMID:24324584

Mastering cognitive development theory in computer science education

NASA Astrophysics Data System (ADS)

Gluga, Richard; Kay, Judy; Lister, Raymond; Simon; Kleitman, Sabina

2013-03-01

To design an effective computer science curriculum, educators require a systematic method of classifying the difficulty level of learning activities and assessment tasks. This is important for curriculum design and implementation and for communication between educators. Different educators must be able to use the method consistently, so that classified activities and assessments are comparable across the subjects of a degree, and, ideally, comparable across institutions. One widespread approach to supporting this is to write learning objects in terms of Bloom's Taxonomy. This, or other such classifications, is likely to be more effective if educators can use them consistently, in the way experts would use them. To this end, we present the design and evaluation of our online interactive web-based tutorial system, which can be configured and used to offer training in different classification schemes. We report on results from three evaluations. First, 17 computer science educators complete a tutorial on using Bloom's Taxonomy to classify programming examination questions. Second, 20 computer science educators complete a Neo-Piagetian tutorial. Third evaluation was a comparison of inter-rater reliability scores of computer science educators classifying programming questions using Bloom's Taxonomy, before and after taking our tutorial. Based on the results from these evaluations, we discuss the effectiveness of our tutorial system design for teaching computer science educators how to systematically and consistently classify programming examination questions. We also discuss the suitability of Bloom's Taxonomy and Neo-Piagetian theory for achieving this goal. The Bloom's and Neo-Piagetian tutorials are made available as a community resource. The contributions of this paper are the following: the tutorial system for learning classification schemes for the purpose of coding the difficulty of computing learning materials; its evaluation; new insights into the consistency that computing educators can achieve using Bloom; and first insights into the use of Neo-Piagetian theory by a group of classifiers.

Portfolio assessment and evaluation: implications and guidelines for clinical nursing education.

PubMed

Chabeli, M M

2002-08-01

With the advent of Outcomes-Based Education in South Africa, the quality of nursing education is debatable, especially with regard to the assessment and evaluation of clinical nursing education, which is complex and renders the validity and reliability of the methods used questionable. This paper seeks to explore and describe the use of portfolio assessment and evaluation, its implications and guidelines for its effective use in nursing education. Firstly, the concepts of assessment, evaluation, portfolio and alternative methods of evaluation are defined. Secondly, a comparison of the characteristics of the old (traditional) methods and the new alternative methods of evaluation is made. Thirdly, through deductive analysis, synthesis and inference, implications and guidelines for the effective use of portfolio assessment and evaluation are described. In view of the qualitative, descriptive and exploratory nature of the study, a focus group interview with twenty students following a post-basic degree at a university in Gauteng regarding their perceptions on the use of portfolio assessment and evaluation method in clinical nursing education was used. A descriptive method of qualitative data analysis of open coding in accordance with Tesch's protocol (in Creswell 1994:155) was used. Resultant implications and guidelines were conceptualised and described within the existing theoretical framework. Principles of trustworthiness were maintained as described by (Lincoln & Guba 1985:290-327). Ethical considerations were in accordance with DENOSA's standards of research (1998:7).

Comparison of Alcohol Use Disorder Screens During College Athlete Pre-Participation Evaluations.

PubMed

Majka, Erek; Graves, Travis; Diaz, Vanessa A; Player, Marty S; Dickerson, Lori M; Gavin, Jennifer K; Wessell, Andrea

2016-05-01

The US Preventive Services Task Force (USPSTF) recommends screening adults for alcohol misuse, a challenge among young adults who may not have regular primary care. The pre-participation evaluation (PPE) provides an opportunity for screening, but traditional screening tools require extra time in an already busy visit. The objective of this study was to compare the 10-item Alcohol Use Disorders Identification Test (AUDIT) with a single-question alcohol misuse screen in a population of college-aged athletes. This cross-sectional study was performed during an athletic PPE clinic at a college in the Southeastern United States among athletes ages 18 years and older. Written AUDIT and single-question screen "How many times in the past year have you had X or more drinks in a day?" (five for men, four for women) asked orally were administered to each participant. Sensitivity, specificity, and positive and negative predictive values for the single-question screen were compared to AUDIT. A total of 225 athletes were screened; 60% were female; 29% screened positive by AUDIT; 59% positive by single-question instrument. Males were more likely to screen positive by both methods. Compared to the AUDIT, the brief single-question screen had 92% sensitivity for alcohol misuse and 55% specificity. The negative predictive value of the single-question screen was 95% compared to AUDIT. A single-question screen for alcohol misuse in college-aged athletes had a high sensitivity and negative predictive value compared to the more extensive AUDIT screen. Ease of administration of this screening tool is ideal for use within the pre-participation physical among college-aged athletes who may not seek regular medical care.

Evaluation of generic medical information accessed via mobile phones at the point of care in resource-limited settings

PubMed Central

Goldbach, Hayley; Chang, Aileen Y; Kyer, Andrea; Ketshogileng, Dineo; Taylor, Lynne; Chandra, Amit; Dacso, Matthew; Kung, Shiang-Ju; Rijken, Taatske; Fontelo, Paul; Littman-Quinn, Ryan; Seymour, Anne K; Kovarik, Carrie L

2014-01-01

Objective Many mobile phone resources have been developed to increase access to health education in the developing world, yet few studies have compared these resources or quantified their performance in a resource-limited setting. This study aims to compare the performance of resident physicians in answering clinical scenarios using PubMed abstracts accessed via the PubMed for Handhelds (PubMed4Hh) website versus medical/drug reference applications (Medical Apps) accessed via software on the mobile phone. Methods A two-arm comparative study with crossover design was conducted. Subjects, who were resident physicians at the University of Botswana, completed eight scenarios, each with multi-part questions. The primary outcome was a grade for each question. The primary independent variable was the intervention arm and other independent variables included residency and question. Results Within each question type there were significant differences in ‘percentage correct’ between Medical Apps and PubMed4Hh for three of the six types of questions: drug-related, diagnosis/definitions, and treatment/management. Within each of these question types, Medical Apps had a higher percentage of fully correct responses than PubMed4Hh (63% vs 13%, 33% vs 12%, and 41% vs 13%, respectively). PubMed4Hh performed better for epidemiologic questions. Conclusions While mobile access to primary literature remains important and serves an information niche, mobile applications with condensed content may be more appropriate for point-of-care information needs. Further research is required to examine the specific information needs of clinicians in resource-limited settings and to evaluate the appropriateness of current resources in bridging location- and context-specific information gaps. PMID:23535665

The Generic Short Patient Experiences Questionnaire (GS-PEQ): identification of core items from a survey in Norway

PubMed Central

2011-01-01

Background Questionnaires are commonly used to collect patient, or user, experiences with health care encounters; however, their adaption to specific target groups limits comparison between groups. We present the construction of a generic questionnaire (maximum of ten questions) for user evaluation across a range of health care services. Methods Based on previous testing of six group-specific questionnaires, we first constructed a generic questionnaire with 23 items related to user experiences. All questions included a "not applicable" response option, as well as a follow-up question about the item's importance. Nine user groups from one health trust were surveyed. Seven groups received questionnaires by mail and two by personal distribution. Selection of core questions was based on three criteria: applicability (proportion "not applicable"), importance (mean scores on follow-up questions), and comprehensiveness (content coverage, maximum two items per dimension). Results 1324 questionnaires were returned providing subsample sizes ranging from 52 to 323. Ten questions were excluded because the proportion of "not applicable" responses exceeded 20% in at least one user group. The number of remaining items was reduced to ten by applying the two other criteria. The final short questionnaire included items on outcome (2), clinician services (2), user involvement (2), incorrect treatment (1), information (1), organisation (1), and accessibility (1). Conclusion The Generic Short Patient Experiences Questionnaire (GS-PEQ) is a short, generic set of questions on user experiences with specialist health care that covers important topics for a range of groups. It can be used alone or with other instruments in quality assessment or in research. The psychometric properties and the relevance of the GS-PEQ in other health care settings and countries need further evaluation. PMID:21510871

How good are the internal controls in your group practice? Ten questions to contemplate.

PubMed

Grant, Barbara J; Foley, Lori A

2002-01-01

Internal controls are the methods and procedures used by any business to prevent or detect errors, safeguard assets (especially cash) from being misappropriated, and encourage staff adherence to prescribed managerial policies. Internal controls in a medical practice differ depending on the size and complexity of the practice. The key, however, is that they prevent or detect errors and efforts to circumvent the established policies and procedures of the organization. How good are the internal controls in your group practice? This article identifies ten questions you should use to evaluate your risk of asset misappropriation.

Users' perception as a tool to improve urban beach planning and management.

PubMed

Cervantes, Omar; Espejel, Ileana; Arellano, Evarista; Delhumeau, Sheila

2008-08-01

Four beaches that share physiographic characteristics (sandy, wide, and long) but differ in socioeconomic and cultural terms (three are located in northwestern Mexico and one in California, USA) were evaluated by beach users. Surveys (565) composed of 36 questions were handed out to beach users on weekends and holidays in 2005. The 25 questions that revealed the most information were selected by factor analysis and classified by cluster analysis. Beach users' preferences were assigned a value by comparing the present survey results with the characteristics of an "ideal" recreational urban beach. Cluster analysis separated three groups of questions: (a) services and infrastructure, (b) recreational activities, and (c) beach conditions. Cluster linkage distance (r=0.82, r=0.78, r=0.67) was used as a weight and multiplied by the value of beach descriptive factors. Mazatlán and Oceanside obtained the highest values because there are enough infrastructure and services; on the contrary, Ensenada and Rosarito were rated medium and low because infrastructure and services are lacking. The presently proposed method can contribute to improving current beach evaluations because the final score represents the beach users' evaluation of the quality of the beach. The weight considered in the present study marks the beach users' preferences among the studied beaches. Adding this weight to beach evaluation will contribute to more specific beach planning in which users' perception is considered.

Users' Perception as a Tool to Improve Urban Beach Planning and Management

NASA Astrophysics Data System (ADS)

Cervantes, Omar; Espejel, Ileana; Arellano, Evarista; Delhumeau, Sheila

2008-08-01

Four beaches that share physiographic characteristics (sandy, wide, and long) but differ in socioeconomic and cultural terms (three are located in northwestern Mexico and one in California, USA) were evaluated by beach users. Surveys (565) composed of 36 questions were handed out to beach users on weekends and holidays in 2005. The 25 questions that revealed the most information were selected by factor analysis and classified by cluster analysis. Beach users’ preferences were assigned a value by comparing the present survey results with the characteristics of an “ideal” recreational urban beach. Cluster analysis separated three groups of questions: (a) services and infrastructure, (b) recreational activities, and (c) beach conditions. Cluster linkage distance ( r = 0.82, r = 0.78, r = 0.67) was used as a weight and multiplied by the value of beach descriptive factors. Mazatlán and Oceanside obtained the highest values because there are enough infrastructure and services; on the contrary, Ensenada and Rosarito were rated medium and low because infrastructure and services are lacking. The presently proposed method can contribute to improving current beach evaluations because the final score represents the beach users’ evaluation of the quality of the beach. The weight considered in the present study marks the beach users’ preferences among the studied beaches. Adding this weight to beach evaluation will contribute to more specific beach planning in which users’ perception is considered.

The implementation and evaluation of teacher training in gaming instruction for secondary science: An action research project

NASA Astrophysics Data System (ADS)

Sanders, Veronica

This study implemented and evaluated gaming instruction as a professional development for science teachers at a Georgia high school. It was guided by four research questions that (a) assessed the impact of training in gaming instruction and evaluation of that training on science teachers' ability to use games; (b) examined evidence showing that science teachers used games; (c) assessed the impact of the implementation and subsequent evaluation of games-based training on how science teachers instruct their students; and (d) explored the use of change management principles to help teachers transition from traditional to gaming instruction. The study included a purposive sampling of 10 volunteer science teachers who received the professional development of training in gaming instruction and were observed as they used games to instruct their students. Quantitative data were collected from interviews, observations, and reviews of student assignments and teacher plans, and were statistically analyzed to answer the research questions. These same methods were used to obtain qualitative data, which were also analyzed to answer the research questions as well as to understand the meaning, beliefs and experience behind the numbers. Ultimately, data analysis revealed that the science teachers not only used gaming instruction but also that the training helped them to use gaming instruction and that they considered gaming instruction a viable instruction methodology. Finally, data analysis revealed that change management was successfully used in the study.

A comparison of web-based and paper-based survey methods: testing assumptions of survey mode and response cost.

PubMed

Greenlaw, Corey; Brown-Welty, Sharon

2009-10-01

Web-based surveys have become more prevalent in areas such as evaluation, research, and marketing research to name a few. The proliferation of these online surveys raises the question, how do response rates compare with traditional surveys and at what cost? This research explored response rates and costs for Web-based surveys, paper surveys, and mixed-mode surveys. The participants included evaluators from the American Evaluation Association (AEA). Results included that mixed-mode, while more expensive, had higher response rates.

Key Questions for Reviewing Virtual Charter School Proposals

ERIC Educational Resources Information Center

National Association of Charter School Authorizers (NJ1), 2011

2011-01-01

This document contains key questions to consider when evaluating virtual charter school proposals.The insert is designed to help authorizers evaluate school quality and capacity in light of the distinctive attributes of virtual schooling. These questions address major functional areas as well essential questions that authorizers should ask of…

Storytelling as an Instructional Method: Descriptions and Research Questions

DTIC Science & Technology

2009-01-01

heavily used today in educa- tion and training of all types. We see evidence of this in dentistry (Whipp, Ferguson, Wells & Iacopino, 2000), the... Ergonomics Society, 2596-2598. Barnes, L. B., Christensen, C. R., & Hansen, A. J. (1994). Teaching and the case method: Text, cases, and readings. Boston...delivery, and evaluation of training systems. In G. Salvendy (Ed.), Handbook of human factors and ergonomics (pp. 472-512). (3rd ed.). New York: John

Car-to-pedestrian collision reconstruction with injury as an evaluation index.

PubMed

Weng, Yiliu; Jin, Xianlong; Zhao, Zhijie; Zhang, Xiaoyun

2010-07-01

Reconstruction of accidents is currently considered as a useful means in the analysis of accidents. By multi-body dynamics and numerical methods, and by adopting vehicle and pedestrian models, the scenario of the crash can often be simulated. When reconstructing the collisions, questions often arise regarding the criteria for the evaluation of simulation results. This paper proposes a reconstruction method for car-to-pedestrian collisions based on injuries of the pedestrians. In this method, pedestrian injury becomes a critical index in judging the correctness of the reconstruction result and guiding the simulation process. Application of this method to a real accident case is also presented in this paper. The study showed a good agreement between injuries obtained by numerical simulation and that by forensic identification. Copyright 2010 Elsevier Ltd. All rights reserved.

Traditional and nontraditional evaluations of student outcomes in a practical final examination of veterinary radiology.

PubMed

Koskinen, Heli I

2010-01-01

The Faculty of Veterinary Medicine at the University of Helsinki recognized the lack of systems to measure the quality of education. At the department level, this meant lack of systems to measure the quality of students' outcomes. The aim of this article was to compare the quality of outcomes of a final examination in veterinary radiology by calculating the correlations between traditional (quantitative scores traditionally given by veterinary teachers) and nontraditional (qualitative Structure of the Observed Learning Outcome, or SOLO, method) grading results. Evaluation of the quality of the questions is also included. The results indicate that SOLO offers criteria for quality evaluation, especially for questions. A correlation of 0.60 (p<0.01) existed between qualitative and quantitative estimations, and a correlation of 0.79 (p<0.01) existed between evaluators, both using traditional scores. Two suggestions for a better system to evaluate quality in the future: First, development of problem-solving skills during the learning process should also be assessed. Second, both the scoring of factual correctness of answers (knowledge) and the grammatical structure of an answer and the quality of presentation should be included in the quality evaluation process.

Health literacy screening instruments for eHealth applications: a systematic review.

PubMed

Collins, Sarah A; Currie, Leanne M; Bakken, Suzanne; Vawdrey, David K; Stone, Patricia W

2012-06-01

To systematically review current health literacy (HL) instruments for use in consumer-facing and mobile health information technology screening and evaluation tools. The databases, PubMed, OVID, Google Scholar, Cochrane Library and Science Citation Index, were searched for health literacy assessment instruments using the terms "health", "literacy", "computer-based," and "psychometrics". All instruments identified by this method were critically appraised according to their reported psychometric properties and clinical feasibility. Eleven different health literacy instruments were found. Screening questions, such as asking a patient about his/her need for assistance in navigating health information, were evaluated in seven different studies and are promising for use as a valid, reliable, and feasible computer-based approach to identify patients that struggle with low health literacy. However, there was a lack of consistency in the types of screening questions proposed. There is also a lack of information regarding the psychometric properties of computer-based health literacy instruments. Only English language health literacy assessment instruments were reviewed and analyzed. Current health literacy screening tools demonstrate varying benefits depending on the context of their use. In many cases, it seems that a single screening question may be a reliable, valid, and feasible means for establishing health literacy. A combination of screening questions that assess health literacy and technological literacy may enable tailoring eHealth applications to user needs. Further research should determine the best screening question(s) and the best synthesis of various instruments' content and methodologies for computer-based health literacy screening and assessment. Copyright © 2012 Elsevier Inc. All rights reserved.

Health Literacy Screening Instruments for eHealth Applications: A Systematic Review

PubMed Central

Collins, Sarah A.; Currie, Leanne M.; Bakken, Suzanne; Vawdrey, David K.; Stone, Patricia W.

2012-01-01

Objective To systematically review current health literacy (HL) instruments for use in consumer-facing and mobile health information technology screening and evaluation tools. Design The databases, PubMed, OVID, Google Scholar, Cochrane Library and Science Citation Index, were searched for health literacy assessment instruments using the terms “health”, “literacy”, “computer-based,” and “psychometrics”. All instruments identified by this method were critically appraised according to their reported psychometric properties and clinical feasibility. Results Eleven different health literacy instruments were found. Screening questions, such as asking a patient about his/her need for assistance in navigating health information, were evaluated in 7 different studies and are promising for use as a valid, reliable, and feasible computer-based approach to identify patients that struggle with low health literacy. However, there was a lack of consistency in the types of screening questions proposed. There is also a lack of information regarding the psychometric properties of computer-based health literacy instruments. Limitations Only English language health literacy assessment instruments were reviewed and analyzed. Conclusions Current health literacy screening tools demonstrate varying benefits depending on the context of their use. In many cases, it seems that a single screening question may be a reliable, valid, and feasible means for establishing health literacy. A combination of screening questions that assess health literacy and technological literacy may enable tailoring eHealth applications to user needs. Further research should determine the best screening question(s) and the best synthesis of various instruments’ content and methodologies for computer-based health literacy screening and assessment. PMID:22521719

Modeling fate and transport of "Contaminants of Emerging Concern" (CECs): is the Soil Water Assessment Tool (SWAT) the appropriate model?

USDA-ARS?s Scientific Manuscript database

Background/Question/Methods As the scientific and regulatory communities realize the significant environmental impacts and ubiquity of “contaminants of emerging concern” (CECs), it is increasingly imperative to develop quantitative assessment tools to evaluate and predict the fate and transport of...

An Evidence-Based Systematic Review on Cognitive Interventions for Individuals with Dementia

ERIC Educational Resources Information Center

Hopper, Tammy; Bourgeois, Michelle; Pimentel, Jane; Qualls, Constance Dean; Hickey, Ellen; Frymark, Tobi; Schooling, Tracy

2013-01-01

Purpose: To evaluate the current state of research evidence related to cognitive interventions for individuals with Alzheimer's disease or related dementias. Method: A systematic search of the literature was conducted across 27 electronic databases based on a set of a priori questions, inclusion/exclusion criteria, and search parameters. Studies…

Outdoor Experiential Environmental Education: An Adult-Centred Intervention for the Affective Domain

ERIC Educational Resources Information Center

Okur-Berberoglu, Emel

2017-01-01

The aim of this research is to evaluate the impact of an outdoor experiential environmental education (OEEE) programme on the affective domain of adult participants--namely, in-service teachers from Turkey. Data collection methods such as; psychodrama, non-participant observation, open-ended questions and content analysis were used within a…

Exploring Secondary Students' Understanding of Chemical Kinetics through Inquiry-Based Learning Activities

ERIC Educational Resources Information Center

Chairam, Sanoe; Klahan, Nutsuda; Coll, Richard K.

2015-01-01

This research is trying to evaluate the feedback of Thai secondary school students to inquiry-based teaching and learning methods, exemplified by the study of chemical kinetics. This work used the multiple-choice questions, scientifically practical diagram and questionnaire to assess students' understanding of chemical kinetics. The findings…

CEE in DC: Tracings of a Conference Experiment and the Questions They Raise.

ERIC Educational Resources Information Center

Christenbury, Leila; Newman, Judith M.

1993-01-01

Describes the changes made in the traditional structure of the conference format as instituted at the 1992 Conference on English Education in Washington, DC. Outlines the methods by which feedback and criticism by participants were solicited, analyzed, and evaluated. Discusses conclusions based on the data gathered. (HB)

An Assessment of Propensity Score Matching as a Nonexperimental Impact Estimator: Evidence from Mexico's PROGRESA Program

ERIC Educational Resources Information Center

Diaz, Juan Jose; Handa, Sudhanshu

2006-01-01

Not all policy questions can be addressed by social experiments. Nonexperimental evaluation methods provide an alternative to experimental designs but their results depend on untestable assumptions. This paper presents evidence on the reliability of propensity score matching (PSM), which estimates treatment effects under the assumption of…

Evaluating the Performance of Repeated Measures Approaches in Replicating Experimental Benchmark Results

ERIC Educational Resources Information Center

McConeghy, Kevin; Wing, Coady; Wong, Vivian C.

2015-01-01

Randomized experiments have long been established as the gold standard for addressing causal questions. However, experiments are not always feasible or desired, so observational methods are also needed. When multiple observations on the same variable are available, a repeated measures design may be used to assess whether a treatment administered…

Clinical Psychology Ph.D. Program Rankings: Evaluating Eminence on Faculty Publications and Citations

ERIC Educational Resources Information Center

Matson, Johnny L.; Malone, Carrie J.; Gonzalez, Melissa L.; McClure, David R.; Laud, Rinita B.; Minshawi, Noha F.

2005-01-01

Program rankings and their visibility have taken on greater and greater significance. Rarely is the accuracy of these rankings, which are typically based on a small subset of university faculty impressions, questioned. This paper presents a more comprehensive survey method based on quantifiable measures of faculty publications and citations. The…

An Integrated Evaluation of Nonspeech Oral Motor Treatments

ERIC Educational Resources Information Center

Powell, Thomas W.

2008-01-01

Purpose: This article functions as an epilogue to the clinical forum examining the use of nonspeech oral motor treatments (NSOMTs) to remediate speech sound disorders in children. Method: Conclusions to eight clinical questions are formed based on the findings that were reported in the clinical forum. Theoretical and clinical challenges are also…

IDEA Technical Report No. 2. Description of Data Base, 1976-77.

ERIC Educational Resources Information Center

Cashin, William E.; Slawson, Hugh M.

The data and computational procedures used by the IDEA system at Kansas State University (during the 1976-77 academic year) to interpret ratings of teacher performance are described in this technical report. The computations for each of the seven parts (evaluation, course description, students' self ratings, methods, additional questions,…

IDEA Technical Report No. 3. Description of Data Base, 1977-78.

ERIC Educational Resources Information Center

Cashin, William E.; Slawson, Hugh M.

The data and computational procedures used by the IDEA System during the 1977-78 academic year at Kansas State University to interpret ratings of teacher performance are described in this technical report. The computations for each of the seven parts (evaluation, course description, students' self-ratings, methods, additional questions, diagnostic…

Active drug safety surveillance: a tool to improve public health.

PubMed

Platt, Richard; Madre, Leanne; Reynolds, Robert; Tilson, Hugh

2008-12-01

Ensuring that drugs have an acceptable safety profile and are used safely is a major public health priority. The Centers for Education and Research on Therapeutics (CERTs) convened experts from academia, government, and industry to assess strategies to increase the speed and predictive value of generating and evaluating safety signals, and to identify next steps to improve the US system for identifying and evaluating potential safety signals. The CERTs convened a think tank comprising representatives of the groups noted above to address these goals. Participants observed that, with the increasing availability of electronic health data, opportunities have emerged to more accurately characterize and confirm potential safety issues. The gain for public health from a highly coordinated network of population-based databases for active surveillance is great and within reach, although operational questions remain. A collaborative network must create a working definition of a safety signal, screening algorithms, and criteria and strategies to confirm or refute a signal once identified through screening. Guidelines are needed for when and how to communicate a signal exists and is being evaluated, as well as the outcome of that evaluation. A public-private partnership to create a network of government and private databases to routinely evaluate and prioritize safety questions is in the public interest. Better methods are needed, and a knowledgeable workforce is required to conduct the surveillance and understand how to interpret the results. The international community will benefit from the availability of better methods and more experts. Copyright (c) 2008 John Wiley & Sons, Ltd.

Descriptive Question Answering with Answer Type Independent Features

NASA Astrophysics Data System (ADS)

Yoon, Yeo-Chan; Lee, Chang-Ki; Kim, Hyun-Ki; Jang, Myung-Gil; Ryu, Pum Mo; Park, So-Young

In this paper, we present a supervised learning method to seek out answers to the most frequently asked descriptive questions: reason, method, and definition questions. Most of the previous systems for question answering focus on factoids, lists or definitional questions. However, descriptive questions such as reason questions and method questions are also frequently asked by users. We propose a system for these types of questions. The system conducts an answer search as follows. First, we analyze the user's question and extract search keywords and the expected answer type. Second, information retrieval results are obtained from an existing search engine such as Yahoo or Google. Finally, we rank the results to find snippets containing answers to the questions based on a ranking SVM algorithm. We also propose features to identify snippets containing answers for descriptive questions. The features are adaptable and thus are not dependent on answer type. Experimental results show that the proposed method and features are clearly effective for the task.

On the question of economic efficiency and how to assess it

NASA Astrophysics Data System (ADS)

Petrosyan, M. O.; Kovalev, I. V.; Zelenkov, P. V.; Chuvashova, M. N.; Grishina, I. A.; Pershakova, K. K.

2016-04-01

Over the years, since the world moved to a system of market-based economic relations scientists and businessmen worldwide wonder how to ease and modernize the process of analyzing business performance. During this time, a variety of methods and tools have been developed to deal with this problem. After all, the productivity of the company is directly related to it’s competitive qualities, so the managers are interested in the quality of evaluation of their companies. The article deals with the traditional methods of evaluating the performance of enterprises and some of the tools used in order to improve its quality.

Using Benchmarking To Strengthen the Assessment of Persistence.

PubMed

McLachlan, Michael S; Zou, Hongyan; Gouin, Todd

2017-01-03

Chemical persistence is a key property for assessing chemical risk and chemical hazard. Current methods for evaluating persistence are based on laboratory tests. The relationship between the laboratory based estimates and persistence in the environment is often unclear, in which case the current methods for evaluating persistence can be questioned. Chemical benchmarking opens new possibilities to measure persistence in the field. In this paper we explore how the benchmarking approach can be applied in both the laboratory and the field to deepen our understanding of chemical persistence in the environment and create a firmer scientific basis for laboratory to field extrapolation of persistence test results.

Level of evidence gap in orthopedic research.

PubMed

Baldwin, Keith D; Bernstein, Joseph; Ahn, Jaimo; McKay, Scott D; Sankar, Wudbhav N

2012-09-01

Level of evidence is the most widely used metric for the quality of a publication, but instances exist in which a Level I study is neither feasible nor desirable. The goal of this study was to evaluate the level of evidence gap in current orthopedic research, which the authors defined as the disparity between the level of evidence that would be required to optimally answer the primary research question and the level of evidence that was actually used. Five orthopedic surgeons (K.D.B., J.B., J.A., S.D.M., W.N.S.) evaluated blinded articles from the first 6 months of 2010 in the Journal of Bone and Joint Surgery (American Volume) (JBJS-Am), classifying the study type and design and extracting a primary research question from each article. Each evaluator then defined the study type and method, along with the level of evidence that would ideally be used to address the primary research question. The level of evidence gap was then calculated by subtracting the actual level of evidence of the manuscript from the level of evidence of the idealized study. Of the 64 JBJS-Am manuscripts eligible for analysis, the average level of evidence was between Level II and III (mean, 2.73). The average level of evidence gap was 1.06 compared with the JBJS-Am-designated level of evidence and 1.28 compared with the evaluators' assessment. Because not all questions require Level I studies, level of evidence alone may not be the best metric for the quality of orthopedic surgery literature. Instead, the authors' concept of a level of evidence gap may be a better tool for assessing the state of orthopedic research publications. Copyright 2012, SLACK Incorporated.

Guide to conducting tinnitus retraining therapy initial and follow-up interviews.

PubMed

Henry, James A; Jastreboff, Margaret M; Jastreboff, Pawel J; Schechter, Martin A; Fausti, Stephen A

2003-01-01

Tinnitus Retraining Therapy (TRT) is a structured method of tinnitus treatment that has been performed since 1990. The TRT Initial Interview form was developed to guide clinicians in obtaining essential information from patients that would specify treatment needs. The TRT Follow-up Interview form is similar to the initial interview form and is designed to evaluate outcomes of treatment. The clinician administers these forms verbally. The forms have been used in a highly abbreviated format with the potential for inconsistent interview administration between examiners. This project was to expand the forms to provide specific wording for each question. The expanded forms are presented in this article, and the intent of each question is explained. Standardized administration of these interview forms will facilitate greater uniformity in the initial evaluation and outcomes analyses of patients treated with TRT.

A review of CDC's Web-based Injury Statistics Query and Reporting System (WISQARS™): Planning for the future of injury surveillance✩

PubMed Central

Ballesteros, Michael F.; Webb, Kevin; McClure, Roderick J.

2017-01-01

Introduction The Centers for Disease Control and Prevention (CDC) developed the Web-based Injury Statistics Query and Reporting System (WISQARSTM) to meet the data needs of injury practitioners. In 2015, CDC completed a Portfolio Review of this system to inform its future development. Methods Evaluation questions addressed utilization, technology and innovation, data sources, and tools and training. Data were collected through environmental scans, a review of peer-reviewed and grey literature, a web search, and stakeholder interviews. Results Review findings led to specific recommendations for each evaluation question. Response CDC reviewed each recommendation and initiated several enhancements that will improve the ability of injury prevention practitioners to leverage these data, better make sense of query results, and incorporate findings and key messages into prevention practices. PMID:28454867

The effect of distance learning via SMS on academic achievement and satisfaction of medical students.

PubMed

Sichani, Mehrdad Mohammadi; Mobarakeh, Shadi Reissizadeh; Omid, Athar

2018-01-01

Recently, medical education has made significant progress, and medical teachers are trying to find methods that have most impressive effects on learning. One of the useful learning methods is student active participation. One of the helpful teaching aids in this method is mobile technology. The present study aimed to determine the effect of sending educational questions through short message service (SMS) on academic achievement and satisfaction of medical students and compare that with lecture teaching. In an semi-experimental, two chapters of urology reference book, Smiths General Urology 17 th edition, were taught to 47 medical students of Isfahan University of Medical Sciences in urology course in 2013 academic year. Kidney tumors chapter was educated by sending questions through SMS, and bladder tumors part was taught in a lecture session. For each method, pretest and posttest were held, each consisting of thirty multiple choice questions. To examine the knowledge retention, a test session was held on the same terms for each chapter, 1 month later. At the end, survey forms were distributed to assess student's satisfaction with SMS learning method. Data were analyzed through using SPSS 20. The findings demonstrated a statistically significant difference between the two learning methods in the medication test scores. Evaluation of the satisfaction showed 78.72% of participants were not satisfied. The results of the study showed that distance learning through SMS in medical students could lead to increase knowledge, however, it was not effective on their satisfaction.

Web-Based Question Bank in Indian Higher Education: An Open Educational Resource

ERIC Educational Resources Information Center

Parthasarathy, M.; Ananthasayanam, R.

2012-01-01

Question bank provides an opportunity to improve the evaluation, learning and answering skills among the students, questioning logics among the teachers, and examination patterns among curriculum designers. The question bank is not a learning material but supports as self-evaluation guide. In India, almost 259 State universities administer…

Using Multiple-Choice Questions to Evaluate In-Depth Learning of Economics

ERIC Educational Resources Information Center

Buckles, Stephen; Siegfried, John J.

2006-01-01

Multiple-choice questions are the basis of a significant portion of assessment in introductory economics courses. However, these questions, as found in course assessments, test banks, and textbooks, often fail to evaluate students' abilities to use and apply economic analysis. The authors conclude that multiple-choice questions can be used to…

Dental students' evaluations of an interactive histology software.

PubMed

Rosas, Cristian; Rubí, Rafael; Donoso, Manuel; Uribe, Sergio

2012-11-01

This study assessed dental students' evaluations of a new Interactive Histology Software (IHS) developed by the authors and compared students' assessment of the extent to which this new software, as well as other histology teaching methods, supported their learning. The IHS is a computer-based tool for histology learning that presents high-resolution images of histology basics as well as specific oral histologies at different magnifications and with text labels. Survey data were collected from 204 first-year dental students at the Universidad Austral de Chile. The survey consisted of questions for the respondents to evaluate the characteristics of the IHS and the contribution of various teaching methods to their histology learning. The response rate was 85 percent. Student evaluations were positive for the design, usability, and theoretical-practical integration of the IHS, and the students reported they would recommend the method to future students. The students continued to value traditional teaching methods for histological lab work and did not think this new technology would replace traditional methods. With respect to the contribution of each teaching method to students' learning, no statistically significant differences (p>0.05) were found for an evaluation of IHS, light microscopy, and slide presentations. However, these student assessments were significantly more positive than the evaluations of other digital or printed materials. Overall, the students evaluated the IHS very positively in terms of method quality and contribution to their learning; they also evaluated use of light microscopy and teacher slide presentations positively.

Neuroleptic-induced movement disorders in a naturalistic schizophrenia population: diagnostic value of actometric movement patterns.

PubMed

Janno, Sven; Holi, Matti M; Tuisku, Katinka; Wahlbeck, Kristian

2008-04-18

Neuroleptic-induced movement disorders (NIMDs) have overlapping co-morbidity. Earlier studies have described typical clinical movement patterns for individual NIMDs. This study aimed to identify specific movement patterns for each individual NIMD using actometry. A naturalistic population of 99 schizophrenia inpatients using conventional antipsychotics and clozapine was evaluated. Subjects with NIMDs were categorized using the criteria for NIMD found in the Diagnostic and Statistical Manual for Mental Disorders - Fourth Edition (DSM-IV).Two blinded raters evaluated the actometric-controlled rest activity data for activity periods, rhythmical activity, frequencies, and highest acceleration peaks. A simple subjective question was formulated to test patient-based evaluation of NIMD. The patterns of neuroleptic-induced akathisia (NIA) and pseudoakathisia (PsA) were identifiable in actometry with excellent inter-rater reliability. The answers to the subjective question about troubles with movements distinguished NIA patients from other patients rather well. Also actometry had rather good screening performances in distinguishing akathisia from other NIMD. Actometry was not able to reliably detect patterns of neuroleptic-induced parkinsonism and tardive dyskinesia. The present study showed that pooled NIA and PsA patients had a different pattern in lower limb descriptive actometry than other patients in a non-selected sample. Careful questioning of patients is a useful method of diagnosing NIA in a clinical setting.

A framework for improving the cost-effectiveness of DSM program evaluations

DOE Office of Scientific and Technical Information (OSTI.GOV)

Sonnenblick, R.; Eto, J.

The prudence of utility demand-side management (DSM) investments hinges on their performance, yet evaluating performance is complicated because the energy saved by DSM programs can never be observed directly but only inferred. This study frames and begins to answer the following questions: (1) how well do current evaluation methods perform in improving confidence in the measurement of energy savings produced by DSM programs; (2) in view of this performance, how can limited evaluation resources be best allocated to maximize the value of the information they provide? The authors review three major classes of methods for estimating annual energy savings: trackingmore » database (sometimes called engineering estimates), end-use metering, and billing analysis and examine them in light of the uncertainties in current estimates of DSM program measure lifetimes. The authors assess the accuracy and precision of each method and construct trade-off curves to examine the costs of increases in accuracy or precision. Several approaches for improving evaluations for the purpose of assessing program cost effectiveness are demonstrated. The methods can be easily generalized to other evaluation objectives, such as shared savings incentive payments.« less

Evaluation of and Feedback for Academic Medicine Leaders: Developing and Implementing the Memorial Method.

PubMed

Rourke, James; Bornstein, Stephen; Vardy, Cathy; Speed, David; White, Tyrone; Corbett, Paula

2017-11-01

Giving and receiving honest and helpful feedback for leadership development is a common challenge in all types of organizations but particularly in academic medicine. At Memorial University of Newfoundland, in 2014, a consensus emerged to develop a new method for evaluating the leadership performance of the discipline chairs, dean, and vice dean, and to provide these leaders with the evaluation results to help them improve their performance. The leaders responsible for developing and implementing this method (called the Memorial Method) decided to use a survey to obtain faculty members' perceptions about their leader's performance. Beginning in October 2014, a portion of several regular meetings of the discipline chairs with the dean and vice dean was used to develop the survey, by first discussing the broad dimensions of leadership performance, then discussing these dimensions in more detail and drafting specific questions. The resulting survey included 44 quantitative questions addressing eight leadership dimensions. In March-April 2015, the survey was administered electronically to full-time faculty members on a confidential basis. The results were compiled and reported to each discipline chair and to the dean and vice dean. In total, 144/249 faculty responded to the survey (response rate: 58%). For the various dimensions, individual chairs' mean scores ranged from 2.82 to 4.70, and overall mean scores ranged from 3.57 to 4.24. Psychometric properties of the survey suggested it was both reliable and valid. The survey will be repeated, this time with part-time as well as full-time faculty included.

Extracting similar terms from multiple EMR-based semantic embeddings to support chart reviews.

PubMed

Cheng Ye, M S; Fabbri, Daniel

2018-05-21

Word embeddings project semantically similar terms into nearby points in a vector space. When trained on clinical text, these embeddings can be leveraged to improve keyword search and text highlighting. In this paper, we present methods to refine the selection process of similar terms from multiple EMR-based word embeddings, and evaluate their performance quantitatively and qualitatively across multiple chart review tasks. Word embeddings were trained on each clinical note type in an EMR. These embeddings were then combined, weighted, and truncated to select a refined set of similar terms to be used in keyword search and text highlighting. To evaluate their quality, we measured the similar terms' information retrieval (IR) performance using precision-at-K (P@5, P@10). Additionally a user study evaluated users' search term preferences, while a timing study measured the time to answer a question from a clinical chart. The refined terms outperformed the baseline method's information retrieval performance (e.g., increasing the average P@5 from 0.48 to 0.60). Additionally, the refined terms were preferred by most users, and reduced the average time to answer a question. Clinical information can be more quickly retrieved and synthesized when using semantically similar term from multiple embeddings. Copyright © 2018. Published by Elsevier Inc.

To Click or Not to Click

NASA Astrophysics Data System (ADS)

Abramzon, Nina; Sadaghiani, Homeyra

2009-03-01

A comparison of clickers v. flashcards in a controlled setting was done to test a) whether clickers show an improvement over flashcards in students learning the following concepts: i) Coulomb's force law and ii) magnetic fields caused by currents, and b) if students using clickers are more open towards conceptual questions and the peer instruction method compared to students using flashcards. Two classes taught concurrently by the same instructor were taught identically, except that in one class the collection of answers to concept questions was done using clickers, and in the other using flashcards. To test which students learned the concepts better, a few multiple choice questions from a standard exam used in physics education were included in the final exam of both classes, and the performance of the two classes was compared. In addition, a questionnaire was given to each class to evaluate students' opinions about the benefits of lectures including conceptual TPS questions and the use of related conceptual questions on exams. The results of the survey were compared between the two classes. The experimental design and results of the study will be presented.

Bringing Evaluative Learning to Life

ERIC Educational Resources Information Center

King, Jean A.

2008-01-01

This excerpt from the opening plenary asks evaluators to consider two questions regarding learning and evaluation: (a) How do evaluators know if, how, when, and what people are learning during an evaluation? and (b) In what ways can evaluation be a learning experience? To answer the first question, evaluators can apply the commonplaces of…

A question prompt list for patients with advanced cancer in the final year of life: development and cross-cultural evaluation.

PubMed

Walczak, Adam; Mazer, Benjamin; Butow, Phyllis N; Tattersall, Martin H N; Clayton, Josephine M; Davidson, Patricia M; Young, Jane; Ladwig, Susan; Epstein, Ronald M

2013-09-01

Clinicians and patients find prognosis and end-of-life care discussions challenging. Misunderstanding one's prognosis can contribute to poor decision-making and end-of-life quality of life. A question prompt list (booklet of questions patients can ask clinicians) targeting these issues may help overcome communication barriers. None exists for end-of-life discussions outside the palliative care setting. To develop/pilot a question prompt list facilitating discussion/planning of end-of-life care for oncology patients with advanced cancer from Australia and the United States and to explore acceptability, perceived benefits/challenges of using the question prompt list, suggestions for improvements and the necessity of country-specific adaptations. An expert panel developed a question prompt list targeting prognosis and end-of-life issues. Australian/US semi-structured interviews and one focus group elicited feedback about the question prompt list. Transcribed data were analysed using qualitative methods. Thirty-four patients with advanced cancer (15 Australian/19 US) and 13 health professionals treating such patients (7 Australian/6 US) from two Australian and one US cancer centre participated. Most endorsed the entire question prompt list, though a minority queried the utility/appropriateness of some questions. Analysis identified four global themes: (1) reinforcement of known benefits of question prompt lists, (2) appraisal of content and suggestions for further developments, (3) perceived benefits and challenges in using the question prompt list and (4) contrasts in Australian/US feedback. These contrasts necessitated distinct Australian/US final versions of the question prompt list. Participants endorsed the question prompt list as acceptable and useful. Feedback resulted in two distinct versions of the question prompt list, accommodating differences between Australian and US approaches to end-of-life discussions, highlighting the appropriateness of tailoring communication aides to individual populations.

Using Bloom's Taxonomy to Evaluate the Cognitive Levels of Master Class Textbook's Questions

ERIC Educational Resources Information Center

Assaly, Ibtihal R.; Smadi, Oqlah M.

2015-01-01

This study aimed at evaluating the cognitive levels of the questions following the reading texts of Master Class textbook. A checklist based on Bloom's Taxonomy was the instrument used to categorize the cognitive levels of these questions. The researchers used proper statistics to rank the cognitive levels of the comprehension questions. The…

Beyond Description in Interpersonal Construct Validation: Methodological Advances in the Circumplex Structural Summary Approach.

PubMed

Zimmermann, Johannes; Wright, Aidan G C

2017-01-01

The interpersonal circumplex is a well-established structural model that organizes interpersonal functioning within the two-dimensional space marked by dominance and affiliation. The structural summary method (SSM) was developed to evaluate the interpersonal nature of other constructs and measures outside the interpersonal circumplex. To date, this method has been primarily descriptive, providing no way to draw inferences when comparing SSM parameters across constructs or groups. We describe a newly developed resampling-based method for deriving confidence intervals, which allows for SSM parameter comparisons. In a series of five studies, we evaluated the accuracy of the approach across a wide range of possible sample sizes and parameter values, and demonstrated its utility for posing theoretical questions on the interpersonal nature of relevant constructs (e.g., personality disorders) using real-world data. As a result, the SSM is strengthened for its intended purpose of construct evaluation and theory building. © The Author(s) 2015.

The comparison of predictive scheduling algorithms for different sizes of job shop scheduling problems

NASA Astrophysics Data System (ADS)

Paprocka, I.; Kempa, W. M.; Grabowik, C.; Kalinowski, K.; Krenczyk, D.

2016-08-01

In the paper a survey of predictive and reactive scheduling methods is done in order to evaluate how the ability of prediction of reliability characteristics influences over robustness criteria. The most important reliability characteristics are: Mean Time to Failure, Mean Time of Repair. Survey analysis is done for a job shop scheduling problem. The paper answers the question: what method generates robust schedules in the case of a bottleneck failure occurrence before, at the beginning of planned maintenance actions or after planned maintenance actions? Efficiency of predictive schedules is evaluated using criteria: makespan, total tardiness, flow time, idle time. Efficiency of reactive schedules is evaluated using: solution robustness criterion and quality robustness criterion. This paper is the continuation of the research conducted in the paper [1], where the survey of predictive and reactive scheduling methods is done only for small size scheduling problems.

Dynamic analysis environment for nuclear forensic analyses

NASA Astrophysics Data System (ADS)

Stork, C. L.; Ummel, C. C.; Stuart, D. S.; Bodily, S.; Goldblum, B. L.

2017-01-01

A Dynamic Analysis Environment (DAE) software package is introduced to facilitate group inclusion/exclusion method testing, evaluation and comparison for pre-detonation nuclear forensics applications. Employing DAE, the multivariate signatures of a questioned material can be compared to the signatures for different, known groups, enabling the linking of the questioned material to its potential process, location, or fabrication facility. Advantages of using DAE for group inclusion/exclusion include built-in query tools for retrieving data of interest from a database, the recording and documentation of all analysis steps, a clear visualization of the analysis steps intelligible to a non-expert, and the ability to integrate analysis tools developed in different programming languages. Two group inclusion/exclusion methods are implemented in DAE: principal component analysis, a parametric feature extraction method, and k nearest neighbors, a nonparametric pattern recognition method. Spent Fuel Isotopic Composition (SFCOMPO), an open source international database of isotopic compositions for spent nuclear fuels (SNF) from 14 reactors, is used to construct PCA and KNN models for known reactor groups, and 20 simulated SNF samples are utilized in evaluating the performance of these group inclusion/exclusion models. For all 20 simulated samples, PCA in conjunction with the Q statistic correctly excludes a large percentage of reactor groups and correctly includes the true reactor of origination. Employing KNN, 14 of the 20 simulated samples are classified to their true reactor of origination.

The effect of distance learning via SMS on academic achievement and satisfaction of medical students

PubMed Central

Sichani, Mehrdad Mohammadi; Mobarakeh, Shadi Reissizadeh; Omid, Athar

2018-01-01

INTRODUCTION: Recently, medical education has made significant progress, and medical teachers are trying to find methods that have most impressive effects on learning. One of the useful learning methods is student active participation. One of the helpful teaching aids in this method is mobile technology. The present study aimed to determine the effect of sending educational questions through short message service (SMS) on academic achievement and satisfaction of medical students and compare that with lecture teaching. SUBJECTS AND METHODS: In an semi-experimental, two chapters of urology reference book, Smiths General Urology 17th edition, were taught to 47 medical students of Isfahan University of Medical Sciences in urology course in 2013 academic year. Kidney tumors chapter was educated by sending questions through SMS, and bladder tumors part was taught in a lecture session. For each method, pretest and posttest were held, each consisting of thirty multiple choice questions. To examine the knowledge retention, a test session was held on the same terms for each chapter, 1 month later. At the end, survey forms were distributed to assess student's satisfaction with SMS learning method. Data were analyzed through using SPSS 20. RESULTS: The findings demonstrated a statistically significant difference between the two learning methods in the medication test scores. Evaluation of the satisfaction showed 78.72% of participants were not satisfied. CONCLUSIONS: The results of the study showed that distance learning through SMS in medical students could lead to increase knowledge, however, it was not effective on their satisfaction. PMID:29629390

Impact of parasitic thermal effects on thermoelectric property measurements by Harman method

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kwon, Beomjin, E-mail: bkwon@kist.re.kr; Baek, Seung-Hyub; Keun Kim, Seong

2014-04-15

Harman method is a rapid and simple technique to measure thermoelectric properties. However, its validity has been often questioned due to the over-simplified assumptions that this method relies on. Here, we quantitatively investigate the influence of the previously ignored parasitic thermal effects on the Harman method and develop a method to determine an intrinsic ZT. We expand the original Harman relation with three extra terms: heat losses via both the lead wires and radiation, and Joule heating within the sample. Based on the expanded Harman relation, we use differential measurement of the sample geometry to measure the intrinsic ZT. Tomore » separately evaluate the parasitic terms, the measured ZTs with systematically varied sample geometries and the lead wire types are fitted to the expanded relation. A huge discrepancy (∼28%) of the measured ZTs depending on the measurement configuration is observed. We are able to separately evaluate those parasitic terms. This work will help to evaluate the intrinsic thermoelectric property with Harman method by eliminating ambiguities coming from extrinsic effects.« less

Defining the methodological challenges and opportunities for an effective science of sociotechnical systems and safety

PubMed Central

Waterson, Patrick; Robertson, Michelle M.; Cooke, Nancy J.; Militello, Laura; Roth, Emilie; Stanton, Neville A.

2015-01-01

An important part of the application of sociotechnical systems theory (STS) is the development of methods, tools and techniques to assess human factors and ergonomics workplace requirements. We focus in this paper on describing and evaluating current STS methods for workplace safety, as well as outlining a set of six case studies covering the application of these methods to a range of safety contexts. We also describe an evaluation of the methods in terms of ratings of their ability to address a set of theoretical and practical questions (e.g. the degree to which methods capture static/dynamic aspects of tasks and interactions between system levels). The outcomes from the evaluation highlight a set of gaps relating to the coverage and applicability of current methods for STS and safety (e.g. coverage of external influences on system functioning; method usability). The final sections of the paper describe a set of future challenges, as well as some practical suggestions for tackling these. Practitioner Summary: We provide an up-to-date review of STS methods, a set of case studies illustrating their use and an evaluation of their strengths and weaknesses. The paper concludes with a ‘roadmap’ for future work. PMID:25832121

Integrated methods for teaching population health.

PubMed

Sistrom, Maria Gilson; Zeigen, Laura; Jones, Melissa; Durham, Korana Fiol; Boudrot, Thomas

2011-01-01

The Institute of Medicine recommends reforms to public health education to better prepare the public health workforce. This study addresses the application of two of the recommended reforms in the population health nursing curriculum at one university: use of an ecological model and distance learning methods. Using interdisciplinary faculty, integrated teaching and learning methods, and a multimedia curriculum, this study examined the following question: can distance learning be designed to support learning goals and outcomes specific to an ecological approach and population health concepts in general? Course content was evaluated using students' perception of practice utility and understanding of population health concepts. Integrated teaching methods were evaluated using a scale as well as comparison to other student distance learning experiences within the university. Findings demonstrated that both the ecological model and distance learning methods were successfully used to teach population health to a large nursing student cohort. 2011, SLACK Incorporated.

“How Did You Feel?”: Increasing Child Sexual Abuse Witnesses’ Production of Evaluative Information

PubMed Central

Lyon, Thomas D.; Scurich, Nicholas; Choi, Karen; Handmaker, Sally; Blank, Rebecca

2014-01-01

In child sexual abuse cases, the victim’s testimony is essential, because the victim and the perpetrator tend to be the only eyewitnesses to the crime. A potentially important component of an abuse report is the child’s subjective reactions to the abuse. Attorneys may ask suggestive questions or avoid questioning children about their reactions, assuming that children, given their immaturity and reluctance, are incapable of articulation. We hypothesized that How questions referencing reactions to abuse (e.g., “how did you feel”) would increase the productivity of children’s descriptions of abuse reactions. Two studies compared the extent to which children provided evaluative content, defined as descriptions of emotional, cognitive, and physical reactions, in response to different question-types, including How questions, Wh-questions, Option-posing questions (yes–no or forced-choice), and Suggestive questions. The first study examined children’s testimony (ages 5–18) in 80 felony child sexual abuse cases. How questions were more productive yet the least prevalent, and Option-posing and Suggestive questions were less productive but the most common. The second study examined interview transcripts of 61 children (ages 6 –12) suspected of being abused, in which children were systematically asked How questions regarding their reactions to abuse, thus controlling for the possibility that in the first study, attorneys selectively asked How questions of more articulate children. Again, How questions were most productive in eliciting evaluative content. The results suggest that interviewers and attorneys interested in eliciting evaluative reactions should ask children “how did you feel?” rather than more direct or suggestive questions. PMID:22309936

Energy Options: A Curriculum Guide. Curriculum Resources for the Alaskan Environment.

ERIC Educational Resources Information Center

Pfisterer, William, Ed.

The nine units in the curriculum guide on energy options focus on energy from coal, the sun, petroleum, the wind, water, and trees; and on heat pumps, energy conservation, and agriculture and energy. The units are comprised of from two to seven lessons. Each lesson contains objectives, approximate time, procedure, evaluation questions or methods,…

Evaluation of Interactive Computerized Training to Teach Parents to Implement Photographic Activity Schedules with Children with Autism Spectrum Disorder

ERIC Educational Resources Information Center

Gerencser, Kristina R.; Higbee, Thomas S.; Akers, Jessica S.; Contreras, Bethany P.

2017-01-01

Training parents of children with autism spectrum disorder can be a challenge due to limited resources, time, and money. Interactive computerized training (ICT)--a self-paced program that incorporates instructions, videos, and interactive questions--is one method professionals can use to disseminate trainings to broader populations. This study…

Principal-Generated YouTube Video as a Method of Improving Parental Involvement

ERIC Educational Resources Information Center

Richards, Joey

2013-01-01

The purpose of this study was to evaluate the involvement level of parents and reveal whether principal-generated YouTube videos for regular communication would enhance levels of parental involvement at one North Texas Christian Middle School (pseudonym). The following questions guided this study: 1. What is the beginning level of parental…

Evaluation of Washington State Commission for Vocational Education Vocational Guidance Publications and Media.

ERIC Educational Resources Information Center

Lawrence, George L.

The purpose of this study was to provide information for program planning to improve the quality of occupational information disseminated in Washington schools. The study tried to answer the following questions: Are the materials being used? Who is using them? Are they considered helpful? What changes are recommended? The method used was a survey…

A method of evaluating forest site quality from soil, forest cover, and indicator plants

Treesearch

Marinus Westveld

1952-01-01

Foresters have overlooked too long the importance of soil as a factor in successful timber production. Greatest production in amount and quality of wood at the smallest cost can be attained by growing the tree species that are best suited to the climate and the soil of the locality in question.

Evaluation of an Instruction Program on Diabetes Diet by Means of a Teaching Machine

ERIC Educational Resources Information Center

Teuscher, A.; Heidecker, Barbara

1976-01-01

A study of 119 diabetic patients, student nurses, social workers, dieticians, and medical students indicates that programmed self-teaching with feedback by multiple-choice questions is an efficient method of instruction of basic facts of nutrition for diabetes. It enables the physician to spend more time on the patient's personal problems.…

The Case Study Method: Guidelines, Practices, and Applications for Vocational Education. Research and Development Series No. 189.

ERIC Educational Resources Information Center

Spirer, Janet E.

In comparison with traditional experimental design, which is concerned with what happened, a case study approach is more appropriate for answering the question of why or how something happened. As an alternative complementary-vocational-education-evaluation approach, the case study attempts to describe and analyze some program in comprehensive…

Effect of Using Smartphones as Clickers and Tablets as Digital Whiteboards on Students' Engagement and Learning

ERIC Educational Resources Information Center

Remón, Javier; Sebastián, Víctor; Romero, Enrique; Arauzo, Jesús

2017-01-01

This work addresses the use of tablets and smartphones to enhance both student learning and engagement. Tablets were tested as potential substitutes for digital whiteboards, while smartphones were tested as potential survey media in the classroom using a question and answer method. Two teaching strategies were evaluated and compared: (1)…

Evaluating Group Therapy for Aphasia: What Is the Evidence? EBP Briefs. Volume 7, Issue 5

ERIC Educational Resources Information Center

Layfield, Claire A.; Ballard, Kirrie J.; Robin, Donald A.

2013-01-01

Clinical Question: For people with aphasia following a stroke, is group therapy shown to be more effective on communication outcome measures reflecting impairment, activity, and/or participation than individual therapy or no therapy? Method: Review of treatment efficacy research for group intervention in aphasia Study Sources: Cochrane Database of…

The gap between what we measure and what we want to know about ecosystem services in streams and their linkages of human values

EPA Science Inventory

Background/Questions/Methods (190 words) We evaluate national stream monitoring data based on its capacity to link stream ecosystems to analyses of human well-being. As a nation we invest substantially in monitoring, but may miss opportunities to collect information that effecti...

Using Cognitive Interviews to Explore the Many Different Meanings Undergraduate Students in China Attribute to the Term "College Teacher"

ERIC Educational Resources Information Center

Yuhao, Cen

2013-01-01

The cognitive interview method was applied to evaluate survey questions translated and adapted from a US-based college student survey instrument. This paper draws data from cognitive interviews with 45 undergraduate students in China and explores the different meanings they attribute to the term "college teacher." Students understood…

The Questions about Behavioral Function (QABF): Current Status as a Method of Functional Assessment

ERIC Educational Resources Information Center

Matson, Johnny L.; Tureck, Kimberly; Rieske, Robert

2012-01-01

Functional assessment has now entered the mainstream for evaluation and to aid in the treatment of challenging behaviors, while experimental functional analysis was at the forefront of this movement, this particular methodology has proven to be impractical, and thus has limited utility in real world settings. As a result of these factors…

Examining Residence Status as a Risk Factor for Health Risk Behaviors among College Students

ERIC Educational Resources Information Center

DiBello, Angelo M.; Benz, Madeline B.; Miller, Mary Beth; Merrill, Jennifer E.; Carey, Kate B.

2018-01-01

Objective: The current study is aimed to evaluate college student residence as a unique risk factor for a range of negative health behaviors. Participants: We examined data from 63,555 students (66% females) from 157 campuses who completed the National College Health Assessment Survey in Spring 2011. Methods: Participants answered questions about…

The effectiveness of scoliosis screening programs: methods for systematic review and expert panel recommendations formulation

PubMed Central

2013-01-01

Background Literature on scoliosis screening is vast, however because of the observational nature of available data and methodological flaws, data interpretation is often complex, leading to incomplete and sometimes, somewhat misleading conclusions. The need to propose a set of methods for critical appraisal of the literature about scoliosis screening, a comprehensive summary and rating of the available evidence appeared essential. Methods To address these gaps, the study aims were: i) To propose a framework for the assessment of published studies on scoliosis screening effectiveness; ii) To suggest specific questions to be answered on screening effectiveness instead of trying to reach a global position for or against the programs; iii) To contextualize the knowledge through expert panel consultation and meaningful recommendations. The general methodological approach proceeds through the following steps: Elaboration of the conceptual framework; Formulation of the review questions; Identification of the criteria for the review; Selection of the studies; Critical assessment of the studies; Results synthesis; Formulation and grading of recommendations in response to the questions. This plan follows at best GRADE Group (Grades of Recommendation, Assessment, Development and Evaluation) requirements for systematic reviews, assessing quality of evidence and grading the strength of recommendations. Conclusions In this article, the methods developed in support of this work are presented since they may be of some interest for similar reviews in scoliosis and orthopaedic fields. PMID:23883346

The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments.

PubMed

Tarrant, Marie; Knierim, Aimee; Hayes, Sasha K; Ware, James

2006-12-01

Multiple-choice questions are a common assessment method in nursing examinations. Few nurse educators, however, have formal preparation in constructing multiple-choice questions. Consequently, questions used in baccalaureate nursing assessments often contain item-writing flaws, or violations to accepted item-writing guidelines. In one nursing department, 2770 MCQs were collected from tests and examinations administered over a five-year period from 2001 to 2005. Questions were evaluated for 19 frequently occurring item-writing flaws, for cognitive level, for question source, and for the distribution of correct answers. Results show that almost half (46.2%) of the questions contained violations of item-writing guidelines and over 90% were written at low cognitive levels. Only a small proportion of questions were teacher generated (14.1%), while 36.2% were taken from testbanks and almost half (49.4%) had no source identified. MCQs written at a lower cognitive level were significantly more likely to contain item-writing flaws. While there was no relationship between the source of the question and item-writing flaws, teacher-generated questions were more likely to be written at higher cognitive levels (p<0.001). Correct answers were evenly distributed across all four options and no bias was noted in the placement of correct options. Further training in item-writing is recommended for all faculty members who are responsible for developing tests. Pre-test review and quality assessment is also recommended to reduce the occurrence of item-writing flaws and to improve the quality of test questions.

Development and validation of a questionnaire to evaluate infection control in oral radiology.

PubMed

da Costa, Eliana D; Pinelli, Camila; da Silva Tagliaferro, Elaine P; Corrente, José E; Ambrosano, Glaucia M B

2017-04-01

To create and validate a questionnaire to evaluate infection control in oral radiology. The questionnaire was developed after review of the literature, which included published articles and the biosafety protocols available from healthcare agencies. The initial version of the questionnaire was composed of 14 multiple choice questions and was divided into 3 domains on handwashing, disinfection/protection of surfaces and disinfectant used. Content validity was assessed by two expert committees, which reviewed the content and scope of the questionnaire and the relevance of each item, respectively. Reliability was evaluated using test-retest and internal consistency methods with 115 undergraduate dentistry students. Construct validity was assessed using the known-groups technique and factor analysis. The known-groups technique involved 641 undergraduate dentistry students, 20 PhD students and 15 oral radiology professors. In the factor analysis, 3 radiology technicians also participated in addition to the 641 undergraduates, 20 PhD students and 15 oral radiology professors. The content validity results were found to be satisfactory to excellent for the ordinal variables (intraclass correlation coefficient = 0.722-1.000) and good to great for the yes/no questions (kappa = 0.662-0.913) in terms of reliability and good internal consistency (Cronbach's alpha = 0.88). After a factor analysis, some questions were excluded, and the questions were grouped into new domains. Significant differences were observed between answers from different groups. The final version of the questionnaire was composed of nine domains. The questionnaire created was found to exhibit good psychometric properties for assessing infection control in oral radiology.

Observational Studies: Cohort and Case-Control Studies

PubMed Central

Song, Jae W.; Chung, Kevin C.

2010-01-01

Observational studies are an important category of study designs. To address some investigative questions in plastic surgery, randomized controlled trials are not always indicated or ethical to conduct. Instead, observational studies may be the next best method to address these types of questions. Well-designed observational studies have been shown to provide results similar to randomized controlled trials, challenging the belief that observational studies are second-rate. Cohort studies and case-control studies are two primary types of observational studies that aid in evaluating associations between diseases and exposures. In this review article, we describe these study designs, methodological issues, and provide examples from the plastic surgery literature. PMID:20697313

Sign changes in sums of the Liouville function

NASA Astrophysics Data System (ADS)

Borwein, Peter; Ferguson, Ron; Mossinghoff, Michael J.

2008-09-01

The Liouville function λ(n) is the completely multiplicative function whose value is -1 at each prime. We develop some algorithms for computing the sum T(n)Dsum_{kD1}^n λ(k)/k , and use these methods to determine the smallest positive integer n where T(n)<0 . This answers a question originating in some work of Turan, who linked the behavior of T(n) to questions about the Riemann zeta function. We also study the problem of evaluating Polya's sum L(n)Dsum_{kD1}^nλ(k) , and we determine some new local extrema for this function, including some new positive values.

Applicability of common measures in multifocus image fusion comparison

NASA Astrophysics Data System (ADS)

Vajgl, Marek

2017-11-01

Image fusion is an image processing area aimed at fusion of multiple input images to achieve an output image somehow better then each of the input ones. In the case of "multifocus fusion", input images are capturing the same scene differing ina focus distance. The aim is to obtain an image, which is sharp in all its areas. The are several different approaches and methods used to solve this problem. However, it is common question which one is the best. This work describes a research covering the field of common measures with a question, if some of them can be used as a quality measure of the fusion result evaluation.

An analysis of the influence of framework aspects on the study design of health economic modeling evaluations.

PubMed

Gurtner, Sebastian

2013-04-01

Research and practical guidelines have many implications for how to structure a health economic study. A major focus in recent decades has been the quality of health economic research. In practice, the factors influencing a study design are not limited to the quest for quality. Moreover, the framework of the study is important. This research addresses three major questions related to these framework aspects. First, we want to know whether the design of health economic studies has changed over time. Second, we want to know how the subject of a study, whether it is a process or product innovation, influences the parameters of the study design. Third, one of the most important questions we will answer is whether and how the study's source of funding has an impact on the design of the research. To answer these questions, a total of 234 health economic studies were analyzed using a correspondence analysis and a logistic regression analysis. All three categories of framework factors have an influence on the aspects of the study design. Health economic studies have evolved over time, leading to the use of more advanced methods like complex sensitivity analyses. Additionally, the patient's point of view has increased in importance. The evaluation of product innovations has focused more on utility concepts. On the other hand, the source of funding may influence only a few aspects of the study design, such as the use of evaluation methods, the source of data, and the use of certain utility measures. The most important trends in health care, such as the emphasis on the patients' point of view, become increasingly established in health economic evaluations with the passage of time. Although methodological challenges remain, modern information and communication technologies provide a basis for increasing the complexity and quality of health economic studies if used frequently.

Transgender-inclusive measures of sex/gender for population surveys: Mixed-methods evaluation and recommendations.

PubMed

Bauer, Greta R; Braimoh, Jessica; Scheim, Ayden I; Dharma, Christoffer

2017-01-01

Given that an estimated 0.6% of the U.S. population is transgender (trans) and that large health disparities for this population have been documented, government and research organizations are increasingly expanding measures of sex/gender to be trans inclusive. Options suggested for trans community surveys, such as expansive check-all-that-apply gender identity lists and write-in options that offer maximum flexibility, are generally not appropriate for broad population surveys. These require limited questions and a small number of categories for analysis. Limited evaluation has been undertaken of trans-inclusive population survey measures for sex/gender, including those currently in use. Using an internet survey and follow-up of 311 participants, and cognitive interviews from a maximum-diversity sub-sample (n = 79), we conducted a mixed-methods evaluation of two existing measures: a two-step question developed in the United States and a multidimensional measure developed in Canada. We found very low levels of item missingness, and no indicators of confusion on the part of cisgender (non-trans) participants for both measures. However, a majority of interview participants indicated problems with each question item set. Agreement between the two measures in assessment of gender identity was very high (K = 0.9081), but gender identity was a poor proxy for other dimensions of sex or gender among trans participants. Issues to inform measure development or adaptation that emerged from analysis included dimensions of sex/gender measured, whether non-binary identities were trans, Indigenous and cultural identities, proxy reporting, temporality concerns, and the inability of a single item to provide a valid measure of sex/gender. Based on this evaluation, we recommend that population surveys meant for multi-purpose analysis consider a new Multidimensional Sex/Gender Measure for testing that includes three simple items (one asked only of a small sub-group) to assess gender identity and lived gender, with optional additions. We provide considerations for adaptation of this measure to different contexts.

[Sensitisation about condom use in Gabon (1999): evaluation of the impact of a comic book].

PubMed

Milleliri, Jean-Marie; Krentel, Alison; Rey, Jean-Loup

2003-01-01

The authors report the evaluation of the impact of a comic book about condom use distributed to Gabonese high school students in Libreville and Lambarene in 1999. This evaluation was conducted through a self-administered questionnaire completed by 954 students in 11 high schools immediately before distribution of a comic book about condoms and by 771 students 15-30 days afterwards. The anonymous questionnaire contained multiple-choice and open questions about knowledge, attitudes and practices. During the second survey (same schools and same classes), the questions tested knowledge about AIDS and about the stories in the book. The student populations who responded to the two questionnaires were homogeneous for sex, age, school class, and province of residence. Knowledge about the modes of HIV/AIDS contamination improved substantially between the two questionnaires, with knowledge about the mother-child transmission pathway increasing from 47% to 75% of responders. At the same time, and without any significant difference by sex, class or province, individual adhesion to the role of the condom as a means of prevention against AIDS progressed from 64% to 95%. The students questioned wanted AIDS prevention information to be better integrated into their curriculum and, in particular, they wanted educational activities in this area in their school, either by their teachers or in special information areas. Thus, the 48-page comic book by young Gabonese artists was perceived as a good method of condom education for the young (75%) and as an excellent method for inducing awareness about it among them (89%). The book's contents had been absorbed, and the students found that the stories and the message were well matched. Moreover, the extension of the readership beyond the initial distribution at the first evaluation (7.5 readers reported per copy) showed that the messages in the book spread well beyond the student group.

Transgender-inclusive measures of sex/gender for population surveys: Mixed-methods evaluation and recommendations

PubMed Central

Bauer, Greta R.; Braimoh, Jessica; Scheim, Ayden I.; Dharma, Christoffer

2017-01-01

Given that an estimated 0.6% of the U.S. population is transgender (trans) and that large health disparities for this population have been documented, government and research organizations are increasingly expanding measures of sex/gender to be trans inclusive. Options suggested for trans community surveys, such as expansive check-all-that-apply gender identity lists and write-in options that offer maximum flexibility, are generally not appropriate for broad population surveys. These require limited questions and a small number of categories for analysis. Limited evaluation has been undertaken of trans-inclusive population survey measures for sex/gender, including those currently in use. Using an internet survey and follow-up of 311 participants, and cognitive interviews from a maximum-diversity sub-sample (n = 79), we conducted a mixed-methods evaluation of two existing measures: a two-step question developed in the United States and a multidimensional measure developed in Canada. We found very low levels of item missingness, and no indicators of confusion on the part of cisgender (non-trans) participants for both measures. However, a majority of interview participants indicated problems with each question item set. Agreement between the two measures in assessment of gender identity was very high (K = 0.9081), but gender identity was a poor proxy for other dimensions of sex or gender among trans participants. Issues to inform measure development or adaptation that emerged from analysis included dimensions of sex/gender measured, whether non-binary identities were trans, Indigenous and cultural identities, proxy reporting, temporality concerns, and the inability of a single item to provide a valid measure of sex/gender. Based on this evaluation, we recommend that population surveys meant for multi-purpose analysis consider a new Multidimensional Sex/Gender Measure for testing that includes three simple items (one asked only of a small sub-group) to assess gender identity and lived gender, with optional additions. We provide considerations for adaptation of this measure to different contexts. PMID:28542498

Medical Literature Evaluation Education at US Schools of Pharmacy

PubMed Central

Phillips, Jennifer; Demaris, Kendra

2016-01-01

Objective. To determine how medical literature evaluation (MLE) is being taught across the United States and to summarize methods for teaching and assessing MLE. Methods. An 18-question survey was administered to faculty members whose primary responsibility was teaching MLE at schools and colleges of pharmacy. Results. Responses were received from 90 (71%) US schools of pharmacy. The most common method of integrating MLE into the curriculum was as a stand-alone course (49%). The most common placement was during the second professional year (43%) or integrated throughout the curriculum (25%). The majority (77%) of schools used a team-based approach. The use of active-learning strategies was common as was the use of multiple methods of evaluation. Responses varied regarding what role the course director played in incorporating MLE into advanced pharmacy practice experiences (APPEs). Conclusion. There is a trend toward incorporating MLE education components throughout the pre-APPE curriculum and placement of literature review/evaluation exercises into therapeutics practice skills laboratories to help students see how this skill integrates into other patient care skills. Several pre-APPE educational standards for MLE education exist, including journal club activities, a team-based approach to teaching and evaluation, and use of active-learning techniques. PMID:26941431

Best methods for evaluating educational impact: a comparison of the efficacy of commonly used measures of library instruction.

PubMed

Schilling, Katherine; Applegate, Rachel

2012-10-01

Libraries are increasingly called upon to demonstrate student learning outcomes and the tangible benefits of library educational programs. This study reviewed and compared the efficacy of traditionally used measures for assessing library instruction, examining the benefits and drawbacks of assessment measures and exploring the extent to which knowledge, attitudes, and behaviors actually paralleled demonstrated skill levels. An overview of recent literature on the evaluation of information literacy education addressed these questions: (1) What evaluation measures are commonly used for evaluating library instruction? (2) What are the pros and cons of popular evaluation measures? (3) What are the relationships between measures of skills versus measures of attitudes and behavior? Research outcomes were used to identify relationships between measures of attitudes, behaviors, and skills, which are typically gathered via attitudinal surveys, written skills tests, or graded exercises. Results provide useful information about the efficacy of instructional evaluation methods, including showing significant disparities between attitudes, skills, and information usage behaviors. This information can be used by librarians to implement the most appropriate evaluation methods for measuring important variables that accurately demonstrate students' attitudes, behaviors, or skills.

Longitudinal construct validity: establishment of clinical meaning in patient evaluative instruments.

PubMed

Liang, M H

2000-09-01

Although widely used and reported in research for the evaluation of groups, measures of health status and health-related quality of life have had little application in clinical practice for the assessment of individual patients. One of the principal barriers is the demonstration that these measures add clinically significant information to measures of function or symptoms alone. Here, we review the methods for evaluation of construct validity in longitudinal studies and make recommendations for nomenclature, reporting of study results, and future research agenda. Analytical review. The terms "sensitivity" and "responsiveness" have been used interchangeably, and there are few studies that evaluate the extent to which health status or health-related quality-of life measures capture clinically important changes ("responsiveness"). Current methods of evaluating responsiveness are not standardized or evaluated. Approaches for the assessment of a clinically significant or meaningful change are described; rather than normative information, however, standardized transition questions are proposed. They would be reported routinely and as separate axes of description to capture individual perceptions. Research in methods to assess the subject's evaluation of the importance and magnitude of a measured change are critical if health status and health-related quality-of-life measures are to have an impact on patient care.

How well will you FIT? Use of a modified MMI to assess applicants' compatibility with an emergency medicine residency program.

PubMed

Min, Alice A; Leetch, Aaron; Nuño, Tomas; Fiorello, Albert B

2016-01-01

Purpose Emergency medicine residency programs have evaluated the use of Multiple Mini Interviews (MMIs) for applicants. The authors developed an MMI-style method called the Fast Interview Track (FIT) to predict an applicant's 'fit' within an individual residency program. Methods Applicants meet with up to five residents and are asked one question by each. Residents score the applicant using a Likert scale from 1 to 5 on two questions: 'How well does the applicant think on his/her feet?' and 'How well do you think the applicant will fit in here?'. To assess how well these questions predicted a resident's 'fit', current residents scored fellow residents on these same questions. These scores were compared with the residents' interview FIT scores. A postmatch survey of applicants who did not match at this program solicited applicants' attitudes toward the FIT sessions. Results Among the junior class, the correlation between interview and current scores was significant for question 1 (rho=0.5192 [p=0.03]) and question 2 (rho=0.5753 [p=0.01]). Among seniors, Spearman's rho was statistically significant for question 2, though not statistically significant for question 1. The chi-square measure of high scores (4-5) versus low scores (1-3) found a statistically significant association between interview and current scores for interns and juniors. Of the 29 responses to the postmatch survey, 16 (55%) felt FIT sessions provided a good sense of the program's personality and only 6 (21%) disagreed. Nine (31%) felt FIT sessions positively impacted our program's ranking and 11 (38%) were 'Neutral'. Only two (7%) reported that FIT sessions negatively impacted their ranking of our program. Conclusions FIT provided program leadership with a sense of an applicant's 'fit' within this program. Interview day scores correlated with scores received during residency. Most applicants report a positive experience with FIT sessions. FIT provides a useful tool to recruit applicants who fit with the residency program.

Assessing the effects of employee assistance programs: a review of employee assistance program evaluations.

PubMed

Colantonio, A

1989-01-01

Employee assistance programs have grown at a dramatic rate, yet the effectiveness of these programs has been called into question. The purpose of this paper was to assess the effectiveness of employee assistance programs (EAPs) by reviewing recently published EAP evaluations. All studies evaluating EAPs published since 1975 from peer-reviewed journals in the English language were included in this analysis. Each of the articles was assessed in the following areas: (a) program description (subjects, setting, type of intervention, format), (b) evaluation design (research design, variables measured, operational methods), and (c) program outcomes. Results indicate numerous methodological and conceptual weaknesses and issues. These weaknesses included lack of controlled research designs and short time lags between pre- and post-test measures. Other problems identified are missing information regarding subjects, type of intervention, how variables are measured (operational methods), and reliability and validity of evaluation instruments. Due to the aforementioned weaknesses, positive outcomes could not be supported. Recommendations are made for future EAP evaluations.

Assessing the effects of employee assistance programs: a review of employee assistance program evaluations.

PubMed Central

Colantonio, A.

1989-01-01

Employee assistance programs have grown at a dramatic rate, yet the effectiveness of these programs has been called into question. The purpose of this paper was to assess the effectiveness of employee assistance programs (EAPs) by reviewing recently published EAP evaluations. All studies evaluating EAPs published since 1975 from peer-reviewed journals in the English language were included in this analysis. Each of the articles was assessed in the following areas: (a) program description (subjects, setting, type of intervention, format), (b) evaluation design (research design, variables measured, operational methods), and (c) program outcomes. Results indicate numerous methodological and conceptual weaknesses and issues. These weaknesses included lack of controlled research designs and short time lags between pre- and post-test measures. Other problems identified are missing information regarding subjects, type of intervention, how variables are measured (operational methods), and reliability and validity of evaluation instruments. Due to the aforementioned weaknesses, positive outcomes could not be supported. Recommendations are made for future EAP evaluations. PMID:2728498

How Do SPJC Administrative and Support Systems and Procedures Need To Change in Order To Overcome Organizational Obstacles to E-Learning Access? Project Eagle Evaluation Question #2. Benchmarking St. Petersburg Junior College: A Report to Leadership.

ERIC Educational Resources Information Center

Burkhart, Joyce

St. Petersburg Junior College (SPJC), Florida, identified critical issues in e-learning practices and posed six questions in order to formulate an evaluation process. SPJC considered one question per quarter for 18 months. The questions were reviewed using the following steps: (1) examine best e-learning practices related to that question, using…

The Method of Manufacturing Nonmetallic Test-Blocks on Different Sensitivity Classes

NASA Astrophysics Data System (ADS)

Kalinichenko, N. P.; Kalinichenko, A. N.; Lobanova, I. S.; Zaitseva, A. A.; Loboda, E. L.

2016-01-01

Nowadays in our modern world there is a vital question of quality control of details made from nonmetallic materials due to their wide spreading. Nondestructive penetrant testing is effective, and in some cases it is the only possible method of accidents prevention at high- risk sites. A brief review of check sample necessary for quality evaluation of penetrant materials is considered. There was offered a way of making agents for quality of penetrant materials testing according to different liquid penetrant testing sensibility classes.

Development of the National Healthy Sleep Awareness Project Sleep Health Surveillance Questions

PubMed Central

Morgenthaler, Timothy I.; Croft, Janet B.; Dort, Leslie C.; Loeding, Lauren D.; Mullington, Janet M.; Thomas, Sherene M.

2015-01-01

Objectives: For the first time ever, as emphasized by inclusion in the Healthy People 2020 goals, sleep health is an emphasis of national health aims. The National Healthy Sleep Awareness Project (NHSAP) was tasked to propose questions for inclusion in the next Behavioral Risk Factor Surveillance System (BRFSS), a survey that includes a number of questions that target behaviors thought to impact health, as a means to measure community sleep health. The total number of questions could not exceed five, and had to include an assessment of the risk for obstructive sleep apnea (OSA). Methods: An appointed workgroup met via teleconference and face-to-face venues to develop an inventory of published survey questions being used to identify sleep health, to develop a framework on which to analyze the strengths and weaknesses of current survey questions concerning sleep, and to develop recommendations for sleep health and disease surveillance questions going forward. Results: The recommendation was to focus on certain existing BRFSS questions pertaining to sleep duration, quality, satisfaction, daytime alertness, and to add to these other BRFSS existing questions to make a modified STOP-BANG questionnaire (minus the N for neck circumference) to assess for risk of OSA. Conclusions: Sleep health is an important dimension of health that has previously received less attention in national health surveys. We believe that 5 questions recommended for the upcoming BRFSS question banks will assist as important measures of sleep health, and may help to evaluate the effectiveness of interventions to improve sleep health in our nation. Citation: Morgenthaler TI, Croft JB, Dort LC, Loeding LD, Mullington JM, Thomas SM. Development of the National Healthy Sleep Awareness Project sleep health surveillance questions. J Clin Sleep Med 2015;11(9):1057–1062. PMID:26235156

Methodological Issues in Questionnaire Design.

PubMed

Song, Youngshin; Son, Youn Jung; Oh, Doonam

2015-06-01

The process of designing a questionnaire is complicated. Many questionnaires on nursing phenomena have been developed and used by nursing researchers. The purpose of this paper was to discuss questionnaire design and factors that should be considered when using existing scales. Methodological issues were discussed, such as factors in the design of questions, steps in developing questionnaires, wording and formatting methods for items, and administrations methods. How to use existing scales, how to facilitate cultural adaptation, and how to prevent socially desirable responding were discussed. Moreover, the triangulation method in questionnaire development was introduced. Steps were recommended for designing questions such as appropriately operationalizing key concepts for the target population, clearly formatting response options, generating items and confirming final items through face or content validity, sufficiently piloting the questionnaire using item analysis, demonstrating reliability and validity, finalizing the scale, and training the administrator. Psychometric properties and cultural equivalence should be evaluated prior to administration when using an existing questionnaire and performing cultural adaptation. In the context of well-defined nursing phenomena, logical and systematic methods will contribute to the development of simple and precise questionnaires.

Lunar-base construction equipment and methods evaluation

NASA Technical Reports Server (NTRS)

Boles, Walter W.; Ashley, David B.; Tucker, Richard L.

1993-01-01

A process for evaluating lunar-base construction equipment and methods concepts is presented. The process is driven by the need for more quantitative, systematic, and logical methods for assessing further research and development requirements in an area where uncertainties are high, dependence upon terrestrial heuristics is questionable, and quantitative methods are seldom applied. Decision theory concepts are used in determining the value of accurate information and the process is structured as a construction-equipment-and-methods selection methodology. Total construction-related, earth-launch mass is the measure of merit chosen for mathematical modeling purposes. The work is based upon the scope of the lunar base as described in the National Aeronautics and Space Administration's Office of Exploration's 'Exploration Studies Technical Report, FY 1989 Status'. Nine sets of conceptually designed construction equipment are selected as alternative concepts. It is concluded that the evaluation process is well suited for assisting in the establishment of research agendas in an approach that is first broad, with a low level of detail, followed by more-detailed investigations into areas that are identified as critical due to high degrees of uncertainty and sensitivity.

Evaluation of the level of difficulty of patient cases for veterinary problem-solving examination: a preliminary comparison of three taxonomies of learning.

PubMed

Koskinen, Heli

2007-01-01

An important issue that has received insufficient attention in the use of problem-based learning in the medical curriculum is the mode of assessing the level of difficulty of patient cases. In the present study, the level of difficulty of case-based questions in a veterinary degree final examination in reproduction was evaluated. First, cognitive taxonomies were evaluated to clarify whether qualitative methods such as Bloom's taxonomy, the Structure of the Observed Learning Outcome (SOLO) taxonomy, and the Amsterdam Clinical Challenge Scale (ACCS) differed from each other as evaluation tools for problem-based cases. Using these taxonomies, 30 case-based questions from the final examination in reproduction in the Helsinki veterinary program were initially evaluated to determine which one was best suited to the evaluation of the difficulty of cases. In follow-up, the same cases were also evaluated by an experienced veterinary instructor in reproduction, with the aim of gaining insight into using these approaches to evaluating difficulty. It would appear, from this preliminary assessment, that the SOLO taxonomy may be the most suitable for evaluating the difficulty of patient cases, since the instructor's quality rating resembled more closely the SOLO than the Bloom taxonomy or the ACCS. It is to be emphasized that the purpose of this study was to provide a preliminary evaluation of possible approaches that might be used to assess patient-case difficulty. Resolving all issues will require a greater number of evaluations of all components.

Intimate partner violence (IPV): The validity of an IPV screening instrument utilized among pregnant women in Tanzania and Vietnam

PubMed Central

Rasch, Vibeke; Van, Toan Ngo; Nguyen, Hanh Thi Thuy; Manongi, Rachel; Mushi, Declare; Meyrowitsch, Dan W.; Gammeltoft, Tine; Wu, Chun Sen

2018-01-01

Background Intimate partner violence (IPV) is a global problem that affects one-third of all women. The present study aims to develop and determine the validity of a screening instrument for the detection of IPV in pregnant women in Tanzania and Vietnam and to determine the minimum number of questions needed to identify IPV. Method An IPV screening instrument based on eight questions was tested on 1,116 Tanzanian and 1,309 Vietnamese women who attended antenatal care before 24 gestational weeks. The women were re-interviewed during their 30th-34th gestational week where the World Health Organization (WHO) IPV questionnaire was used as the gold standard. In all, 255 combinations of eight different questions were first tested on the Tanzanian study population where sensitivity, specificity, positive predictive value, negative predictive value and accuracy were calculated. In the evaluation of the performance of the question combinations, different IPV types and the frequency of abusive acts were considered. The question combinations that performed best in Tanzania were subsequently evaluated in the Vietnamese study population. Results In Tanzania, a combination of three selected questions including one question on emotional IPV, one on physical IPV and one on sexual IPV was found to be most effective in identifying women who are exposed to at least one type of IPV during pregnancy (sensitivity = .80; specificity = .74). The performance of the identified combination was slightly less effective in Vietnam (sensitivity = .74; specificity = .68). Focusing on different IPV types, the best performance was found for exposure to physical IPV in both Tanzania (sensitivity = .93; specificity = .70) and Vietnam (sensitivity = .96; specificity = .55). In both countries, the sensitivity increased with the frequency of abuse whereas the specificity decreased. Conclusion By asking pregnant women three simple questions we were able to identify women who were exposed to IPV during pregnancy in two different countries. The question combination performed best in assessing physical IPV where it identified 93% and 96% of Vietnamese and Tanzanian women, respectively, who were exposed to physical IPV. PMID:29389954

The GRADE approach for assessing new technologies as applied to apheresis devices in ulcerative colitis

PubMed Central

2010-01-01

Background In the last few years, a new non-pharmacological treatment, termed apheresis, has been developed to lessen the burden of ulcerative colitis (UC). Several methods can be used to establish treatment recommendations, but over the last decade an informal collaboration group of guideline developers, methodologists, and clinicians has developed a more sensible and transparent approach known as the Grading of Recommendations, Assessment, Development and Evaluation (GRADE). GRADE has mainly been used in clinical practice guidelines and systematic reviews. The aim of the present study is to describe the use of this approach in the development of recommendations for a new health technology, and to analyse the strengths, weaknesses, opportunities, and threats found when doing so. Methods A systematic review of the use of apheresis for UC treatment was performed in June 2004 and updated in May 2008. Two related clinical questions were selected, the outcomes of interest defined, and the quality of the evidence assessed. Finally, the overall quality of each question was taken into account to formulate recommendations following the GRADE approach. To evaluate this experience, a SWOT (strengths, weaknesses, opportunities and threats) analysis was performed to enable a comparison with our previous experience with the SIGN (Scottish Intercollegiate Guidelines Network) method. Results Application of the GRADE approach allowed recommendations to be formulated and the method to be clarified and made more explicit and transparent. Two weak recommendations were proposed to answer to the formulated questions. Some challenges, such as the limited number of studies found for the new technology and the difficulties encountered when searching for the results for the selected outcomes, none of which are specific to GRADE, were identified. GRADE was considered to be a more time-consuming method, although it has the advantage of taking into account patient values when defining and grading the relevant outcomes, thereby avoiding any influence from literature precedents, which could be considered to be a strength of this method. Conclusions The GRADE approach could be appropriate for making the recommendation development process for Health Technology Assessment (HTA) reports more explicit, especially with regard to new technologies. PMID:20553616

Evaluation of Modified Essay Questions (MEQ) and Multiple Choice Questions (MCQ) as a tool for Assessing the Cognitive Skills of Undergraduate Medical Students

PubMed Central

Khan, Moeen-uz-Zafar; Aljarallah, Badr Muhammad

2011-01-01

Objectives: Developing and testing the cognitive skills and abstract thinking of undergraduate medical students are the main objectives of problem based learning. Modified Essay Questions (MEQ) and Multiple Choice Questions (MCQ) may both be designed to test these skills. The objectives of this study were to assess the effectiveness of both forms of questions in testing the different levels of the cognitive skills of undergraduate medical students and to detect any item writing flaws in the questions. Methods: A total of 50 MEQs and 50 MCQs were evaluated. These questions were chosen randomly from various examinations given to different batches of undergraduate medical students taking course MED 411–412 at the Department of Medicine, Qassim University from the years 2005 to 2009. The effectiveness of the questions was determined by two assessors and was defined by the question’s ability to measure higher cognitive skills, as determined by modified Bloom’s taxonomy, and its quality as determined by the presence of item writing flaws. ‘SPSS15’ and ‘Medcalc’ programs were used to tabulate and analyze the data. Results: The percentage of questions testing the level III (problem solving) cognitive skills of the students was 40% for MEQs and 60% for the MCQs; the remaining questions merely assessed the recall and comprehension. No significant difference was found between MEQ and MCQ in relation to the type of questions (recall; comprehension or problem solving x2 = 5.3, p = 0.07).The agreement between the two assessors was quite high in case of MCQ (kappa=0.609; SE 0.093; 95%CI 0.426 – 0.792) but lower in case of MEQ (kappa=0.195; SE 0.073; 95%CI 0.052 – 0.338). 16% of the MEQs and 12% of the MCQs had item writing flaws. Conclusion: A well constructed MCQ is superior to MEQ in testing the higher cognitive skills of undergraduate medical students in a problem based learning setup. Constructing an MEQ for assessing the cognitive skills of a student is not a simple task and is more frequently associated with item writing flaws. PMID:22489228

Issues in the incorporation of economic perspectives and evidence into Cochrane reviews

PubMed Central

2013-01-01

Background Methods for systematic reviews of the effects of health interventions have focused mainly on addressing the question of 'What works?’ or 'Is this intervention effective in achieving one or more specific outcomes?’ Addressing the question 'Is it worth it given the resources available?’ has received less attention. This latter question can be addressed by applying an economic lens to the systematic review process. This paper reflects on the value and desire for the consideration by end users for coverage of an economic perspective in a Cochrane review and outlines two potential approaches and future directions. Methods Two frameworks to guide review authors who are seeking to include an economic perspective are outlined. The first involves conducting a full systematic review of economic evaluations that is integrated into a review of intervention effects. The second involves developing a brief economic commentary. The two approaches share a set of common stages but allow the tailoring of the economic component of the Cochrane review to the skills and resources available to the review team. Results The number of studies using the methods outlined in the paper is limited, and further examples are needed both to explore the value of these approaches and to further develop them. The rate of progress will hinge on the organisational leadership, capacity and resources available to the CCEMG, author teams and other Cochrane entities. Particular methodological challenges to overcome relate to understanding the key economic trade-offs and casual relationships for a given decision problem and informing the development of evaluations designed to support local decision-makers. Conclusions Methods for incorporating economic perspectives and evidence into Cochrane intervention reviews are established. Their role is not to provide a precise estimate of 'cost-effectiveness’ but rather to help end-users of Cochrane reviews to determine the implications of the economic components of reviews for their own specific decisions. PMID:24050504

Which Form of Medical Training is the Best in Improving Interns’ knowledge Related to Advanced Cardiac Life Support Drugs Pharmacology? An Educational Analytical Intervention Study Between Electronic Learning and Lecture-Based Education

PubMed Central

Khoshbaten, Manouchehr; Soleimanpour, Hassan; Ala, Alireza; Shams Vahdati, Samad; Ebrahimian, Kimia; Safari, Saeid; Golzari, Samad EJ; Salek Ranjbarzadeh, Fariba; Mehdizadeh Esfanjani, Robab

2014-01-01

Background: Conventional educational systems seem to be improper throughout the cardiopulmonary resuscitation (CPR) teaching process. The most common causes of failed resuscitation are unfamiliarity with cardiopulmonary resuscitation algorithms, poor performance of leader of the CPR team and lack of skilled personnel, coordination among members during resuscitation, and responsibility of staff. Objectives: Electronic learning, as a new educational method is controversial issue in medical education for improving physicians’ practical knowledge and it is inevitable that further research on its effectiveness should be done. Materials and Methods: The present study is a prospective, pre- and post-educational, cross-sectional research, in which 84 interns were randomly divided into two groups. pre- and post- educational interventions that took place in the Department of Emergency Medicine, interns were evaluated by 21 multiple choice questions related to American Heart Association guidelineson cardiopulmonary resuscitation drugs. Questions were assessed in terms of routes for CPR drugs administration, CPR drug dosage forms, clinical judgment and appropriate CPR drug administration, and the alternative drugs in emergency situations. Data were analyzed by generalized estimating equations regression models and P < 0.05 was considered statistically significant. Results: Evaluating the effectiveness of both educational methods revealed that the mean answering score for 21 questions before education was 7.5 ± 2.6 and no significant difference was observed in groups (P = 0.55). However, after education, the average scores significantly increased to 11.0 ± 3.9 (P < 0.001). Electronic learning method was not associated with considerable increase in the knowledge of interns in this group compared with the lecture-based group (P = 0.49). Conclusions: No significant differences were observed between electronic learning and lecture-based education in improving interns’ knowledge of CPR drugs. PMID:24719802

Observer Bias: An Interaction of Temperament Traits with Biases in the Semantic Perception of Lexical Material

PubMed Central

Trofimova, Ira

2014-01-01

The lexical approach is a method in differential psychology that uses people's estimations of verbal descriptors of human behavior in order to derive the structure of human individuality. The validity of the assumptions of this method about the objectivity of people's estimations is rarely questioned. Meanwhile the social nature of language and the presence of emotionality biases in cognition are well-recognized in psychology. A question remains, however, as to whether such an emotionality-capacities bias is strong enough to affect semantic perception of verbal material. For the lexical approach to be valid as a method of scientific investigations, such biases should not exist in semantic perception of the verbal material that is used by this approach. This article reports on two studies investigating differences between groups contrasted by 12 temperament traits (i.e. by energetic and other capacities, as well as emotionality) in the semantic perception of very general verbal material. Both studies contrasted the groups by a variety of capacities: endurance, lability and emotionality separately in physical, social-verbal and mental aspects of activities. Hypotheses of “background emotionality” and a “projection through capacities” were supported. Non-evaluative criteria for categorization (related to complexity, organization, stability and probability of occurrence of objects) followed the polarity of evaluative criteria, and did not show independence from this polarity. Participants with stronger physical or social endurance gave significantly more positive ratings to a variety of concepts, and participants with faster physical tempo gave more positive ratings to timing-related concepts. The results suggest that people's estimations of lexical material related to human behavior have emotionality, language- and dynamical capacities-related biases and therefore are unreliable. This questions the validity of the lexical approach as a method for the objective study of stable individual differences. PMID:24475048

Does An Education Seminar Intervention Improve the Parents’ Knowledge on Vaccination? Evidence from Yiwu, East China

PubMed Central

Hu, Yu

2015-01-01

Background: caregivers’ knowledge on vaccination is an important impact factor for their children’s vaccination status. The aims of this study were to evaluate the caregivers’ knowledge of vaccination, and to assess effectiveness of a health education seminar for improving caregivers’ knowledge on immunization. Methods: pre- and post-assessment design was adopted for a single group to evaluate the effectiveness of the health education seminar on vaccination. The seminar consisted of a lecture using simple understandable language. Improvements in total knowledge score before and after the seminar were assessed using a validated questionnaire that included ten questions. Description analysis and non-parametric tests were applied to evaluate and compare the vaccination knowledge level before and after the seminar. Results: 378 caregivers participated in this study. The majority were mothers. Of the ten questions, the correct response rates had significantly increased for nine questions after the education seminar. The mean total score of the assessment before the seminar was 5.2 ± 1.2 while that was 8.4 ± 0.9 for the assessment after the seminar, with a significant increase of 3.18 points. Conclusion: a short education seminar designed for caregivers had a remarkable effect on their vaccination knowledge. Health education on vaccination targeting migrant caregivers, caregivers with lower education level or household income, and employed caregivers are needed in future. PMID:25811770

Transcultural adaptation and validation of the Celiac Dietary Adherence Test. A simple questionnaire to measure adherence to a gluten-free diet.

PubMed

Fueyo-Díaz, Ricardo; Gascón-Santos, Santiago; Asensio-Martínez, Ángela; Sánchez-Calavera, María Antonia; Magallón-Botaya, Rosa

2016-03-01

A gluten-free diet is to date the only treatment available to celiac disease sufferers. However, systematic reviews indicate that, depending on the method of evaluation used, only 42% to 91% of patients adhere to the diet strictly. Transculturally adapted tools that evaluate adherence beyond simple self-informed questions or invasive analyses are, therefore, of importance. The aim is to obtain a Spanish transcultural adaption and validation of Leffler's Celiac Dietary Adherence Test. A two-stage observational transversal study: translation and back translation by four qualified translators followed by a validation stage in which the questionnaire was administered to 306 celiac disease patients aged between 12 and 72 years and resident in Aragon. Factorial structure, criteria validity and internal consistency were evaluated. The Spanish version maintained the 7 items in a 3-factor structure. Reliability was very high in all the questions answered and the floor and ceiling effects were very low (4.3% and 1%, respectively). The Spearman correlation with the self-efficacy and life quality scales and the self-informed question were statistically significant (p < 0.01). According to the questionnaire criteria, adherence was 72.3%. The Spanish version of the Celiac Dietary Adherence Test shows appropriate psychometric properties and is, therefore, suitable for studying adherence to a gluten-free diet in clinical and research environments.

Evaluation of five full-text drug databases by pharmacy students, faculty, and librarians: do the groups agree?

PubMed Central

Kupferberg, Natalie; Hartel, Lynda Jones

2004-01-01

Objectives: The purpose of this study is to assess the usefulness of five full-text drug databases as evaluated by medical librarians, pharmacy faculty, and pharmacy students at an academic health center. Study findings and recommendations are offered as guidance to librarians responsible for purchasing decisions. Methods: Four pharmacy students, four pharmacy faculty members, and four medical librarians answered ten drug information questions using the databases AHFS Drug Information (STAT!Ref); DRUGDEX (Micromedex); eFacts (Drug Facts and Comparisons); Lexi-Drugs Online (Lexi-Comp); and the PDR Electronic Library (Micromedex). Participants noted whether each database contained answers to the questions and evaluated each database on ease of navigation, screen readability, overall satisfaction, and product recommendation. Results: While each study group found that DRUGDEX provided the most direct answers to the ten questions, faculty members gave Lexi-Drugs the highest overall rating. Students favored eFacts. The faculty and students found the PDR least useful. Librarians ranked DRUGDEX the highest and AHFS the lowest. The comments of pharmacy faculty and students show that these groups preferred concise, easy-to-use sources; librarians focused on the comprehensiveness, layout, and supporting references of the databases. Conclusion: This study demonstrates the importance of consulting with primary clientele before purchasing databases. Although there are many online drug databases to consider, present findings offer strong support for eFacts, Lexi-Drugs, and DRUGDEX. PMID:14762464

Evaluation of an interactive, case-based review session in teaching medical microbiology

PubMed Central

Blewett, Earl L; Kisamore, Jennifer L

2009-01-01

Background Oklahoma State University-Center for Health Sciences (OSU-CHS) has replaced its microbiology wet laboratory with a variety of tutorials including a case-based interactive session called Microbial Jeopardy!. The question remains whether the time spent by students and faculty in the interactive case-based tutorial is worthwhile? This study was designed to address this question by analyzing both student performance data and assessing students' perceptions regarding the tutorial. Methods Both quantitative and qualitative data were used in the current study. Part One of the study involved assessing student performance using archival records of seven case-based exam questions used in the 2004, 2005, 2006, and 2007 OSU-CHS Medical Microbiology course. Two sample t-tests for proportions were used to test for significant differences related to tutorial usage. Part Two used both quantitative and qualitative means to assess student's perceptions of the Microbial Jeopardy! session. First, a retrospective survey was administered to students who were enrolled in Medical Microbiology in 2006 or 2007. Second, responses to open-ended items from the 2008 course evaluations were reviewed for comments regarding the Microbial Jeopardy! session. Results Both student performance and student perception data support continued use of the tutorials. Quantitative and qualitative data converge to suggest that students like and learn from the interactive, case-based session. Conclusion The case-based tutorial appears to improve student performance on case-based exam questions. Additionally, students perceived the tutorial as helpful in preparing for exam questions and reviewing the course material. The time commitment for use of the case-based tutorial appears to be justified. PMID:19712473

48 CFR 315.306 - Exchanges with offerors after receipt of proposals.

Code of Federal Regulations, 2010 CFR

2010-10-01

... establishment of the competitive range. The Project Officer or technical evaluation panel shall develop technical questions as part of the technical evaluation report. The questions shall disclose the ambiguities... practicable, all questions and answers shall be in writing. The Contracting Officer shall give each offeror in...

An Application of Digital Portfolio with the Peer, Self and Instructor Assessments in Art Education

ERIC Educational Resources Information Center

Dikici, Ayhan

2009-01-01

Problem Statement: In art education, one of the most difficult tasks is to evaluate the artistic works of students. Portfolio assessment is a method of assessment that is commonly used in artistic education. The major research question investigated in the study is: "What are the advantages and disadvantages of digital portfolio in an art…

Extensions of Existing Methods for Use with a New Class of Experimental Designs Useful when There Is Treatment Effect Contamination

ERIC Educational Resources Information Center

Rhoads, Christopher

2011-01-01

Researchers planning a randomized field trial to evaluate the effectiveness of an educational intervention often face the following dilemma. They plan to recruit schools to participate in their study. The question is, "Should the researchers randomly assign individuals (either students or teachers, depending on the intervention) within schools to…

Scientific Reputation and "The Golden Standards": Quality Management System Impact and the Teaching-Research Nexus

ERIC Educational Resources Information Center

Moraru, Luminita

2012-01-01

In the academic environment the reputation is linked to research performance. However, the questions "what reputation actually is, how it is formed and which are the best ways to measure it?" still remain and they raise tension and confusion between academics. The evaluation methods tend to over emphasise research and fail to address the…

Content-Related Issues Pertaining to Teaching Statistics: Making Decisions about Educational Objectives in Statistics Courses.

ERIC Educational Resources Information Center

Bliss, Leonard B.; Tashakkori, Abbas

This paper discusses the objectives that would be appropriate for statistics classes for students who are not majoring in statistics, evaluation, or quantitative research design. These "non-majors" should be able to choose appropriate analytical methods for specific sets of data based on the research question and the nature of the data, and they…

Evaluating the Impact of Professional Development and Curricular Implementation on Student Mathematics Achievement: A Mixed Methods Study

ERIC Educational Resources Information Center

Krupa, Erin Elizabeth

2011-01-01

In this era of high-stakes testing and accountability, curricula are viewed as catalysts to improve high school students' mathematics performances and a critical question is whether single subject or integrated curricula produce stronger student outcomes. This study was designed to investigate the effects of an integrated reform-based curriculum,…

Reading Intervention with K-4 Struggling Readers: The Effect of Using Minimally Trained Tutors in One Inner-City Public School

ERIC Educational Resources Information Center

Lipsey, Tammy

2010-01-01

The purpose of this quasi-experimental study was to evaluate the effectiveness of an early reading intervention method designed for students who were in need of extra help in reading. The early reading intervention (tutoring) was conducted for Kindergarten through fourth grade students. Specifically, this dissertation investigated the question: To…

Small Groups, Significant Impact: A Review of Peer-Led Team Learning Research with Implications for STEM Education Researchers and Faculty

ERIC Educational Resources Information Center

Wilson, Sarah Beth; Varma-Nelson, Pratibha

2016-01-01

Peer-led team learning (PLTL) research has expanded from its roots in program evaluation of student success measures in Workshop Chemistry to a spectrum of research questions and qualitative, quantitative, and mixed methods study approaches. In order to develop recommendations for PLTL research and propose best practices for faculty who will…

A Decision Tool to Evaluate Budgeting Methodologies for Estimating Facility Recapitalization Requirements

DTIC Science & Technology

2008-03-01

1 . Maintenance Practices Influence Service Life .......................................................... 11 2 . Expectations or Standards May...BRB, 1991, p. 1 - 2 ) Additionally, public sector organizations typically have a larger inventory of facilities to maintain, making asset management...questions were answered. 1 . What are the long term causes and effects of under-funding the maintenance of facilities? 2 . What methods currently

Teaching New Words to Children with Poor Existing Vocabulary Knowledge: A Controlled Evaluation of the Definition and Context Methods

ERIC Educational Resources Information Center

Nash, Hannah; Snowling, Margaret

2006-01-01

Background: Children who have poor vocabulary knowledge are at risk of wider language weaknesses and reading comprehension difficulties, which will impact upon their educational achievement. The central question addressed in this paper is how best to teach new vocabulary items to these children. Aims: To investigate the effects of two different…

A Phenomenological Study on the Lived Experience of First and Second Year Teachers in Standards-Based Grading Districts

ERIC Educational Resources Information Center

Battistone, William A., Jr.

2017-01-01

Problem: There is an existing cycle of questionable grading practices at the K-12 level. As a result, districts continue to search for innovative methods of evaluating and reporting student progress. One result of this effort has been the adoption of a standards-based grading approach. Research concerning standards-based grading implementation has…

Does attitude hinder or help selecting evaluation questions?

PubMed

Shams, Behzad; Dehghani, Mostafa

2015-06-01

Positive attitude leads to a more successfully implementation of a change. We investigated the effect of attitudes of stakeholders toward a program on their prioritization of the program components for selecting the key question of a theory-driven evaluation with concept mapping method. During a brainstorming session, stated statements defined the program components. Then they were sorted and rated regarding the importance and feasibility of them. In addition, the attitudes of participants were assessed by a 30 items questionnaire extracted from a pool named as "50 reasons not to change." We determined and compared the consensus points of participants both with and without of considering their attitudes toward the program. The participants were divided into two groups of high (45% - above the mean) and low (55% - below the mean) attitude. Brainstorming discussions generated a pool of almost 120 statements which were subsequently refined to 44 statements. Matching the rating scores between two attitude groups yielded a consensus at a higher priority than the other method. In the concept mapping procedure, it is crucial to reach the consensus with respect to the participants' attitude, rather than the similarity of mean scores of feasibility and importance.

The Application of Intensive Longitudinal Methods to Investigate Change: Stimulating the Field of Applied Family Research.

PubMed

Bamberger, Katharine T

2016-03-01

The use of intensive longitudinal methods (ILM)-rapid in situ assessment at micro timescales-can be overlaid on RCTs and other study designs in applied family research. Particularly, when done as part of a multiple timescale design-in bursts over macro timescales-ILM can advance the study of the mechanisms and effects of family interventions and processes of family change. ILM confers measurement benefits in accurately assessing momentary and variable experiences and captures fine-grained dynamic pictures of time-ordered processes. Thus, ILM allows opportunities to investigate new research questions about intervention effects on within-subject (i.e., within-person, within-family) variability (i.e., dynamic constructs) and about the time-ordered change process that interventions induce in families and family members beginning with the first intervention session. This paper discusses the need and rationale for applying ILM to family intervention evaluation, new research questions that can be addressed with ILM, example research using ILM in the related fields of basic family research and the evaluation of individual-based interventions. Finally, the paper touches on practical challenges and considerations associated with ILM and points readers to resources for the application of ILM.

The Application of Intensive Longitudinal Methods to Investigate Change: Stimulating the Field of Applied Family Research

PubMed Central

Bamberger, Katharine T.

2015-01-01

The use of intensive longitudinal methods (ILM)—rapid in situ assessment at micro timescales—can be overlaid on RCTs and other study designs in applied family research. Especially when done as part of a multiple timescale design—in bursts over macro timescales, ILM can advance the study of the mechanisms and effects of family interventions and processes of family change. ILM confers measurement benefits in accurately assessing momentary and variable experiences and captures fine-grained dynamic pictures of time-ordered processes. Thus, ILM allows opportunities to investigate new research questions about intervention effects on within-subject (i.e., within-person, within-family) variability (i.e., dynamic constructs) and about the time-ordered change process that interventions induce in families and family members beginning with the first intervention session. This paper discusses the need and rationale for applying ILM to intervention evaluation, new research questions that can be addressed with ILM, example research using ILM in the related fields of basic family research and the evaluation of individual-based (rather than family-based) interventions. Finally, the paper touches on practical challenges and considerations associated with ILM and points readers to resources for the application of ILM. PMID:26541560

Using wireless handheld computers to seek information at the point of care: an evaluation by clinicians.

PubMed

Hauser, Susan E; Demner-Fushman, Dina; Jacobs, Joshua L; Humphrey, Susanne M; Ford, Glenn; Thoma, George R

2007-01-01

To evaluate: (1) the effectiveness of wireless handheld computers for online information retrieval in clinical settings; (2) the role of MEDLINE in answering clinical questions raised at the point of care. A prospective single-cohort study: accompanying medical teams on teaching rounds, five internal medicine residents used and evaluated MD on Tap, an application for handheld computers, to seek answers in real time to clinical questions arising at the point of care. All transactions were stored by an intermediate server. Evaluators recorded clinical scenarios and questions, identified MEDLINE citations that answered the questions, and submitted daily and summative reports of their experience. A senior medical librarian corroborated the relevance of the selected citation to each scenario and question. Evaluators answered 68% of 363 background and foreground clinical questions during rounding sessions using a variety of MD on Tap features in an average session length of less than four minutes. The evaluator, the number and quality of query terms, the total number of citations found for a query, and the use of auto-spellcheck significantly contributed to the probability of query success. Handheld computers with Internet access are useful tools for healthcare providers to access MEDLINE in real time. MEDLINE citations can answer specific clinical questions when several medical terms are used to form a query. The MD on Tap application is an effective interface to MEDLINE in clinical settings, allowing clinicians to quickly find relevant citations.

Using Wireless Handheld Computers to Seek Information at the Point of Care: An Evaluation by Clinicians

PubMed Central

Hauser, Susan E.; Demner-Fushman, Dina; Jacobs, Joshua L.; Humphrey, Susanne M.; Ford, Glenn; Thoma, George R.

2007-01-01

Objective To evaluate: (1) the effectiveness of wireless handheld computers for online information retrieval in clinical settings; (2) the role of MEDLINE® in answering clinical questions raised at the point of care. Design A prospective single-cohort study: accompanying medical teams on teaching rounds, five internal medicine residents used and evaluated MD on Tap, an application for handheld computers, to seek answers in real time to clinical questions arising at the point of care. Measurements All transactions were stored by an intermediate server. Evaluators recorded clinical scenarios and questions, identified MEDLINE citations that answered the questions, and submitted daily and summative reports of their experience. A senior medical librarian corroborated the relevance of the selected citation to each scenario and question. Results Evaluators answered 68% of 363 background and foreground clinical questions during rounding sessions using a variety of MD on Tap features in an average session length of less than four minutes. The evaluator, the number and quality of query terms, the total number of citations found for a query, and the use of auto-spellcheck significantly contributed to the probability of query success. Conclusion Handheld computers with Internet access are useful tools for healthcare providers to access MEDLINE in real time. MEDLINE citations can answer specific clinical questions when several medical terms are used to form a query. The MD on Tap application is an effective interface to MEDLINE in clinical settings, allowing clinicians to quickly find relevant citations. PMID:17712085

The application of knowledge synthesis methods in agri-food public health: recent advancements, challenges and opportunities.

PubMed

Young, Ian; Waddell, Lisa; Sanchez, Javier; Wilhelm, Barbara; McEwen, Scott A; Rajić, Andrijana

2014-03-01

Knowledge synthesis refers to the integration of findings from individual research studies on a given topic or question into the global knowledge base. The application of knowledge synthesis methods, particularly systematic reviews and meta-analysis, has increased considerably in the agri-food public health sector over the past decade and this trend is expected to continue. The objectives of our review were: (1) to describe the most promising knowledge synthesis methods and their applicability in agri-food public health, and (2) to summarize the recent advancements, challenges, and opportunities in the use of systematic review and meta-analysis methods in this sector. We performed a structured review of knowledge synthesis literature from various disciplines to address the first objective, and used comprehensive insights and experiences in applying these methods in the agri-food public health sector to inform the second objective. We describe five knowledge synthesis methods that can be used to address various agri-food public health questions or topics under different conditions and contexts. Scoping reviews describe the main characteristics and knowledge gaps in a broad research field and can be used to evaluate opportunities for prioritizing focused questions for related systematic reviews. Structured rapid reviews are streamlined systematic reviews conducted within a short timeframe to inform urgent decision-making. Mixed-method and qualitative reviews synthesize diverse sources of contextual knowledge (e.g. socio-cognitive, economic, and feasibility considerations). Systematic reviews are a structured and transparent method used to summarize and synthesize literature on a clearly-defined question, and meta-analysis is the statistical combination of data from multiple individual studies. We briefly describe and discuss key advancements in the use of systematic reviews and meta-analysis, including: risk-of-bias assessments; an overall quality-of-evidence approach; engagement of stakeholders; Bayesian, multivariate, and network meta-analysis; and synthesis of diagnostic test accuracy studies. We also highlight several challenges and opportunities in the conduct of systematic reviews (e.g. inclusion of grey literature, minimizing language bias, and optimizing search strategies) and meta-analysis (e.g. inclusion of observational studies and approaches to address the insufficient reporting of data and significant heterogeneity). Many of these developments have yet to be comprehensively applied and evaluated in an agri-food public health context, and more research is needed in this area. There is a need to strengthen knowledge synthesis capacity and infrastructure at the regional, national, and international levels in this sector to ensure that the best available knowledge is used to inform future decision-making about agri-food public health issues. Crown Copyright © 2013. Published by Elsevier B.V. All rights reserved.

Towards sound epistemological foundations of statistical methods for high-dimensional biology.

PubMed

Mehta, Tapan; Tanik, Murat; Allison, David B

2004-09-01

A sound epistemological foundation for biological inquiry comes, in part, from application of valid statistical procedures. This tenet is widely appreciated by scientists studying the new realm of high-dimensional biology, or 'omic' research, which involves multiplicity at unprecedented scales. Many papers aimed at the high-dimensional biology community describe the development or application of statistical techniques. The validity of many of these is questionable, and a shared understanding about the epistemological foundations of the statistical methods themselves seems to be lacking. Here we offer a framework in which the epistemological foundation of proposed statistical methods can be evaluated.

HyperCard to SPSS: improving data integrity.

PubMed

Gostel, R

1993-01-01

This article describes a database design that captures responses in a HyperCard stack and moves the data to SPSS for the Macintosh without the need to rekey data. Pregnant women used an interactive computer application with a touch screen to answer questions and receive educational information about fetal alcohol syndrome. A database design was created to capture survey responses through interaction with a computer by a sample of prenatal women during formative evaluation trials. The author does not compare this method of data collection to other methods. This article simply describes the method of data collection as a useful research tool.

Sensitivity Analysis of earth and environmental models: a systematic review to guide scientific advancement

NASA Astrophysics Data System (ADS)

Wagener, Thorsten; Pianosi, Francesca

2016-04-01

Sensitivity Analysis (SA) investigates how the variation in the output of a numerical model can be attributed to variations of its input factors. SA is increasingly being used in earth and environmental modelling for a variety of purposes, including uncertainty assessment, model calibration and diagnostic evaluation, dominant control analysis and robust decision-making. Here we provide some practical advice regarding best practice in SA and discuss important open questions based on a detailed recent review of the existing body of work in SA. Open questions relate to the consideration of input factor interactions, methods for factor mapping and the formal inclusion of discrete factors in SA (for example for model structure comparison). We will analyse these questions using relevant examples and discuss possible ways forward. We aim at stimulating the discussion within the community of SA developers and users regarding the setting of good practices and on defining priorities for future research.

Process-oriented guided inquiry learning strategy enhances students' higher level thinking skills in a pharmaceutical sciences course.

PubMed

Soltis, Robert; Verlinden, Nathan; Kruger, Nicholas; Carroll, Ailey; Trumbo, Tiffany

2015-02-17

To determine if the process-oriented guided inquiry learning (POGIL) teaching strategy improves student performance and engages higher-level thinking skills of first-year pharmacy students in an Introduction to Pharmaceutical Sciences course. Overall examination scores and scores on questions categorized as requiring either higher-level or lower-level thinking skills were compared in the same course taught over 3 years using traditional lecture methods vs the POGIL strategy. Student perceptions of the latter teaching strategy were also evaluated. Overall mean examination scores increased significantly when POGIL was implemented. Performance on questions requiring higher-level thinking skills was significantly higher, whereas performance on questions requiring lower-level thinking skills was unchanged when the POGIL strategy was used. Student feedback on use of this teaching strategy was positive. The use of the POGIL strategy increased student overall performance on examinations, improved higher-level thinking skills, and provided an interactive class setting.

Experiences of using an interactive audience response system in lectures

PubMed Central

Uhari, Matti; Renko, Marjo; Soini, Hannu

2003-01-01

Background Lectures are good for presenting information and providing explanations, but because they lack active participation they have been neglected. Methods Students' experiences were evaluated after exposing them to the use of voting during lectures in their paediatrics course. Questions were delivered to the students taking paediatrics course. Thirty-six students out of the total of 40 (90%) attended the opening lecture, at which the first survey concerning previous experiences of lectures was performed. Thirty-nine students (98%) answered the second series of questions at the end of the paediatrics course. Results Most of the students felt that voting improved their activity during lectures, enhanced their learning, and that it was easier to make questions during lectures than earlier. Conclusions The students gained new, exciting insights much more often during the paediatrics course than before. We as teachers found that voting during lectures could easily overcome some of the obstacles of good lecturing. PMID:14678571

Assigning Resources to Health Care Use for Health Services Research: Options and Consequences

PubMed Central

Fishman, Paul A.; Hornbrook, Mark C.

2013-01-01

Aims Our goals are threefold: 1) to review the leading options for assigning resource coefficients to health services utilization; 2) to discuss the relative advantages of each option; and, 3) provide examples where the research question had marked implications for the choice of which resource measure to employ. Methods Three approaches have been used to establish relative resource weights in health services research: a) direct estimation of production costs through micro-costing or step down allocation methods; b) macro-costing/regression analysis; and, c) standardized resource assignment. We describe each of these methods and provide examples of how the study question drove the choice of resource use measure. Findings All empirical resource-intensity weighting systems contain distortions that limit their universal application. Hence, users must select the weighting system that matches the needs of their specific analysis. All systems require significant data resources and data processing. However, inattention to the distortions contained in a complex resource weighting system may undermine the validity and generalizability of an economic evaluation. Conclusions Direct estimation of production costs are useful for empirical analyses, but they contain distortions that undermine optimal resource allocation decisions. Researchers must ensure that the data being used meets both the study design and the question being addressed. They also should ensure that the choice of resource measure is the best fit for the analysis. Implications for Research and Policy Researchers should consider which of the available measures is the most appropriate for the question being addressed rather than take ‘cost’ or utilization as a variable over which they have no control PMID:19536002

Does Moral Case Deliberation Help Professionals in Care for the Homeless in Dealing with Their Dilemmas? A Mixed-Methods Responsive Study.

PubMed

Spijkerboer, R P; van der Stel, J C; Widdershoven, G A M; Molewijk, A C

2017-03-01

Health care professionals often face moral dilemmas. Not dealing constructively with moral dilemmas can cause moral distress and can negatively affect the quality of care. Little research has been documented with methodologies meant to support professionals in care for the homeless in dealing with their dilemmas. Moral case deliberation (MCD) is a method for systematic reflection on moral dilemmas and is increasingly being used as ethics support for professionals in various health-care domains. This study deals with the question: What is the contribution of MCD in helping professionals in an institution for care for the homeless to deal with their moral dilemmas? A mixed-methods responsive evaluation design was used to answer the research question. Five teams of professionals from a Dutch care institution for the homeless participated in MCD three times. Professionals in care for the homeless value MCD positively. They report that MCD helped them to identify the moral dilemma/question, and that they learned from other people's perspectives while reflecting and deliberating on the values at stake in the dilemma or moral question. They became aware of the moral dimension of moral dilemmas, of related norms and values, of other perspectives, and learned to formulate a moral standpoint. Some experienced the influence of MCD in the way they dealt with moral dilemmas in daily practice. Half of the professionals expect MCD will influence the way they deal with moral dilemmas in the future. Most of them were in favour of further implementation of MCD in their organization.

Do Collaborative Exams Really Promote Learning?

NASA Astrophysics Data System (ADS)

Miller, Scott; James, C. Renee

2018-01-01

Collaborative, two-stage exams are becoming more popular in physics and astronomy courses, and their supposed benefits in terms of collaborative learning have been reported in the field of physics. In a collaborative, two-stage exam, students first complete an exam individually. Once that portion of the exam is over, students then retake all or part of the exam within a group, where they are able to discuss the questions with their peers and arrive at a common answer. While there are a number of papers that discuss the purported benefits of this method from a collaborative point of view, few, if any discuss the actual benefits in terms of student learning. One paper found that when students were presented with previous exam questions a few weeks later, they performed better on questions covered previously in the group portion of the exam compared to similar questions which were tested but not part of the group portion. But, when students were retested on exam questions which were administered earlier, roughly six to seven weeks beforehand, no difference was found in their performance on the two sets of questions.We present preliminary findings comparing student performance levels on multiple sets of exam questions administered to students in an introductory astronomy course where two-stage exams are administered. Questions were administered first in an exam during the course of the semester, then again during a final exam. During the semester exams, one set of questions was also contained within the group portion of the exam, while questions similar in concept and difficulty were not. A comparison of student performance on these two sets of questions are compared to evaluate the usefulness of collaborative exams to promote learning.

Development and Early Piloting of a CanMEDS Competency-Based Feedback Tool for Surgical Grand Rounds.

PubMed

Fahim, Christine; Bhandari, Mohit; Yang, Ilun; Sonnadara, Ranil

2016-01-01

Grand rounds offer an excellent opportunity for the evaluation of medical expertise, and other competencies, such as communication and professionalism. The purpose of this study was to develop a tool that would facilitate the provision of formative feedback for grand rounds to improve learning. The resulting CanMEDS-based evaluation tool was piloted in an academic surgical department. This study employed the use of a 3-phase, qualitatively-focused, embedded mixed methods approach. In Phase 1, an intrinsic case study was conducted to identify preliminary themes. These findings were crystallized using a quantitative survey. Following interpretation of these data, a grand rounds evaluation tool was developed in Phase 2. The tool was piloted in the Phase 3 focus group. This study was piloted at an academic surgical center among members of the Department of Surgery, McMaster University, Ontario, Canada. Purposive sampling was used for this study. A total of n = 7 individuals participated in the Phase 1 interviews, and n = 24 participants completed the Phase 1 survey. Participants included a representative sample of medical students, residents, fellows, and staff. The tool was piloted among n = 19 participants. The proposed evaluation tool contains 13 Likert-scale questions and 2 open-ended questions. The tool outlines specific questions to assess grand rounds presenters within the structure of the 7 CanMEDS competency domains. "Evaluation fatigue" was identified as a major barrier in the willingness to provide effective feedback. Further, a number of factors regarding the preferred content, structure, and format of surgical grand rounds were identified. This pilot study presents a CanMEDS-specific evaluation tool that can be applied to surgical grand rounds. With the increasing adoption of competency-based medical education, comprehensive evaluation of surgical activities is required. This form provides a template for the development of competency-based evaluation tools for medical and surgical learning activities. Copyright © 2016 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

A mentor-based portfolio program to evaluate pharmacy students' self-assessment skills.

PubMed

Kalata, Lindsay R; Abate, Marie A

2013-05-13

Objective. To evaluate pharmacy students' self-assessment skills with an electronic portfolio program using mentor evaluators. Design. First-year (P1) and second-year (P2) pharmacy students used online portfolios that required self-assessments of specific graded class assignments. Using a rubric, faculty and alumni mentors evaluated students' self-assessments and provided feedback. Assessment. Eighty-four P1 students, 74 P2 students, and 59 mentors participated in the portfolio program during 2010-2011. Both student groups performed well overall, with only a small number of resubmissions required. P1 students showed significant improvements across semesters for 2 of the self-assessment questions; P2 students' scores did not differ significantly. The P1 scores were significantly higher than P2 scores for 3 questions during spring 2011. Mentors and students had similar levels of agreement with the extent to which students put forth their best effort on the self-assessments. Conclusion. An electronic portfolio using mentors based inside and outside the school provided students with many opportunities to practice their self-assessment skills. This system represents a useful method of incorporating self-assessments into the curriculum that allows for feedback to be provided to the students.

A Mentor-Based Portfolio Program to Evaluate Pharmacy Students’ Self-Assessment Skills

PubMed Central

Kalata, Lindsay R.

2013-01-01

Objective. To evaluate pharmacy students' self-assessment skills with an electronic portfolio program using mentor evaluators. Design. First-year (P1) and second-year (P2) pharmacy students used online portfolios that required self-assessments of specific graded class assignments. Using a rubric, faculty and alumni mentors evaluated students' self-assessments and provided feedback. Assessment. Eighty-four P1 students, 74 P2 students, and 59 mentors participated in the portfolio program during 2010-2011. Both student groups performed well overall, with only a small number of resubmissions required. P1 students showed significant improvements across semesters for 2 of the self-assessment questions; P2 students' scores did not differ significantly. The P1 scores were significantly higher than P2 scores for 3 questions during spring 2011. Mentors and students had similar levels of agreement with the extent to which students put forth their best effort on the self-assessments. Conclusion. An electronic portfolio using mentors based inside and outside the school provided students with many opportunities to practice their self-assessment skills. This system represents a useful method of incorporating self-assessments into the curriculum that allows for feedback to be provided to the students. PMID:23716749

BRIEF REPORT: Screening Items to Identify Patients with Limited Health Literacy Skills

PubMed Central

Wallace, Lorraine S; Rogers, Edwin S; Roskos, Steven E; Holiday, David B; Weiss, Barry D

2006-01-01

BACKGROUND Patients with limited literacy skills are routinely encountered in clinical practice, but they are not always identified by clinicians. OBJECTIVE To evaluate 3 candidate questions to determine their accuracy in identifying patients with limited or marginal health literacy skills. METHODS We studied 305 English-speaking adults attending a university-based primary care clinic. Demographic items, health literacy screening questions, and the Rapid Estimate of Adult Literacy in Medicine (REALM) were administered to patients. To determine the accuracy of the candidate questions for identifying limited or marginal health literacy skills, we plotted area under the receiver operating characteristic (AUROC) curves for each item, using REALM scores as a reference standard. RESULTS The mean age of subjects was 49.5; 67.5% were female, 85.2% Caucasian, and 81.3% insured by TennCare and/or Medicare. Fifty-four (17.7%) had limited and 52 (17.0%) had marginal health literacy skills. One screening question, “How confident are you filling out medical forms by yourself?” was accurate in detecting limited (AUROC of 0.82; 95% confidence interval [CI]=0.77 to 0.86) and limited/marginal (AUROC of 0.79; 95% CI=0.74 to 0.83) health literacy skills. This question had significantly greater AUROC than either of the other questions (P<.01) and also a greater AUROC than questions based on demographic characteristics. CONCLUSIONS One screening question may be sufficient for detecting limited and marginal health literacy skills in clinic populations. PMID:16881950

Evaluation of gene expression classification studies: factors associated with classification performance.

PubMed

Novianti, Putri W; Roes, Kit C B; Eijkemans, Marinus J C

2014-01-01

Classification methods used in microarray studies for gene expression are diverse in the way they deal with the underlying complexity of the data, as well as in the technique used to build the classification model. The MAQC II study on cancer classification problems has found that performance was affected by factors such as the classification algorithm, cross validation method, number of genes, and gene selection method. In this paper, we study the hypothesis that the disease under study significantly determines which method is optimal, and that additionally sample size, class imbalance, type of medical question (diagnostic, prognostic or treatment response), and microarray platform are potentially influential. A systematic literature review was used to extract the information from 48 published articles on non-cancer microarray classification studies. The impact of the various factors on the reported classification accuracy was analyzed through random-intercept logistic regression. The type of medical question and method of cross validation dominated the explained variation in accuracy among studies, followed by disease category and microarray platform. In total, 42% of the between study variation was explained by all the study specific and problem specific factors that we studied together.

Robust Derivation of Risk Reduction Strategies

NASA Technical Reports Server (NTRS)

Richardson, Julian; Port, Daniel; Feather, Martin

2007-01-01

Effective risk reduction strategies can be derived mechanically given sufficient characterization of the risks present in the system and the effectiveness of available risk reduction techniques. In this paper, we address an important question: can we reliably expect mechanically derived risk reduction strategies to be better than fixed or hand-selected risk reduction strategies, given that the quantitative assessment of risks and risk reduction techniques upon which mechanical derivation is based is difficult and likely to be inaccurate? We consider this question relative to two methods for deriving effective risk reduction strategies: the strategic method defined by Kazman, Port et al [Port et al, 2005], and the Defect Detection and Prevention (DDP) tool [Feather & Cornford, 2003]. We performed a number of sensitivity experiments to evaluate how inaccurate knowledge of risk and risk reduction techniques affect the performance of the strategies computed by the Strategic Method compared to a variety of alternative strategies. The experimental results indicate that strategies computed by the Strategic Method were significantly more effective than the alternative risk reduction strategies, even when knowledge of risk and risk reduction techniques was very inaccurate. The robustness of the Strategic Method suggests that its use should be considered in a wide range of projects.

Using experimental design modules for process characterization in manufacturing/materials processes laboratories

NASA Technical Reports Server (NTRS)

Ankenman, Bruce; Ermer, Donald; Clum, James A.

1994-01-01

Modules dealing with statistical experimental design (SED), process modeling and improvement, and response surface methods have been developed and tested in two laboratory courses. One course was a manufacturing processes course in Mechanical Engineering and the other course was a materials processing course in Materials Science and Engineering. Each module is used as an 'experiment' in the course with the intent that subsequent course experiments will use SED methods for analysis and interpretation of data. Evaluation of the modules' effectiveness has been done by both survey questionnaires and inclusion of the module methodology in course examination questions. Results of the evaluation have been very positive. Those evaluation results and details of the modules' content and implementation are presented. The modules represent an important component for updating laboratory instruction and to provide training in quality for improved engineering practice.

Evaluation of a national quality use of medicines service in Australia: an evolving model.

PubMed

Beilby, Justin; Wutzke, Sonia E; Bowman, Jenny; Mackson, Judith M; Weekes, Lynn M

2006-04-01

To describe the first phase of a global evaluation framework for the National Prescribing Service (NPS), with a focus on services for health professionals, and in particular highlight the lessons learnt from evaluation around the establishment and implementation of this national program. The agreed evaluation framework used mixed methods focused around a series of evaluation questions, aimed at measuring the overall effect of this new organization as well as the individual programs within it. The evaluation questions were determined a priori and were based on the objectives established by the organization in its first year of operation. A detailed analyses has been completed of: the process, scope and reach of program delivery using both quantitative and qualitative measures; changes in attitudes and knowledge measured through key informant interviews and surveys of professional groups and consumers; and changes in prescribing behaviour and savings to the Pharmaceutical Benefits Scheme (PBS) through analysis of prescription data. The evaluation period for this report was mid-1998 to mid-2004. The NPS has successfully implemented a complex, multi-faceted program across Australia. From 1998 to 2004, in addition to print material provided to all general practitioners (GPs) and pharmacists, 90% of all GPs have been actively involved in one or more educational activity; 116 of 120 divisions of general practice have coordinated local NPS programs; and 9% of pharmacists have actively participated in at least one educational activity. Sixty per cent of GPs and pharmacists rated the printed educational materials as good or very good. In the last three years, the NPS activities have generated savings in the range of $121-163 million to the PBS, owing to changes in prescribing practices. The national evaluation framework has informed program delivery and ongoing design and development. Continued refinement of existing evaluation methods and further exploration of new techniques will remain a priority for the organization.

[Myths and evidence on the use of botulinum toxin: spasticity in adults and in children with cerebral palsy].

PubMed

Martinez-Castrillo, J C; Pena-Segura, J L; Sanz-Cartagena, P; Alonso-Curco, X; Arbelo-Gonzalez, J M; Arriola-Pereda, G; Coll-Bosch, M D; Conejero-Casares, J A; Garcia-Ribes, A; Jauma-Classen, S; Pagonabarraga, J; Vidal-Valls, J; Garcia-Ruiz, P J

2017-05-16

Spasticity is a medical problem with a high incidence that significantly impact on the quality of life of patients and their families. To analyze and to answer different questions about the use of botulinum toxin type A (BTA) in our clinical practice. A group of experts in neurology develop a list of topics related with the use of BTA. Two big groups were considered: spasticity in adults and in children with cerebral palsy. A literature search at PubMed for English, French, and Spanish language articles published up to June 2016 was performed. The manuscript was structured as a questionnaire that includes those questions that, according to the panel opinion, could generate more controversy or doubt. The initial draft was reviewed by the expert panel members to allow for modifications, and after subsequent revisions for achieving the highest degree of consensus, the final text was then validated. Different questions about diverse aspects of spasticity in adults, such as methods for evaluating spasticity, infiltration techniques, doses, number of infiltration points, etc. Regarding spasticity in children with cerebral palsy, the document included questions about minimum age of infiltration, methods of analgesia, etc. This review is a tool for continuous training for neurologist and rehabilitation specialist and residents of both specialties, about different specific areas of the management of BTA.

Perovskite Technology is Scalable, But Questions Remain about the Best

Science.gov Websites

Methods | News | NREL Perovskite Technology is Scalable, But Questions Remain about the Best Methods News Release: Perovskite Technology is Scalable, But Questions Remain about the Best Methods NREL be used on a larger surface. The NREL researchers examined potential scalable deposition methods

An assessment of the risk of foreign animal disease introduction into the United States of America through garbage from Alaskan cruise ships.

PubMed

McElvaine, M D; McDowell, R M; Fite, R W; Miller, L

1993-12-01

The United States Department of Agriculture, Animal and Plant Health Inspection Service (USDA-APHIS) has been exploring methods of quantitative risk assessment to support decision-making, provide risk management options and identify research needs. With current changes in world trade, regulatory decisions must have a scientific basis which is transparent, consistent, documentable and defensible. These quantitative risk assessment methods are described in an accompanying paper in this issue. In the present article, the authors provide an illustration by presenting an application of these methods. Prior to proposing changes in regulations, USDA officials requested an assessment of the risk of introduction of foreign animal disease to the United States of America through garbage from Alaskan cruise ships. The risk assessment team used a combination of quantitative and qualitative methods to evaluate this question. Quantitative risk assessment methods were used to estimate the amount of materials of foreign origin being sent to Alaskan landfills. This application of quantitative risk assessment illustrates the flexibility of the methods in addressing specific questions. By applying these methods, specific areas were identified where more scientific information and research were needed. Even with limited information, the risk assessment provided APHIS management with a scientific basis for a regulatory decision.

Outcome Evaluation: Student Development Program, Special Studies Division, Cleveland State University.

ERIC Educational Resources Information Center

Pasch, Marvin

Techniques and procedures used to evaluate the outcomes of the student development program, and to use the evaluation results, are presented. Specific evaluation questions are posed that address overall outcomes, not individual student outcomes, and quantitative measures are suggested to accompany the questions. The measures include statistical…

Measuring Coverage in MNCH: A Prospective Validation Study in Pakistan and Bangladesh on Measuring Correct Treatment of Childhood Pneumonia

PubMed Central

el Arifeen, Shams; Khan, Amira M.; Huque, M. Hamidul; Kazmi, Narjis; Roy, Sushmita; Abbasi, Saleem; Rahman, Qazi Sadeq-ur; Theodoratou, Evropi; Khorshed, Mahmuda Shayema; Rahman, Kazi Mizanur; Bari, Sanwarul; Kaiser, M. Mahfuzul Islam; Saha, Samir K.; Ahmed, A. S. M. Nawshad Uddin; Rudan, Igor; Bryce, Jennifer; Qazi, Shamim Ahmad; Campbell, Harry

2013-01-01

Background Antibiotic treatment for pneumonia as measured by Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS) is a key indicator for tracking progress in achieving Millennium Development Goal 4. Concerns about the validity of this indicator led us to perform an evaluation in urban and rural settings in Pakistan and Bangladesh. Methods and Findings Caregivers of 950 children under 5 y with pneumonia and 980 with “no pneumonia” were identified in urban and rural settings and allocated for DHS/MICS questions 2 or 4 wk later. Study physicians assigned a diagnosis of pneumonia as reference standard; the predictive ability of DHS/MICS questions and additional measurement tools to identify pneumonia versus non-pneumonia cases was evaluated. Results at both sites showed suboptimal discriminative power, with no difference between 2- or 4-wk recall. Individual patterns of sensitivity and specificity varied substantially across study sites (sensitivity 66.9% and 45.5%, and specificity 68.8% and 69.5%, for DHS in Pakistan and Bangladesh, respectively). Prescribed antibiotics for pneumonia were correctly recalled by about two-thirds of caregivers using DHS questions, increasing to 72% and 82% in Pakistan and Bangladesh, respectively, using a drug chart and detailed enquiry. Conclusions Monitoring antibiotic treatment of pneumonia is essential for national and global programs. Current (DHS/MICS questions) and proposed new (video and pneumonia score) methods of identifying pneumonia based on maternal recall discriminate poorly between pneumonia and children with cough. Furthermore, these methods have a low yield to identify children who have true pneumonia. Reported antibiotic treatment rates among these children are therefore not a valid proxy indicator of pneumonia treatment rates. These results have important implications for program monitoring and suggest that data in its current format from DHS/MICS surveys should not be used for the purpose of monitoring antibiotic treatment rates in children with pneumonia at the present time. Please see later in the article for the Editors' Summary PMID:23667339

Critiquing qualitative research.

PubMed

Beck, Cheryl Tatano

2009-10-01

The ability to critique research is a valuable skill that is fundamental to a perioperative nurse's ability to base his or her clinical practice on evidence derived from research. Criteria differ for critiquing a quantitative versus a qualitative study (ie, statistics are evaluated in a quantitative study, but not in a qualitative study). This article provides on guidelines for assessing qualitative research. Excerpts from a published qualitative research report are summarized and then critiqued. Questions are provided that help evaluate different sections of a research study (eg, sample, data collection methods, data analysis).

Economic Evaluation as a Component of Quality Effectiveness Research: Methodological and Practical Benefits

ERIC Educational Resources Information Center

Olsson, Tina M.

2012-01-01

Background: As research moves from questions of efficacy (can an intervention work) to questions of effectiveness (does an intervention work in practice), questions of efficiency (what are the costs and consequences of the intervention) become increasingly important. The incorporation of economic evaluation into the planning and execution of…

An Evaluation of University World Geography Textbook Questions for Components of Spatial Thinking

ERIC Educational Resources Information Center

Scholz, Michael A.; Huynh, Niem Tu; Brysch, Carmen P.; Scholz, Ruojing Wang

2014-01-01

Geography textbooks contain chapter or review questions that may engage students in spatial thinking. This research used Jo and Bednarz's (2009) "Taxonomy of Spatial Thinking" to evaluate the percentage of spatial thinking questions in four university-level world geography course textbooks. The results from this study were then…

Examining Adolescents' Strategic Processing during Online Reading with a Question-Generating Task

ERIC Educational Resources Information Center

Cho, Byeong-Young; Woodward, Lindsay; Li, Dan; Barlow, Wendy

2017-01-01

Forty-three high school students participated in an online reading task to generate a critical question on a controversial topic. Participants' concurrent verbal reports of strategy use (i.e., information location, meaning making, source evaluation, self-monitoring) and their reading outcome (i.e., the generated question) were evaluated with…

Integrating health technology assessment requirements in the clinical development of medicines: the experience from NICE scientific advice.

PubMed

Maignen, François; Osipenko, Leeza; Pinilla-Dominguez, Pilar; Crowe, Emily

2017-03-01

The primary objective of the study was to analyse the proposed clinical development and economic evaluation plans for investigational medicinal products for which pharmaceutical companies have sought health technology assessment (HTA) scientific advice (SA). We have selected and analysed all the scientific advice procedures undertaken by National Institute for Health and Care Excellence (NICE) SA between 1 January 2009 and 3 December 2015 for investigational medicinal products. We have mapped the questions asked by the companies and the areas of advice highlighted in the advice reports to the sections of the NICE methods guide to the technology appraisals (2013). An overwhelming proportion of SA procedures have addressed questions related to the clinical development and specifically the main pivotal efficacy studies. Approximately a quarter of the questions relate to the approaches to economic evaluation. Questions raised in European Medicines Agency-HTA procedures generally focus on clinical efficacy issues whereas cost-effectiveness ones tend to dominate in NICE-only procedures. Our analysis shows that the issues mostly discussed in the HTA SA are the choice of comparator, the generalisability of the clinical trial evidence to the NHS practice and the impact of the clinical trial outcomes on quality of life and survival. Less disagreement with the developers' plans was seen in the choice of clinical endpoints, population definition, position of the technology in the treatment pathway and study design. Scientific advice is designed to improve the quality of evidence and approaches to evidence generation for future regulatory approval and HTA evaluation. Our experience to date suggests that payer requirements are inconsistently integrated in the clinical development programmes. More efforts should be dedicated to demonstrating the clinical value of new medicinal products to patients and key decision-makers.

Are results of randomized controlled trials useful to psychotherapists?

PubMed

Persons, J B; Silberschatz, G

1998-02-01

Two clinicians provided opposite answers to the title question: Persons argued that information from randomized controlled trials (RCTs) is vital to clinicians, and Silberschatz argued that information from RCTs is irrelevant to clinicians. Persons argued that clinicians cannot provide top quality care to their patients without attending to findings of RCTs and that clinicians have an ethical responsibility to inform patients about, recommend, and provide treatments supported by RCTs before informing patients about, recommending, and providing treatments shown to be inferior in RCTs or not evaluated in RCTs. Silberschatz argued that RCTs do not and cannot answer questions that concern practicing clinicians. He advocates alternative research approaches (effectiveness studies, quasi-experimental methods, case-specific research) for studying psychotherapy.

Toward automated classification of consumers' cancer-related questions with a new taxonomy of expected answer types.

PubMed

McRoy, Susan; Jones, Sean; Kurmally, Adam

2016-09-01

This article examines methods for automated question classification applied to cancer-related questions that people have asked on the web. This work is part of a broader effort to provide automated question answering for health education. We created a new corpus of consumer-health questions related to cancer and a new taxonomy for those questions. We then compared the effectiveness of different statistical methods for developing classifiers, including weighted classification and resampling. Basic methods for building classifiers were limited by the high variability in the natural distribution of questions and typical refinement approaches of feature selection and merging categories achieved only small improvements to classifier accuracy. Best performance was achieved using weighted classification and resampling methods, the latter yielding an accuracy of F1 = 0.963. Thus, it would appear that statistical classifiers can be trained on natural data, but only if natural distributions of classes are smoothed. Such classifiers would be useful for automated question answering, for enriching web-based content, or assisting clinical professionals to answer questions. © The Author(s) 2015.

Evaluating the feasibility and acceptability of sending pregnancy and abortion history surveys through SMS text messaging to help reach sustainable development goal 3.

PubMed

Leidich, Aimee; Jayaweera, Ruvani; Arcara, Jennet; Clawson, Shannon; Chalker, Chad; Rochat, Roger

2018-06-01

In Kenya, abortion is illegal under most circumstances, yet about 48 abortions per 1,000 women occurred in 2012. Given the stigma around abortion, little is known about the availability of safe abortion. Thus, we explored the feasibility and acceptability of using SMS mobile surveys to collect information about women's pregnancy and abortion histories in Kenya. We sent a one-time mobile survey to 500 18-24year old women who had opted-in to a 16,000-person mobile survey panel. Women elected to answer questions about pregnancy only (number of pregnancies, number of births, age at first pregnancy) or pregnancy and abortion (ever tried to obtain an abortion and whether medical treatment was sought for any complications). The final question for all survey versions was an open-ended question asking respondents how they felt answering these questions. 356 (71%) responded to the initial survey question and 333 (94%) consented to answer questions about pregnancy. Of these, 233 (70%) agreed to answer questions about just pregnancy and 76 (23%) agreed to answer questions about pregnancy and abortion. Three-quarters of respondents indicated that they had positive or neutral feelings about answering these questions over SMS. Lastly, all data was retrieved within 24h of survey distribution allowing for immediate analysis and dissemination. Women's willingness to share their pregnancy history suggests mobile surveys are a cost-effective and accessible method for gathering information on abortion in Kenya. Copyright © 2017 Elsevier B.V. All rights reserved.

A two-question tool to assess the risk of repeated falls in the elderly

PubMed Central

Gálvez-Barrón, César; Narvaiza, Leire; Miñarro, Antonio; Ruiz, Jorge; Valldosera, Esther; Gonzalo, Natalia; Ng, Thalia; Sanguino, María Jesús; Yuste, Antonio

2017-01-01

Introduction Older adults’ perception of their own risk of fall has never been included into screening tools. The goal of this study was to evaluate the predictive validity of questions on subjects’ self-perception of their own risk of fall. Methods This prospective study was conducted on a probabilistic sample of 772 Spanish community-dwelling older adults, who were followed-up for a one year period. At a baseline visit, subjects were asked about their recent history of falls (question 1: “Have you fallen in the last 6 months?”), as well as on their perception of their own risk of fall by using two questions (question 2: “Do you think you may fall in the next few months?” possible answers: yes/no; question 3: “What is the probability that you fall in the next few months?” possible answers: low/intermediate/high). The follow-up consisted of quarterly telephone calls, where the number of falls occurred in that period was recorded. Results A short questionnaire built with questions 1 and 3 showed 70% sensitivity (95% CI: 56%-84%), 72% specificity (95% CI: 68%-76%) and 0.74 area under the ROC curve (95% CI: 0.66–0.82) for prediction of repeated falls in the subsequent year. Conclusions The estimation of one’s own risk of fall has predictive validity for the occurrence of repeated falls in older adults. A short questionnaire including a question on perception of one’s own risk of fall and a question on the recent history of falls had good predictive validity. PMID:28489888

Use of simulated patients to evaluate combined oral contraceptive dispensing practices of community pharmacists.

PubMed

Obreli-Neto, Paulo Roque; Pereira, Leonardo Régis Leira; Guidoni, Camilo Molino; de Oliveira Baldoni, André; Marusic, Srecko; de Lyra-Júnior, Divaldo Pereira; de Almeida, Kelsen Luis; Pazete, Ana Claudia Montolezi; do Nascimento, Janaina Dutra; Kos, Mitja; Girotto, Edmarlon; Cuman, Roberto Kenji Nakamura

2013-01-01

Combined oral contraceptive (COC) use is the most commonly used reversible method of birth control. The incorrect use of COCs is frequent and one of the most common causes of unintended pregnancies. Community pharmacists (CPs) are in a strategic position to improve COC use because they are the last health professional to interact with patients before drug use. To evaluate the COC dispensing practices of CPs in a developing country. A cross-sectional study was conducted in community pharmacies of Assis and Ourinhos microregions, Brazil, between June 1, 2012, and October 30, 2012. Four simulated patients (SPs) (with counseled audio recording) visited community pharmacies with a prescription for Ciclo 21(®) (a COC containing ethinyl estradiol 30 mcg + levonorgestrel 15 mcg). The audio recording of every SP visit was listened to independently by 3 researchers to evaluate the COC dispensing practice. The percentage of CPs who performed a screening for safe use of COCs (i.e., taking of patients' medical and family history, and measuring of blood pressure) and provided counseling, as well as the quality of the screening and counseling, were evaluated. Of the 185 CPs contacted, 41 (22.2%) agreed to participate in the study and finished the study protocol. Only 3 CPs asked the SP a question (1 question asked by each professional), and all of the questions were closed-ended, viz., "do you smoke?" (n = 2) and "what is your age?" (n = 1). None of the CPs measured the patient's blood pressure. Six CPs provided counseling when dispensing COCs (drug dosing, 5 CPs; possible adverse effects, 2 CPs), and one CP provided counseling regarding both aspects. The CPs evaluated did not dispense COC appropriately and could influence in the occurrence of negatives therapeutic outcomes such as adverse effects and treatment failure.

Radiology Residents' Awareness about Ionizing Radiation Doses in Imaging Studies and Their Cancer Risk during Radiological Examinations

PubMed Central

Divrik Gökçe, Senem; Coşkun, Melek

2012-01-01

Objective Imaging methods that use ionizing radiation have been more frequent in various medical fields with advances in imaging technology. The aim of our study was to make residents be aware of the radiation dose they are subjected to when they conduct radiological imaging methods, and of cancer risk. Materials and Methods A total of 364 residents participated in this descriptive study which was conducted during the period between October, 2008 and January, 2009. The questionnaires were completed under strict control on a one-to-one basis from each department. A χ2-test was used for the evaluation of data obtained. Results Only 7% of residents correctly answered to the question about the ionizing radiation dose of a posteroanterior (PA) chest X-ray. The question asking about the equivalent number of PA chest X-rays to the ionizing dose of a brain CT was answered correctly by 24% of residents; the same question regarding abdominal CT was answered correctly by 16% of residents, thorax CT by 16%, thyroid scintigraphy by 15%, intravenous pyelography by 9%, and lumbar spine radiography by 2%. The risk of developing a cancer throughout lifetime by a brain and abdominal CT were 33% and 28%, respectively. Conclusion Radiologic residents should have updated knowledge about radiation dose content and attendant cancer risks of various radiological imaging methods during both basic medical training period and following practice period. PMID:22438688

App Chronic Disease Checklist: Protocol to Evaluate Mobile Apps for Chronic Disease Self-Management

PubMed Central

Anderson, Kevin; Burford, Oksana

2016-01-01

Background The availability of mobile health apps for self-care continues to increase. While little evidence of their clinical impact has been published, there is general agreement among health authorities and authors that consumers’ use of health apps assist in self-management and potentially clinical decision making. A consumer’s sustained engagement with a health app is dependent on the usability and functionality of the app. While numerous studies have attempted to evaluate health apps, there is a paucity of published methods that adequately recognize client experiences in the academic evaluation of apps for chronic conditions. Objective This paper reports (1) a protocol to shortlist health apps for academic evaluation, (2) synthesis of a checklist to screen health apps for quality and reliability, and (3) a proposed method to theoretically evaluate usability of health apps, with a view towards identifying one or more apps suitable for clinical assessment. Methods A Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram was developed to guide the selection of the apps to be assessed. The screening checklist was thematically synthesized with reference to recurring constructs in published checklists and related materials for the assessment of health apps. The checklist was evaluated by the authors for face and construct validity. The proposed method for evaluation of health apps required the design of procedures for raters of apps, dummy data entry to test the apps, and analysis of raters’ scores. Results The PRISMA flow diagram comprises 5 steps: filtering of duplicate apps; eliminating non-English apps; removing apps requiring purchase, filtering apps not updated within the past year; and separation of apps into their core functionality. The screening checklist to evaluate the selected apps was named the App Chronic Disease Checklist, and comprises 4 sections with 6 questions in each section. The validity check verified classification of, and ambiguity in, wording of questions within constructs. The proposed method to evaluate shortlisted and downloaded apps comprises instructions to attempt set-up of a dummy user profile, and dummy data entry to represent in-range and out-of-range clinical measures simulating a range of user behaviors. A minimum score of 80% by consensus (using the Intraclass Correlation Coefficient) between raters is proposed to identify apps suitable for clinical trials. Conclusions The flow diagram allows researchers to shortlist health apps that are potentially suitable for formal evaluation. The evaluation checklist enables quantitative comparison of shortlisted apps based on constructs reported in the literature. The use of multiple raters, and comparison of their scores, is proposed to manage inherent subjectivity in assessing user experiences. Initial trial of the combined protocol is planned for apps pertaining to the self-monitoring of asthma; these results will be reported elsewhere. PMID:27815233

Implants in bone: part II. Research on implant osseointegration: material testing, mechanical testing, imaging and histoanalytical methods.

PubMed

von Wilmowsky, Cornelius; Moest, Tobias; Nkenke, Emeka; Stelzle, Florian; Schlegel, Karl Andreas

2014-12-01

In order to determine whether a newly developed implant material conforms to the requirements of biocompatibility, it must undergo rigorous testing. To correctly interpret the results of studies on implant material osseointegration, it is necessary to have a sound understanding of all the testing methods. The aim of this overview is to elucidate the methods that are used for the experimental evaluation of the osseointegration of implant materials. In recent decades, there has been a constant proliferation of new materials and surface modifications in the field of dental implants. This continuous development of innovative biomaterials requires a precise and detailed evaluation in terms of biocompatibility and implant healing before clinical use. The current gold standard is in vivo animal testing on well validated animal models. However, long-term outcome studies on patients have to follow to finally validate and show patient benefit. No experimental set-up can provide answers for all possible research questions. However, a certain transferability of the results to humans might be possible if the experimental set-up is carefully chosen for the aspects and questions being investigated. To enhance the implant survival rate in the rising number of patients with chronic diseases which compromise wound healing and osseointegration, dental implant research on compromised animal models will further gain importance in future.

Sensitivity and Calibration of Non-Destructive Evaluation Method That Uses Neural-Net Processing of Characteristic Fringe Patterns

NASA Technical Reports Server (NTRS)

Decker, Arthur J.; Weiland, Kenneth E.

2003-01-01

This paper answers some performance and calibration questions about a non-destructive-evaluation (NDE) procedure that uses artificial neural networks to detect structural damage or other changes from sub-sampled characteristic patterns. The method shows increasing sensitivity as the number of sub-samples increases from 108 to 6912. The sensitivity of this robust NDE method is not affected by noisy excitations of the first vibration mode. A calibration procedure is proposed and demonstrated where the output of a trained net can be correlated with the outputs of the point sensors used for vibration testing. The calibration procedure is based on controlled changes of fastener torques. A heterodyne interferometer is used as a displacement sensor for a demonstration of the challenges to be handled in using standard point sensors for calibration.

Clinically Unjustified Diagnostic Imaging – a Worrisome Tendency in Today’s Medical Practice

PubMed Central

Sobiecka, Aleksandra; Bekiesińska-Figatowska, Monika; Rutkowska, Milena; Latos, Tomasz; Walecki, Jerzy

2016-01-01

Summary Background The purpose of the study was to evaluate the percentage of unjustified examinations among all the CT and MRI studies performed by two radiology departments and to determine the types of examinations which are most commonly carried out unnecessarily. Material/Methods Three radiologists assessed the justification of CT and MRI examinations performed during a period of 14 days based on the referrals. The radiologists assessed 799 referrals for CT scans (847 examinations of a particular part of the body) and 269 MRI referrals (269 examinations). The criteria for justification were: medical expertise and the guidelines. During the first stage radiologists divided the examinations into 3 groups: justified, unjustified and the examinations of questionable justification. The second step was to determine the reasons why the studies were considered as unjustified or of questionable justification. Results 73 of 1116 examinations (6.54%) were considered to be unjustified or of a questionable justification. There were 59 CT scans (59/847=6.97%) and 14 MRI studies (14/269=5.20%). The most common reasons to consider them as unjustified or of questionable justification were: inadequate method of diagnostic imaging chosen as a first-line tool and lacking or insufficient clinical details. Conclusions In our investigation 6.54% of both CT and MRI examinations were considered as unjustified or of questionable justification, which is lower than described in other studies (from 7% to 26%). The assessment was based only on referrals, therefore a total share of these examinations is likely to be higher. PMID:27471577

Targeted neonatal echocardiography services: need for standardized training and quality assurance.

PubMed

Finan, Emer; Sehgal, Arvind; Khuffash, Afif El; McNamara, Patrick J

2014-10-01

Targeted neonatal echocardiography refers to a focused assessment of myocardial performance and hemodynamics directed by a specific clinical question. It has become the standard of care in many parts of the world, but practice is variable, and there has been a lack of standardized training and evaluation to date. Targeted neonatal echocardiography was first introduced to Canada in 2006. The purpose of this study was to examine the characteristics of targeted neonatal echocardiography practice and training methods in Canadian neonatal intensive care units (NICUs). A total of 142 Canadian neonatologists were invited to participate in an online survey, which was conducted in September 2010. The survey consisted of questions related to the availability of targeted neonatal echocardiography, clinical indications, benefits and risks, and training methods. The overall survey response rate was 65%. Forty-eight respondents (34%) indicated that targeted neonatal echocardiography was available in their units, and the program was introduced within the preceding 1 to 5 years. In centers where it was unavailable, lack of on-site echocardiography expertise was cited as the major barrier to implementation. The most common indications for targeted neonatal echocardiography included evaluation of a hemodynamically significant ductus arteriosus, systemic or pulmonary blood flow, and response to cardiovascular treatments. Only 27% of respondents, working in centers where targeted neonatal echocardiography existed, actually performed the studies themselves; most individuals completed 11 to 20 studies per month. Almost half of the respondents said that training was available in their institutions, but methods of training and evaluation were inconsistent. Eighty-seven percent of respondents reported no formalized process for assessment of ongoing competency after the initial training period. Targeted neonatal echocardiography is becoming more widely available and is gaining acceptance in Canadian NICUs. Although training is provided in many institutions, the process is not well established, and formal evaluation is rarely performed. This study emphasizes the need for development of standards for formalized training, evaluation, and quality assurance. © 2014 by the American Institute of Ultrasound in Medicine.

Item Difficulty in the Evaluation of Computer-Based Instruction: An Example from Neuroanatomy

PubMed Central

Chariker, Julia H.; Naaz, Farah; Pani, John R.

2012-01-01

This article reports large item effects in a study of computer-based learning of neuroanatomy. Outcome measures of the efficiency of learning, transfer of learning, and generalization of knowledge diverged by a wide margin across test items, with certain sets of items emerging as particularly difficult to master. In addition, the outcomes of comparisons between instructional methods changed with the difficulty of the items to be learned. More challenging items better differentiated between instructional methods. This set of results is important for two reasons. First, it suggests that instruction may be more efficient if sets of consistently difficult items are the targets of instructional methods particularly suited to them. Second, there is wide variation in the published literature regarding the outcomes of empirical evaluations of computer-based instruction. As a consequence, many questions arise as to the factors that may affect such evaluations. The present paper demonstrates that the level of challenge in the material that is presented to learners is an important factor to consider in the evaluation of a computer-based instructional system. PMID:22231801

Item difficulty in the evaluation of computer-based instruction: an example from neuroanatomy.

PubMed

Chariker, Julia H; Naaz, Farah; Pani, John R

2012-01-01

This article reports large item effects in a study of computer-based learning of neuroanatomy. Outcome measures of the efficiency of learning, transfer of learning, and generalization of knowledge diverged by a wide margin across test items, with certain sets of items emerging as particularly difficult to master. In addition, the outcomes of comparisons between instructional methods changed with the difficulty of the items to be learned. More challenging items better differentiated between instructional methods. This set of results is important for two reasons. First, it suggests that instruction may be more efficient if sets of consistently difficult items are the targets of instructional methods particularly suited to them. Second, there is wide variation in the published literature regarding the outcomes of empirical evaluations of computer-based instruction. As a consequence, many questions arise as to the factors that may affect such evaluations. The present article demonstrates that the level of challenge in the material that is presented to learners is an important factor to consider in the evaluation of a computer-based instructional system. Copyright © 2011 American Association of Anatomists.

Global research priorities for interpersonal violence prevention: a modified Delphi study

PubMed Central

Tanaka, Masako; Tomlinson, Mark; Streiner, David L; Tonmyr, Lil; Lee, Bandy X; Fisher, Jane; Hegadoren, Kathy; Pim, Joam Evans; Wang, Shr-Jie Sharlenna; MacMillan, Harriet L

2017-01-01

Abstract Objective To establish global research priorities for interpersonal violence prevention using a systematic approach. Methods Research priorities were identified in a three-round process involving two surveys. In round 1, 95 global experts in violence prevention proposed research questions to be ranked in round 2. Questions were collated and organized according to the four-step public health approach to violence prevention. In round 2, 280 international experts ranked the importance of research in the four steps, and the various substeps, of the public health approach. In round 3, 131 international experts ranked the importance of detailed research questions on the public health step awarded the highest priority in round 2. Findings In round 2, “developing, implementing and evaluating interventions” was the step of the public health approach awarded the highest priority for four of the six types of violence considered (i.e. child maltreatment, intimate partner violence, armed violence and sexual violence) but not for youth violence or elder abuse. In contrast, “scaling up interventions and evaluating their cost–effectiveness” was ranked lowest for all types of violence. In round 3, research into “developing, implementing and evaluating interventions” that addressed parenting or laws to regulate the use of firearms was awarded the highest priority. The key limitations of the study were response and attrition rates among survey respondents. However, these rates were in line with similar priority-setting exercises. Conclusion These findings suggest it is premature to scale up violence prevention interventions. Developing and evaluating smaller-scale interventions should be the funding priority. PMID:28053363

Earth recovery mode analysis for a Martian sample return mission

NASA Technical Reports Server (NTRS)

Green, J. P.

1978-01-01

The analysis has concerned itself with evaluating alternative methods of recovering a sample module from a trans-earth trajectory originating in the vicinity of Mars. The major modes evaluated are: (1) direct atmospheric entry from trans-earth trajectory; (2) earth orbit insertion by retropropulsion; and (3) atmospheric braking to a capture orbit. In addition, the question of guided vs. unguided entry vehicles was considered, as well as alternative methods of recovery after orbit insertion for modes (2) and (3). A summary of results and conclusions is presented. Analytical results for aerodynamic and propulsive maneuvering vehicles are discussed. System performance requirements and alternatives for inertial systems implementation are also discussed. Orbital recovery operations and further studies required to resolve the recovery mode issue are described.

Task-oriented evaluation of electronic medical records systems: development and validation of a questionnaire for physicians

PubMed Central

2004-01-01

Background Evaluation is a challenging but necessary part of the development cycle of clinical information systems like the electronic medical records (EMR) system. It is believed that such evaluations should include multiple perspectives, be comparative and employ both qualitative and quantitative methods. Self-administered questionnaires are frequently used as a quantitative evaluation method in medical informatics, but very few validated questionnaires address clinical use of EMR systems. Methods We have developed a task-oriented questionnaire for evaluating EMR systems from the clinician's perspective. The key feature of the questionnaire is a list of 24 general clinical tasks. It is applicable to physicians of most specialties and covers essential parts of their information-oriented work. The task list appears in two separate sections, about EMR use and task performance using the EMR, respectively. By combining these sections, the evaluator may estimate the potential impact of the EMR system on health care delivery. The results may also be compared across time, site or vendor. This paper describes the development, performance and validation of the questionnaire. Its performance is shown in two demonstration studies (n = 219 and 80). Its content is validated in an interview study (n = 10), and its reliability is investigated in a test-retest study (n = 37) and a scaling study (n = 31). Results In the interviews, the physicians found the general clinical tasks in the questionnaire relevant and comprehensible. The tasks were interpreted concordant to their definitions. However, the physicians found questions about tasks not explicitly or only partially supported by the EMR systems difficult to answer. The two demonstration studies provided unambiguous results and low percentages of missing responses. In addition, criterion validity was demonstrated for a majority of task-oriented questions. Their test-retest reliability was generally high, and the non-standard scale was found symmetric and ordinal. Conclusion This questionnaire is relevant for clinical work and EMR systems, provides reliable and interpretable results, and may be used as part of any evaluation effort involving the clinician's perspective of an EMR system. PMID:15018620

The Viewpoints of Students and Evaluation Experts About Performance Processes of Faculty Member Evaluation at Mazandaran University of Medical Sciences, 2014

PubMed Central

Ghahrani, Nassim; Balaghafari, Azita; Aligolbandi, Kobra; Vahedi, Mohammad; Siamian, Hasan

2015-01-01

Background and purpose: One of the most common ways used in most of the countries and Iran to determine the status of teacher training is the evaluation by students. The most common method of evaluation is the survey questionnaire, the content of a number of questions about educational activities provided to the students. The researchers plan to evaluate the students’ and experts’ performances at Mazandaran University of Medical Sciences on the process of evaluating the performance of teachers, they examined in 2014. Materials and methods: This study surveys the students and experts in the evaluation of faculty members’ performance process. The study subjects were 3904 students and 37 evaluation expert of Mazandaran University of Medical Sciences. Using Cochran sampling formula of 350 students through proportional stratified random sampling were selected. The experts’ viewpoint, method was used. Data collection tools consisted of 14 questions with answers Yes, or, I don’t know. Descriptive Statistical analysis of the data and chi-square test was performed. Results: From total of 350 students, 346 and the entire 37 evaluations expert participated in this study. Most of the students, 80 (23.12%) and the largest number of experts, 8 (21.62%) were from Sari Allied Medical Sciences Faculty. Most of the demographic information about gender were, 255 female students (74.56%) and 29 female experts (78.37%). In most age groups of students, 188 (55.62 percent) were in the category of 18 to 20 years, and the experts, 19 (51.35%) were in the category of 22 and 31 years. Most students, 232 of them (70.95%) were in semester 2 and 4. Most experts, 20 (54.05 percent) were under 10 years of work experience. The comparison between the views of students and experts in the evaluation process between the schools of Mazandaran University of Medical Sciences, Sari School of Nursing and Midwifery, there was difference between the opinions of experts and students (p-value=0.01. It showed 86.7% student and 33.3% of experts is satisfied with the evaluation process. Conclusion: on comparison of students and experts viewpoints on the implementation of the evaluation process, it is noteworthy that among students of different opinions on how the evaluation process. It worth to mention that there is insignificant difference between their viewpoints and majority of students and evaluation experts with the evaluation the process. In addition, the experts evaluated at different schools, most of them are satisfied the process. PMID:26236169

Why, and how, mixed methods research is undertaken in health services research in England: a mixed methods study

PubMed Central

O'Cathain, Alicia; Murphy, Elizabeth; Nicholl, Jon

2007-01-01

Background Recently, there has been a surge of international interest in combining qualitative and quantitative methods in a single study – often called mixed methods research. It is timely to consider why and how mixed methods research is used in health services research (HSR). Methods Documentary analysis of proposals and reports of 75 mixed methods studies funded by a research commissioner of HSR in England between 1994 and 2004. Face-to-face semi-structured interviews with 20 researchers sampled from these studies. Results 18% (119/647) of HSR studies were classified as mixed methods research. In the documentation, comprehensiveness was the main driver for using mixed methods research, with researchers wanting to address a wider range of questions than quantitative methods alone would allow. Interviewees elaborated on this, identifying the need for qualitative research to engage with the complexity of health, health care interventions, and the environment in which studies took place. Motivations for adopting a mixed methods approach were not always based on the intrinsic value of mixed methods research for addressing the research question; they could be strategic, for example, to obtain funding. Mixed methods research was used in the context of evaluation, including randomised and non-randomised designs; survey and fieldwork exploratory studies; and instrument development. Studies drew on a limited number of methods – particularly surveys and individual interviews – but used methods in a wide range of roles. Conclusion Mixed methods research is common in HSR in the UK. Its use is driven by pragmatism rather than principle, motivated by the perceived deficit of quantitative methods alone to address the complexity of research in health care, as well as other more strategic gains. Methods are combined in a range of contexts, yet the emerging methodological contributions from HSR to the field of mixed methods research are currently limited to the single context of combining qualitative methods and randomised controlled trials. Health services researchers could further contribute to the development of mixed methods research in the contexts of instrument development, survey and fieldwork, and non-randomised evaluations. PMID:17570838

GUIDE FOR SELF-EXAMINATION AND PLANNING, THE CLUB ANALYSIS PROGRAM OF INDUSTRIAL MANAGEMENT CLUBS OF THE YMCA.

ERIC Educational Resources Information Center

MEYER, WILLIAM F.

THIS GUIDE PROVIDES A QUESTION AND ANSWER METHOD IN WHICH LEADERS OF INDUSTRIAL MANAGEMENT CLUBS (IMC) MAY EVALUATE CLUB PROGRESS AND ACCOMPLISHMENTS. A REPORT OF A NATIONAL STUDY MADE BY GEORGE WILLIAMS COLLEGE ON I.M.C. MEMBERS AND THEIR ATTITUDES TOWARD CLUB PROGRESS IS PROVIDED AS A GUIDE FOR GATHERING INFORMATION. A SMALL CORE STEERING…

A Comparison of Web-Based and Paper-Based Survey Methods: Testing Assumptions of Survey Mode and Response Cost

ERIC Educational Resources Information Center

Greenlaw, Corey; Brown-Welty, Sharon

2009-01-01

Web-based surveys have become more prevalent in areas such as evaluation, research, and marketing research to name a few. The proliferation of these online surveys raises the question, how do response rates compare with traditional surveys and at what cost? This research explored response rates and costs for Web-based surveys, paper surveys, and…

A Method for Writing Open-Ended Curved Arrow Notation Questions for Multiple-Choice Exams and Electronic-Response Systems

ERIC Educational Resources Information Center

Ruder, Suzanne M.; Straumanis, Andrei R.

2009-01-01

A critical stage in the process of developing a conceptual understanding of organic chemistry is learning to use curved arrow notation. From this stems the ability to predict reaction products and mechanisms beyond the realm of memorization. Since evaluation (i.e., testing) is known to be a key driver of student learning, it follows that a new…

Student Perception on a Student Response System Formed by Combining Mobile Phone and a Polling Website

ERIC Educational Resources Information Center

Wong, Adam

2016-01-01

Every teacher understands the importance of getting timely student feedback for effective and efficient teaching and learning. However, students are not always keen to answer questions in the classroom in front of their peers. There is a need for an efficient method to engage all the students in a classroom and quickly evaluate the progress of…

Evaluation of the Citizenship Consciousness of the 8th Year Students

ERIC Educational Resources Information Center

Tonga, Deniz; Keles, Hamza

2014-01-01

The prime purpose of this study is to elucidate the awareness level of citizenship of 8th year students. That was why the answer of the question "what is the level of citizenship consciousness of the 8th year students" was sought. The study was designed according to the descriptive survey method with the use of a scale with 7 open ended…

Technology Insertion for Recapitalization of Legacy Systems

DTIC Science & Technology

2015-09-30

peened, and 4) an Abcite coating will be flame sprayed on the component. The ALCM program (B) has 1) evaluated data provided, 2) gathered questions...Report Technology Insertion for the Recapitalization of Legacy Systems Laser sintering, thermal spray and cold spray are additive manufacturing methods... coatings Need an experienced operator Requires a special spray booth to limit overspray and protect operator Requires primer or surface treatment

Statistical Evaluation of Biometric Evidence in Forensic Automatic Speaker Recognition

NASA Astrophysics Data System (ADS)

Drygajlo, Andrzej

Forensic speaker recognition is the process of determining if a specific individual (suspected speaker) is the source of a questioned voice recording (trace). This paper aims at presenting forensic automatic speaker recognition (FASR) methods that provide a coherent way of quantifying and presenting recorded voice as biometric evidence. In such methods, the biometric evidence consists of the quantified degree of similarity between speaker-dependent features extracted from the trace and speaker-dependent features extracted from recorded speech of a suspect. The interpretation of recorded voice as evidence in the forensic context presents particular challenges, including within-speaker (within-source) variability and between-speakers (between-sources) variability. Consequently, FASR methods must provide a statistical evaluation which gives the court an indication of the strength of the evidence given the estimated within-source and between-sources variabilities. This paper reports on the first ENFSI evaluation campaign through a fake case, organized by the Netherlands Forensic Institute (NFI), as an example, where an automatic method using the Gaussian mixture models (GMMs) and the Bayesian interpretation (BI) framework were implemented for the forensic speaker recognition task.

Analyzing Electronic Question/Answer Services: Framework and Evaluations of Selected Services.

ERIC Educational Resources Information Center

White, Marilyn Domas, Ed.

This report develops an analytical framework based on systems analysis for evaluating electronic question/answer or AskA services operated by a wide range of types of organizations, including libraries. Version 1.0 of this framework was applied in June 1999 to a selective sample of 11 electronic question/answer services, which cover a range of…

Rasch Model Analysis on the Effectiveness of Early Evaluation Questions as a Benchmark for New Students Ability

ERIC Educational Resources Information Center

Arsad, Norhana; Kamal, Noorfazila; Ayob, Afida; Sarbani, Nizaroyani; Tsuey, Chong Sheau; Misran, Norbahiah; Husain, Hafizah

2013-01-01

This paper discusses the effectiveness of the early evaluation questions conducted to determine the academic ability of the new students in the Department of Electrical, Electronics and Systems Engineering. Questions designed are knowledge based--on what the students have learned during their pre-university level. The results show students have…

Why, and how, mixed methods research is undertaken in health services research in England: a mixed methods study.

PubMed

O'Cathain, Alicia; Murphy, Elizabeth; Nicholl, Jon

2007-06-14

Recently, there has been a surge of international interest in combining qualitative and quantitative methods in a single study--often called mixed methods research. It is timely to consider why and how mixed methods research is used in health services research (HSR). Documentary analysis of proposals and reports of 75 mixed methods studies funded by a research commissioner of HSR in England between 1994 and 2004. Face-to-face semi-structured interviews with 20 researchers sampled from these studies. 18% (119/647) of HSR studies were classified as mixed methods research. In the documentation, comprehensiveness was the main driver for using mixed methods research, with researchers wanting to address a wider range of questions than quantitative methods alone would allow. Interviewees elaborated on this, identifying the need for qualitative research to engage with the complexity of health, health care interventions, and the environment in which studies took place. Motivations for adopting a mixed methods approach were not always based on the intrinsic value of mixed methods research for addressing the research question; they could be strategic, for example, to obtain funding. Mixed methods research was used in the context of evaluation, including randomised and non-randomised designs; survey and fieldwork exploratory studies; and instrument development. Studies drew on a limited number of methods--particularly surveys and individual interviews--but used methods in a wide range of roles. Mixed methods research is common in HSR in the UK. Its use is driven by pragmatism rather than principle, motivated by the perceived deficit of quantitative methods alone to address the complexity of research in health care, as well as other more strategic gains. Methods are combined in a range of contexts, yet the emerging methodological contributions from HSR to the field of mixed methods research are currently limited to the single context of combining qualitative methods and randomised controlled trials. Health services researchers could further contribute to the development of mixed methods research in the contexts of instrument development, survey and fieldwork, and non-randomised evaluations.

How effective are selection methods in medical education? A systematic review.

PubMed

Patterson, Fiona; Knight, Alec; Dowell, Jon; Nicholson, Sandra; Cousans, Fran; Cleland, Jennifer

2016-01-01

Selection methods used by medical schools should reliably identify whether candidates are likely to be successful in medical training and ultimately become competent clinicians. However, there is little consensus regarding methods that reliably evaluate non-academic attributes, and longitudinal studies examining predictors of success after qualification are insufficient. This systematic review synthesises the extant research evidence on the relative strengths of various selection methods. We offer a research agenda and identify key considerations to inform policy and practice in the next 50 years. A formalised literature search was conducted for studies published between 1997 and 2015. A total of 194 articles met the inclusion criteria and were appraised in relation to: (i) selection method used; (ii) research question(s) addressed, and (iii) type of study design. Eight selection methods were identified: (i) aptitude tests; (ii) academic records; (iii) personal statements; (iv) references; (v) situational judgement tests (SJTs); (vi) personality and emotional intelligence assessments; (vii) interviews and multiple mini-interviews (MMIs), and (viii) selection centres (SCs). The evidence relating to each method was reviewed against four evaluation criteria: effectiveness (reliability and validity); procedural issues; acceptability, and cost-effectiveness. Evidence shows clearly that academic records, MMIs, aptitude tests, SJTs and SCs are more effective selection methods and are generally fairer than traditional interviews, references and personal statements. However, achievement in different selection methods may differentially predict performance at the various stages of medical education and clinical practice. Research into selection has been over-reliant on cross-sectional study designs and has tended to focus on reliability estimates rather than validity as an indicator of quality. A comprehensive framework of outcome criteria should be developed to allow researchers to interpret empirical evidence and compare selection methods fairly. This review highlights gaps in evidence for the combination of selection tools that is most effective and the weighting to be given to each tool. © 2015 John Wiley & Sons Ltd.

[The evaluation of pharmacological drugs, medical devices, and non-pharmacological or public health interventions: Experimental design limitations. Moving towards new methods?

PubMed

Villeval, M; Carayol, M; Lamy, S; Lepage, B; Lang, T

2016-12-01

In the field of health, evidence-based medicine and associated methods like randomised controlled trials (RCTs) have become widely used. RCT has become the gold standard for evaluating causal links between interventions and health results. Originating in pharmacology, this method has been progressively expanded to medical devices, non-pharmacological individual interventions, as well as collective public health interventions. Its use in these domains has led to the formulation of several limits, and it has been called into question as an undisputed gold standard. Some of those limits (e.g. confounding biases and external validity) are common to these four different domains, while others are more specific. This paper describes the different limits, as well as several research avenues. Some are methodological reflections aiming at adapting RCT to the complexity of the tested interventions, and at overcoming some of its limits. Others are alternative methods. The objective is not to remove RCT from the range of evaluation methodologies, but to resituate it within this range. The aim is to encourage choosing between different methods according to the features and the level of the intervention to evaluate, thereby calling for methodological pluralism. Copyright © 2016 Elsevier Masson SAS. All rights reserved.

Evaluation of best practices in the design of online evidence-based practice instructional modules*

PubMed Central

Foster, Margaret J.; Shurtz, Suzanne; Pepper, Catherine

2014-01-01

Objectives: The research determined to what extent best practices are being followed by freely available online modules aimed at teaching critical thinking and evidence-based practices (EBPs) in health sciences fields. Methods: In phase I, an evaluation rubric was created after reviewing the literature. Individual rubric questions were assigned point values and grouped into sections, and the sections weighted. Phase II involved searching Internet platforms to locate online EBP modules, which were screened to determine if they met predetermined criteria for inclusion. Phase III comprised a first evaluation, in which two authors assessed each module, followed by a second evaluation of the top-scoring modules by five representatives from different health sciences units. Results: The rubric's 28 questions were categorized into 4 sections: content, design, interactivity, and usability. After retrieving 170 online modules and closely screening 91, 42 were in the first evaluation and 8 modules were in the second evaluation. Modules in the first evaluation earned, on average, 59% of available points; modules in the second earned an average of 68%. Both evaluations had a moderate level of inter-rater reliability. Conclusions: The rubric was effective and reliable in evaluating the modules. Most modules followed best practices for content and usability but not for design and interactivity. Implications: By systematically collecting and evaluating instructional modules, the authors found many potentially useful elements for module creation. Also, by reviewing the limitations of the evaluated modules, the authors were able to anticipate and plan ways to overcome potential issues in module design. PMID:24415917

Development of a self-report questionnaire designed for population-based surveillance of gingivitis in adolescents: assessment of content validity and reliability

PubMed Central

QUIROZ, Viviana; REINERO, Daniela; HERNÁNDEZ, Patricia; CONTRERAS, Johanna; VERNAL, Rolando; CARVAJAL, Paola

2017-01-01

Abstract The major infectious diseases in Chile encompass the periodontal diseases, with a combined prevalence that rises up to 90% of the population. Thus, the population-based surveillance of periodontal diseases plays a central role for assessing their prevalence and for planning, implementing, and evaluating preventive and control programs. Self-report questionnaires have been proposed for the surveillance of periodontal diseases in adult populations world-wide. Objective This study aimed to develop and assess the content validity and reliability of a cognitively adapted self-report questionnaire designed for surveillance of gingivitis in adolescents. Material and Methods Ten predetermined self-report questions evaluating early signs and symptoms of gingivitis were preliminary assessed by a panel of clinical experts. Eight questions were selected and cognitively tested in 20 adolescents aged 12 to 18 years from Santiago de Chile. The questionnaire was then conducted and answered by 178 Chilean adolescents. Internal consistency was measured using the Cronbach’s alpha and temporal stability was calculated using the Kappa-index. Results A reliable final self-report questionnaire consisting of 5 questions was obtained, with a total Cronbach’s alpha of 0.73 and a Kappa-index ranging from 0.41 to 0.77 between the different questions. Conclusions The proposed questionnaire is reliable, with an acceptable internal consistency and a temporal stability from moderate to substantial, and it is promising for estimating the prevalence of gingivitis in adolescents. PMID:28877279

Review of reviews on exposures to synthetic organic chemicals and children's neurodevelopment: Methodological and interpretation challenges.

PubMed

LaKind, Judy S; Anthony, Laura G; Goodman, Michael

2017-01-01

Environmental epidemiology data are becoming increasingly important in public health decision making, which commonly incorporates a systematic review of multiple studies. This review addresses two fundamental questions: What is the quality of available reviews on associations between exposure to synthetic organic chemicals and neurodevelopmental outcomes? What is the value (e.g., quality and consistency) of the underlying literature? Published reviews on associations between synthetic organic environmental chemical exposures and neurodevelopmental outcomes in children were systematically evaluated. Seventy-four relevant reviews were identified, and these were evaluated with respect to four methodological characteristics: (1) systematic inclusion/exclusion criteria and reproducible methods for search and retrieval of studies; (2) structured evaluation of underlying data quality; (3) systematic assessment of consistency across specific exposure-outcome associations; and (4) evaluation of reporting/publication bias. None of the 74 reviews fully met the criteria for all four methodological characteristics. Only four reviews met two criteria, and six reviews fulfilled only one criterion. Perhaps more importantly, the higher quality reviews were not able to meet all of the criteria owing to the shortcomings of underlying studies, which lacked comparability in terms of specific research question of interest, overall design, exposure assessment, outcome ascertainment, and analytic methods. Thus, even the most thoughtful and rigorous review may be of limited value if the underlying literature includes investigations that address different hypotheses and are beset by methodological inconsistencies and limitations. Issues identified in this review of reviews illustrate considerable challenges that are facing assessments of epidemiological evidence.

Using Qualitative Research to Inform Development of Professional Guidelines: A Case Study of the Society of Critical Care Medicine Family-Centered Care Guidelines.

PubMed

Coombs, Maureen A; Davidson, Judy E; Nunnally, Mark E; Wickline, Mary A; Curtis, J Randall

2017-08-01

To explore the importance, challenges, and opportunities using qualitative research to enhance development of clinical practice guidelines, using recent guidelines for family-centered care in the ICU as an example. In developing the Society of Critical Care Medicine guidelines for family-centered care in the neonatal ICU, PICU, and adult ICU, we developed an innovative adaptation of the Grading of Recommendations, Assessments, Development and Evaluations approach to explicitly incorporate qualitative research. Using Grading of Recommendations, Assessments, Development and Evaluations and the Council of Medical Specialty Societies principles, we conducted a systematic review of qualitative research to establish family-centered domains and outcomes. Thematic analyses were undertaken on study findings and used to support Population, Intervention, Comparison, Outcome question development. We identified and employed three approaches using qualitative research in these guidelines. First, previously published qualitative research was used to identify important domains for the Population, Intervention, Comparison, Outcome questions. Second, this qualitative research was used to identify and prioritize key outcomes to be evaluated. Finally, we used qualitative methods, member checking with patients and families, to validate the process and outcome of the guideline development. In this, a novel report, we provide direction for standardizing the use of qualitative evidence in future guidelines. Recommendations are made to incorporate qualitative literature review and appraisal, include qualitative methodologists in guideline taskforce teams, and develop training for evaluation of qualitative research into guideline development procedures. Effective methods of involving patients and families as members of guideline development represent opportunities for future work.

Simple neck pain questions used in surveys, evaluated in relation to health outcomes: a cohort study

PubMed Central

2012-01-01

Background The high prevalence of pain reported in many epidemiological studies, and the degree to which this prevalence reflects severe pain is under discussion in the literature. The aim of the present study was to evaluate use of the simple neck pain questions commonly included in large epidemiological survey studies with respect to aspects of health. We investigated if and how an increase in number of days with pain is associated with reduction in health outcomes. Methods A cohort of university students (baseline age 19–25 years) were recruited in 2002 and followed annually for 4 years. The baseline response rate was 69% which resulted in 1200 respondents (627 women, 573 men). Participants were asked about present and past pain and perceptions of their general health, sleep disturbance, stress and energy levels, and general performance. The data were analyzed using a mixed model for repeated measurements and a random intercept logistic model. Results When reporting present pain, participants also reported lower prevalence of very good health, higher stress and sleep disturbance scores and lower energy score. Among those with current neck pain, additional questions characterizing the pain such as duration (categorized), additional pain sites and decreased general performance were associated with lower probability of very good health and higher amounts of sleep disturbance. Knowing about the presence or not of pain explains more of the variation in health between individuals, than within individuals. Conclusion This study of young university students has demonstrated that simple neck pain survey questions capture features of pain that affect aspects of health such as perceived general health, sleep disturbance, mood in terms of stress and energy. Simple pain questions are more useful for group descriptions than for describing or following pain in an individual. PMID:23102060

Optimizing household survey methods to monitor the Sustainable Development Goals targets 6.1 and 6.2 on drinking water, sanitation and hygiene: A mixed-methods field-test in Belize.

PubMed

Khan, Shane M; Bain, Robert E S; Lunze, Karsten; Unalan, Turgay; Beshanski-Pedersen, Bo; Slaymaker, Tom; Johnston, Richard; Hancioglu, Attila

2017-01-01

The Sustainable Development Goals (SDGs) require household survey programmes such as the UNICEF-supported Multiple Indicator Cluster Surveys (MICS) to enhance data collection to cover new indicators. This study aims to evaluated methods for assessing water quality, water availability, emptying of sanitation facilities, menstrual hygiene management and the acceptability of water quality testing in households which are key to monitoring SDG targets 6.1 and 6.2 on drinking Water, Sanitation and Hygiene (WASH) and emerging issues. As part of a MICS field test, we interviewed 429 households and 267 women age 15-49 in Stann Creek, Belize in a split-sample experiment. In a concurrent qualitative component, we conducted focus groups with interviewers and cognitive interviews with respondents during and immediately following questionnaire administration in the field to explore their question comprehension and response processes. About 88% of respondents agreed to water quality testing but also desired test results, given the potential implications for their own health. Escherichia coli was present in 36% of drinking water collected at the source, and in 47% of samples consumed in the household. Both questions on water availability necessitated probing by interviewers. About one quarter of households reported emptying of pit latrines and septic tanks, though one-quarter could not provide an answer to the question. Asking questions on menstrual hygiene was acceptable to respondents, but required some clarification and probing. In the context of Belize, this study confirmed the feasibility of collecting information on the availability and quality of drinking water, emptying of sanitation facilities and menstrual hygiene in a multi-purpose household survey, indicating specific areas to improve question formulation and field protocols. Improvements have been incorporated into the latest round of MICS surveys which will be a major source of national data for monitoring of SDG targets for drinking water, sanitation and hygiene and emerging issues for WASH sector programming.

A Deep Learning-Based Method for Similar Patient Question Retrieval in Chinese.

PubMed

Tang, Guo Yu; Ni, Yuan; Xie, Guo Tong; Fan, Xin Li; Shi, Yan Ling

2017-01-01

The online patient question and answering (Q&A) system, either as a website or a mobile application, attracts an increasing number of users in China. Patients will post their questions and the registered doctors then provide the corresponding answers. A large amount of questions with answers from doctors are accumulated. Instead of awaiting the response from a doctor, the newly posted question could be quickly answered by finding a semantically equivalent question from the Q&A achive. In this study, we investigated a novel deep learning based method to retrieve the similar patient question in Chinese. An unsupervised learning algorithm using deep neural network is performed on the corpus to generate the word embedding. The word embedding was then used as the input to a supervised learning algorithm using a designed deep neural network, i.e. the supervised neural attention model (SNA), to predict the similarity between two questions. The experimental results showed that our SNA method achieved P@1 = 77% and P@5 = 84%, which outperformed all other compared methods.

Multiple Questions Require Multiple Designs: An Evaluation of the 1981 Changes to the AFDC Program.

ERIC Educational Resources Information Center

Hedrick, Terry E.; Shipman, Stephanie L.

1988-01-01

Changes made in 1981 to the Aid to Families with Dependent Children (AFDC) program under the Omnibus Budget Reconciliation Act were evaluated. Multiple quasi-experimental designs (interrupted time series, non-equivalent comparison groups, and simple pre-post designs) used to address evaluation questions illustrate the issues faced by evaluators in…

The Effects of Head Start Health Services: Executive Summary of the Head Start Health Evaluation.

ERIC Educational Resources Information Center

Fosburg, Linda B.; And Others

This report summarizes findings of an evaluation of Head Start health services. Chapter one presents an overview of the background of the evaluation project. Chapter two highlights findings for the major evaluation questions. These questions focus specifically on children's health status prior to entry into Head Start, health services subsequently…

Defining the Scope of Your Evaluation

ERIC Educational Resources Information Center

Wood, Elizabeth

2015-01-01

One challenge of conducting evaluations is finding the right questions to guide the work. A clear purpose for a study gives the evaluator a good sense of what information can answer the questions, and helps frame the scope of the project as a whole. Knowing the scope of the evaluation project provides a sense of the resources needed. A common…

Reflect before you act: providing structure to the evaluation of rehabilitation programmes.

PubMed

Velema, Johan P; Cornielje, Huib

2003-11-18

This paper is concerned with understanding and evaluating potentially diverse rehabilitation programmes. It helps evaluators and programme managers to focus attention on specific aspects of the rehabilitation process and select evaluation questions relevant to each. Distinction is made between the rehabilitation programme itself, the programme environment and the relationships between the two. For each of these areas, evaluation questions have been formulated. For services offered to individual clients, questions address whether the status of clients has improved, what interventions are offered and who benefit from them, the relationships between the service providers and the clients, and who may be involved in the rehabilitation process besides the client. To assess the programme environment, questions address the epidemiology of disability, the resources available to persons with disabilities, the inclusiveness of education and employment and a number of eco-social variables. Relationships between the programme and its environment concern the support of the community for the programme, the way the programme seeks to influence the community, the referral of clients to other services available in the community and the extent to which the programme is a learning organization. Lists of evaluation questions are presented from which the evaluator can select those most relevant to the programme to be evaluated. This provides a framework for the evaluation and for the information to be gathered. Rather than providing a blue print, this framework permits flexibility to adapt to the specific situation of the programme to be evaluated. This paper presents a useful guideline that stimulates the thinking of those preparing for the evaluation of rehabilitation programmes.

Cervical vertebrae maturation method morphologic criteria: poor reproducibility.

PubMed

Nestman, Trenton S; Marshall, Steven D; Qian, Fang; Holton, Nathan; Franciscus, Robert G; Southard, Thomas E

2011-08-01

The cervical vertebrae maturation (CVM) method has been advocated as a predictor of peak mandibular growth. A careful review of the literature showed potential methodologic errors that might influence the high reported reproducibility of the CVM method, and we recently established that the reproducibility of the CVM method was poor when these potential errors were eliminated. The purpose of this study was to further investigate the reproducibility of the individual vertebral patterns. In other words, the purpose was to determine which of the individual CVM vertebral patterns could be classified reliably and which could not. Ten practicing orthodontists, trained in the CVM method, evaluated the morphology of cervical vertebrae C2 through C4 from 30 cephalometric radiographs using questions based on the CVM method. The Fleiss kappa statistic was used to assess interobserver agreement when evaluating each cervical vertebrae morphology question for each subject. The Kendall coefficient of concordance was used to assess the level of interobserver agreement when determining a "derived CVM stage" for each subject. Interobserver agreement was high for assessment of the lower borders of C2, C3, and C4 that were either flat or curved in the CVM method, but interobserver agreement was low for assessment of the vertebral bodies of C3 and C4 when they were either trapezoidal, rectangular horizontal, square, or rectangular vertical; this led to the overall poor reproducibility of the CVM method. These findings were reflected in the Fleiss kappa statistic. Furthermore, nearly 30% of the time, individual morphologic criteria could not be combined to generate a final CVM stage because of incompatible responses to the 5 questions. Intraobserver agreement in this study was only 62%, on average, when the inconclusive stagings were excluded as disagreements. Intraobserver agreement was worse (44%) when the inconclusive stagings were included as disagreements. For the group of subjects that could be assigned a CVM stage, the level of interobserver agreement as measured by the Kendall coefficient of concordance was only 0.45, indicating moderate agreement. The weakness of the CVM method results, in part, from difficulty in classifying the vertebral bodies of C3 and C4 as trapezoidal, rectangular horizontal, square, or rectangular vertical. This led to the overall poor reproducibility of the CVM method and our inability to support its use as a strict clinical guideline for the timing of orthodontic treatment. Copyright © 2011 American Association of Orthodontists. Published by Mosby, Inc. All rights reserved.

"Follow Through" Evaluation.

ERIC Educational Resources Information Center

Glass, Gene V.; Camilli, Gregory A.

Two questions are addressed in this document: What is worth knowing about Project Follow Through? and, How should the National Institute of Education (NIE) evaluate the Follow Through program? Discussion of the first question focuses on findings of past Follow Through evaluations, problems associated with the use of experimental design and…

Cochrane Qualitative and Implementation Methods Group guidance series-paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings.

PubMed

Noyes, Jane; Booth, Andrew; Flemming, Kate; Garside, Ruth; Harden, Angela; Lewin, Simon; Pantoja, Tomas; Hannes, Karin; Cargo, Margaret; Thomas, James

2018-05-01

The Cochrane Qualitative and Implementation Methods Group develops and publishes guidance on the synthesis of qualitative and mixed-method implementation evidence. Choice of appropriate methodologies, methods, and tools is essential when developing a rigorous protocol and conducting the synthesis. Cochrane authors who conduct qualitative evidence syntheses have thus far used a small number of relatively simple methods to address similarly written questions. Cochrane has invested in methodological work to develop new tools and to encourage the production of exemplar reviews to show the value of more innovative methods that address a wider range of questions. In this paper, in the series, we report updated guidance on the selection of tools to assess methodological limitations in qualitative studies and methods to extract and synthesize qualitative evidence. We recommend application of Grades of Recommendation, Assessment, Development, and Evaluation-Confidence in the Evidence from Qualitative Reviews to assess confidence in qualitative synthesized findings. This guidance aims to support review authors to undertake a qualitative evidence synthesis that is intended to be integrated subsequently with the findings of one or more Cochrane reviews of the effects of similar interventions. The review of intervention effects may be undertaken concurrently with or separate to the qualitative evidence synthesis. We encourage further development through reflection and formal testing. Copyright © 2017 Elsevier Inc. All rights reserved.

The effect of question order on evaluations of test performance: Can the bias dissolve?

PubMed

Bard, Gabriele; Weinstein, Yana

2017-10-01

Question difficulty order has been shown to affect students' global postdictions of test performance. We attempted to eliminate the bias by letting participants experience the question order manipulation multiple times. In all three experiments, participants answered general knowledge questions and self-evaluated their performance. In Experiment 1, participants studied questions and answers in easy-hard or hard-easy question order prior to taking a test in the same order. In Experiment 2, participants took the same test twice in the opposite question order (easy-hard then hard-easy, or hard-easy then easy-hard). In Experiment 3, participants took two different tests in the opposite question order (easy-hard then hard-easy, or hard-easy then easy-hard). In all three experiments, we were unable to eliminate the bias, which suggests that repeated exposure is insufficient to overcome a strong initial anchor.

Optimizing household survey methods to monitor the Sustainable Development Goals targets 6.1 and 6.2 on drinking water, sanitation and hygiene: A mixed-methods field-test in Belize

PubMed Central

Bain, Robert E. S.; Lunze, Karsten; Unalan, Turgay; Beshanski-Pedersen, Bo; Slaymaker, Tom; Johnston, Richard; Hancioglu, Attila

2017-01-01

Background The Sustainable Development Goals (SDGs) require household survey programmes such as the UNICEF-supported Multiple Indicator Cluster Surveys (MICS) to enhance data collection to cover new indicators. This study aims to evaluated methods for assessing water quality, water availability, emptying of sanitation facilities, menstrual hygiene management and the acceptability of water quality testing in households which are key to monitoring SDG targets 6.1 and 6.2 on drinking Water, Sanitation and Hygiene (WASH) and emerging issues. Methods As part of a MICS field test, we interviewed 429 households and 267 women age 15–49 in Stann Creek, Belize in a split-sample experiment. In a concurrent qualitative component, we conducted focus groups with interviewers and cognitive interviews with respondents during and immediately following questionnaire administration in the field to explore their question comprehension and response processes. Findings About 88% of respondents agreed to water quality testing but also desired test results, given the potential implications for their own health. Escherichia coli was present in 36% of drinking water collected at the source, and in 47% of samples consumed in the household. Both questions on water availability necessitated probing by interviewers. About one quarter of households reported emptying of pit latrines and septic tanks, though one-quarter could not provide an answer to the question. Asking questions on menstrual hygiene was acceptable to respondents, but required some clarification and probing. Conclusions In the context of Belize, this study confirmed the feasibility of collecting information on the availability and quality of drinking water, emptying of sanitation facilities and menstrual hygiene in a multi-purpose household survey, indicating specific areas to improve question formulation and field protocols. Improvements have been incorporated into the latest round of MICS surveys which will be a major source of national data for monitoring of SDG targets for drinking water, sanitation and hygiene and emerging issues for WASH sector programming. PMID:29216244

Duct Leakage Repeatability Testing

DOE Office of Scientific and Technical Information (OSTI.GOV)

Walker, Iain; Sherman, Max

2014-08-01

The purpose of this report is to evaluate the repeatability of the three most significant measurement techniques for duct leakage using data from the literature and recently obtained field data. We will also briefly discuss the first two factors. The main question to be answered by this study is to determine if differences in the repeatability of these tests methods is sufficient to indicate that any of these methods is so poor that it should be excluded from consideration as an allowed procedure in codes and standards. The three duct leak measurement methods assessed in this report are the twomore » duct pressurization methods that are commonly used by many practitioners and the DeltaQ technique. These are methods B, C and A, respectively of the ASTM E1554 standard. Although it would be useful to evaluate other duct leak test methods, this study focused on those test methods that are commonly used and are required in various test standards, such as BPI (2010), RESNET (2014), ASHRAE 62.2 (2013), California Title 24 (CEC 2012), DOE Weatherization and many other energy efficiency programs.« less

Intercultural Ethics: Questions of Methods in Language and Intercultural Communication

ERIC Educational Resources Information Center

Phipps, Alison

2013-01-01

This paper explores how questions of ethics and questions of method are intertwined and unavoidable in any serious study of language and intercultural communication. It argues that the focus on difference and solution orientations to intercultural conflict has been a fundamental driver for theory, data collection and methods in the field. These…

A Comparison of Methods for Transforming Sentences into Test Questions for Instructional Materials. Technical Report #1.

ERIC Educational Resources Information Center

Roid, Gale; And Others

Several measurement theorists have convincingly argued that methods of writing test questions, particularly for criterion-referenced tests, should be based on operationally defined rules. This study was designed to examine and further refine a method for objectively generating multiple-choice questions for prose instructional materials. Important…

Evaluating the green practice of food service supply chain management based on fuzzy DEMATEL-ANP model

NASA Astrophysics Data System (ADS)

Li, Xiaoying; Zhu, Qinghua

2017-01-01

The question on how to evaluate a company's green practice has recently become a key strategic consideration for the food service supply chain management. This paper proposed a novel hybrid model that combines a fuzzy Decision Making Trial And Evaluation Laboratory(DEMATEL) and Analysis Network Process(ANP) methods, which developed the green restaurant criteria and demonstrated the complicated relations among various criteria to help the food service operation to better analyze the real-world situation and determine the different weight value of the criteria .The analysis of the evaluation of green practices will help the food service operation to be clear about the key measures of green practice to improve supply chain management.

Variation in choice of study design: findings from the Epidemiology Design Decision Inventory and Evaluation (EDDIE) survey.

PubMed

Stang, Paul E; Ryan, Patrick B; Overhage, J Marc; Schuemie, Martijn J; Hartzema, Abraham G; Welebob, Emily

2013-10-01

Researchers using observational data to understand drug effects must make a number of analytic design choices that suit the characteristics of the data and the subject of the study. Review of the published literature suggests that there is a lack of consistency even when addressing the same research question in the same database. To characterize the degree of similarity or difference in the method and analysis choices made by observational database research experts when presented with research study scenarios. On-line survey using research scenarios on drug-effect studies to capture method selection and analysis choices that follow a dependency branching based on response to key questions. Voluntary participants experienced in epidemiological study design solicited for participation through registration on the Observational Medical Outcomes Partnership website, membership in particular professional organizations, or links in relevant newsletters. Description (proportion) of respondents selecting particular methods and making specific analysis choices based on individual drug-outcome scenario pairs. The number of questions/decisions differed based on stem questions of study design, time-at-risk, outcome definition, and comparator. There is little consistency across scenarios, by drug or by outcome of interest, in the decisions made for design and analyses in scenarios using large healthcare databases. The most consistent choice was the cohort study design but variability in the other critical decisions was common. There is great variation among epidemiologists in the design and analytical choices that they make when implementing analyses in observational healthcare databases. These findings confirm that it will be important to generate empiric evidence to inform these decisions and to promote a better understanding of the impact of standardization on research implementation.

Ethical issues related to screening for preeclampsia.

PubMed

Jørgensen, Jennifer M; Hedley, Paula L; Gjerris, Mickey; Christiansen, Michael

2014-09-01

The implementation of new methods of treating and preventing disease raises many question of both technical and moral character. Currently, many studies focus on developing a screening test for preeclampsia (PE), a disease complicating 2-8% of pregnancies, potentially causing severe consequences for pregnant women and their fetuses. The purpose is to develop a test that can identify pregnancies at high risk for developing PE sufficiently early in pregnancy to allow for prophylaxis. However, the question of implementing a screening test for PE does not only involve an evaluation of technical feasibility and clinical efficacy, it also requires an analysis of how the test influences the conditions and choices for those tested. This study evaluates state-of-the-art techniques for preeclampsia screening in an ethical framework, pointing out the central areas of moral relevance within the context of such screening activity. Furthermore, we propose ethical guidelines that a screening programme for PE should meet in order to become an uncontroversial addition to prenatal health care. © 2012 John Wiley & Sons Ltd.

Immunology and Homeopathy. 4. Clinical Studies—Part 2

PubMed Central

Bellavite, Paolo; Ortolani, Riccardo; Pontarollo, Francesco; Piasere, Valeria; Benato, Giovanni; Conforti, Anita

2006-01-01

The clinical studies on the effectiveness of homeopathy in respiratory allergy (18 randomized trials and 9 observational studies) are described. The literature of common immunologic disorders including also upper respiratory tract infections (URTI) and otorhinolaryngology (reported in part 1), is evaluated and discussed. Most of initial evidence-based research was addressed to the question of whether homeopathic high dilutions are placebos or possess specific effects, but this question has been often equivocal and is still a matter of debate. The evidence demonstrates that in some conditions homeopathy shows significant promise, e.g. Galphimia glauca (low dilutions/potencies) in allergic oculorhinitis, classical individualized homeopathy in otitis and possibly in asthma and allergic complaints, and a few low-potency homeopathic complexes in sinusitis and rhinoconjunctivitis. A general weakness of evidence derives from lack of independent confirmation of reported trials and from presence of conflicting results, as in case of homeopathic immunotherapy and of classical homeopathy for URTI. The suitable methods to evaluate homeopathy effectiveness, without altering the setting of cure, are also analyzed. PMID:17173103

Modeling the hydrophysical soil properties as a part of self-regulated flood dams projection in gis-environment for sustainable urban development

NASA Astrophysics Data System (ADS)

Nikonorov, Aleksandr; Terleev, Vitaly; Badenko, Vladimir; Mirschel, Wilfried; Abakumov, Evgeny; Ginevsky, Roman; Lazarev, Viktor; Togo, Issa; Volkova, Yulia; Melnichuk, Aleksandr; Dunaieva, Ielizaveta; Akimov, Luka

2017-10-01

The problem of flood protection measures are considered in the paper. The regulation of river flow by the system of Self-Regulated Flood Dams (SRFD) is analyzed. The method of SRFD modeling in GIS environment is proposed. The question of the ecological aspect of the SRFD management is considered based on the hydrophysical properties of the soil. The improved Mualem-Van Genuchted method is proposed for the evaluation of the possible SRFD location influence on the soil of flooded territory - the temporary reservoirs. The importance and utility of the proposed complex method is stated.

How well will you FIT? Use of a modified MMI to assess applicants’ compatibility with an emergency medicine residency program

PubMed Central

Min, Alice A.; Leetch, Aaron; Nuño, Tomas; Fiorello, Albert B.

2016-01-01

Purpose Emergency medicine residency programs have evaluated the use of Multiple Mini Interviews (MMIs) for applicants. The authors developed an MMI-style method called the Fast Interview Track (FIT) to predict an applicant's ‘fit’ within an individual residency program. Methods Applicants meet with up to five residents and are asked one question by each. Residents score the applicant using a Likert scale from 1 to 5 on two questions: ‘How well does the applicant think on his/her feet?’ and ‘How well do you think the applicant will fit in here?’. To assess how well these questions predicted a resident's ‘fit’, current residents scored fellow residents on these same questions. These scores were compared with the residents’ interview FIT scores. A postmatch survey of applicants who did not match at this program solicited applicants’ attitudes toward the FIT sessions. Results Among the junior class, the correlation between interview and current scores was significant for question 1 (rho=0.5192 [p=0.03]) and question 2 (rho=0.5753 [p=0.01]). Among seniors, Spearman's rho was statistically significant for question 2, though not statistically significant for question 1. The chi-square measure of high scores (4–5) versus low scores (1–3) found a statistically significant association between interview and current scores for interns and juniors. Of the 29 responses to the postmatch survey, 16 (55%) felt FIT sessions provided a good sense of the program's personality and only 6 (21%) disagreed. Nine (31%) felt FIT sessions positively impacted our program's ranking and 11 (38%) were ‘Neutral’. Only two (7%) reported that FIT sessions negatively impacted their ranking of our program. Conclusions FIT provided program leadership with a sense of an applicant's ‘fit’ within this program. Interview day scores correlated with scores received during residency. Most applicants report a positive experience with FIT sessions. FIT provides a useful tool to recruit applicants who fit with the residency program. PMID:26842824

DREAMTools: a Python package for scoring collaborative challenges

PubMed Central

Cokelaer, Thomas; Bansal, Mukesh; Bare, Christopher; Bilal, Erhan; Bot, Brian M.; Chaibub Neto, Elias; Eduati, Federica; de la Fuente, Alberto; Gönen, Mehmet; Hill, Steven M.; Hoff, Bruce; Karr, Jonathan R.; Küffner, Robert; Menden, Michael P.; Meyer, Pablo; Norel, Raquel; Pratap, Abhishek; Prill, Robert J.; Weirauch, Matthew T.; Costello, James C.; Stolovitzky, Gustavo; Saez-Rodriguez, Julio

2016-01-01

DREAM challenges are community competitions designed to advance computational methods and address fundamental questions in system biology and translational medicine. Each challenge asks participants to develop and apply computational methods to either predict unobserved outcomes or to identify unknown model parameters given a set of training data. Computational methods are evaluated using an automated scoring metric, scores are posted to a public leaderboard, and methods are published to facilitate community discussions on how to build improved methods. By engaging participants from a wide range of science and engineering backgrounds, DREAM challenges can comparatively evaluate a wide range of statistical, machine learning, and biophysical methods. Here, we describe DREAMTools, a Python package for evaluating DREAM challenge scoring metrics. DREAMTools provides a command line interface that enables researchers to test new methods on past challenges, as well as a framework for scoring new challenges. As of March 2016, DREAMTools includes more than 80% of completed DREAM challenges. DREAMTools complements the data, metadata, and software tools available at the DREAM website http://dreamchallenges.org and on the Synapse platform at https://www.synapse.org. Availability:  DREAMTools is a Python package. Releases and documentation are available at http://pypi.python.org/pypi/dreamtools. The source code is available at http://github.com/dreamtools/dreamtools. PMID:27134723

Using Multiple Types of Studies in Systematic Reviews of Health Care Interventions – A Systematic Review

PubMed Central

Peinemann, Frank; Tushabe, Doreen Allen; Kleijnen, Jos

2013-01-01

Background A systematic review may evaluate different aspects of a health care intervention. To accommodate the evaluation of various research questions, the inclusion of more than one study design may be necessary. One aim of this study is to find and describe articles on methodological issues concerning the incorporation of multiple types of study designs in systematic reviews on health care interventions. Another aim is to evaluate methods studies that have assessed whether reported effects differ by study types. Methods and Findings We searched PubMed, the Cochrane Database of Systematic Reviews, and the Cochrane Methodology Register on 31 March 2012 and identified 42 articles that reported on the integration of single or multiple study designs in systematic reviews. We summarized the contents of the articles qualitatively and assessed theoretical and empirical evidence. We found that many examples of reviews incorporating multiple types of studies exist and that every study design can serve a specific purpose. The clinical questions of a systematic review determine the types of design that are necessary or sufficient to provide the best possible answers. In a second independent search, we identified 49 studies, 31 systematic reviews and 18 trials that compared the effect sizes between randomized and nonrandomized controlled trials, which were statistically different in 35%, and not different in 53%. Twelve percent of studies reported both, different and non-different effect sizes. Conclusions Different study designs addressing the same question yielded varying results, with differences in about half of all examples. The risk of presenting uncertain results without knowing for sure the direction and magnitude of the effect holds true for both nonrandomized and randomized controlled trials. The integration of multiple study designs in systematic reviews is required if patients should be informed on the many facets of patient relevant issues of health care interventions. PMID:24416098

Public Opinion Poll Question Databases: An Evaluation

ERIC Educational Resources Information Center

Woods, Stephen

2007-01-01

This paper evaluates five polling resource: iPOLL, Polling the Nations, Gallup Brain, Public Opinion Poll Question Database, and Polls and Surveys. Content was evaluated on disclosure standards from major polling organizations, scope on a model for public opinion polls, and presentation on a flow chart discussing search limitations and usability.

Evaluating the One-in-Five Statistic: Women's Risk of Sexual Assault While in College.

PubMed

Muehlenhard, Charlene L; Peterson, Zoë D; Humphreys, Terry P; Jozkowski, Kristen N

In 2014, U.S. president Barack Obama announced a White House Task Force to Protect Students From Sexual Assault, noting that "1 in 5 women on college campuses has been sexually assaulted during their time there." Since then, this one-in-five statistic has permeated public discourse. It is frequently reported, but some commentators have criticized it as exaggerated. Here, we address the question, "What percentage of women are sexually assaulted while in college?" After discussing definitions of sexual assault, we systematically review available data, focusing on studies that used large, representative samples of female undergraduates and multiple behaviorally specific questions. We conclude that one in five is a reasonably accurate average across women and campuses. We also review studies that are inappropriately cited as either supporting or debunking the one-in-five statistic; we explain why they do not adequately address this question. We identify and evaluate several assumptions implicit in the public discourse (e.g., the assumption that college students are at greater risk than nonstudents). Given the empirical support for the one-in-five statistic, we suggest that the controversy occurs because of misunderstandings about studies' methods and results and because this topic has implications for gender relations, power, and sexuality; this controversy is ultimately about values.

A Pilot Study for Evaluation of Digital Systems as an Adjunct to Sphygmomanometry for Undergraduate Teaching

PubMed Central

Sharma, Renuka; Kapoor, Raj

2016-01-01

Objectives: Blood pressure estimation is a key skill for medical practitioners. It is routinely taught to undergraduate medical students using an aneroid sphygmomanometer. However, the conceptual understanding in the practical remains limited. We conducted the following study to evaluate the efficacy of digital data acquisition systems as an adjunct to the sphygmomanometer to teach blood pressure. Methods: Fifty-seven first-year medical students participated in the study. An MCQ test of 15 questions, consisting of 10 conceptual and five factual questions, was administered twice – pre- and post-demonstration of blood pressure measurement using a digital data acquisition system. In addition, qualitative feedback was also obtained. Results: Median scores were 7 (6 - 8) and 3 (1.5 - 4) in pre-test sessions for conceptual and factual questions, respectively. Post-test scores showed a significant improvement in both categories (10 (9 - 10) and 4 (4 - 4.5), respectively, Mann-Whitney U test, p < 0.0001). Student feedback also indicated that the digital system enhanced learning and student participation. Conclusions: Student feedback regarding the demonstrations was uniformly positive, which was also reflected in significantly improved post-test scores. We conclude that parallel demonstration on digital systems and the sphygmomanometer will enhance student engagement and understanding of blood pressure measurement. PMID:27660735

Performative, Arts-Based, or Arts-Informed? Reflections on the Development of Arts-Based Research in Music Therapy.

PubMed

Ledger, Alison; McCaffrey, Tríona

2015-01-01

Arts-based research (ABR) has emerged in music therapy in diverse ways, employing a range of interpretive paradigms and artistic media. It is notable that no consensus exists as to when and where the arts are included in the research process, or which music therapy topics are most suited to arts-based study. This diversity may pose challenges for music therapists who are developing, reading, and evaluating arts-based research. This paper provides an updated review of arts-based research literature in music therapy, along with four questions for researchers who are developing arts-based research. These questions are 1) When should the arts be introduced? 2) Which artistic medium is appropriate? 3) How should the art be understood? and 4) What is the role of the audience? We argue that these questions are key to understanding arts-based research, justifying methods, and evaluating claims arising from arts-based research. Rather than defining arts-based research in music therapy, we suggest that arts-based research should be understood as a flexible research strategy appropriate for exploring the complexities of music therapy practice. © the American Music Therapy Association 2015. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Tree Alignment Based on Needleman-Wunsch Algorithm for Sensor Selection in Smart Homes.

PubMed

Chua, Sook-Ling; Foo, Lee Kien

2017-08-18

Activity recognition in smart homes aims to infer the particular activities of the inhabitant, the aim being to monitor their activities and identify any abnormalities, especially for those living alone. In order for a smart home to support its inhabitant, the recognition system needs to learn from observations acquired through sensors. One question that often arises is which sensors are useful and how many sensors are required to accurately recognise the inhabitant's activities? Many wrapper methods have been proposed and remain one of the popular evaluators for sensor selection due to its superior accuracy performance. However, they are prohibitively slow during the evaluation process and may run into the risk of overfitting due to the extent of the search. Motivated by this characteristic, this paper attempts to reduce the cost of the evaluation process and overfitting through tree alignment. The performance of our method is evaluated on two public datasets obtained in two distinct smart home environments.

The case for randomized controlled trials to assess the impact of clinical information systems.

PubMed

Liu, Joseph L Y; Wyatt, Jeremy C

2011-01-01

There is a persistent view of a significant minority in the medical informatics community that the randomized controlled trial (RCT) has a limited role to play in evaluating clinical information systems. A common reason voiced by skeptics is that these systems are fundamentally different from drug interventions, so the RCT is irrelevant. There is an urgent need to promote the use of RCTs, given the shift to evidence-based policy and the need to demonstrate cost-effectiveness of these systems. The authors suggest returning to first principles and argue that what is required is clarity about how to match methods to evaluation questions. The authors address common concerns about RCTs, and the extent to which they are fallacious, and also discuss the challenges of conducting RCTs in informatics and alternative study designs when randomized trials are infeasible. While neither a perfect nor universal evaluation method, RCTs form an important part of an evaluator's toolkit.

The case for randomized controlled trials to assess the impact of clinical information systems

PubMed Central

Wyatt, Jeremy C

2011-01-01

There is a persistent view of a significant minority in the medical informatics community that the randomized controlled trial (RCT) has a limited role to play in evaluating clinical information systems. A common reason voiced by skeptics is that these systems are fundamentally different from drug interventions, so the RCT is irrelevant. There is an urgent need to promote the use of RCTs, given the shift to evidence-based policy and the need to demonstrate cost-effectiveness of these systems. The authors suggest returning to first principles and argue that what is required is clarity about how to match methods to evaluation questions. The authors address common concerns about RCTs, and the extent to which they are fallacious, and also discuss the challenges of conducting RCTs in informatics and alternative study designs when randomized trials are infeasible. While neither a perfect nor universal evaluation method, RCTs form an important part of an evaluator's toolkit. PMID:21270132

Designs of Empirical Evaluations of Nonexperimental Methods in Field Settings.

PubMed

Wong, Vivian C; Steiner, Peter M

2018-01-01

Over the last three decades, a research design has emerged to evaluate the performance of nonexperimental (NE) designs and design features in field settings. It is called the within-study comparison (WSC) approach or the design replication study. In the traditional WSC design, treatment effects from a randomized experiment are compared to those produced by an NE approach that shares the same target population. The nonexperiment may be a quasi-experimental design, such as a regression-discontinuity or an interrupted time-series design, or an observational study approach that includes matching methods, standard regression adjustments, and difference-in-differences methods. The goals of the WSC are to determine whether the nonexperiment can replicate results from a randomized experiment (which provides the causal benchmark estimate), and the contexts and conditions under which these methods work in practice. This article presents a coherent theory of the design and implementation of WSCs for evaluating NE methods. It introduces and identifies the multiple purposes of WSCs, required design components, common threats to validity, design variants, and causal estimands of interest in WSCs. It highlights two general approaches for empirical evaluations of methods in field settings, WSC designs with independent and dependent benchmark and NE arms. This article highlights advantages and disadvantages for each approach, and conditions and contexts under which each approach is optimal for addressing methodological questions.

Measuring cognition in teams: a cross-domain review.

PubMed

Wildman, Jessica L; Salas, Eduardo; Scott, Charles P R

2014-08-01

The purpose of this article is twofold: to provide a critical cross-domain evaluation of team cognition measurement options and to provide novice researchers with practical guidance when selecting a measurement method. A vast selection of measurement approaches exist for measuring team cognition constructs including team mental models, transactive memory systems, team situation awareness, strategic consensus, and cognitive processes. Empirical studies and theoretical articles were reviewed to identify all of the existing approaches for measuring team cognition. These approaches were evaluated based on theoretical perspective assumed, constructs studied, resources required, level of obtrusiveness, internal consistency reliability, and predictive validity. The evaluations suggest that all existing methods are viable options from the point of view of reliability and validity, and that there are potential opportunities for cross-domain use. For example, methods traditionally used only to measure mental models may be useful for examining transactive memory and situation awareness. The selection of team cognition measures requires researchers to answer several key questions regarding the theoretical nature of team cognition and the practical feasibility of each method. We provide novice researchers with guidance regarding how to begin the search for a team cognition measure and suggest several new ideas regarding future measurement research. We provide (1) a broad overview and evaluation of existing team cognition measurement methods, (2) suggestions for new uses of those methods across research domains, and (3) critical guidance for novice researchers looking to measure team cognition.

Evaluation of a method for comparing phosphorus loads from barnyards and croplands in Otter Creek Watershed, Wisconsin

USGS Publications Warehouse

Wierl, Judy A.; Giddings, Elise M.P.; Bannerman, Roger T.

1998-01-01

Control of phosphorus from rural nonpoint sources is a major focus of current efforts to improve and protect water resources in Wisconsin and is recommended in almost every priority watershed plan prepared for the State's Nonpoint Source (NFS) Program. Barnyards and crop- lands usually are identified as the primary rural sources of phosphorus. Numerous questions have arisen about which of these two sources to control and about the method currently being used by the NFS program to compare phosphorus loads from barnyards and croplands. To evaluate the method, the U.S. Geological Survey (USGS). in cooperation with the Wisconsin Department of Natural Resources, used phosphorus-load and sediment-load data from streams and phosphorus concentrations in soils from the Otter Creek Watershed (located in the Sheboygan River Basin: fig. 1) in conjunction with two computer-based models. 

Multicriteria decision-aid method to evaluate the performance of stormwater infiltration systems over the time.

PubMed

Moura, P; Barraud, S; Baptista, M B; Malard, F

2011-01-01

Nowadays, stormwater infiltration systems are frequently used because of their ability to reduce flows and volumes in downstream sewers, decrease overflows in surface waters and make it possible to recharge groundwater. Moreover, they come in various forms with different uses. Despite these advantages the long term sustainability of these systems is questionable and their real performances have to be assessed taking into account various and sometimes conflicting aspects. To address this problem a decision support system is proposed. It is based on a multicriteria method built to help managers to evaluate the performance of an existing infiltration system at different stages of its lifespan and identify whether it performs correctly or not, according to environmental, socio-economic, technical and sanitary aspects. The paper presents successively: the performance indicators and the way they were built, the multicriteria method to identify if the system works properly and a case study.

What Makes a Scientific Research Question Worth Investigating? Students' Epistemic Criteria and Considerations of Contribution

NASA Astrophysics Data System (ADS)

Berson, Eric Bruckner

This dissertation introduces the construct of worthwhileness as an important aspect of students' practical epistemologies of science (Sandoval, 2005). Specifically, it examines how students conceptualize what makes a scientific research question worthwhile, through a close analysis of the criteria they use for evaluating scientific research questions. Elementary (n=21) and high school students (n=21) participated in this study. As part of semi-structured interviews, students engaged in three novel tasks designed to elicit the epistemic criteria they use to evaluate scientific research questions in a variety of contexts. Findings indicate that elementary and high school students alike could engage in the practice of evaluating the worth of scientific questions. The criteria they employed included degree of interest, difficulty, and the contribution of questions to knowledge or to solving a problem. The criteria students considered varied by context. Several key differences emerged between the reasoning of the two grade cohorts. High school students tended to place more weight on the contribution of the research question. Also, the criteria reflected in the high school students' judgments of the scientific value of individual questions more closely accorded with the criteria they identified retrospectively as the basis of their judgments. Furthermore, the older cohort more often rationalized the selection and sequence of research questions within a single domain on the basis of epistemic contingency between questions. How students conceptualize what makes a scientific research question worthwhile constitutes a key aspect of students' epistemic reasoning. It is particularly important to understand how students judge the worthwhilness of scientific research questions given the central epistemic role of research questions in scientific inquiry.

Influence of Images on the Evaluation of Jams Using Conjoint Analysis Combined with Check-All-That-Apply (CATA) Questions.

PubMed

Miraballes, Marcelo; Gámbaro, Adriana

2018-01-01

A study of the influence of the use of images in a conjoint analysis combined with check-all-that apply (CATA) questions on jams was carried out. The relative importance of flavor and the information presented in the label in the willingness to purchase and the perception of how healthy the product is has been evaluated. Sixty consumers evaluated the stimuli presented only in text format (session 1), and another group of 60 consumers did so by receiving the stimuli in text format along with an image of the product (session 2). In addition, for each stimulus, consumers answered a CATA question consisting of 20 terms related to their involvement with the product. The perception of healthy increased when the texts were accompanied with images and also increased when the text included information. Willingness to purchase was only influenced by the flavor of the jams. The presence of images did not influence the CATA question's choice of terms, which were influenced by the information presented in the text. The use of a check-all-that-apply question in concepts provided an interesting possibility when they were combined with the results from the conjoint analysis, improving the comprehension of consumers' perception. Using CATA questions as an alternative way of evaluating consumer involvement seems to be beneficial and should be evaluated much further. © 2017 Institute of Food Technologists®.

Supporting Mediated Peer-Evaluation to Grade Answers to Open-Ended Questions

ERIC Educational Resources Information Center

De Marsico, Maria; Sciarrone, Filippo; Sterbini, Andrea; Temperini, Marco

2017-01-01

We show an approach to semi-automatic grading of answers given by students to open ended questions (open answers). We use both peer-evaluation and teacher evaluation. A learner is modeled by her Knowledge and her assessments quality (Judgment). The data generated by the peer- and teacher-evaluations, and by the learner models is represented by a…

User Experience Evaluations in Rehabilitation Video Games for Children: A Systematic Mapping of the Literature.

PubMed

Rico-Olarte, Carolina; López, Diego M; Blobel, Bernd; Kepplinger, Sara

2017-01-01

In recent years, the interest in user experience (UX) evaluation methods for assessing technology solutions, especially in health systems for children with special needs like cognitive disabilities, has increased. Conduct a systematic mapping study to provide an overview in the field of UX evaluations in rehabilitation video games for children. The definition of research questions, the search for primary studies and the extraction of those studies by inclusion and exclusion criteria lead to the mapping of primary papers according to a classification scheme. Main findings from this study include the detection of the target population of the selected studies, the recognition of two different ways of evaluating UX: (i) user evaluation and (ii) system evaluation, and UX measurements and devices used. This systematic mapping specifies the research gaps identified for future research works in the area.

How do we interest students in science?

NASA Astrophysics Data System (ADS)

Murray, L.

2016-02-01

In today's world science literacy is now, more than ever, critical to society. However, today's technically savvy student tends to be bored by "cook-book" laboratory exercises and dated lecture style, which typifies the way that most science courses are taught. To enhance student interest in and understanding of the sciences, we developed two unique programs, in which teachers were provided with the tools and hands-on experience that enabled them to implement research- and inquiry-based projects with their students. The approach was based a framework that is student driven and enables active participation and innovation in the study of the environment. The framework involved selection of a theme and an activity that captured the interest of the participants, participant development of research or investigative questions based on the theme, experimentation to address the research questions, formulation of conclusions, and communication of these results. The projects consisted of two parts: a professional development institute for teachers and the classroom implementation of student research projects, both of which incorporated the framework process. The institutes focused on modeling the framework process, with teachers actively developing questions, researching the question, formulating results and conclusions. This method empowered teachers to be confident in the implementation of the process with their students. With support from project staff, teachers followed up by incorporating the method of teaching with their students. Evaluation results from the programs concluded that projects such as these can increase student interest in and understanding of the scientific process.

Make pupils young researchers!

NASA Astrophysics Data System (ADS)

Gouhier, Armelle

2015-04-01

With the 2011 educational reform in France, a new course has been created in secondary schools : Methods & Practices in Science (MPS). The main goal was to improve the pupils working methods in science, including laboratory and field works. In addition, the pedagogy develops pupils autonomy and creativity, a key factor in a research process. Three teachers are working together (Mathematics, Physics and Geology-Biology), showing how different disciplines complement one another. Eventually, this is aimed at attracting more students in scientific sections. This course is optional, in the "seconde" class in French secondary schools (i.e., for 15 years old students). For the next class, they will have to choose between scientific, economic and literature sections : it is a useful option for them to decide which section has their preference. In my high-school in Clermont-Ferrand, we have chosen a research subject on hydrogeology & water quality improvement in region "Auvergne". The pupils will have to develop and set up appropriate tools to check and improve the water quality, related to different disciplines : - Geology & Biology: hydrogeology, effects of different pollutants on aquatic life, solutions to improve water quality (example of the natural water treatment zone in the lake of "Aydat, Auvergne, France"). - Physics & Chemistry: water potability criteria, pollution tests in water, water treatment plants working. - Mathematics: algorithm development, modeling on excel of the dispersion of pollutants The pedagogy of this course is new in French high-schools : pupils work in groups of three, so as to develop cooperation and autonomy. The teachers give the guidelines at the beginning of each working session, and answer the students questions when necessary. The evaluation is competence-based : instead of a mark, which is the main evaluation method in France, the pupils have to evaluate their own skills. Then, the teachers make an evaluation, and the global process is called "co-evaluation". The final purpose for each group of pupils is to present their scientific results to the others (poster, slide show). They will have to answer the questions from their colleagues & the three teachers. Eventually, the teachers evaluate specific skills about this oral presentation.

Enhancing Critical Thinking Skills for Army Leaders Using Blended-Learning Methods

DTIC Science & Technology

2013-01-01

delivering, and evaluating leader education and those who develop and implement distributed learning courses that incorporate group collaboration on topics...Circumstances Numerous studies comparing outcomes of collocated and virtual groups show that collocated groups perform better on interdependent tasks, such as...in class or “cold call” on students to answer questions. Third, using small (rather than large) groups for interactive activities can alleviate free

Cause for Concern: A Mixed-Methods Study of Campus Safety and Security Practices in United States-Mexico Border Institutions of Higher Education

ERIC Educational Resources Information Center

Holmes, Ryan Clevis

2014-01-01

Campus safety has been a source of concern since the 1990s. However, in 2007, the tragedy at the Virginia Polytechnic and State University sent a sense of alarm through many institutions of higher education. Immediately following this tragedy, institutions across the country began to evaluate and question their safety and security practices. While…

The Effect of Practical Presentations' Strategy on the Achievement of the Ninth Graders Students of the Pre-Vocational Education Subject in Shobak District Schools

ERIC Educational Resources Information Center

Al Tawarah, Haroon M.; Mahasneh, Omar M.; Al-Hawartheh, Nourah A.

2017-01-01

The present research was conducted to evaluate the effect of using practical presentations on the achievement of the ninth graders students in pre-vocational education subject in Shobak district schools, in comparison to the prevailing method, and identify whether there are differences attributed to the gender variable. To answer the questions,…

Label-indicator morpheme growth on LSTM for Chinese healthcare question department classification.

PubMed

Hu, Yang; Wen, Guihua; Ma, Jiajiong; Li, Danyang; Wang, Changjun; Li, Huihui; Huan, Eryang

2018-04-26

Current Chinese medicine has an urgent demand for convenient medical services. When facing a large number of patients, understanding patients' questions automatically and precisely is useful. Different from the high professional medical text, patients' questions contain only a small amount of descriptions regarding the symptoms, and the questions are slightly professional and colloquial. The aim of this paper is to implement a department classification system for patient questions. Patients' questions will be classified into 11 departments, such as surgery and others. This paper presents a morpheme growth model that enhances the memories of key elements in questions, and later extracts the "label-indicators" and germinates the expansion vectors around them. Finally, the model inputs the expansion vectors into a neural network to assign department labels for patients' questions. All compared methods are validated by experiments on three datasets that are composed of real patient questions. The proposed method has some ability to improve the performance of the classification. The proposed method is effective for the departments classification of patients questions and serves as a useful system for the automatic understanding of patient questions. Copyright © 2018. Published by Elsevier Inc.

Questioning Our Questions: Assessing Question Asking Practices to Evaluate a yPAR Program

ERIC Educational Resources Information Center

Grace, Sarah; Langhout, Regina Day

2014-01-01

The purpose of this research was to examine question asking practices in a youth participatory action research (yPAR) after school program housed at an elementary school. The research question was: In which ways did the adult question asking practices in a yPAR setting challenge and/or reproduce conventional models of power in educational…

Information retrieval, critical appraisal and knowledge of evidence-based dentistry among Finnish dental students.

PubMed

Nieminen, P; Virtanen, J I

2017-11-01

One of the core skills of competent dentist is the ability to search and analyse high-quality evidence. Problems in understanding the basic aspects of knowledge-based information may impede its implementation into clinical practice. We examined how Finnish dental students acquire scientific information and how familiar they are with methods for evaluating scientific evidence related to clinical questions. All fifth-year dental students (n = 120) at the three universities in Finland received a self-administered questionnaire. The three most commonly used sources of information were colleagues, the commercial Health Gate Portal for dental practitioners and personal lecture notes. Although students rarely read scientific journals, they did find that they possess at least passable or even good skills in literature retrieval. Three questions related to the appraisal of evidence in dentistry revealed that students' knowledge of evidence-based dentistry was inadequate to critically evaluate clinical research findings. Most students seem to lack knowledge of key methodological evidence-based terms. The present curricula in dental schools fail to encourage the students to search and acquire knowledge wider than their patients themselves do. Universities have the responsibility to teach dentists various methods of critical appraisal to cope with scientific information. © 2016 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Evidence acquisition and evaluation for a U.S. Government Evidence Summit on Protecting Children Outside Family Care.

PubMed

Higgs, Elizabeth S; Zlidar, Vera M; Balster, Robert L

2012-10-01

Recognizing the need for evidence to inform policies, strategies, and programs to care for vulnerable children, the U.S. Government convened an Evidence Summit on Protecting Children Outside of Family Care on December 12-13, 2011, in Washington, DC, USA. This paper summarizes the background and methods for the acquisition and evaluation of the evidence used to achieve the goals of the Summit. A multistep process was undertaken to identify the appropriate evidence for review. It began by identifying crucial focal questions intended to inform low and middle income governments and the U.S. Government about effective systems for protecting children outside family care. This was followed by a systematic attempt to gather relevant peer reviewed and gray literature that would inform these focal questions. The search processes, methods used for screening and quality reviews are described. In addition, members of the Evidence Review Teams were invited to add relevant papers not identified in the initial literature review to complete the bibliographies. These teams were asked to comply with a specific evaluation framework for recommendations on practice and policy based on both expert opinion and the quality of the data. This was the first U.S. Government Evidence Summit originating in the U.S. Agency for International Development Global Health Bureau and valuable lessons were learned on the identification and assessment of evidence informing complex development challenges. Published by Elsevier Ltd.

Reliable Analysis of Single-Unit Recordings from the Human Brain under Noisy Conditions: Tracking Neurons over Hours

PubMed Central

Boström, Jan; Elger, Christian E.; Mormann, Florian

2016-01-01

Recording extracellulary from neurons in the brains of animals in vivo is among the most established experimental techniques in neuroscience, and has recently become feasible in humans. Many interesting scientific questions can be addressed only when extracellular recordings last several hours, and when individual neurons are tracked throughout the entire recording. Such questions regard, for example, neuronal mechanisms of learning and memory consolidation, and the generation of epileptic seizures. Several difficulties have so far limited the use of extracellular multi-hour recordings in neuroscience: Datasets become huge, and data are necessarily noisy in clinical recording environments. No methods for spike sorting of such recordings have been available. Spike sorting refers to the process of identifying the contributions of several neurons to the signal recorded in one electrode. To overcome these difficulties, we developed Combinato: a complete data-analysis framework for spike sorting in noisy recordings lasting twelve hours or more. Our framework includes software for artifact rejection, automatic spike sorting, manual optimization, and efficient visualization of results. Our completely automatic framework excels at two tasks: It outperforms existing methods when tested on simulated and real data, and it enables researchers to analyze multi-hour recordings. We evaluated our methods on both short and multi-hour simulated datasets. To evaluate the performance of our methods in an actual neuroscientific experiment, we used data from from neurosurgical patients, recorded in order to identify visually responsive neurons in the medial temporal lobe. These neurons responded to the semantic content, rather than to visual features, of a given stimulus. To test our methods with multi-hour recordings, we made use of neurons in the human medial temporal lobe that respond selectively to the same stimulus in the evening and next morning. PMID:27930664

Evaluation of objective structured clinical examination for advanced orthodontic education 12 years after introduction.

PubMed

Fields, Henry W; Kim, Do-Gyoon; Jeon, Minjeong; Firestone, Allen R; Sun, Zongyang; Shanker, Shiva; Mercado, Ana M; Deguchi, Toru; Vig, Katherine W L

2017-05-01

Advanced education programs in orthodontics must ensure student competency in clinical skills. An objective structure clinical examination has been used in 1 program for over a decade. The results were analyzed cross-sectionally and longitudinally to provide insights regarding the achievement of competency, student growth, question difficulty, question discrimination, and question predictive ability. In this study, we analyzed 218 (82 first-year, 68 second-year, and 68 third-year classes) scores of each station from 85 orthodontic students. The grades originated from 13 stations and were collected anonymously for 12 consecutive years during the first 2 decades of the 2000s. The stations tested knowledge and skills regarding dental relationships, analyzing a cephalometric tracing, performing a diagnostic skill, identifying cephalometric points, bracket placement, placing first-order and second-order bends, forming a loop, placing accentuated third-order bends, identifying problems and planning mixed dentition treatment, identifying problems and planning adolescent dentition treatment, identifying problems and planning nongrowing skeletal treatment, superimposing cephalometric tracings, and interpreting cephalometric superimpositions. Results were evaluated using multivariate analysis of variance, chi-square tests, and latent growth analysis. The multivariate analysis of variance showed that all stations except 3 (analyzing a cephalometric tracing, forming a loop, and identifying cephalometric points) had significantly lower mean scores for the first-year student class than the second- and third-year classes (P <0.028); scores between the second- and third-year student classes were not significantly different (P >0.108). The chi-square analysis of the distribution of the number of noncompetent item responses decreased from the first to the second years (P <0.0003), from the second to the third years (P <0.0042), and from the first to the third years (P <0.00003). The latent growth analysis showed a wide range of difficulty and discrimination between questions. It also showed continuous growth for some areas and the ability of 6 questions to predict competency at greater than the 80% level. Objective structure clinical examinations can provide a method of evaluating student performance and curriculum impact over time, but cross-sectional and longitudinal analyses of the results may not be complementary. Significant learning appears to occur during all years of a 3-year program. Valuable questions were both easy and difficult, discriminating and not discriminating, and came from all domains: diagnostic, technical, and evaluation/synthesis. Copyright © 2017 American Association of Orthodontists. Published by Elsevier Inc. All rights reserved.

Study protocol of a mixed-methods evaluation of a cluster randomized trial to improve the safety of NSAID and antiplatelet prescribing: data-driven quality improvement in primary care.

PubMed

Grant, Aileen; Dreischulte, Tobias; Treweek, Shaun; Guthrie, Bruce

2012-08-28

Trials of complex interventions are criticized for being 'black box', so the UK Medical Research Council recommends carrying out a process evaluation to explain the trial findings. We believe it is good practice to pre-specify and publish process evaluation protocols to set standards and minimize bias. Unlike protocols for trials, little guidance or standards exist for the reporting of process evaluations. This paper presents the mixed-method process evaluation protocol of a cluster randomized trial, drawing on a framework designed by the authors. This mixed-method evaluation is based on four research questions and maps data collection to a logic model of how the data-driven quality improvement in primary care (DQIP) intervention is expected to work. Data collection will be predominately by qualitative case studies in eight to ten of the trial practices, focus groups with patients affected by the intervention and quantitative analysis of routine practice data, trial outcome and questionnaire data and data from the DQIP intervention. We believe that pre-specifying the intentions of a process evaluation can help to minimize bias arising from potentially misleading post-hoc analysis. We recognize it is also important to retain flexibility to examine the unexpected and the unintended. From that perspective, a mixed-methods evaluation allows the combination of exploratory and flexible qualitative work, and more pre-specified quantitative analysis, with each method contributing to the design, implementation and interpretation of the other.As well as strengthening the study the authors hope to stimulate discussion among their academic colleagues about publishing protocols for evaluations of randomized trials of complex interventions. DATA-DRIVEN QUALITY IMPROVEMENT IN PRIMARY CARE TRIAL REGISTRATION: ClinicalTrials.gov: NCT01425502.

A Framework for the Study of Complex mHealth Interventions in Diverse Cultural Settings

PubMed Central

Yeates, Karen; Perkins, Nancy; Boesch, Lisa; Hua-Stewart, Diane; Liu, Peter; Sleeth, Jessica; Tobe, Sheldon W

2017-01-01

Background To facilitate decision-making capacity between options of care under real-life service conditions, clinical trials must be pragmatic to evaluate mobile health (mHealth) interventions under the variable conditions of health care settings with a wide range of participants. The mHealth interventions require changes in the behavior of patients and providers, creating considerable complexity and ambiguity related to causal chains. Process evaluations of the implementation are necessary to shed light on the range of unanticipated effects an intervention may have, what the active ingredients in everyday practice are, how they exert their effect, and how these may vary among recipients or between sites. Objective Building on the CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile HEalth Applications and onLine TeleHealth) statement and participatory evaluation theory, we present a framework for the process evaluations for mHealth interventions in multiple cultural settings. We also describe the application of this evaluation framework to the implementation of DREAM-GLOBAL (Diagnosing hypertension—Engaging Action and Management in Getting Lower BP in Indigenous and LMIC [low- and middle-income countries]), a pragmatic randomized controlled trial (RCT), and mHealth intervention designed to improve hypertension management in low-resource environments. We describe the evaluation questions and the data collection processes developed by us. Methods Our literature review revealed that there is a significant knowledge gap related to the development of a process evaluation framework for mHealth interventions. We used community-based participatory research (CBPR) methods and formative research data to develop a process evaluation framework nested within a pragmatic RCT. Results Four human organizational levels of participants impacted by the mHealth intervention were identified that included patients, providers, community and organizations actors, and health systems and settings. These four levels represent evaluation domains and became the core focus of the evaluation. In addition, primary implementation themes to explore in each of the domains were identified as follows: (1) the major active components of the intervention, (2) technology of the intervention, (3) cultural congruence, (4) task shifting, and (5) unintended consequences. Using the four organizational domains and their interaction with primary implementation themes, we developed detailed evaluation research questions and identified the data or information sources to best answer our questions. Conclusions Using DREAM-GLOBAL to illustrate our approach, we succeeded in developing an uncomplicated process evaluation framework for mHealth interventions that provide key information to stakeholders, which can optimize implementation of a pragmatic trial as well as inform scale up. The human organizational level domains used to focus the primary implementation themes in the DREAM-GLOBAL process evaluation framework are sufficiently supported in our research, and the literature and can serve as a valuable tool for other mHealth process evaluations. Trial Registration ClinicalTrials.gov NCT02111226; https://clinicaltrials.gov/ct2/show/NCT02111226 (Archived by WebCite at http://www.webcitation.org/6oxfHXege) PMID:28428165

Creation and pilot testing of cases for case-based learning: A pedagogical approach for pathology cancer diagnosis

PubMed Central

Wilson, Michael; Masia, Ricard; Moloo, Zahir; Stall, Jennifer; Eslan, Alexia; Ayers, Stephanie; Mutuku, Angela

2017-01-01

Background Case-based learning (CBL) is an established pedagogical active learning method used in various disciplines and defined based on the field of study and type of case. The utility of CBL for teaching specific aspects of cancer diagnosis to practising pathologists has not been previously studied in sub-Saharan Africa. Objectives We aimed to pilot test standardised cancer cases on a group of practising pathologists in sub-Saharan Africa to evaluate case content, clarity of questions and delivery of content. Methods Expert faculty created cases for the four most commonly diagnosed cancers. The format included mini-cases and bullet cases which were all open-ended. The questions dealt with interpretation of clinical information, gross specimen examination, morphologic characteristics of tumours, ancillary testing, reporting and appropriate communication to clinicians. Results Cases on breast, cervical, prostate and colorectal cancers were tested on seven practising pathologists. Each case took an average of 45–90 min to complete. Questions that were particularly challenging to testers were on: Specimens they should have been but for some reason were not exposed to in routine practice.Ancillary testing and appropriate tumour staging. New knowledge gained included tumour grading and assessment of radial margins. Revisions to cases were made based on testers’ feedback, which included rewording of questions to reduce ambiguity and adding of tables to clarify concepts. Conclusion Cases were created for CBL in Kenya, but these are applicable elsewhere in Africa and beyond to teach cancer diagnosis. The pilot testing of cases prepared faculty for the actual CBL course and feedback provided by the testers assisted in improving the questions and impact on day-to-day practice. PMID:29147646

[Evaluation of organization of the military medical support in the peace time].

PubMed

Korshever, N G

2001-09-01

The work is devoted to investigation of some theoretical and practical aspects of organizational diagnosis of troops medical support during the peace-time. The analysis of literature, the materials obtained during the practical training of the students of Saratov Military Medical Institute, the expert questioning of the teaching stuff (14 persons) allowed to characterize the difficulties in diagnosis of different objects as well as the qualities necessary for the diagnostician. It also permitted to determine the functions, methods, classification and stages of organizational diagnosis, to substantiate the requirements necessary for evaluation of troops medical support.

Evaluating Payments for Environmental Services: Methodological Challenges

PubMed Central

2016-01-01

Over the last fifteen years, Payments for Environmental Services (PES) schemes have become very popular environmental policy instruments, but the academic literature has begun to question their additionality. The literature attempts to estimate the causal effect of these programs by applying impact evaluation (IE) techniques. However, PES programs are complex instruments and IE methods cannot be directly applied without adjustments. Based on a systematic review of the literature, this article proposes a framework for the methodological process of designing an IE for PES schemes. It revises and discusses the methodological choices at each step of the process and proposes guidelines for practitioners. PMID:26910850

Using evaluation theory in priority setting and resource allocation.

PubMed

Smith, Neale; Mitton, Craig; Cornelissen, Evelyn; Gibson, Jennifer; Peacock, Stuart

2012-01-01

Public sector interest in methods for priority setting and program or policy evaluation has grown considerably over the last several decades, given increased expectations for accountable and efficient use of resources and emphasis on evidence-based decision making as a component of good management practice. While there has been some occasional effort to conduct evaluation of priority setting projects, the literatures around priority setting and evaluation have largely evolved separately. In this paper, the aim is to bring them together. The contention is that evaluation theory is a means by which evaluators reflect upon what it is they are doing when they do evaluation work. Theories help to organize thinking, sort out relevant from irrelevant information, provide transparent grounds for particular implementation choices, and can help resolve problematic issues which may arise in the conduct of an evaluation project. A detailed review of three major branches of evaluation theory--methods, utilization, and valuing--identifies how such theories can guide the development of efforts to evaluate priority setting and resource allocation initiatives. Evaluation theories differ in terms of their guiding question, anticipated setting or context, evaluation foci, perspective from which benefits are calculated, and typical methods endorsed. Choosing a particular theoretical approach will structure the way in which any priority setting process is evaluated. The paper suggests that explicitly considering evaluation theory makes key aspects of the evaluation process more visible to all stakeholders, and can assist in the design of effective evaluation of priority setting processes; this should iteratively serve to improve the understanding of priority setting practices themselves.

Pharmacy Student Performance on Constructed-Response Versus Selected-Response Calculations Questions

PubMed Central

Addo, Richard T.

2013-01-01

Objective. To introduce PharmD students to changes in calculations question types (constructed-response versus selected-response questions); measure and compare student performance on constructed-response and selected-response questions in a pharmaceutics course; and collect student feedback on the use of differing question types. Methods A pharmaceutics/pharmaceutical calculations examination was administered that included 15 pairs of questions; each pair consisted of a constructed-response question and a similar selected-response question. An online questionnaire was conducted to collect student feedback. Results. Of the 15 topics, the class scored higher on the constructed-response question for 4 topics and higher on the selected-response question for 10 topics. Eighty percent of the class preferred selected-response questions, although 47.8% felt constructed-response questions better prepared them for a career in healthcare. Conclusions. Students correctly answered more selected-response questions than constructed-response questions and felt more confident in doing so. Additional constructed-response teaching and testing methods should be incorporated into pharmacy education. PMID:23459503

Issues in the incorporation of economic perspectives and evidence into Cochrane reviews.

PubMed

Shemilt, Ian; McDaid, David; Marsh, Kevin; Henderson, Catherine; Bertranou, Evelina; Mallander, Jacqueline; Drummond, Mike; Mugford, Miranda; Vale, Luke

2013-09-20

Methods for systematic reviews of the effects of health interventions have focused mainly on addressing the question of 'What works?' or 'Is this intervention effective in achieving one or more specific outcomes?' Addressing the question 'Is it worth it given the resources available?' has received less attention. This latter question can be addressed by applying an economic lens to the systematic review process.This paper reflects on the value and desire for the consideration by end users for coverage of an economic perspective in a Cochrane review and outlines two potential approaches and future directions. Two frameworks to guide review authors who are seeking to include an economic perspective are outlined. The first involves conducting a full systematic review of economic evaluations that is integrated into a review of intervention effects. The second involves developing a brief economic commentary. The two approaches share a set of common stages but allow the tailoring of the economic component of the Cochrane review to the skills and resources available to the review team. The number of studies using the methods outlined in the paper is limited, and further examples are needed both to explore the value of these approaches and to further develop them. The rate of progress will hinge on the organisational leadership, capacity and resources available to the CCEMG, author teams and other Cochrane entities. Particular methodological challenges to overcome relate to understanding the key economic trade-offs and casual relationships for a given decision problem and informing the development of evaluations designed to support local decision-makers. Methods for incorporating economic perspectives and evidence into Cochrane intervention reviews are established. Their role is not to provide a precise estimate of 'cost-effectiveness' but rather to help end-users of Cochrane reviews to determine the implications of the economic components of reviews for their own specific decisions.

[Gender-sensitive epidemiological data analysis: methodological aspects and empirical outcomes. Illustrated by a health reporting example].

PubMed

Jahn, I; Foraita, R

2008-01-01

In Germany gender-sensitive approaches are part of guidelines for good epidemiological practice as well as health reporting. They are increasingly claimed to realize the gender mainstreaming strategy in research funding by the federation and federal states. This paper focuses on methodological aspects of data analysis, as an empirical data example of which serves the health report of Bremen, a population-based cross-sectional study. Health reporting requires analysis and reporting methods that are able to discover sex/gender issues of questions, on the one hand, and consider how results can adequately be communicated, on the other hand. The core question is: Which consequences do a different inclusion of the category sex in different statistical analyses for identification of potential target groups have on the results? As evaluation methods logistic regressions as well as a two-stage procedure were exploratively conducted. This procedure combines graphical models with CHAID decision trees and allows for visualising complex results. Both methods are analysed by stratification as well as adjusted by sex/gender and compared with each other. As a result, only stratified analyses are able to detect differences between the sexes and within the sex/gender groups as long as one cannot resort to previous knowledge. Adjusted analyses can detect sex/gender differences only if interaction terms have been included in the model. Results are discussed from a statistical-epidemiological perspective as well as in the context of health reporting. As a conclusion, the question, if a statistical method is gender-sensitive, can only be answered by having concrete research questions and known conditions. Often, an appropriate statistic procedure can be chosen after conducting a separate analysis for women and men. Future gender studies deserve innovative study designs as well as conceptual distinctiveness with regard to the biological and the sociocultural elements of the category sex/gender.

Problems and challenges of nursing students’ clinical evaluation: A qualitative study

PubMed Central

Rafiee, Ghazanfar; Moattari, Marzieh; Nikbakht, Alireza N; Kojuri, Javad; Mousavinasab, Masoud

2014-01-01

Background: The purpose of this qualitative exploratory study was to explore the views of nursing trainers and students about nursing students’ clinical evaluation problems and drawbacks in Shiraz Nursing and Midwifery School. Materials and Methods: A qualitative exploratory approach was used in this study at Shiraz Nursing and Midwifery School in 2012. A purposeful sample of 8 nursing instructors and 40 nursing students was interviewed and the data on their opinions about the problems of the clinical evaluation were collected through semi-structured deep interviews. Initially, four open-ended questions, which were related to the clinical evaluation status, problems, were used to stimulate discussions in the interview sessions. Content analysis was employed in order to analyze the transcribed data. The recorded interviews were initially transcribed, read, and reread on a number of occasions to get an overall feeling of what the participants were saying. Each line or incident was described, and then a code, which reflected the essence of the participants’ comments, was given. Results: The codes were compared for similarity and differences, merged together, and categorized. Finally, five themes emerged: In appropriate clinical evaluation method, problems of clinical evaluation Process, problems related to clinical instructors, unsuitable programming of clinical education, and organizational shortcomings. Conclusion: Besides focusing on upgrading the current clinical evaluation forms, nursing trainers should improve their knowledge about a complete and comprehensive clinical evaluation. They should also apply other appropriate and objective clinical evaluation methods and tools, and perform a formative and summative clinical evaluation. Also, workload adjustment of the nursing trainers needs revision. Therefore, despite using traditional and sometimes limited evaluation methods for assessing nursing students, a co mprehensive and appropriate evaluation of nursing students’ clinical competencies seems necessary. PMID:24554959

Identifying Drivers of Overall Satisfaction in Patients Receiving HIV Primary Care: A Cross-Sectional Study

PubMed Central

Dang, Bich N.; Westbrook, Robert A.; Rodriguez-Barradas, Maria C.; Giordano, Thomas P.

2012-01-01

Objective This study seeks to understand the drivers of overall patient satisfaction in a predominantly low-income, ethnic-minority population of HIV primary care patients. The study’s primary aims were to determine 1) the component experiences which contribute to patients’ evaluations of their overall satisfaction with care received, and 2) the relative contribution of each component experience in explaining patients’ evaluation of overall satisfaction. Methods We conducted a cross-sectional study of 489 adult patients receiving HIV primary care at two clinics in Houston, Texas, from January 13–April 21, 2011. The participation rate among eligible patients was 94%. The survey included 15 questions about various components of the care experience, 4 questions about the provider experience and 3 questions about overall care. To ensure that the survey was appropriately tailored to our clinic population and the list of component experiences reflected all aspects of the care experience salient to patients, we conducted in-depth interviews with key providers and clinic staff and pre-tested the survey instrument with patients. Results Patients’ evaluation of their provider correlated the strongest with their overall satisfaction (standardized β = 0.445, p<0.001) and accounted for almost half of the explained variance. Access and availability, like clinic hours and ease of calling the clinic, also correlated with overall satisfaction, but less strongly. Wait time and parking, despite receiving low patient ratings, did not correlate with overall satisfaction. Conclusions The patient-provider relationship far exceeds other component experiences of care in its association with overall satisfaction. Our study suggests that interventions to improve overall patient satisfaction should focus on improving patients’ evaluation of their provider. PMID:22912770

Comparative exploration of multidimensional flow cytometry software: a model approach evaluating T cell polyfunctional behavior.

PubMed

Spear, Timothy T; Nishimura, Michael I; Simms, Patricia E

2017-08-01

Advancement in flow cytometry reagents and instrumentation has allowed for simultaneous analysis of large numbers of lineage/functional immune cell markers. Highly complex datasets generated by polychromatic flow cytometry require proper analytical software to answer investigators' questions. A problem among many investigators and flow cytometry Shared Resource Laboratories (SRLs), including our own, is a lack of access to a flow cytometry-knowledgeable bioinformatics team, making it difficult to learn and choose appropriate analysis tool(s). Here, we comparatively assess various multidimensional flow cytometry software packages for their ability to answer a specific biologic question and provide graphical representation output suitable for publication, as well as their ease of use and cost. We assessed polyfunctional potential of TCR-transduced T cells, serving as a model evaluation, using multidimensional flow cytometry to analyze 6 intracellular cytokines and degranulation on a per-cell basis. Analysis of 7 parameters resulted in 128 possible combinations of positivity/negativity, far too complex for basic flow cytometry software to analyze fully. Various software packages were used, analysis methods used in each described, and representative output displayed. Of the tools investigated, automated classification of cellular expression by nonlinear stochastic embedding (ACCENSE) and coupled analysis in Pestle/simplified presentation of incredibly complex evaluations (SPICE) provided the most user-friendly manipulations and readable output, evaluating effects of altered antigen-specific stimulation on T cell polyfunctionality. This detailed approach may serve as a model for other investigators/SRLs in selecting the most appropriate software to analyze complex flow cytometry datasets. Further development and awareness of available tools will help guide proper data analysis to answer difficult biologic questions arising from incredibly complex datasets. © Society for Leukocyte Biology.

Evaluation Design 1977-1978 ESEA Title VII Bilingual Program.

ERIC Educational Resources Information Center

Matuszek, Paula; And Others

A design for the evaluation of ESEA Title VII bilingual programs in the Austin Independent School District is presented. The following topics are included: (1) Evaluation Design Review Form; (2) decision questions including the actual questions addressed on three levels: the Office of Education-Level, the system-level and the project-level; (3) a…

An online expert network for high quality information on occupational safety and health: cross-sectional study of user satisfaction and impact

PubMed Central

2011-01-01

Background Many people have difficulties finding information on health questions, including occupational safety and health (OSH) issues. One solution to alleviate these difficulties could be to offer questioners free-of-charge, online access to a network of OSH experts who provide tailored, high-quality information. The aim of this study was to assess whether network quality, respectively information quality, as perceived by the questioners, is associated with questioners' overall satisfaction and to explore the impact of the information received on questioners' knowledge, work and work functioning. Methods We evaluated the experiences of OSH questioners with the online network ArboAntwoord.com over a two-year period. In this network, approximately 80 qualified experts are available to answer OSH questions. By means of a questionnaire, we assessed questioners' overall satisfaction with the network, whether the network was user-friendly, easily accessible and easy to handle and whether the information provided was complete, applicable and received in a timely manner. The impact of the information on questioners' knowledge, work or work functioning was explored with seven questions. In the study period, 460 unique OSH questioners asked 851 OSH questions. In total, 205 of the 460 questioners completed the questionnaire (response rate 45%). Results Of the responders, 71% were satisfied with the ArboAntwoord network. Multiple logistic regression analysis showed that the applicability of the information had a positive influence on the questioners' overall satisfaction (OR = 16.0, 95% CI: 7.0-36.4). Also, user friendliness of the network (OR = 3.3, 95% CI: 1.3-8.6) and completeness of the information provided (OR = 3.0, 95% CI: 1.3-6.8) were positively related to the questioners' satisfaction. For 74% of the questioners, the information helped to increase their knowledge and understanding. Overall, 25% of the questioners indicated that the received information improved their work, work functioning or health. Conclusions A free-of-charge, online expert network in the field of OSH can be a useful strategy to provide OSH questioners with applicable, complete and timely information that may help improve safety and health at work. This study provides more insight in how to satisfy network questioners and about the potential impact of provided information on OSH. PMID:22111587

Mirroring co-evolving trees in the light of their topologies.

PubMed

Hajirasouliha, Iman; Schönhuth, Alexander; de Juan, David; Valencia, Alfonso; Sahinalp, S Cenk

2012-05-01

Determining the interaction partners among protein/domain families poses hard computational problems, in particular in the presence of paralogous proteins. Available approaches aim to identify interaction partners among protein/domain families through maximizing the similarity between trimmed versions of their phylogenetic trees. Since maximization of any natural similarity score is computationally difficult, many approaches employ heuristics to evaluate the distance matrices corresponding to the tree topologies in question. In this article, we devise an efficient deterministic algorithm which directly maximizes the similarity between two leaf labeled trees with edge lengths, obtaining a score-optimal alignment of the two trees in question. Our algorithm is significantly faster than those methods based on distance matrix comparison: 1 min on a single processor versus 730 h on a supercomputer. Furthermore, we outperform the current state-of-the-art exhaustive search approach in terms of precision, while incurring acceptable losses in recall. A C implementation of the method demonstrated in this article is available at http://compbio.cs.sfu.ca/mirrort.htm

When ab ≠ c - c': published errors in the reports of single-mediator models.

PubMed

Petrocelli, John V; Clarkson, Joshua J; Whitmire, Melanie B; Moon, Paul E

2013-06-01

Accurate reports of mediation analyses are critical to the assessment of inferences related to causality, since these inferences are consequential for both the evaluation of previous research (e.g., meta-analyses) and the progression of future research. However, upon reexamination, approximately 15% of published articles in psychology contain at least one incorrect statistical conclusion (Bakker & Wicherts, Behavior research methods, 43, 666-678 2011), disparities that beget the question of inaccuracy in mediation reports. To quantify this question of inaccuracy, articles reporting standard use of single-mediator models in three high-impact journals in personality and social psychology during 2011 were examined. More than 24% of the 156 models coded failed an equivalence test (i.e., ab = c - c'), suggesting that one or more regression coefficients in mediation analyses are frequently misreported. The authors cite common sources of errors, provide recommendations for enhanced accuracy in reports of single-mediator models, and discuss implications for alternative methods.

Recent trends in the development and evaluation of assistive robotic manipulation devices.

PubMed

Allin, Sonya; Eckel, Emily; Markham, Heather; Brewer, Bambi R

2010-02-01

This review explores recent trends in the development and evaluation of assistive robotic arms, both prosthetic and externally mounted. Evaluations have been organized according to the CATOR taxonomy of assistive device outcomes, which takes into consideration device effectiveness, social significance, and impact on subjective well-being. Questions that have informed the review include: (1) Are robotic arms being comprehensively evaluated along axes of the CATOR taxonomy? (2) Are definitions of effectiveness in accordance with the priorities of users? (3) What gaps in robotic arm evaluation exist, and how might these best be addressed? (4) What further advances can be expected in the next 15 years? Results highlight the need for increased standardization of evaluation methods, increased emphasis on the social significance (i.e., social cost) of devices, and increased emphasis on device impact on quality of life. Several open areas for future research, in terms of both device evaluation and device development, are also discussed.

Tobacco and Nicotine Product Testing

PubMed Central

Biener, Lois; Leischow, Scott J.; Zeller, Mitch R.

2012-01-01

Introduction: Tobacco product testing is a critical component of the Family Smoking Prevention and Tobacco Control Act (FSPTCA), which grants the Food and Drug Administration the authority to regulate tobacco products. The availability of methods and measures that can provide accurate data on the relative health risks across types of tobacco products, brands, and subbrands of tobacco products on the validity of any health claims associated with a product, and on how consumers perceive information on products toxicity or risks is crucial for making decisions on the product's potential impact on public health. These tools are also necessary for making assessments of the impact of new indications for medicinal products (other than cessation) but more importantly of tobacco products that may in the future be marketed as cessation tools. Objective: To identify research opportunities to develop empirically based and comprehensive methods and measures for testing tobacco and other nicotine-containing products so that the best science is available when decisions are made about products or policies. Methods: Literature was reviewed to address sections of the FSPTCA relevant to tobacco product evaluation; research questions were generated and then reviewed by a committee of research experts. Results: A research agenda was developed for tobacco product evaluation in the general areas of toxicity and health risks, abuse liability, consumer perception, and population effects. Conclusion: A cohesive, systematic, and comprehensive assessment of tobacco products is important and will require building consensus and addressing some crucial research questions. PMID:21460383

Association between framing of the research question using the PICOT format and reporting quality of randomized controlled trials

PubMed Central

2010-01-01

Background Experts recommend formulating a structured research question to guide the research design. However, the basis for this recommendation has not been formally evaluated. The aim of this study was to examine if a structured research question using the PICOT (Population, Intervention, Comparator, Outcome, Time-frame) format is associated with a better reporting quality of randomized controlled trials (RCTs). Methods We evaluated 89 RCTs reports published in three endocrinology journals in 2005 and 2006, the quality of reporting of which was assessed in a previous study. We examined whether the reports stated each of the five elements of a structured research question: population, intervention, comparator, outcome and time-frame. A PICOT score was created with a possible score between 0 and 5. Outcomes were: 1) a 14-point overall reporting quality score (OQS) based on the Consolidated Standards for Reporting Trials; and 2) a 3-point key score (KS), based on allocation concealment, blinding and use of intention-to-treat analysis. We conducted multivariable regression analyses using generalized estimating equations to determine if a higher PICOT score or the use of a structured research question were independently associated with a better reporting quality. Journal of publication, funding source and sample size were identified as factors associated with OQS in our previous report on this dataset, and therefore included in the model. Results A higher PICOT score was independently associated with OQS (incidence rate ratio (IRR) = 1.021, 95% CI: 1.012 to 1.029) and KS (IRR = 1.142, 95% CI: 1.079 to 1.210). A structured research question was present in 33.7% of the reports and it was associated with a better OQS (IRR = 1.095, 95% CI 1.059-1.132) and KS (IRR = 1.530, 95% CI 1.311-1.786). Conclusions Better framing of the research question using the PICOT format is independently associated with better overall reporting quality - although the effect is small - and better reporting of key methodologies. PMID:20137069

Modeling ozone episodes in the Baltimore-Washington region

NASA Technical Reports Server (NTRS)

Ryan, William F.

1994-01-01

Surface ozone (O3) concentrations in excess of the National Ambient Air Quality Standard (NAAQS) continue to occur in metropolitan areas in the United States despite efforts to control emissions of O3 precursors. Future O3 control strategies will be based on results from modeling efforts that have just begun in many areas. Two initial questions that arise are model sensitivity to domain-specific conditions and the selection of episodes for model evaluation and control strategy development. For the Baltimore-Washington region (B-W), the presence of the Chesapeake Bay introduces a number of issues relevant to model sensitivity. In this paper, the specific questions of the determination of model volume (mixing height) for the Urban Airshed Model (UAM) is discussed and various alternative methods compared. For the latter question, several analytic approaches, Cluster Analysis and classification and Regression Tree (CART) analysis are undertaken to determine meteorological conditions associated with severe O3 events in the B-W domain.

Carbon Materials for Lithium Sulfur Batteries-Ten Critical Questions.

PubMed

Borchardt, Lars; Oschatz, Martin; Kaskel, Stefan

2016-05-23

Lithium-sulfur batteries are among the most promising electrochemical energy storage devices of the near future. Especially the low price and abundant availability of sulfur as the cathode material and the high theoretical capacity in comparison to state-of-the art lithium-ion technologies are attractive features. Despite significant research achievements that have been made over the last years, fundamental (electro-) chemical questions still remain unanswered. This review addresses ten crucial questions associated with lithium-sulfur batteries and critically evaluates current research with respect to them. The sulfur-carbon composite cathode is a particular focus, but its complex interplay with other hardware components in the cell, such as the electrolyte and the anode, necessitates a critical discussion of other cell components. Modern in situ characterisation methods are ideally suited to illuminate the role of each component. This article does not pretend to summarise all recently published data, but instead is a critical overview over lithium-sulfur batteries based on recent research findings. © 2016 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim.

Process-Oriented Guided Inquiry Learning Strategy Enhances Students’ Higher Level Thinking Skills in a Pharmaceutical Sciences Course

PubMed Central

Verlinden, Nathan; Kruger, Nicholas; Carroll, Ailey; Trumbo, Tiffany

2015-01-01

Objective. To determine if the process-oriented guided inquiry learning (POGIL) teaching strategy improves student performance and engages higher-level thinking skills of first-year pharmacy students in an Introduction to Pharmaceutical Sciences course. Design. Overall examination scores and scores on questions categorized as requiring either higher-level or lower-level thinking skills were compared in the same course taught over 3 years using traditional lecture methods vs the POGIL strategy. Student perceptions of the latter teaching strategy were also evaluated. Assessment. Overall mean examination scores increased significantly when POGIL was implemented. Performance on questions requiring higher-level thinking skills was significantly higher, whereas performance on questions requiring lower-level thinking skills was unchanged when the POGIL strategy was used. Student feedback on use of this teaching strategy was positive. Conclusion. The use of the POGIL strategy increased student overall performance on examinations, improved higher-level thinking skills, and provided an interactive class setting. PMID:25741027

Analyzing the BBOB results by means of benchmarking concepts.

PubMed

Mersmann, O; Preuss, M; Trautmann, H; Bischl, B; Weihs, C

2015-01-01

We present methods to answer two basic questions that arise when benchmarking optimization algorithms. The first one is: which algorithm is the "best" one? and the second one is: which algorithm should I use for my real-world problem? Both are connected and neither is easy to answer. We present a theoretical framework for designing and analyzing the raw data of such benchmark experiments. This represents a first step in answering the aforementioned questions. The 2009 and 2010 BBOB benchmark results are analyzed by means of this framework and we derive insight regarding the answers to the two questions. Furthermore, we discuss how to properly aggregate rankings from algorithm evaluations on individual problems into a consensus, its theoretical background and which common pitfalls should be avoided. Finally, we address the grouping of test problems into sets with similar optimizer rankings and investigate whether these are reflected by already proposed test problem characteristics, finding that this is not always the case.

Simulated patients in audiology education: student reports.

PubMed

Naeve-Velguth, Susan; Christensen, Sara A; Woods, Suzanne

2013-09-01

Despite increased attention in recent years to audiology counseling education, students remain concerned about their abilities to interact with patients in challenging situations, such as when breaking difficult news. Simulated patients, or actors trained to portray patients in clinical scenarios, have been used for many years in medical schools to teach and assess students' interpersonal skills, and are just beginning to be used in audiology programs. Although research suggests that medical students value simulated patient experiences, little is known about whether the same is true for audiology students. The purpose of this study was to survey audiology students who had completed a simulated patient counseling experience as part of their graduate coursework at Central Michigan University, to learn about their experiences and views of this instructional format. This study used descriptive and comparative statistics to report student observations and to determine if student responses to evaluative questions differed from chance. Study participants included 29 audiology students who had completed a "breaking difficult news" simulated patient experience as part of their required graduate coursework in patient counseling. Participants completed an online survey that included seven evaluative five-point Likert-scale questions about their simulated patient counseling experience. Participants also completed one multiple-choice question on suggestions for future simulated-patient sessions. For each of the seven evaluative questions, a majority of participants (76-100%) responded positively, agreeing or strongly agreeing that the experience was helpful to their learning. For each of these evaluative questions, a χ² analysis revealed that the distribution of positive (i.e., strongly agree and agree) to nonpositive (i.e., neutral, disagree, and strongly disagree) responses differed significantly from chance (p < .0001, df = 1). The results also indicated that when asked which of several suggested clinical scenarios would be helpful for future sessions, simulations of challenging patient types (i.e., hostile, rambling, and noncommunicative patients) were supported by most (62-90%) respondents. The results of the present study are consistent with findings of medical students' positive perceptions of simulated patient experiences as well as those previously reported for audiology students. Together, these data support the continued use of simulated patients as a method of instruction for audiology counseling education for breaking difficult news, and suggest a potential value of using simulated patient interactions for training counseling skills in other clinical situations and scenarios. American Academy of Audiology.

Evaluation of methods for managing censored results when calculating the geometric mean.

PubMed

Mikkonen, Hannah G; Clarke, Bradley O; Dasika, Raghava; Wallis, Christian J; Reichman, Suzie M

2018-01-01

Currently, there are conflicting views on the best statistical methods for managing censored environmental data. The method commonly applied by environmental science researchers and professionals is to substitute half the limit of reporting for derivation of summary statistics. This approach has been criticised by some researchers, raising questions around the interpretation of historical scientific data. This study evaluated four complete soil datasets, at three levels of simulated censorship, to test the accuracy of a range of censored data management methods for calculation of the geometric mean. The methods assessed included removal of censored results, substitution of a fixed value (near zero, half the limit of reporting and the limit of reporting), substitution by nearest neighbour imputation, maximum likelihood estimation, regression on order substitution and Kaplan-Meier/survival analysis. This is the first time such a comprehensive range of censored data management methods have been applied to assess the accuracy of calculation of the geometric mean. The results of this study show that, for describing the geometric mean, the simple method of substitution of half the limit of reporting is comparable or more accurate than alternative censored data management methods, including nearest neighbour imputation methods. Copyright © 2017 Elsevier Ltd. All rights reserved.

The need for transparency and reproducibility in documenting values for regulatory decision making and evaluating causality: The example of formaldehyde.

PubMed

Van Landingham, Cynthia; Mundt, Kenneth A; Allen, Bruce C; Gentry, P Robinan

2016-11-01

Reproducibility and transparency in scientific reporting is paramount to advancing science and providing the foundation required for sound regulation. Recent examples demonstrate that pivotal scientific findings cannot be replicated, due to poor documentation or methodological bias, sparking debate across scientific and regulatory communities. However, there is general agreement that improvements in communicating and documenting research and risk assessment methods are needed. In the case of formaldehyde, the peer-review conducted by a National Academy of Sciences (NAS) Committee questioned the approaches used by the Integrated Risk Information System (IRIS) in developing draft unit risk values. Using the original data from the key study (Beane Freeman et al., 2009) and documentation provided in the draft IRIS profile, we attempted to duplicate the reported inhalation unit risk values and address the NAS Committee's questions regarding application of the appropriate dose-response model. Overall, documentation of the methods lacked sufficient detail to allow for replication of the unit risk estimates, specifically for Hodgkin lymphoma and leukemias, the key systemic endpoints selected by IRIS. The lack of apparent exposure-response relationships for selected endpoints raises the question whether quantitative analyses are appropriate for these endpoints, and if so, how results are to be interpreted. Copyright © 2016. Published by Elsevier Inc.

Quality of Prostate Cancer Treatment Information on Cancer Center Websites.

PubMed

Dulaney, Caleb; Barrett, Olivia Claire; Rais-Bahrami, Soroush; Wakefield, Daniel; Fiveash, John; Dobelbower, Michael

2016-04-20

Cancer center websites are trusted sources of internet information about treatment options for prostate cancer. The quality of information on these websites is unknown. The objective of this study was to evaluate the quality of information on cancer center websites addressing prostate cancer treatment options, outcomes, and toxicity. We evaluated the websites of all National Cancer Institute-designated cancer centers to determine if sufficient information was provided to address eleven decision-specific knowledge questions from the validated Early Prostate Cancer Treatment Decision Quality Instrument. We recorded the number of questions addressed, the number of clicks to reach the prostate cancer-specific webpage, evaluation time, and Spanish and mobile accessibility. Correlation between evaluation time and questions addressed were calculated using the Pearson coefficient. Sixty-three websites were reviewed. Eighty percent had a prostate cancer-specific webpage reached in a median of three clicks. The average evaluation time was 6.5 minutes. Information was available in Spanish on 24% of sites and 59% were mobile friendly. Websites provided sufficient information to address, on average, 19% of questions. No website addressed all questions. Evaluation time correlated with the number of questions addressed (R(2) = 0.42, p < 0.001). Cancer center websites provide insufficient information for men with localized prostate cancer due to a lack of information about and direct comparison of specific treatment outcomes and toxicities. Information is also less accessible in Spanish and on mobile devices. These data can be used to improve the quality and accessibility of prostate cancer treatment information on cancer center websites.

The Practice of Health Program Evaluation.

PubMed

Lewis, Sarah R

2017-11-01

The Practice of Health Program Evaluation provides an overview of the evaluation process for public health programs while diving deeper to address select advanced concepts and techniques. The book unfolds evaluation as a three-phased process consisting of identification of evaluation questions, data collection and analysis, and dissemination of results and recommendations. The text covers research design, sampling methods, as well as quantitative and qualitative approaches. Types of evaluation are also discussed, including economic assessment and systems research as relative newcomers. Aspects critical to conducting a successful evaluation regardless of type or research design are emphasized, such as stakeholder engagement, validity and reliability, and adoption of sound recommendations. The book encourages evaluators to document their approach by developing an evaluation plan, a data analysis plan, and a dissemination plan, in order to help build consensus throughout the process. The evaluative text offers a good bird's-eye view of the evaluation process, while offering guidance for evaluation experts on how to navigate political waters and advocate for their findings to help affect change.

To Refer or Not To Refer...What Are the Questions? Evaluative Questions for Procedural Implementation of an Effective Prereferral Phase for Culturally Diverse and Limited English Proficient Students.

ERIC Educational Resources Information Center

Ruz, Magdalena

Culturally specific behaviors must be considered in the pre-referral phase of assessing culturally diverse/limited-English-proficient students. Inferences regarding the reasons for students' performance reflect the knowledge base acquired by the school staff concerning a student's culture. Evaluative questions can gather data to assure non-biased…

HEALTH TECHNOLOGY ASSESSMENT OF PUBLIC HEALTH INTERVENTIONS: A SYNTHESIS OF METHODOLOGICAL GUIDANCE.

PubMed

Mathes, Tim; Antoine, Sunya-Lee; Prengel, Peggy; Bühn, Stefanie; Polus, Stephanie; Pieper, Dawid

2017-01-01

The evaluation of public health interventions poses some challenges. As a consequence, health technology assessment (HTA) methods for public health interventions (PHI) have to be adapted. This study aimed to summarize the available guidance on methods for HTA of PHI. We systematically searched for methodological guidance on HTA of PHIs. Our focus was on research synthesis methods to evaluate effectiveness. Relevant information was synthesized narratively in a standardized way. Only four guidance documents were identified specifically for HTAs of PHI. The approaches used for HTAs of PHIs are broader and more flexible than those for medical interventions. For this reason, there is a tendency to identify the intervention components and context factors that influence the effectiveness and transferability of an intervention rather than to assess its effectiveness in general. The details in the guidance vary without justification. Unjustified heterogeneity between the different guidance approaches is most pronounced for quality assessment, assessment of applicability, and methods to integrate qualitative and quantitative evidence. Descriptions for the assessment of integrity, heterogeneity, sustainability, context factors, and applicability are often vague. The heterogeneity in approaches indicates that there is currently no consensus on methods to deal with the challenges of the PHI evaluations. A possible explanation for this may be that the methods are not sufficiently developed, and advantages and disadvantages of a certain method in relation to the research question (e.g., broad/focused) have not yet been sufficiently evaluated.

Canadian Guidelines for Controlled Pediatric Donation After Circulatory Determination of Death—Summary Report*

PubMed Central

Hornby, Laura; Rochwerg, Bram; van Manen, Michael; Dhanani, ; Sonny; Sivarajan, V. Ben; Appleby, Amber; Bennett, Mary; Buchman, Daniel; Farrell, Catherine; Goldberg, Aviva; Greenberg, Rebecca; Singh, Ram; Nakagawa, Thomas A.; Witteman, William; Barter, Jill; Beck, Allon; Coughlin, Kevin; Conradi, Alf; Cupido, Cynthia; Dawson, Rosanne; Dipchand, Anne; Freed, Darren; Hornby, Karen; Langlois, Valerie; Mack, Cheryl; Mahoney, Meagan; Manhas, Deepak; Tomlinson, Christopher; Zavalkoff, Samara; Shemie, Sam D.

2017-01-01

Objectives: Create trustworthy, rigorous, national clinical practice guidelines for the practice of pediatric donation after circulatory determination of death in Canada. Methods: We followed a process of clinical practice guideline development based on World Health Organization and Canadian Medical Association methods. This included application of Grading of Recommendations Assessment, Development, and Evaluation methodology. Questions requiring recommendations were generated based on 1) 2006 Canadian donation after circulatory determination of death guidelines (not pediatric specific), 2) a multidisciplinary symposium of national and international pediatric donation after circulatory determination of death leaders, and 3) a scoping review of the pediatric donation after circulatory determination of death literature. Input from these sources drove drafting of actionable questions and Good Practice Statements, as defined by the Grading of Recommendations Assessment, Development, and Evaluation group. We performed additional literature reviews for all actionable questions. Evidence was assessed for quality using Grading of Recommendations Assessment, Development, and Evaluation and then formulated into evidence profiles that informed recommendations through the evidence-to-decision framework. Recommendations were revised through consensus among members of seven topic-specific working groups and finalized during meetings of working group leads and the planning committee. External review was provided by pediatric, critical care, and critical care nursing professional societies and patient partners. Results: We generated 63 Good Practice Statements and seven Grading of Recommendations Assessment, Development, and Evaluation recommendations covering 1) ethics, consent, and withdrawal of life-sustaining therapy, 2) eligibility, 3) withdrawal of life-sustaining therapy practices, 4) ante and postmortem interventions, 5) death determination, 6) neonatal pediatric donation after circulatory determination of death, 7) cardiac and innovative pediatric donation after circulatory determination of death, and 8) implementation. For brevity, 48 Good Practice Statement and truncated justification are included in this summary report. The remaining recommendations, detailed methodology, full Grading of Recommendations Assessment, Development, and Evaluation tables, and expanded justifications are available in the full text report. Conclusions: This process showed that rigorous, transparent clinical practice guideline development is possible in the domain of pediatric deceased donation. Application of these recommendations will increase access to pediatric donation after circulatory determination of death across Canada and may serve as a model for future clinical practice guideline development in deceased donation. PMID:28925929

[Docimologic analysis of 4th-year preclinical exam questions].

PubMed

Gnagne-Agnero, Koffi N; Zinsou, E M; Assoumou, N M; Adiko, E F

2003-12-01

Operative Dentistry and Endodontics' Department of the School of Dentistry of Abidjan experienced pre-clinical exam in fourth year of dentistry with MCQ following guided courses which aim was to lead student to be correctly in charge of the patients when they start their first clinical performance. The objective off his this work is to show how one's can analyse exams questions efficiently. In this work the authors present et discuss the results of the evaluation of this preclinical exam performed through calculation of index of success (Ir) which gives us information on the difficulty of a question for all the students who answered, the discriminative index (Id) which allow to determine when a question is selective enough to distinguish weak to strong students in a group. The mean to evaluate is well chosen because the questions asked has a Ir between 46% et 80% (satisfying Ir) and the average Id is between 0.30 and 0.53 (Id discriminates well among 0.30 et 1). This methodology allows an evaluation of a high number of students by stocked questions.

Questioning ORACLE: An Assessment of ORACLE's Analysis of Teachers' Questions and [A Comment on "Questioning ORACLE"].

ERIC Educational Resources Information Center

Scarth, John; And Others

1986-01-01

Analysis of teachers' questions, part of the ORACLE (Observation Research and Classroom Learning Evaluation) project research, is examined in detail. Scarth and Hammersley argue that the rules ORACLE uses for identifying different types of questions involve levels of ambiguity and inference that threaten reliability and validity of the study's…

Types and Cognitive Levels of Questions Asked by Professors during College of Agriculture Class Sessions

ERIC Educational Resources Information Center

Ewing, John C.; Whittington, M. Susie

2007-01-01

One common teacher behavior exhibited in college of agriculture class sessions is oral questioning of students. Belland, Belland, and Price (1971) believed that if questioning was a noted teacher behavior, then it was important to evaluate and analyze questions asked by professors. Professors use questions to control classroom interactions,…