Step by Step: Biology Undergraduates' Problem-Solving Procedures during Multiple-Choice Assessment

ERIC Educational Resources Information Center

Prevost, Luanna B.; Lemons, Paula P.

2016-01-01

This study uses the theoretical framework of domain-specific problem solving to explore the procedures students use to solve multiple-choice problems about biology concepts. We designed several multiple-choice problems and administered them on four exams. We trained students to produce written descriptions of how they solved the problem, and this…

Developing, Analyzing, and Using Distractors for Multiple-Choice Tests in Education: A Comprehensive Review

ERIC Educational Resources Information Center

Gierl, Mark J.; Bulut, Okan; Guo, Qi; Zhang, Xinxin

2017-01-01

Multiple-choice testing is considered one of the most effective and enduring forms of educational assessment that remains in practice today. This study presents a comprehensive review of the literature on multiple-choice testing in education focused, specifically, on the development, analysis, and use of the incorrect options, which are also…

Measuring more than we know? An examination of the motivational and situational influences in science achievement

NASA Astrophysics Data System (ADS)

Haydel, Angela Michelle

The purpose of this dissertation was to advance theoretical understanding about fit between the personal resources of individuals and the characteristics of science achievement tasks. Testing continues to be pervasive in schools, yet we know little about how students perceive tests and what they think and feel while they are actually working on test items. This study focused on both the personal (cognitive and motivational) and situational factors that may contribute to individual differences in achievement-related outcomes. 387 eighth grade students first completed a survey including measures of science achievement goals, capability beliefs, efficacy related to multiple-choice items and performance assessments, validity beliefs about multiple-choice items and performance assessments, and other perceptions of these item formats. Students then completed science achievement tests including multiple-choice items and two performance assessments. A sample of students was asked to verbalize both thoughts and feelings as they worked through the test items. These think-alouds were transcribed and coded for evidence of cognitive, metacognitive and motivational engagement. Following each test, all students completed measures of effort, mood, energy level and strategy use during testing. Students reported that performance assessments were more challenging, authentic, interesting and valid than multiple-choice tests. They also believed that comparisons between students were easier using multiple-choice items. Overall, students tried harder, felt better, had higher levels of energy and used more strategies while working on performance assessments. Findings suggested that performance assessments might be more congruent with a mastery achievement goal orientation, while multiple-choice tests might be more congruent with a performance achievement goal orientation. A variable-centered analytic approach including regression analyses provided information about how students, on average, who differed in terms of their teachers' ratings of their science ability, achievement goals, capability beliefs and experiences with science achievement tasks perceived, engaged in, and performed on multiple-choice items and performance assessments. Person-centered analyses provided information about the perceptions, engagement and performance of subgroups of individuals who had different motivational characteristics. Generally, students' personal goals and capability beliefs related more strongly to test perceptions, but not performance, while teacher ratings of ability and test-specific beliefs related to performance.

Application of a Utility Analysis to Evaluate a Novel Assessment Tool for Clinically Oriented Physiology and Pharmacology

ERIC Educational Resources Information Center

Cramer, Nicholas; Asmar, Abdo; Gorman, Laurel; Gros, Bernard; Harris, David; Howard, Thomas; Hussain, Mujtaba; Salazar, Sergio; Kibble, Jonathan D.

2016-01-01

Multiple-choice questions are a gold-standard tool in medical school for assessment of knowledge and are the mainstay of licensing examinations. However, multiple-choice questions items can be criticized for lacking the ability to test higher-order learning or integrative thinking across multiple disciplines. Our objective was to develop a novel…

The Effect of Position and Format on the Difficulty of Assessment Exercises.

ERIC Educational Resources Information Center

Burton, Nancy W.; And Others

Assessment exercises (items) in three different formats--multiple-choice with an "I don't know" (IDK) option, multiple-choice without the IDK, and open-ended--were placed at the beginning, middle and end of 45-minute assessment packages (instruments). A balanced incomplete blocks analysis of variance was computed to determine the biasing…

Reducing the Need for Guesswork in Multiple-Choice Tests

ERIC Educational Resources Information Center

Bush, Martin

2015-01-01

The humble multiple-choice test is very widely used within education at all levels, but its susceptibility to guesswork makes it a suboptimal assessment tool. The reliability of a multiple-choice test is partly governed by the number of items it contains; however, longer tests are more time consuming to take, and for some subject areas, it can be…

Mining Diagnostic Assessment Data for Concept Similarity

ERIC Educational Resources Information Center

Madhyastha, Tara; Hunt, Earl

2009-01-01

This paper introduces a method for mining multiple-choice assessment data for similarity of the concepts represented by the multiple choice responses. The resulting similarity matrix can be used to visualize the distance between concepts in a lower-dimensional space. This gives an instructor a visualization of the relative difficulty of concepts…

How to Assess Student Performance in Science: Going beyond Multiple-Choice Tests. Third Edition

ERIC Educational Resources Information Center

Butler, Susan M.; McColskey, Wendy; O'Sullivan, Rita

2005-01-01

Educational systems promote student growth in a variety of dimensions. Basic content knowledge can be effectively assessed with multiple-choice and completion tests. However educational reforms have become more concerned with higher-order cognitive dimensions (problem-solving, creativity), social dimensions (communication skills, ability to work…

Measuring University students' understanding of the greenhouse effect - a comparison of multiple-choice, short answer and concept sketch assessment tools with respect to students' mental models

NASA Astrophysics Data System (ADS)

Gold, A. U.; Harris, S. E.

2013-12-01

The greenhouse effect comes up in most discussions about climate and is a key concept related to climate change. Existing studies have shown that students and adults alike lack a detailed understanding of this important concept or might hold misconceptions. We studied the effectiveness of different interventions on University-level students' understanding of the greenhouse effect. Introductory level science students were tested for their pre-knowledge of the greenhouse effect using validated multiple-choice questions, short answers and concept sketches. All students participated in a common lesson about the greenhouse effect and were then randomly assigned to one of two lab groups. One group explored an existing simulation about the greenhouse effect (PhET-lesson) and the other group worked with absorption spectra of different greenhouse gases (Data-lesson) to deepen the understanding of the greenhouse effect. All students completed the same assessment including multiple choice, short answers and concept sketches after participation in their lab lesson. 164 students completed all the assessments, 76 completed the PhET lesson and 77 completed the data lesson. 11 students missed the contrasting lesson. In this presentation we show the comparison between the multiple-choice questions, short answer questions and the concept sketches of students. We explore how well each of these assessment types represents student's knowledge. We also identify items that are indicators of the level of understanding of the greenhouse effect as measured in correspondence of student answers to an expert mental model and expert responses. Preliminary data analysis shows that student who produce concept sketch drawings that come close to expert drawings also choose correct multiple-choice answers. However, correct multiple-choice answers are not necessarily an indicator that a student produces an expert-like correlating concept sketch items. Multiple-choice questions that require detailed knowledge of the greenhouse effect (e.g. direction of re-emission of infrared energy from greenhouse gas) are significantly more likely to be answered correctly by students who also produce expert-like concept sketch items than by students who don't include this aspect in their sketch and don't answer the multiple choice questions correctly. This difference is not as apparent for less technical multiple-choice questions (e.g. type of radiation emitted by Sun). Our findings explore the formation of student's mental models throughout different interventions and how well the different assessment techniques used in this study represent the student understanding of the overall concept.

A Better Benchmark Assessment: Multiple-Choice versus Project-Based

ERIC Educational Resources Information Center

Peariso, Jamon F.

2006-01-01

The purpose of this literature review and Ex Post Facto descriptive study was to determine which type of benchmark assessment, multiple-choice or project-based, provides the best indication of general success on the history portion of the CST (California Standards Tests). The result of the study indicates that although the project-based benchmark…

Cognitive Validity: Can Multiple-Choice Items Tap Historical Thinking Processes?

ERIC Educational Resources Information Center

Smith, Mark D.

2017-01-01

Cognitive validity examines the relationship between what an assessment aims to measure and what it actually elicits from test takers. The present study examined whether multiple-choice items from the National Assessment of Educational Progress (NAEP) grade 12 U.S. history exam elicited the historical thinking processes they were designed to…

An Australian Study Comparing the Use of Multiple-Choice Questionnaires with Assignments as Interim, Summative Law School Assessment

ERIC Educational Resources Information Center

Huang, Vicki

2017-01-01

To the author's knowledge, this is the first Australian study to empirically compare the use of a multiple-choice questionnaire (MCQ) with the use of a written assignment for interim, summative law school assessment. This study also surveyed the same student sample as to what types of assessments are preferred and why. In total, 182 undergraduate…

Comparing Assessments of Students' Knowledge by Computerized Open-Ended and Multiple-Choice Tests.

ERIC Educational Resources Information Center

Anbar, Michael

1991-01-01

Interactive computerized tests accepting unrestricted natural-language input were used to assess knowledge of clinical biophysics at the State University of New York at Buffalo. Comparison of responses to open-ended sequential questions and multiple-choice questions on the same material found the two formats test different aspects of competence.…

Multiple-Choice Question Tests: A Convenient, Flexible and Effective Learning Tool? A Case Study

ERIC Educational Resources Information Center

Douglas, Mercedes; Wilson, Juliette; Ennis, Sean

2012-01-01

The research presented in this paper is part of a project investigating assessment practices, funded by the Scottish Funding Council. Using established principles of good assessment and feedback, the use of online formative and summative multiple choice tests (MCT's) was piloted to support independent and self-directed learning and improve…

Fast Assessments with Digital Tools Using Multiple-Choice Questions

ERIC Educational Resources Information Center

Howell, Dusti D.; Tseng, Daphne ChingYu; Colorado-Resa, Jozenia T.

2017-01-01

Multiple Choice Questions (MCQs) have come a long way since they were used in "The Kansas Silent Reading Test" in 1915. After over 100 years of MCQs, new innovative digital tools using this form of assessment can help foster interactivity in today's classrooms. This article describes three free online MCQ tools that are relatively quick…

A Cognitive Diagnosis Model for Cognitively Based Multiple-Choice Options

ERIC Educational Resources Information Center

de la Torre, Jimmy

2009-01-01

Cognitive or skills diagnosis models are discrete latent variable models developed specifically for the purpose of identifying the presence or absence of multiple fine-grained skills. However, applications of these models typically involve dichotomous or dichotomized data, including data from multiple-choice (MC) assessments that are scored as…

Development and Application of a Two-Tier Multiple Choice Diagnostic Instrument To Assess High School Students' Understanding of Inorganic Chemistry Qualitative Analysis.

ERIC Educational Resources Information Center

Tan, Kim Chwee Daniel; Goh, Ngoh Khang; Chia, Lian Sai; Treagust, David F.

2002-01-01

Describes the development and application of a two-tier multiple choice diagnostic instrument to assess high school students' understanding of inorganic chemistry qualitative analysis. Shows that the Grade 10 students had difficulty understanding the reactions involved in the identification of cations and anions, for example, double decomposition…

Improving Measures via Examining the Behavior of Distractors in Multiple-Choice Tests: Assessment and Remediation

ERIC Educational Resources Information Center

Sideridis, Georgios; Tsaousis, Ioannis; Al Harbi, Khaleel

2017-01-01

The purpose of the present article was to illustrate, using an example from a national assessment, the value from analyzing the behavior of distractors in measures that engage the multiple-choice format. A secondary purpose of the present article was to illustrate four remedial actions that can potentially improve the measurement of the…

Multiple-Choice Testing Using Immediate Feedback--Assessment Technique (IF AT®) Forms: Second-Chance Guessing vs. Second-Chance Learning?

ERIC Educational Resources Information Center

Merrel, Jeremy D.; Cirillo, Pier F.; Schwartz, Pauline M.; Webb, Jeffrey A.

2015-01-01

Multiple choice testing is a common but often ineffective method for evaluating learning. A newer approach, however, using Immediate Feedback Assessment Technique (IF AT®, Epstein Educational Enterprise, Inc.) forms, offers several advantages. In particular, a student learns immediately if his or her answer is correct and, in the case of an…

A Systematic Assessment of "None of the Above" on Multiple Choice Tests in a First Year Psychology Classroom

ERIC Educational Resources Information Center

Pachai, Matthew V.; DiBattista, David; Kim, Joseph A.

2015-01-01

Multiple choice writing guidelines are decidedly split on the use of "none of the above" (NOTA), with some authors discouraging and others advocating its use. Moreover, empirical studies of NOTA have produced mixed results. Generally, these studies have utilized NOTA as either the correct response or a distractor and assessed its effect…

The Development of Multiple-Choice Items Consistent with the AP Chemistry Curriculum Framework to More Accurately Assess Deeper Understanding

ERIC Educational Resources Information Center

Domyancich, John M.

2014-01-01

Multiple-choice questions are an important part of large-scale summative assessments, such as the advanced placement (AP) chemistry exam. However, past AP chemistry exam items often lacked the ability to test conceptual understanding and higher-order cognitive skills. The redesigned AP chemistry exam shows a distinctive shift in item types toward…

Do large-scale assessments measure students' ability to integrate scientific knowledge?

NASA Astrophysics Data System (ADS)

Lee, Hee-Sun

2010-03-01

Large-scale assessments are used as means to diagnose the current status of student achievement in science and compare students across schools, states, and countries. For efficiency, multiple-choice items and dichotomously-scored open-ended items are pervasively used in large-scale assessments such as Trends in International Math and Science Study (TIMSS). This study investigated how well these items measure secondary school students' ability to integrate scientific knowledge. This study collected responses of 8400 students to 116 multiple-choice and 84 open-ended items and applied an Item Response Theory analysis based on the Rasch Partial Credit Model. Results indicate that most multiple-choice items and dichotomously-scored open-ended items can be used to determine whether students have normative ideas about science topics, but cannot measure whether students integrate multiple pieces of relevant science ideas. Only when the scoring rubric is redesigned to capture subtle nuances of student open-ended responses, open-ended items become a valid and reliable tool to assess students' knowledge integration ability.

Accommodations for Multiple Choice Tests

ERIC Educational Resources Information Center

Trammell, Jack

2011-01-01

Students with learning or learning-related disabilities frequently struggle with multiple choice assessments due to difficulty discriminating between items, filtering out distracters, and framing a mental best answer. This Practice Brief suggests accommodations and strategies that disability service providers can utilize in conjunction with…

Assessing understanding of relative clauses: a comparison of multiple-choice comprehension versus sentence repetition.

PubMed

Frizelle, Pauline; O'Neill, Clodagh; Bishop, Dorothy V M

2017-11-01

Although sentence repetition is considered a reliable measure of children's grammatical knowledge, few studies have directly compared children's sentence repetition performance with their understanding of grammatical structures. The current study aimed to compare children's performance on these two assessment measures, using a multiple-choice picture-matching sentence comprehension task and a sentence repetition task. Thirty-three typically developing children completed both assessments, which included relative clauses representing a range of syntactic roles. Results revealed a similar order of difficulty of constructions on both measures but little agreement between them when evaluating individual differences. Interestingly, repetition was the easier of the two measures, with children showing the ability to repeat sentences they did not understand. This discrepancy is primarily attributed to the additional processing load resulting from the design of multiple-choice comprehension tasks, and highlights the fact that these assessments are invoking skills beyond those of linguistic competence.

Using Distractor-Driven Standards-Based Multiple-Choice Assessments and Rasch Modeling to Investigate Hierarchies of Chemistry Misconceptions and Detect Structural Problems with Individual Items

ERIC Educational Resources Information Center

Herrmann-Abell, Cari F.; DeBoer, George E.

2011-01-01

Distractor-driven multiple-choice assessment items and Rasch modeling were used as diagnostic tools to investigate students' understanding of middle school chemistry ideas. Ninety-one items were developed according to a procedure that ensured content alignment to the targeted standards and construct validity. The items were administered to 13360…

Optimal assessment of multiple cues.

PubMed Central

Fawcett, Tim W; Johnstone, Rufus A

2003-01-01

In a wide range of contexts from mate choice to foraging, animals are required to discriminate between alternative options on the basis of multiple cues. How should they best assess such complex multicomponent stimuli? Here, we construct a model to investigate this problem, focusing on a simple case where a 'chooser' faces a discrimination task involving two cues. These cues vary in their accuracy and in how costly they are to assess. As an example, we consider a mate-choice situation where females choose between males of differing quality. Our model predicts the following: (i) females should become less choosy as the cost of finding new males increases; (ii) females should prioritize cues differently depending on how choosy they are; (iii) females may sometimes prioritize less accurate cues; and (iv) which cues are most important depends on the abundance of desirable mates. These predictions are testable in mate-choice experiments where the costs of choice can be manipulated. Our findings are applicable to other discrimination tasks besides mate choice, for example a predator's choice between palatable and unpalatable prey, or an altruist's choice between kin and non-kin. PMID:12908986

Assessing Scientific Practices Using Machine-Learning Methods: How Closely Do They Match Clinical Interview Performance?

NASA Astrophysics Data System (ADS)

Beggrow, Elizabeth P.; Ha, Minsu; Nehm, Ross H.; Pearl, Dennis; Boone, William J.

2014-02-01

The landscape of science education is being transformed by the new Framework for Science Education (National Research Council, A framework for K-12 science education: practices, crosscutting concepts, and core ideas. The National Academies Press, Washington, DC, 2012), which emphasizes the centrality of scientific practices—such as explanation, argumentation, and communication—in science teaching, learning, and assessment. A major challenge facing the field of science education is developing assessment tools that are capable of validly and efficiently evaluating these practices. Our study examined the efficacy of a free, open-source machine-learning tool for evaluating the quality of students' written explanations of the causes of evolutionary change relative to three other approaches: (1) human-scored written explanations, (2) a multiple-choice test, and (3) clinical oral interviews. A large sample of undergraduates (n = 104) exposed to varying amounts of evolution content completed all three assessments: a clinical oral interview, a written open-response assessment, and a multiple-choice test. Rasch analysis was used to compute linear person measures and linear item measures on a single logit scale. We found that the multiple-choice test displayed poor person and item fit (mean square outfit >1.3), while both oral interview measures and computer-generated written response measures exhibited acceptable fit (average mean square outfit for interview: person 0.97, item 0.97; computer: person 1.03, item 1.06). Multiple-choice test measures were more weakly associated with interview measures (r = 0.35) than the computer-scored explanation measures (r = 0.63). Overall, Rasch analysis indicated that computer-scored written explanation measures (1) have the strongest correspondence to oral interview measures; (2) are capable of capturing students' normative scientific and naive ideas as accurately as human-scored explanations, and (3) more validly detect understanding than the multiple-choice assessment. These findings demonstrate the great potential of machine-learning tools for assessing key scientific practices highlighted in the new Framework for Science Education.

Government. Maryland High School Assessment.

ERIC Educational Resources Information Center

Maryland State Dept. of Education, Baltimore.

This document is a mostly multiple choice test for content given to Maryland high school students enrolled in a government course. The test is divided into 2 sessions, with 25 questions in session 1 and 56 questions in session 2. The multiple choice questions are designated as selected response questions. Other constructed response questions…

A Case Study on Multiple-Choice Testing in Anatomical Sciences

ERIC Educational Resources Information Center

Golda, Stephanie DuPont

2011-01-01

Objective testing techniques, such as multiple-choice examinations, are a widely accepted method of assessment in gross anatomy. In order to deter cheating on these types of examinations, instructors often design several versions of an examination to distribute. These versions usually involve the rearrangement of questions and their corresponding…

Guide to Developing High-Quality, Reliable, and Valid Multiple-Choice Assessments

ERIC Educational Resources Information Center

Towns, Marcy H.

2014-01-01

Chemistry faculty members are highly skilled in obtaining, analyzing, and interpreting physical measurements, but often they are less skilled in measuring student learning. This work provides guidance for chemistry faculty from the research literature on multiple-choice item development in chemistry. Areas covered include content, stem, and…

Neural bases of human mate choice: multiple value dimensions, sex difference, and self-assessment system.

PubMed

Funayama, Risa; Sugiura, Motoaki; Sassa, Yuko; Jeong, Hyeonjeong; Wakusawa, Keisuke; Horie, Kaoru; Sato, Shigeru; Kawashima, Ryuta

2012-01-01

Mate choice is an example of sophisticated daily decision making supported by multiple componential processes. In mate-choice literature, different characteristics of the value dimensions, including the sex difference in the value dimensions, and the involvement of self-assessment due to the mutual nature of the choice, have been suggested. We examined whether the brain-activation pattern during virtual mate choice would be congruent with these characteristics in terms of stimulus selectivity and activated brain regions. In measuring brain activity, young men and women were shown two pictures of either faces or behaviors, and they indicated which person they would choose either as a spouse or as a friend. Activation selective to spouse choice was observed face-selectively in men's amygdala and behavior-selectively in women's motor system. During both partner-choice conditions, behavior-selective activation was observed in the temporoparietal regions. Taking the available knowledge of these regions into account, these results are congruent with the suggested characteristics of value dimensions for physical attractiveness, parenting resources, and beneficial personality traits for a long-lasting relationship, respectively. The medial prefrontal and posterior cingulate cortices were nonselectively activated during the partner choices, suggesting the involvement of a self-assessment process. The results thus provide neuroscientific support for the multi-component mate-choice mechanism.

Using the Multiple Choice Procedure to Measure College Student Gambling

ERIC Educational Resources Information Center

Butler, Leon Harvey

2010-01-01

Research suggests that gambling is similar to addictive behaviors such as substance use. In the current study, gambling was investigated from a behavioral economics perspective. The Multiple Choice Procedure (MCP) with gambling as the target behavior was used to assess for relative reinforcing value, the effect of alternative reinforcers, and…

Assessing Multiple Choice Question (MCQ) Tests--A Mathematical Perspective

ERIC Educational Resources Information Center

Scharf, Eric M.; Baldwin, Lynne P.

2007-01-01

The reasoning behind popular methods for analysing the raw data generated by multiple choice question (MCQ) tests is not always appreciated, occasionally with disastrous results. This article discusses and analyses three options for processing the raw data produced by MCQ tests. The article shows that one extreme option is not to penalize a…

Cheating on Multiple-Choice Exams: Monitoring, Assessment, and an Optional Assignment

ERIC Educational Resources Information Center

Nath, Leda; Lovaglia, Michael

2009-01-01

Academic dishonesty is unethical. Exam cheating is viewed as more serious than most other forms (Pincus and Schmelkin 2003). The authors review the general cheating problem, introduce a program to conservatively identify likely cheaters on multiple-choice exams, and offer a procedure for handling likely cheaters. Feedback from students who confess…

Optimizing Multiple-Choice Tests as Learning Events

ERIC Educational Resources Information Center

Little, Jeri Lynn

2011-01-01

Although generally used for assessment, tests can also serve as tools for learning--but different test formats may not be equally beneficial. Specifically, research has shown multiple-choice tests to be less effective than cued-recall tests in improving the later retention of the tested information (e.g., see meta-analysis by Hamaker, 1986),…

Teaching Choice Making to Children with Visual Impairments and Multiple Disabilities in Preschool and Kindergarten Classrooms

ERIC Educational Resources Information Center

Clark, Christine; McDonnell, Andrea P.

2008-01-01

This study examined the effectiveness of an intervention package that included visual accommodations, daily preference assessments, and naturalistic instructional strategies on the accuracy of choice-making responses for three participants with visual impairments and multiple disabilities. It also examined the participants' ability to maintain and…

Comparing narrative and multiple-choice formats in online communication skill assessment.

PubMed

Kim, Sara; Spielberg, Freya; Mauksch, Larry; Farber, Stu; Duong, Cuong; Fitch, Wes; Greer, Tom

2009-06-01

We compared multiple-choice and open-ended responses collected from a web-based tool designated 'Case for Change', which had been developed for assessing and teaching medical students in the skills involved in integrating sexual risk assessment and behaviour change discussions into patient-centred primary care visits. A total of 111 Year 3 students completed the web-based tool. A series of videos from one patient encounter illustrated how a clinician uses patient-centred communication and health behaviour change skills while caring for a patient presenting with a urinary tract infection. Each video clip was followed by a request for students to respond in two ways to the question: 'What would you do next?' Firstly, students typed their statements of what they would say to the patient. Secondly, students selected from a multiple-choice list the statements that most closely resembled their free text entries. These two modes of students' answers were analysed and compared. When articulating what they would say to the patient in a narrative format, students frequently used doctor-centred approaches that focused on premature diagnostic questioning or neglected to elicit patient perspectives. Despite the instruction to select a matching statement from the multiple-choice list, students tended to choose the most exemplary patient-centred statement, which was contrary to the doctor-centred approaches reflected in their narrative responses. Open-ended questions facilitate in-depth understanding of students' educational needs, although the scoring of narrative responses is time-consuming. Multiple-choice questions allow efficient scoring and individualised feedback associated with question items but do not fully elicit students' thought processes.

Assessing reservoir operations risk under climate change

USGS Publications Warehouse

Brekke, L.D.; Maurer, E.P.; Anderson, J.D.; Dettinger, M.D.; Townsley, E.S.; Harrison, A.; Pruitt, T.

2009-01-01

Risk-based planning offers a robust way to identify strategies that permit adaptive water resources management under climate change. This paper presents a flexible methodology for conducting climate change risk assessments involving reservoir operations. Decision makers can apply this methodology to their systems by selecting future periods and risk metrics relevant to their planning questions and by collectively evaluating system impacts relative to an ensemble of climate projection scenarios (weighted or not). This paper shows multiple applications of this methodology in a case study involving California's Central Valley Project and State Water Project systems. Multiple applications were conducted to show how choices made in conducting the risk assessment, choices known as analytical design decisions, can affect assessed risk. Specifically, risk was reanalyzed for every choice combination of two design decisions: (1) whether to assume climate change will influence flood-control constraints on water supply operations (and how), and (2) whether to weight climate change scenarios (and how). Results show that assessed risk would motivate different planning pathways depending on decision-maker attitudes toward risk (e.g., risk neutral versus risk averse). Results also show that assessed risk at a given risk attitude is sensitive to the analytical design choices listed above, with the choice of whether to adjust flood-control rules under climate change having considerably more influence than the choice on whether to weight climate scenarios. Copyright 2009 by the American Geophysical Union.

The "None of the Above" Option in Multiple-Choice Testing: An Experimental Study

ERIC Educational Resources Information Center

DiBattista, David; Sinnige-Egger, Jo-Anne; Fortuna, Glenda

2014-01-01

The authors assessed the effects of using "none of the above" as an option in a 40-item, general-knowledge multiple-choice test administered to undergraduate students. Examinees who selected "none of the above" were given an incentive to write the correct answer to the question posed. Using "none of the above" as the…

Initial Correction versus Negative Marking in Multiple Choice Examinations

ERIC Educational Resources Information Center

Van Hecke, Tanja

2015-01-01

Optimal assessment tools should measure in a limited time the knowledge of students in a correct and unbiased way. A method for automating the scoring is multiple choice scoring. This article compares scoring methods from a probabilistic point of view by modelling the probability to pass: the number right scoring, the initial correction (IC) and…

Multiple Choice Test Bias Uncovered by Use of an "I Don't Know" Alternative.

ERIC Educational Resources Information Center

Sherman, Susan W.

The multiple-choice science exercises used by the National Assessment of Educational Progress include an "I Don't Know" (IDK) alternative to estimate more accurately knowledge of groups of respondents. Group percentages of IDK responses were examined and compared with correct responses to see if the IDK introduces bias. Variance common…

Multiple-Choice versus Constructed-Response Tests in the Assessment of Mathematics Computation Skills.

ERIC Educational Resources Information Center

Gadalla, Tahany M.

The equivalence of multiple-choice (MC) and constructed response (discrete) (CR-D) response formats as applied to mathematics computation at grade levels two to six was tested. The difference between total scores from the two response formats was tested for statistical significance, and the factor structure of items in both response formats was…

A Participatory Learning Approach to Biochemistry Using Student Authored and Evaluated Multiple-Choice Questions

ERIC Educational Resources Information Center

Bottomley, Steven; Denny, Paul

2011-01-01

A participatory learning approach, combined with both a traditional and a competitive assessment, was used to motivate students and promote a deep approach to learning biochemistry. Students were challenged to research, author, and explain their own multiple-choice questions (MCQs). They were also required to answer, evaluate, and discuss MCQs…

Coupled Multiple-Response versus Free-Response Conceptual Assessment: An Example from Upper-Division Physics

ERIC Educational Resources Information Center

Wilcox, Bethany R.; Pollock, Steven J.

2014-01-01

Free-response research-based assessments, like the Colorado Upper-division Electrostatics Diagnostic (CUE), provide rich, fine-grained information about students' reasoning. However, because of the difficulties inherent in scoring these assessments, the majority of the large-scale conceptual assessments in physics are multiple choice. To increase…

Effectiveness of an audience response system in teaching pharmacology to baccalaureate nursing students.

PubMed

Vana, Kimberly D; Silva, Graciela E; Muzyka, Diann; Hirani, Lorraine M

2011-06-01

It has been proposed that students' use of an audience response system, commonly called clickers, may promote comprehension and retention of didactic material. Whether this method actually improves students' grades, however, is still not determined. The purpose of this study was to evaluate whether a lecture format utilizing multiple-choice PowerPoint slides and an audience response system was more effective than a lecture format using only multiple-choice PowerPoint slides in the comprehension and retention of pharmacological knowledge in baccalaureate nursing students. The study also assessed whether the additional use of clickers positively affected students' satisfaction with their learning. Results from 78 students who attended lecture classes with multiple-choice PowerPoint slides plus clickers were compared with those of 55 students who utilized multiple-choice PowerPoint slides only. Test scores between these two groups were not significantly different. A satisfaction questionnaire showed that 72.2% of the control students did not desire the opportunity to use clickers. Of the group utilizing the clickers, 92.3% recommend the use of this system in future courses. The use of multiple-choice PowerPoint slides and an audience response system did not seem to improve the students' comprehension or retention of pharmacological knowledge as compared with those who used solely multiple-choice PowerPoint slides.

Optimizing multiple-choice tests as tools for learning.

PubMed

Little, Jeri L; Bjork, Elizabeth Ligon

2015-01-01

Answering multiple-choice questions with competitive alternatives can enhance performance on a later test, not only on questions about the information previously tested, but also on questions about related information not previously tested-in particular, on questions about information pertaining to the previously incorrect alternatives. In the present research, we assessed a possible explanation for this pattern: When multiple-choice questions contain competitive incorrect alternatives, test-takers are led to retrieve previously studied information pertaining to all of the alternatives in order to discriminate among them and select an answer, with such processing strengthening later access to information associated with both the correct and incorrect alternatives. Supporting this hypothesis, we found enhanced performance on a later cued-recall test for previously nontested questions when their answers had previously appeared as competitive incorrect alternatives in the initial multiple-choice test, but not when they had previously appeared as noncompetitive alternatives. Importantly, however, competitive alternatives were not more likely than noncompetitive alternatives to be intruded as incorrect responses, indicating that a general increased accessibility for previously presented incorrect alternatives could not be the explanation for these results. The present findings, replicated across two experiments (one in which corrective feedback was provided during the initial multiple-choice testing, and one in which it was not), thus strongly suggest that competitive multiple-choice questions can trigger beneficial retrieval processes for both tested and related information, and the results have implications for the effective use of multiple-choice tests as tools for learning.

Developing Achievement Test: A Research for Assessment of 5th Grade Biology Subject

ERIC Educational Resources Information Center

Sener, Nilay; Tas, Erol

2017-01-01

The purpose of this study is to prepare a multiple-choice achievement test with high reliability and validity for the "Let's Solve the Puzzle of Our Body" unit. For this purpose, a multiple choice achievement test consisting of 46 items was applied to 178 fifth grade students in total. As a result of the test and material analysis…

The Incidence of Clueing in Multiple Choice Testbank Questions in Accounting: Some Evidence from Australia

ERIC Educational Resources Information Center

Ibbett, Nicole L.; Wheldon, Brett J.

2016-01-01

In 2014 Central Queensland University (CQU) in Australia banned the use of multiple choice questions (MCQs) as an assessment tool. One of the reasons given for this decision was that MCQs provide an opportunity for students to "pass" by merely guessing their answers. The mathematical likelihood of a student passing by guessing alone can…

More than the Verbal Stimulus Matters: Visual Attention in Language Assessment for People with Aphasia Using Multiple-Choice Image Displays

ERIC Educational Resources Information Center

Heuer, Sabine; Ivanova, Maria V.; Hallowell, Brooke

2017-01-01

Purpose: Language comprehension in people with aphasia (PWA) is frequently evaluated using multiple-choice displays: PWA are asked to choose the image that best corresponds to the verbal stimulus in a display. When a nontarget image is selected, comprehension failure is assumed. However, stimulus-driven factors unrelated to linguistic…

A Method for Imputing Response Options for Missing Data on Multiple-Choice Assessments

ERIC Educational Resources Information Center

Wolkowitz, Amanda A.; Skorupski, William P.

2013-01-01

When missing values are present in item response data, there are a number of ways one might impute a correct or incorrect response to a multiple-choice item. There are significantly fewer methods for imputing the actual response option an examinee may have provided if he or she had not omitted the item either purposely or accidentally. This…

Incorporating Multiple-Choice Questions into an AACSB Assurance of Learning Process: A Course-Embedded Assessment Application to an Introductory Finance Course

ERIC Educational Resources Information Center

Santos, Michael R.; Hu, Aidong; Jordan, Douglas

2014-01-01

The authors offer a classification technique to make a quantitative skills rubric more operational, with the groupings of multiple-choice questions to match the student learning levels in knowledge, calculation, quantitative reasoning, and analysis. The authors applied this classification technique to the mid-term exams of an introductory finance…

Predictive Validity of a Multiple-Choice Test for Placement in a Community College

ERIC Educational Resources Information Center

Verbout, Mary F.

2013-01-01

Multiple-choice tests of punctuation and usage are used throughout the United States to assess the writing skills of new community college students in order to place them in either a basic writing course or first-year composition. To determine whether using the COMPASS Writing Test (CWT) is a valid placement at a community college, student test…

Dynamic Testing of Analogical Reasoning in 5- to 6-Year-Olds: Multiple-Choice versus Constructed-Response Training Items

ERIC Educational Resources Information Center

Stevenson, Claire E.; Heiser, Willem J.; Resing, Wilma C. M.

2016-01-01

Multiple-choice (MC) analogy items are often used in cognitive assessment. However, in dynamic testing, where the aim is to provide insight into potential for learning and the learning process, constructed-response (CR) items may be of benefit. This study investigated whether training with CR or MC items leads to differences in the strategy…

Emotion and decision making: multiple modulatory neural circuits.

PubMed

Phelps, Elizabeth A; Lempert, Karolina M; Sokol-Hessner, Peter

2014-01-01

Although the prevalent view of emotion and decision making is derived from the notion that there are dual systems of emotion and reason, a modulatory relationship more accurately reflects the current research in affective neuroscience and neuroeconomics. Studies show two potential mechanisms for affect's modulation of the computation of subjective value and decisions. Incidental affective states may carry over to the assessment of subjective value and the decision, and emotional reactions to the choice may be incorporated into the value calculation. In addition, this modulatory relationship is reciprocal: Changing emotion can change choices. This research suggests that the neural mechanisms mediating the relation between affect and choice vary depending on which affective component is engaged and which decision variables are assessed. We suggest that a detailed and nuanced understanding of emotion and decision making requires characterizing the multiple modulatory neural circuits underlying the different means by which emotion and affect can influence choices.

An Evaluation of the Preferences of Individuals with Severe and Multiple Disabilities and the Teaching of Choice-Making Skills

ERIC Educational Resources Information Center

Eldeniz Çetin, Müzeyyen; Safak, Pinar

2017-01-01

The general purpose of the present study is to determine the relationship between direct and indirect preference assessments of individuals with severe and multiple disabilities (SMD) and the relationship between the direct preference assessments (single-stimulus, paired-stimulus, and multiple-stimulus) as applied to individuals with SMD, and to…

Assessment of item-writing flaws in multiple-choice questions.

PubMed

Nedeau-Cayo, Rosemarie; Laughlin, Deborah; Rus, Linda; Hall, John

2013-01-01

This study evaluated the quality of multiple-choice questions used in a hospital's e-learning system. Constructing well-written questions is fraught with difficulty, and item-writing flaws are common. Study results revealed that most items contained flaws and were written at the knowledge/comprehension level. Few items had linked objectives, and no association was found between the presence of objectives and flaws. Recommendations include education for writing test questions.

Exploring undergraduates' understanding of photosynthesis using diagnostic question clusters.

PubMed

Parker, Joyce M; Anderson, Charles W; Heidemann, Merle; Merrill, John; Merritt, Brett; Richmond, Gail; Urban-Lurain, Mark

2012-01-01

We present a diagnostic question cluster (DQC) that assesses undergraduates' thinking about photosynthesis. This assessment tool is not designed to identify individual misconceptions. Rather, it is focused on students' abilities to apply basic concepts about photosynthesis by reasoning with a coordinated set of practices based on a few scientific principles: conservation of matter, conservation of energy, and the hierarchical nature of biological systems. Data on students' responses to the cluster items and uses of some of the questions in multiple-choice, multiple-true/false, and essay formats are compared. A cross-over study indicates that the multiple-true/false format shows promise as a machine-gradable format that identifies students who have a mixture of accurate and inaccurate ideas. In addition, interviews with students about their choices on three multiple-choice questions reveal the fragility of students' understanding. Collectively, the data show that many undergraduates lack both a basic understanding of the role of photosynthesis in plant metabolism and the ability to reason with scientific principles when learning new content. Implications for instruction are discussed.

Exploring Undergraduates' Understanding of Photosynthesis Using Diagnostic Question Clusters

PubMed Central

Parker, Joyce M.; Anderson, Charles W.; Heidemann, Merle; Merrill, John; Merritt, Brett; Richmond, Gail; Urban-Lurain, Mark

2012-01-01

We present a diagnostic question cluster (DQC) that assesses undergraduates' thinking about photosynthesis. This assessment tool is not designed to identify individual misconceptions. Rather, it is focused on students' abilities to apply basic concepts about photosynthesis by reasoning with a coordinated set of practices based on a few scientific principles: conservation of matter, conservation of energy, and the hierarchical nature of biological systems. Data on students' responses to the cluster items and uses of some of the questions in multiple-choice, multiple-true/false, and essay formats are compared. A cross-over study indicates that the multiple-true/false format shows promise as a machine-gradable format that identifies students who have a mixture of accurate and inaccurate ideas. In addition, interviews with students about their choices on three multiple-choice questions reveal the fragility of students' understanding. Collectively, the data show that many undergraduates lack both a basic understanding of the role of photosynthesis in plant metabolism and the ability to reason with scientific principles when learning new content. Implications for instruction are discussed. PMID:22383617

Teaching Individuals with Profound Multiple Disabilities to Access Preferred Stimuli with Multiple Microswitches

ERIC Educational Resources Information Center

Tam, Gee May; Phillips, Katrina J.; Mudford, Oliver C.

2011-01-01

We replicated and extended previous research on microswitch facilitated choice making by individuals with profound multiple disabilities. Following an assessment of stimulus preferences, we taught 6 adults with profound multiple disabilities to emit 2 different responses to activate highly preferred stimuli. All participants learnt to activate…

Using the Multiple-Choice Procedure to Measure the Relative Reinforcing Efficacy of Gambling: Initial Validity Evidence Among College Students.

PubMed

Butler, Leon H; Irons, Jessica G; Bassett, Drew T; Correia, Christopher J

2018-06-01

The multiple choice procedure (MCP) is used to assess the relative reinforcing value of concurrently available stimuli. The MCP was originally developed to assess the reinforcing value of drugs; the current within-subjects study employed the MCP to assess the reinforcing value of gambling behavior. Participants (N = 323) completed six versions of the MCP that presented hypothetical choices between money to be used while gambling ($10 or $25) versus escalating amounts of guaranteed money available immediately or after delays of either 1 week or 1 month. Results suggest that choices on the MCP are correlated with other measures of gambling behavior, thus providing concurrent validity data for using the MCP to quantify the relative reinforcing value of gambling. The MCP for gambling also displayed sensitivity to reinforcer magnitude and delay effects, which provides evidence of criterion validity. The results are consistent with a behavioral economic model of addiction and suggest that the MCP could be a valid tool for future research on gambling behavior.

Retrieval practice with short-answer, multiple-choice, and hybrid tests.

PubMed

Smith, Megan A; Karpicke, Jeffrey D

2014-01-01

Retrieval practice improves meaningful learning, and the most frequent way of implementing retrieval practice in classrooms is to have students answer questions. In four experiments (N=372) we investigated the effects of different question formats on learning. Students read educational texts and practised retrieval by answering short-answer, multiple-choice, or hybrid questions. In hybrid conditions students first attempted to recall answers in short-answer format, then identified answers in multiple-choice format. We measured learning 1 week later using a final assessment with two types of questions: those that could be answered by recalling information verbatim from the texts and those that required inferences. Practising retrieval in all format conditions enhanced retention, relative to a study-only control condition, on both verbatim and inference questions. However, there were little or no advantages of answering short-answer or hybrid format questions over multiple-choice questions in three experiments. In Experiment 4, when retrieval success was improved under initial short-answer conditions, there was an advantage of answering short-answer or hybrid questions over multiple-choice questions. The results challenge the simple conclusion that short-answer questions always produce the best learning, due to increased retrieval effort or difficulty, and demonstrate the importance of retrieval success for retrieval-based learning activities.

More Than the Verbal Stimulus Matters: Visual Attention in Language Assessment for People With Aphasia Using Multiple-Choice Image Displays

PubMed Central

Ivanova, Maria V.; Hallowell, Brooke

2017-01-01

Purpose Language comprehension in people with aphasia (PWA) is frequently evaluated using multiple-choice displays: PWA are asked to choose the image that best corresponds to the verbal stimulus in a display. When a nontarget image is selected, comprehension failure is assumed. However, stimulus-driven factors unrelated to linguistic comprehension may influence performance. In this study we explore the influence of physical image characteristics of multiple-choice image displays on visual attention allocation by PWA. Method Eye fixations of 41 PWA were recorded while they viewed 40 multiple-choice image sets presented with and without verbal stimuli. Within each display, 3 images (majority images) were the same and 1 (singleton image) differed in terms of 1 image characteristic. The mean proportion of fixation duration (PFD) allocated across majority images was compared against the PFD allocated to singleton images. Results PWA allocated significantly greater PFD to the singleton than to the majority images in both nonverbal and verbal conditions. Those with greater severity of comprehension deficits allocated greater PFD to nontarget singleton images in the verbal condition. Conclusion When using tasks that rely on multiple-choice displays and verbal stimuli, one cannot assume that verbal stimuli will override the effect of visual-stimulus characteristics. PMID:28520866

Do Students Know What They Know and What They Don't Know? Using a Four-Tier Diagnostic Test to Assess the Nature of Students' Alternative Conceptions

ERIC Educational Resources Information Center

Caleon, Imelda S.; Subramaniam, R.

2010-01-01

This study reports on the development and application of a four-tier multiple-choice (4TMC) diagnostic instrument, which has not been reported in the literature. It is an enhanced version of the two-tier multiple-choice (2TMC) test. As in 2TMC tests, its answer and reason tiers measure students' content knowledge and explanatory knowledge,…

Psychometrics of Multiple Choice Questions with Non-Functioning Distracters: Implications to Medical Education.

PubMed

Deepak, Kishore K; Al-Umran, Khalid Umran; AI-Sheikh, Mona H; Dkoli, B V; Al-Rubaish, Abdullah

2015-01-01

The functionality of distracters in a multiple choice question plays a very important role. We examined the frequency and impact of functioning and non-functioning distracters on psychometric properties of 5-option items in clinical disciplines. We analyzed item statistics of 1115 multiple choice questions from 15 summative assessments of undergraduate medical students and classified the items into five groups by their number of non-functioning distracters. We analyzed the effect of varying degree of non-functionality ranging from 0 to 4, on test reliability, difficulty index, discrimination index and point biserial correlation. The non-functionality of distracters inversely affected the test reliability and quality of items in a predictable manner. The non-functioning distracters made the items easier and lowered the discrimination index significantly. Three non-functional distracters in a 5-option MCQ significantly affected all psychometric properties (p < 0.5). The corrected point biserial correlation revealed that the items with 3 functional options were psychometrically as effective as 5-option items. Our study reveals that a multiple choice question with 3 functional options provides lower most limit of item format that has adequate psychometric property. The test containing items with less number of functioning options have significantly lower reliability. The distracter function analysis and revision of nonfunctioning distracters can serve as important methods to improve the psychometrics and reliability of assessment.

Investigating the potential influence of established multiple-choice test-taking cues on item response in a pharmacotherapy board certification examination preparatory manual: a pilot study.

PubMed

Gettig, Jacob P

2006-04-01

To determine the prevalence of established multiple-choice test-taking correct and incorrect answer cues in the American College of Clinical Pharmacy's Updates in Therapeutics: The Pharmacotherapy Preparatory Course, 2005 Edition, as an equal or lesser surrogate indication of the prevalence of such cues in the Pharmacotherapy board certification examination. All self-assessment and patient case question-and-answer sets were assessed individually to determine if they were subject to selected correct and incorrect answer cues commonly seen in multiple-choice question writing. If the question was considered evaluable, correct answer cues-longest answer, mid-range number, one of two similar choices, and one of two opposite choices-were tallied. In addition, incorrect answer cues- inclusionary language and grammatical mismatch-were also tallied. Each cue was counted if it did what was expected or did the opposite of what was expected. Multiple cues could be identified in each question. A total of 237 (47.7%) of 497 questions in the manual were deemed evaluable. A total of 325 correct answer cues and 35 incorrect answer cues were identified in the 237 evaluable questions. Most evaluable questions contained one to two correct and/or incorrect answer cue(s). Longest answer was the most frequently identified correct answer cue; however, it was the least likely to identify the correct answer. Inclusionary language was the most frequently identified incorrect answer cue. Incorrect answer cues were considerably more likely to identify incorrect answer choices than correct answer cues were able to identify correct answer choices. The use of established multiple-choice test-taking cues is unlikely to be of significant help when taking the Pharmacotherapy board certification examination, primarily because of the lack of questions subject to such cues and the inability of correct answer cues to accurately identify correct answers. Incorrect answer cues, especially the use of inclusionary language, almost always will accurately identify an incorrect answer choice. Assuming that questions in the preparatory course manual were equal or lesser surrogates of those in the board certification examination, it is unlikely that intuition alone can replace adequate preparation and studying as the sole determinant of examination success.

Tuning into YouTube in the Classroom: Improving Assessment Scores through Social Media

ERIC Educational Resources Information Center

Younger, Dylinda W.; Duncan, Jan E.; Hart, LaToya M.

2013-01-01

Despite the consistent tendencies of higher-education faculty to utilize single testing measures (i.e. essay or multiple choice), education research indicates effective assessment of student learning must incorporate multiple formats. With the surge of online courses, programs, and universities in the last 20 years, there is an increasing need to…

Insights into Students' Conceptual Understanding Using Textual Analysis: A Case Study in Signal Processing

ERIC Educational Resources Information Center

Goncher, Andrea M.; Jayalath, Dhammika; Boles, Wageeh

2016-01-01

Concept inventory tests are one method to evaluate conceptual understanding and identify possible misconceptions. The multiple-choice question format, offering a choice between a correct selection and common misconceptions, can provide an assessment of students' conceptual understanding in various dimensions. Misconceptions of some engineering…

Gender and Ethnicity Differences in Multiple-Choice Testing. Effects of Self-Assessment and Risk-Taking Propensity

DTIC Science & Technology

1993-05-01

correctness of the response provides I some advantages. They are: i 1. Increased reliability of the test; 2. Examinees pay more attention to the multiple...their choice 3 of test date. Each sign up sheet was divided into four cells: Non-Hispanic males and females and Hispanic males and females. 3 I I I...certain prestige and financial rewards; or entering a conservatory of music for advanced training with a well-known pianist . Mr. H realizes that even

Writing Multiple Choice Outcome Questions to Assess Knowledge and Competence.

PubMed

Brady, Erik D

2015-11-01

Few articles contemplate the need for good guidance in question item-writing in the continuing education (CE) space. Although many of the core principles of sound item design translate to the CE health education team, the need exists for specific examples for nurse educators that clearly describe how to measure changes in competence and knowledge using multiple choice items. In this article, some keys points and specific examples for nursing CE providers are shared. Copyright 2015, SLACK Incorporated.

Assessment of High-school Students Engaged in the EarthLabs Climate Modules using the Climate Concept Inventory

NASA Astrophysics Data System (ADS)

McNeal, K.; Libarkin, J. C.; Ledley, T. S.; Gold, A. U.; Lynds, S. E.; Haddad, N.; Ellins, K.; Dunlap, C.; Bardar, E. W.; Youngman, E.

2015-12-01

Instructors must have on hand appropriate assessments that align with their teaching and learning goals in order to provide evidence of student learning. We have worked with curriculum developers and scientists to develop the Climate Concept Inventory (CCI), which meets goals of the EarthLabs Climate on-line curriculum. The developed concept inventory includes 19 content-driven multiple choice questions, six affective-based multiple choice questions, one confidence question, three open-ended questions, and eight demographic questions. Our analysis of the instrument applies item response theory and uses item characteristic curves. We have assessed over 500 students in nearly twenty high school classrooms in Mississippi and Texas that have engaged in the implementation of the EarthLabs curriculum and completed the CCI. Results indicate that students had pre-post gains on 9 out of 10 of the content-based multiple choice questions with positive gains in answer choice selection ranging from 1.72% to 42%. Students significantly reported increased confidence with 15% more students reporting that they were either very or fairly confident with their answers. Of the six affective questions posed, 5 out of 6 showed significant shifts towards gains in knowledge, awareness, and information about Earth's climate system. The research has resulted in a robust and validated climate concept inventory for use with advanced high school students, where we have been able to apply its use within the EarthLabs project.

Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper

PubMed Central

Palmer, Edward J; Devitt, Peter G

2007-01-01

Background Reliable and valid written tests of higher cognitive function are difficult to produce, particularly for the assessment of clinical problem solving. Modified Essay Questions (MEQs) are often used to assess these higher order abilities in preference to other forms of assessment, including multiple-choice questions (MCQs). MEQs often form a vital component of end-of-course assessments in higher education. It is not clear how effectively these questions assess higher order cognitive skills. This study was designed to assess the effectiveness of the MEQ to measure higher-order cognitive skills in an undergraduate institution. Methods An analysis of multiple-choice questions and modified essay questions (MEQs) used for summative assessment in a clinical undergraduate curriculum was undertaken. A total of 50 MCQs and 139 stages of MEQs were examined, which came from three exams run over two years. The effectiveness of the questions was determined by two assessors and was defined by the questions ability to measure higher cognitive skills, as determined by a modification of Bloom's taxonomy, and its quality as determined by the presence of item writing flaws. Results Over 50% of all of the MEQs tested factual recall. This was similar to the percentage of MCQs testing factual recall. The modified essay question failed in its role of consistently assessing higher cognitive skills whereas the MCQ frequently tested more than mere recall of knowledge. Conclusion Construction of MEQs, which will assess higher order cognitive skills cannot be assumed to be a simple task. Well-constructed MCQs should be considered a satisfactory replacement for MEQs if the MEQs cannot be designed to adequately test higher order skills. Such MCQs are capable of withstanding the intellectual and statistical scrutiny imposed by a high stakes exit examination. PMID:18045500

Electronic Portfolios: Blending Technology, Accountability & Assessment

ERIC Educational Resources Information Center

Ahn, June

2004-01-01

Many educators struggle to discover the proper assessment strategies for students. Systemic reform and the standards movement introduce clarity and accountability in assessing students. Though proven to be efficient, standardized assessment such as multiple-choice tests often turn teachers away as they may not align with their classroom practices…

What We've Learned about Assessing Hands-On Science.

ERIC Educational Resources Information Center

Shavelson, Richard J.; Baxter, Gail P.

1992-01-01

A recent study compared hands-on scientific inquiry assessment to assessments involving lab notebooks, computer simulations, short-answer paper-and-pencil problems, and multiple-choice questions. Creating high quality performance assessments is a costly, time-consuming process requiring considerable scientific and technological know-how. Improved…

Authentic Assessments: Praxis for the Distance Librarian

ERIC Educational Resources Information Center

Twomey, Beth

2015-01-01

Distance librarians continually develop information literacy instruction in a variety of formats. Assessment, when it occurs, tends to be of the traditional multiple-choice variety and does not measure more complex skills. Authentic assessments offer the instruction librarian a way to re-think their instruction strategies and assessment of student…

Case Study of a Computer Based Examination System

ERIC Educational Resources Information Center

Fluck, Andrew; Pullen, Darren; Harper, Colleen

2009-01-01

Electronic supported assessment or e-Assessment is a field of growing importance, but it has yet to make a significant impact in the Australian higher education sector (Byrnes & Ellis, 2006). Current computer based assessment models focus on the assessment of knowledge rather than deeper understandings, using multiple choice type questions,…

The "pHunger Games": Manuscript Review to Assess Graduating Chemistry Majors

ERIC Educational Resources Information Center

Gorin, David J.; Jamieson, Elizabeth R.; Queeney, K. T.; Shea, Kevin M.; Spray, Carrie G. Read

2016-01-01

Numerous options exist to assess student performance using standardized, multiple-choice exams at the course and department levels. This paper describes the development and implementation of an alternative department-level assessment for graduating chemistry majors. The assessment detailed here evaluates students' ability to transfer chemical…

The Assessment of Hands-On Elementary Science Programs.

ERIC Educational Resources Information Center

Hein, George, Ed.

This document contains 15 chapters on various topics related to elementary science assessment. A comprehensive description of efforts to introduce alternatives to multiple-choice, paper and pencil tests to assess science learning is provided. The monograph includes an analysis of assessment issues, descriptions of current practice, and suggestions…

E-Assessment as a Service

ERIC Educational Resources Information Center

Amelung, M.; Krieger, K.; Rosner, D.

2011-01-01

Assessment is an essential element in learning processes. It is therefore not unsurprising that almost all learning management systems (LMSs) offer support for assessment, e.g., for the creation, execution, and evaluation of multiple choice tests. We have designed and implemented generic support for assessment that is based on assignments that…

Science Competencies That Go Unassessed

ERIC Educational Resources Information Center

Gilmer, Penny J.; Sherdan, Danielle M.; Oosterhof, Albert; Rohani, Faranak; Rouby, Aaron

2011-01-01

Present large-scale assessments require the use of item formats, such as multiple choice, that can be administered and scored efficiently. This limits competencies that can be measured by these assessments. An alternative approach to large-scale assessments is being investigated that would include the use of complex performance assessments. As…

Trends in computer applications in science assessment

NASA Astrophysics Data System (ADS)

Kumar, David D.; Helgeson, Stanley L.

1995-03-01

Seven computer applications to science assessment are reviewed. Conventional test administration includes record keeping, grading, and managing test banks. Multiple-choice testing involves forced selection of an answer from a menu, whereas constructed-response testing involves options for students to present their answers within a set standard deviation. Adaptive testing attempts to individualize the test to minimize the number of items and time needed to assess a student's knowledge. Figurai response testing assesses science proficiency in pictorial or graphic mode and requires the student to construct a mental image rather than selecting a response from a multiple choice menu. Simulations have been found useful for performance assessment on a large-scale basis in part because they make it possible to independently specify different aspects of a real experiment. An emerging approach to performance assessment is solution pathway analysis, which permits the analysis of the steps a student takes in solving a problem. Virtually all computer-based testing systems improve the quality and efficiency of record keeping and data analysis.

Teacher Quality and Quality Teaching: Examining the Relationship of a Teacher Assessment to Practice

ERIC Educational Resources Information Center

Hill, Heather C.; Umland, Kristin; Litke, Erica; Kapitula, Laura R.

2012-01-01

Multiple-choice assessments are frequently used for gauging teacher quality. However, research seldom examines whether results from such assessments generalize to practice. To illuminate this issue, we compare teacher performance on a mathematics assessment, during mathematics instruction, and by student performance on a state assessment. Poor…

Validation of an Instrument for Measuring Students' Understanding of Interdisciplinary Science in Grades 4-8 over Multiple Semesters: A Rasch Measurement Study

ERIC Educational Resources Information Center

Yang, Yang; He, Peng; Liu, Xiufeng

2018-01-01

So far, not enough effort has been invested in developing reliable, valid, and engaging assessments in school science, especially assessment of interdisciplinary science based on the new Next Generation Science Standards (NGSS). Furthermore, previous tools rely mostly on multiple-choice items and evaluation of student outcome is linked only to…

Mathematics Assessment Sampler 3-5

ERIC Educational Resources Information Center

National Council of Teachers of Mathematics, 2005

2005-01-01

The sample assessment items in this volume are sorted according to the strands of number and operations, algebra, geometry, measurement, and data analysis and probability. Because one goal of assessment is to determine students' abilities to communicate mathematically, the writing team suggests ways to extend or modify multiple-choice and…

Improving Student Performance through Computer-Based Assessment: Insights from Recent Research.

ERIC Educational Resources Information Center

Ricketts, C.; Wilks, S. J.

2002-01-01

Compared student performance on computer-based assessment to machine-graded multiple choice tests. Found that performance improved dramatically on the computer-based assessment when students were not required to scroll through the question paper. Concluded that students may be disadvantaged by the introduction of online assessment unless care is…

Relative Costs of Various Types of Assessments.

ERIC Educational Resources Information Center

Wheeler, Patricia H.

Issues of the relative costs of multiple choice tests and alternative types of assessment are explored. Before alternative assessments in large-scale or small-scale programs are used, attention must be given to cost considerations and the resources required to develop and implement the assessment. Major categories of cost to be considered are…

Teaching for Successful Intelligence Raises School Achievement.

ERIC Educational Resources Information Center

Sternberg, Robert J.; Torff, Bruce; Grigorenko, Elena

1998-01-01

A "successful intelligence" intervention improved school achievement for a group of 225 ethnically diverse third-graders, both on performance assessments measuring analytical, creative, and practical achievements and on conventional multiple-choice memory assessments. Teaching for triarchic thinking facilitates factual recall, because learning…

A Diagnostic Assessment for Introductory Molecular and Cell Biology

ERIC Educational Resources Information Center

Shi, Jia; Wood, William B.; Martin, Jennifer M.; Guild, Nancy A.; Vicens, Quentin; Knight, Jennifer K.

2010-01-01

We have developed and validated a tool for assessing understanding of a selection of fundamental concepts and basic knowledge in undergraduate introductory molecular and cell biology, focusing on areas in which students often have misconceptions. This multiple-choice Introductory Molecular and Cell Biology Assessment (IMCA) instrument is designed…

A multi-instructor, team-based, active-learning exercise to integrate basic and clinical sciences content.

PubMed

Kolluru, Srikanth; Roesch, Darren M; Akhtar de la Fuente, Ayesha

2012-03-12

To introduce a multiple-instructor, team-based, active-learning exercise to promote the integration of basic sciences (pathophysiology, pharmacology, and medicinal chemistry) and clinical sciences in a doctor of pharmacy curriculum. A team-based learning activity that involved pre-class reading assignments, individual-and team-answered multiple-choice questions, and evaluation and discussion of a clinical case, was designed, implemented, and moderated by 3 faculty members from the pharmaceutical sciences and pharmacy practice departments. Student performance was assessed using a multiple-choice examination, an individual readiness assurance test (IRAT), a team readiness assurance test (TRAT), and a subjective, objective, assessment, and plan (SOAP) note. Student attitudes were assessed using a pre- and post-exercise survey instrument. Students' understanding of possible correct treatment strategies for depression improved. Students were appreciative of this true integration of basic sciences knowledge in a pharmacotherapy course and to have faculty members from both disciplines present to answer questions. Mean student score on the on depression module for the examination was 80.4%, indicating mastery of the content. An exercise led by multiple instructors improved student perceptions of the importance of team-based teaching. Integrated teaching and learning may be achieved when instructors from multiple disciplines work together in the classroom using proven team-based, active-learning exercises.

What is an Objective Structured Practical Examination in Anatomy?

ERIC Educational Resources Information Center

Yaqinuddin, Ahmed; Zafar, Muhammad; Ikram, Muhammad Faisal; Ganguly, Paul

2013-01-01

Assessing teaching-learning outcomes in anatomical knowledge is a complex task that requires the evaluation of multiple domains: theoretical, practical, and clinical knowledge. In general, theoretical knowledge is tested by a written examination system constituted by multiple choice questions (MCQs) and/or short answer questions (SAQ). The…

Methods & Strategies: Deep Assessment

ERIC Educational Resources Information Center

Haas, Alison; Hollimon, Shameka; Lee, Okhee

2015-01-01

The "Next Generation Science Standards" ("NGSS") push students to have "a deeper understanding of content" (NGSS Lead States 2013, Appendix A, p. 4). However, with the reality of high-stakes assessments that rely primarily on multiple-choice questions, how can a science teacher analyze students' written responses…

Primary FRCS Eng

PubMed Central

Hobsley, Michael

1974-01-01

In five consecutive Primary Examinations for the Fellowship of the Royal College of Surgeons of England, the scores of candidates in the multiple choice question paper, written paper, and oral interview have been analysed for mutual correlations and for the reproducibility of the written paper score. The conclusions reached were that all these scores correlate with each other, that no score can be left out without reducing the reliability of the examination, that the marking of written papers in a close-marking system is remarkably reproducible, and that the oral score contributes most, the multiple choice question paper the least, to the overall assessment. PMID:4417893

Data Collection Design for Equivalent Groups Equating: Using a Matrix Stratification Framework for Mixed-Format Assessment

ERIC Educational Resources Information Center

Mbella, Kinge Keka

2012-01-01

Mixed-format assessments are increasingly being used in large scale standardized assessments to measure a continuum of skills ranging from basic recall to higher order thinking skills. These assessments are usually comprised of a combination of (a) multiple-choice items which can be efficiently scored, have stable psychometric properties, and…

Web-Based Dynamic Assessment: Taking Assessment as Teaching and Learning Strategy for Improving Students e-Learning Effectiveness

ERIC Educational Resources Information Center

Wang, Tzu-Hua

2010-01-01

This research combines the idea of cake format dynamic assessment defined by Sternberg and Grigorenko (2001) and the "graduated prompt approach" proposed by (Campione and Brown, 1985) and (Campione and Brown, 1987) to develop a multiple-choice Web-based dynamic assessment system. This research adopts a quasi-experimental design to…

Development and Evaluation of a Questionnaire to Assess Physical Educators' Knowledge of Student Assessment

ERIC Educational Resources Information Center

Emmanouilidou, Kyriaki; Derri, Vassiliki; Aggelousis, Nicolaos; Vassiliadou, Olga

2012-01-01

The purpose of this pilot study was to develop and evaluate an instrument for measuring Greek elementary physical educators' knowledge of student assessment. A multiple-choice questionnaire comprised of items about concepts, methods, tools, and types of student assessment in physical education was designed and tested. The initial 35-item…

Effect of differing PowerPoint slide design on multiple-choice test scores for assessment of knowledge and retention in a theriogenology course.

PubMed

Root Kustritz, Margaret V

2014-01-01

Third-year veterinary students in a required theriogenology diagnostics course were allowed to self-select attendance at a lecture in either the evening or the next morning. One group was presented with PowerPoint slides in a traditional format (T group), and the other group was presented with PowerPoint slides in the assertion-evidence format (A-E group), which uses a single sentence and a highly relevant graphic on each slide to ensure attention is drawn to the most important points in the presentation. Students took a multiple-choice pre-test, attended lecture, and then completed a take-home assignment. All students then completed an online multiple-choice post-test and, one month later, a different online multiple-choice test to evaluate retention. Groups did not differ on pre-test, assignment, or post-test scores, and both groups showed significant gains from pre-test to post-test and from pre-test to retention test. However, the T group showed significant decline from post-test to retention test, while the A-E group did not. Short-term differences between slide designs were most likely unaffected due to required coursework immediately after lecture, but retention of material was superior with the assertion-evidence slide design.

Comprehension of confidence intervals - development and piloting of patient information materials for people with multiple sclerosis: qualitative study and pilot randomised controlled trial.

PubMed

Rahn, Anne C; Backhus, Imke; Fuest, Franz; Riemann-Lorenz, Karin; Köpke, Sascha; van de Roemer, Adrianus; Mühlhauser, Ingrid; Heesen, Christoph

2016-09-20

Presentation of confidence intervals alongside information about treatment effects can support informed treatment choices in people with multiple sclerosis. We aimed to develop and pilot-test different written patient information materials explaining confidence intervals in people with relapsing-remitting multiple sclerosis. Further, a questionnaire on comprehension of confidence intervals was developed and piloted. We developed different patient information versions aiming to explain confidence intervals. We used an illustrative example to test three different approaches: (1) short version, (2) "average weight" version and (3) "worm prophylaxis" version. Interviews were conducted using think-aloud and teach-back approaches to test feasibility and analysed using qualitative content analysis. To assess comprehension of confidence intervals, a six-item multiple choice questionnaire was developed and tested in a pilot randomised controlled trial using the online survey software UNIPARK. Here, the average weight version (intervention group) was tested against a standard patient information version on confidence intervals (control group). People with multiple sclerosis were invited to take part using existing mailing-lists of people with multiple sclerosis in Germany and were randomised using the UNIPARK algorithm. Participants were blinded towards group allocation. Primary endpoint was comprehension of confidence intervals, assessed with the six-item multiple choice questionnaire with six points representing perfect knowledge. Feasibility of the patient information versions was tested with 16 people with multiple sclerosis. For the pilot randomised controlled trial, 64 people with multiple sclerosis were randomised (intervention group: n = 36; control group: n = 28). More questions were answered correctly in the intervention group compared to the control group (mean 4.8 vs 3.8, mean difference 1.1 (95 % CI 0.42-1.69), p = 0.002). The questionnaire's internal consistency was moderate (Cronbach's alpha = 0.56). The pilot-phase shows promising results concerning acceptability and feasibility. Pilot randomised controlled trial results indicate that the patient information is well understood and that knowledge gain on confidence intervals can be assessed with a set of six questions. German Clinical Trials Register: DRKS00008561 . Registered 8th of June 2015.

An international comparison study of pharmacy students' achievement goals and their relationship to assessment type and scores.

PubMed

Alrakaf, Saleh; Anderson, Claire; Coulman, Sion A; John, Dai N; Tordoff, June; Sainsbury, Erica; Rose, Grenville; Smith, Lorraine

2015-04-25

To identify pharmacy students' preferred achievement goals in a multi-national undergraduate population, to investigate achievement goal preferences across comparable degree programs, and to identify relationships between achievement goals, academic performance, and assessment type. The Achievement Goal Questionnaire was administered to second year students in 4 universities in Australia, New Zealand, England, and Wales. Academic performance was measured using total scores, multiple-choice questions, and written answers (short essay). Four hundred eighty-six second year students participated. Students showed an overall preference for the mastery-approach goal orientation across all sites. The predicted relationships between goal orientation and multiple-choice questions, and written answers scores, were significant. This study is the first of its kind to examine pharmacy students' achievement goals at a multi-national level and to differentiate between assessment type and measures of achievement motivation. Students adopting a mastery-approach goal are more likely to gain high scores in assessments that measure understanding and depth of knowledge.

An International Comparison Study of Pharmacy Students’ Achievement Goals and their Relationship to Assessment Type and Scores

PubMed Central

Anderson, Claire; Coulman, Sion A.; John, Dai N.; Tordoff, June; Sainsbury, Erica; Rose, Grenville; Smith, Lorraine

2015-01-01

Objective: To identify pharmacy students’ preferred achievement goals in a multi-national undergraduate population, to investigate achievement goal preferences across comparable degree programs, and to identify relationships between achievement goals, academic performance, and assessment type. Methods: The Achievement Goal Questionnaire was administered to second year students in 4 universities in Australia, New Zealand, England, and Wales. Academic performance was measured using total scores, multiple-choice questions, and written answers (short essay). Results: Four hundred eighty-six second year students participated. Students showed an overall preference for the mastery-approach goal orientation across all sites. The predicted relationships between goal orientation and multiple-choice questions, and written answers scores, were significant. Conclusion: This study is the first of its kind to examine pharmacy students’ achievement goals at a multi-national level and to differentiate between assessment type and measures of achievement motivation. Students adopting a mastery-approach goal are more likely to gain high scores in assessments that measure understanding and depth of knowledge. PMID:25995510

The Precalculus Concept Assessment: A Tool for Assessing Students' Reasoning Abilities and Understandings

ERIC Educational Resources Information Center

Carlson, Marilyn; Oehrtman, Michael; Engelke, Nicole

2010-01-01

This article describes the development of the Precalculus Concept Assessment (PCA) instrument, a 25-item multiple-choice exam. The reasoning abilities and understandings central to precalculus and foundational for beginning calculus were identified and characterized in a series of research studies and are articulated in the PCA Taxonomy. These…

Web-Based Quiz-Game-Like Formative Assessment: Development and Evaluation

ERIC Educational Resources Information Center

Wang, Tzu-Hua

2008-01-01

This research aims to develop a multiple-choice Web-based quiz-game-like formative assessment system, named GAM-WATA. The unique design of "Ask-Hint Strategy" turns the Web-based formative assessment into an online quiz game. "Ask-Hint Strategy" is composed of "Prune Strategy" and "Call-in Strategy".…

Assessment in Immersive Virtual Environments: Cases for Learning, of Learning, and as Learning

ERIC Educational Resources Information Center

Code, Jillianne; Zap, Nick

2017-01-01

The key to education reform lies in exploring alternative forms of assessment. Alternative performance assessments provide a more valid measure than multiple-choice tests of students' conceptual understanding and higher-level skills such as problem solving and inquiry. Advances in game-based and virtual environment technologies are creating new…

How Much Detail Needs to Be Elucidated in Self-Harm Research?

ERIC Educational Resources Information Center

Stanford, Sarah; Jones, Michael P.

2010-01-01

Assessing self-harm through brief multiple choice items is simple and less invasive than more detailed methods of assessment. However, there is currently little validation for brief methods of self-harm assessment. This study evaluates the extent to which adolescents' perceptions of self-harm agree with definitions in the literature, and what…

Innovation of a Reinforcer Preference Assessment with the Difficult to Test

ERIC Educational Resources Information Center

Saunders, Muriel D.; Saunders, Richard R.

2011-01-01

In this study, we continued evaluation of a two-choice preference assessment aimed at identifying a hierarchy of reinforcers for individuals with only one voluntary motor sequence--closing and releasing an adaptive switch. We assessed preferences among types of sensory stimulation in 6 adults with multiple profound impairments using concurrent…

Development and Validation of the Conceptual Assessment of Natural Selection (CANS)

ERIC Educational Resources Information Center

Kalinowski, Steven T.; Leonard, Mary J.; Taper, Mark L.

2016-01-01

We developed and validated the Conceptual Assessment of Natural Selection (CANS), a multiple-choice test designed to assess how well college students understand the central principles of natural selection. The expert panel that reviewed the CANS concluded its questions were relevant to natural selection and generally did a good job sampling the…

Blogging to Learn: Educational Blogs and U.S. History

ERIC Educational Resources Information Center

Manfra, Meghan McGlinn; Gray, George E., Jr.; Lee, John K.

2010-01-01

Social studies teachers assess their students in a number of ways. Among these are formative assessments, authentic assessments, and summative low-level multiple-choice tests. Working with two classrooms of low-achieving U.S. history students, the authors compared student experiences in traditional units to those in units that integrated an…

The positive and negative consequences of multiple-choice testing.

PubMed

Roediger, Henry L; Marsh, Elizabeth J

2005-09-01

Multiple-choice tests are commonly used in educational settings but with unknown effects on students' knowledge. The authors examined the consequences of taking a multiple-choice test on a later general knowledge test in which students were warned not to guess. A large positive testing effect was obtained: Prior testing of facts aided final cued-recall performance. However, prior testing also had negative consequences. Prior reading of a greater number of multiple-choice lures decreased the positive testing effect and increased production of multiple-choice lures as incorrect answers on the final test. Multiple-choice testing may inadvertently lead to the creation of false knowledge.

Diagnostic Opportunities Using Rasch Measurement in the Context of a Misconceptions-Based Physical Science Assessment

ERIC Educational Resources Information Center

Wind, Stefanie A.; Gale, Jessica D.

2015-01-01

Multiple-choice (MC) items that are constructed such that distractors target known misconceptions for a particular domain provide useful diagnostic information about student misconceptions (Herrmann-Abell & DeBoer, 2011, 2014; Sadler, 1998). Item response theory models can be used to examine misconceptions distractor-driven multiple-choice…

Academic Performance in Introductory Accounting: Do Learning Styles Matter?

ERIC Educational Resources Information Center

Tan, Lin Mei; Laswad, Fawzi

2015-01-01

This study examines the impact of learning styles on academic performance using major assessment methods (examinations and assignments including multiple-choice and constructed response questions (CRQs)) in an introductory accounting course. Students' learning styles were assessed using Kolb's Learning Style Inventory Version 3.1. The results…

The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments.

PubMed

Tarrant, Marie; Knierim, Aimee; Hayes, Sasha K; Ware, James

2006-12-01

Multiple-choice questions are a common assessment method in nursing examinations. Few nurse educators, however, have formal preparation in constructing multiple-choice questions. Consequently, questions used in baccalaureate nursing assessments often contain item-writing flaws, or violations to accepted item-writing guidelines. In one nursing department, 2770 MCQs were collected from tests and examinations administered over a five-year period from 2001 to 2005. Questions were evaluated for 19 frequently occurring item-writing flaws, for cognitive level, for question source, and for the distribution of correct answers. Results show that almost half (46.2%) of the questions contained violations of item-writing guidelines and over 90% were written at low cognitive levels. Only a small proportion of questions were teacher generated (14.1%), while 36.2% were taken from testbanks and almost half (49.4%) had no source identified. MCQs written at a lower cognitive level were significantly more likely to contain item-writing flaws. While there was no relationship between the source of the question and item-writing flaws, teacher-generated questions were more likely to be written at higher cognitive levels (p<0.001). Correct answers were evenly distributed across all four options and no bias was noted in the placement of correct options. Further training in item-writing is recommended for all faculty members who are responsible for developing tests. Pre-test review and quality assessment is also recommended to reduce the occurrence of item-writing flaws and to improve the quality of test questions.

M-OSCE as a method to measure dental hygiene students' critical thinking: a pilot study.

PubMed

McComas, Martha J; Wright, Rebecca A; Mann, Nancy K; Cooper, Mary D; Jacks, Mary E

2013-04-01

Educators in all academic disciplines have been encouraged to utilize assessment strategies to evaluate students' critical thinking. The purpose of this study was to assess the viability of the modified objective structured clinical examination (m-OSCE) to evaluate critical thinking in dental hygiene education. This evaluation utilized a convenience sample of senior dental hygiene students. Students participated in the m-OSCE in which portions of a patient case were revealed at four stations. The exam consisted of multiple-choice questions intended to measure students' ability to utilize critical thinking skills. Additionally, there was one fill-in-the-blank question and a treatment plan that was completed at the fifth station. The results of this study revealed that the m-OSCE did not reliably measure dental hygiene students' critical thinking. Statistical analysis found no satisfactory reliability within the multiple-choice questions and moderately reliable results within the treatment planning portion of the examination. In addition, the item analysis found gaps in students' abilities to transfer clinical evidence/data to basic biomedical knowledge as demonstrated through the multiple-choice questioning results. This outcome warrants further investigation of the utility of the m-OSCE, with a focus on modifications to the evaluation questions, grading rubric, and patient case.

Scanner Based Assessment in Exams Organized with Personalized Thesis Randomly Generated via Microsoft Word

ERIC Educational Resources Information Center

Teneqexhi, Romeo; Qirko, Margarita; Sharko, Genci; Vrapi, Fatmir; Kuneshka, Loreta

2017-01-01

Exams assessment is one of the most tedious work for university teachers all over the world. Multiple choice theses make exams assessment a little bit easier, but the teacher cannot prepare more than 3-4 variants; in this case, the possibility of students for cheating from one another becomes a risk for "objective assessment outcome." On…

Identifying work preferences among individuals with severe multiple disabilities prior to beginning supported work.

PubMed

Reid, D H; Parsons, M B; Green, C W

1998-01-01

We evaluated a prework assessment for predicting work-task preferences among workers with severe multiple disabilities prior to beginning supported work. The assessment involved comparing worker selections from pairs of work tasks drawn from their future job duties. Results of workers' choices once they began their jobs in a publishing company indicated that the assessment predicted tasks that the workers preferred to work on during their job routines. Results are discussed regarding other possible means of determining preferred types of supported work.

Assessing secondary science students' knowledge of molecule movement, concentration gradients, and equilibrium through multiple contexts

NASA Astrophysics Data System (ADS)

Raven, Sara

2015-09-01

Background: Studies have shown that students' knowledge of osmosis and diffusion and the concepts associated with these processes is often inaccurate. This is important to address, as these concepts not only provide the foundation for more advanced topics in biology and chemistry, but are also threaded throughout both state and national science standards. Purpose: In this study, designed to determine the completeness and accuracy of three specific students' knowledge of molecule movement, concentration gradients, and equilibrium, I sought to address the following question: Using multiple evaluative methods, how can students' knowledge of molecule movement, concentration gradients, and equilibrium be characterized? Sample: This study focuses on data gathered from three students - Emma, Henry, and Riley - all of whom were gifted/honors ninth-grade biology students at a suburban high school in the southeast United States. Design and Methods: Using various qualitative data analysis techniques, I analyzed multiple sources of data from the three students, including multiple-choice test results, written free-response answers, think-aloud interview responses, and student drawings. Results: Results of the analysis showed that students maintained misconceptions about molecule movement, concentration gradients, and equilibrium. The conceptual knowledge students demonstrated differed depending on the assessment method, with the most distinct differences appearing on the multiple-choice versus the free-response questions, and in verbal versus written formats. Conclusions: Multiple levels of assessment may be required to obtain an accurate picture of content knowledge, as free-response and illustrative tasks made it difficult for students to conceal any misconceptions. Using a variety of assessment methods within a section of the curriculum can arguably help to provide a deeper understanding of student knowledge and learning, as well as illuminate misconceptions that may have remained unknown if only one assessment method was used. Furthermore, beyond simply evaluating past learning, multiple assessment methods may aid in student comprehension of key concepts.

Can a Two-Question Test Be Reliable and Valid for Predicting Academic Outcomes?

ERIC Educational Resources Information Center

Bridgeman, Brent

2016-01-01

Scores on essay-based assessments that are part of standardized admissions tests are typically given relatively little weight in admissions decisions compared to the weight given to scores from multiple-choice assessments. Evidence is presented to suggest that more weight should be given to these assessments. The reliability of the writing scores…

Development and Analysis of an Instrument to Assess Student Understanding of GOB Chemistry Knowledge Relevant to Clinical Nursing Practice

ERIC Educational Resources Information Center

Brown, Corina E.; Hyslop, Richard M.; Barbera, Jack

2015-01-01

The General, Organic, and Biological Chemistry Knowledge Assessment (GOB-CKA) is a multiple-choice instrument designed to assess students' understanding of the chemistry topics deemed important to clinical nursing practice. This manuscript describes the development process of the individual items along with a psychometric evaluation of the…

Assessment and the Learning Brain: What the Research Tells Us

ERIC Educational Resources Information Center

Hardiman, Mariale; Whitman, Glenn

2014-01-01

If you really want to see how innovative a school is, inquire about its thinking and practices regarding assessment. For the students, does the mere thought of assessment trigger stress? Do the teachers rely heavily on high-stakes, multiple-choice, Bell Curve-generating tests? Or do the students seem relaxed and engaged as teachers experiment with…

Small Changes: Using Assessment to Direct Instructional Practices in Large-Enrollment Biochemistry Courses

ERIC Educational Resources Information Center

Xu, Xiaoying; Lewis, Jennifer E.; Loertscher, Jennifer; Minderhout, Vicky; Tienson, Heather L.

2017-01-01

Multiple-choice assessments provide a straightforward way for instructors of large classes to collect data related to student understanding of key concepts at the beginning and end of a course. By tracking student performance over time, instructors receive formative feedback about their teaching and can assess the impact of instructional changes.…

Guide to an Assessment of Consumer Skills.

ERIC Educational Resources Information Center

Education Commission of the States, Denver, CO.

This guide is intended to assist those interested in developing and/or assessing consumer skills. It is an accompanyment to a separate collection of survey items (mostly in a multiple choice format) designed to assess seventeen-year-olds' consumer skills. It is suggested that the items can be used as part of an item pool, as an instructional tool,…

Measuring Up: Online Technology Assessment Tools Ease the Teacher's Burden and Help Students Learn

ERIC Educational Resources Information Center

Roland, Jennifer

2006-01-01

Standards are a reality in all academic disciplines, and they can be hard to measure using conventional methods. Technology skills in particular are hard to assess using multiple-choice, paper-based tests. A new generation of online assessments of student technology skills allows students to prove proficiency by completing tasks in their natural…

High School Students' Physical Education Conceptual Knowledge

ERIC Educational Resources Information Center

Ayers, Suzan F.

2004-01-01

The value of conceptual physical education knowledge has long been acknowledged (American Alliance for Health, Physical Education, and Recreation, 1969; Kneer, 1981; NASPE, 1995) yet has not been formally measured or assessed. Seven multiple choice tests with established validity and reliability (Ayers, 2001b) were used to assess the concepts…

The Collegiate Learning Assessment: Facts and Fantasies

ERIC Educational Resources Information Center

Klein, Stephen; Benjamin, Roger; Shavelson, Richard; Bolus, Roger

2007-01-01

The Collegiate Learning Assessment (CLA) is a computer administered, open-ended (as opposed to multiple-choice) test of analytic reasoning, critical thinking, problem solving, and written communication skills. Because the CLA has been endorsed by several national higher education commissions, it has come under intense scrutiny by faculty members,…

Geography Students Assess Their Learning Using Computer-Marked Tests.

ERIC Educational Resources Information Center

Hogg, Jim

1997-01-01

Reports on a pilot study designed to assess the potential of computer-marked tests for allowing students to monitor their learning. Students' answers to multiple choice tests were fed into a computer that provided a full analysis of their strengths and weaknesses. Students responded favorably to the feedback. (MJP)

Assessing Pupils' Skills in Experimentation

ERIC Educational Resources Information Center

Hammann, Marcus; Phan, Thi Thanh Hoi; Ehmer, Maike; Grimm, Tobias

2008-01-01

This study is concerned with different forms of assessment of pupils' skills in experimentation. The findings of three studies are reported. Study 1 investigates whether it is possible to develop reliable multiple-choice tests for the skills of forming hypotheses, designing experiments and analysing experimental data. Study 2 compares scores from…

Assessing Students' Understanding of Macroevolution: Concerns regarding the Validity of the MUM

ERIC Educational Resources Information Center

Novick, Laura R.; Catley, Kefyn M.

2012-01-01

In a recent article, Nadelson and Southerland (2010. Development and preliminary evaluation of the Measure of Understanding of Macroevolution: Introducing the MUM. "The Journal of Experimental Education", 78, 151-190) reported on their development of a multiple-choice concept inventory intended to assess college students' understanding…

Next-Generation Environments for Assessing and Promoting Complex Science Learning

ERIC Educational Resources Information Center

Quellmalz, Edys S.; Davenport, Jodi L.; Timms, Michael J.; DeBoer, George E.; Jordan, Kevin A.; Huang, Chun-Wei; Buckley, Barbara C.

2013-01-01

How can assessments measure complex science learning? Although traditional, multiple-choice items can effectively measure declarative knowledge such as scientific facts or definitions, they are considered less well suited for providing evidence of science inquiry practices such as making observations or designing and conducting investigations.…

Online Testing: The Dog Sat on My Keyboard.

ERIC Educational Resources Information Center

White, Jacci

This paper will highlight some advantages and disadvantages of several online models for student assessment. These models will include: live exams, multiple choice tests, essay exams, and student projects. In addition, real student responses and "problems" will be used as prompts to improve models of authentic online assessment in mathematics.…

Alternative Assessment Techniques for Blended and Online Courses

ERIC Educational Resources Information Center

Litchfield, Brenda C.; Dempsey, John V.

2013-01-01

Alternative assessment techniques are essential for increasing student learning in blended and online courses. Rather than simply answer multiple-choice questions, students can choose activities in an academic contract. By using a contract, students will be active participants in their own learning. Contracts add a dimension of authenticity to…

Harnessing Technology to Improve Formative Assessment of Student Conceptions in STEM: Forging a National Network

ERIC Educational Resources Information Center

Haudek, Kevin C.; Kaplan, Jennifer J.; Knight, Jennifer; Long, Tammy; Merrill, John; Munn, Alan; Nehm, Ross; Smith, Michelle; Urban-Lurain, Mark

2011-01-01

Concept inventories, consisting of multiple-choice questions designed around common student misconceptions, are designed to reveal student thinking. However, students often have complex, heterogeneous ideas about scientific concepts. Constructed-response assessments, in which students must create their own answer, may better reveal students'…

Nuclear Energy Assessment Battery. Form C.

ERIC Educational Resources Information Center

Showers, Dennis Edward

This publication consists of a nuclear energy assessment battery for secondary level students. The test contains 44 multiple choice items and is organized into four major sections. Parts include: (1) a knowledge scale; (2) attitudes toward nuclear energy; (3) a behaviors and intentions scale; and (4) an anxiety scale. Directions are provided for…

Fundamentals of Marketing Core Curriculum. Test Items and Assessment Techniques.

ERIC Educational Resources Information Center

Smith, Clifton L.; And Others

This document contains multiple choice test items and assessment techniques for Missouri's fundamentals of marketing core curriculum. The core curriculum is divided into these nine occupational duties: (1) communications in marketing; (2) economics and marketing; (3) employment and advancement; (4) human relations in marketing; (5) marketing…

Multimodal Likelihoods in Educational Assessment: Will the Real Maximum Likelihood Score Please Stand up?

ERIC Educational Resources Information Center

Wothke, Werner; Burket, George; Chen, Li-Sue; Gao, Furong; Shu, Lianghua; Chia, Mike

2011-01-01

It has been known for some time that item response theory (IRT) models may exhibit a likelihood function of a respondent's ability which may have multiple modes, flat modes, or both. These conditions, often associated with guessing of multiple-choice (MC) questions, can introduce uncertainty and bias to ability estimation by maximum likelihood…

Effects of Test Expectation on Multiple-Choice Performance and Subjective Ratings

ERIC Educational Resources Information Center

Balch, William R.

2007-01-01

Undergraduates studied the definitions of 16 psychology terms, expecting either a multiple-choice (n = 132) or short-answer (n = 122) test. All students then received the same multiple-choice test, requiring them to recognize the definitions as well as novel examples of the terms. Compared to students expecting a multiple-choice test, those…

Multiple-choice examinations: adopting an evidence-based approach to exam technique.

PubMed

Hammond, E J; McIndoe, A K; Sansome, A J; Spargo, P M

1998-11-01

Negatively marked multiple-choice questions (MCQs) are part of the assessment process in both the Primary and Final examinations for the fellowship of the Royal College of Anaesthetists. It is said that candidates who guess will lose marks in the MCQ paper. We studied candidates attending a pre-examination revision course and have shown that an evaluation of examination technique is an important part of an individual's preparation. All candidates benefited substantially from backing their educated guesses while only 3 out of 27 lost marks from backing their wild guesses. Failure to appreciate the relationship between knowledge and technique may significantly affect a candidate's performance in the examination.

Spatial abilities and anatomy knowledge assessment: A systematic review.

PubMed

Langlois, Jean; Bellemare, Christian; Toulouse, Josée; Wells, George A

2017-06-01

Anatomy knowledge has been found to include both spatial and non-spatial components. However, no systematic evaluation of studies relating spatial abilities and anatomy knowledge has been undertaken. The objective of this study was to conduct a systematic review of the relationship between spatial abilities test and anatomy knowledge assessment. A literature search was done up to March 20, 2014 in Scopus and in several databases on the OvidSP and EBSCOhost platforms. Of the 556 citations obtained, 38 articles were identified and fully reviewed yielding 21 eligible articles and their quality were formally assessed. Non-significant relationships were found between spatial abilities test and anatomy knowledge assessment using essays and non-spatial multiple-choice questions. Significant relationships were observed between spatial abilities test and anatomy knowledge assessment using practical examination, three-dimensional synthesis from two-dimensional views, drawing of views, and cross-sections. Relationships between spatial abilities test and anatomy knowledge assessment using spatial multiple-choice questions were unclear. The results of this systematic review provide evidence for spatial and non-spatial methods of anatomy knowledge assessment. Anat Sci Educ 10: 235-241. © 2016 American Association of Anatomists. © 2016 American Association of Anatomists.

Cocaine choice procedures in animals, humans, and treatment-seekers: Can we bridge the divide?

PubMed Central

Moeller, Scott J.; Stoops, William W.

2015-01-01

Individuals with cocaine use disorder chronically self-administer cocaine to the detriment of other rewarding activities, a phenomenon best modeled in laboratory drug-choice procedures. These procedures can evaluate the reinforcing effects of drugs versus comparably valuable alternatives under multiple behavioral arrangements and schedules of reinforcement. However, assessing drug-choice in treatment-seeking or abstaining humans poses unique challenges: for ethical reasons, these populations typically cannot receive active drugs during research studies. Researchers have thus needed to rely on alternative approaches that approximate drug-choice behavior or assess more general forms of decision-making, but whether these alternatives have relevance to real-world drug-taking that can inform clinical trials is not well-understood. In this mini-review, we (A) summarize several important modulatory variables that influence cocaine choice in nonhuman animals and non-treatment seeking humans; (B) discuss some of the ethical considerations that could arise if treatment-seekers are enrolled in drug-choice studies; (C) consider the efficacy of alternative procedures, including non-drug-related decision-making and ‘simulated’ drug-choice (a choice is made, but no drug is administered) to approximate drug choice; and (D) suggest opportunities for new translational work to bridge the current divide between preclinical and clinical research. PMID:26432174

Developing Web-Based Assessment Strategies for Facilitating Junior High School Students to Perform Self-Regulated Learning in an E-Learning Environment

ERIC Educational Resources Information Center

Wang, Tzu-Hua

2011-01-01

This research refers to the self-regulated learning strategies proposed by Pintrich (1999) in developing a multiple-choice Web-based assessment system, the Peer-Driven Assessment Module of the Web-based Assessment and Test Analysis system (PDA-WATA). The major purpose of PDA-WATA is to facilitate learner use of self-regulatory learning behaviors…

The memorial consequences of multiple-choice testing.

PubMed

Marsh, Elizabeth J; Roediger, Henry L; Bjork, Robert A; Bjork, Elizabeth L

2007-04-01

The present article addresses whether multiple-choice tests may change knowledge even as they attempt to measure it. Overall, taking a multiple-choice test boosts performance on later tests, as compared with non-tested control conditions. This benefit is not limited to simple definitional questions, but holds true for SAT II questions and for items designed to tap concepts at a higher level in Bloom's (1956) taxonomy of educational objectives. Students, however, can also learn false facts from multiple-choice tests; testing leads to persistence of some multiple-choice lures on later general knowledge tests. Such persistence appears due to faulty reasoning rather than to an increase in the familiarity of lures. Even though students may learn false facts from multiple-choice tests, the positive effects of testing outweigh this cost.

Is It Working? Distractor Analysis Results from the Test Of Astronomy STandards (TOAST) Assessment Instrument

NASA Astrophysics Data System (ADS)

Slater, Stephanie

2009-05-01

The Test Of Astronomy STandards (TOAST) assessment instrument is a multiple-choice survey tightly aligned to the consensus learning goals stated by the American Astronomical Society - Chair's Conference on ASTRO 101, the American Association of the Advancement of Science's Project 2061 Benchmarks, and the National Research Council's National Science Education Standards. Researchers from the Cognition in Astronomy, Physics and Earth sciences Research (CAPER) Team at the University of Wyoming's Science and Math Teaching Center (UWYO SMTC) have been conducting a question-by-question distractor analysis procedure to determine the sensitivity and effectiveness of each item. In brief, the frequency each possible answer choice, known as a foil or distractor on a multiple-choice test, is determined and compared to the existing literature on the teaching and learning of astronomy. In addition to having statistical difficulty and discrimination values, a well functioning assessment item will show students selecting distractors in the relative proportions to how we expect them to respond based on known misconceptions and reasoning difficulties. In all cases, our distractor analysis suggests that all items are functioning as expected. These results add weight to the validity of the Test Of Astronomy STandards (TOAST) assessment instrument, which is designed to help instructors and researchers measure the impact of course-length duration instructional strategies for undergraduate science survey courses with learning goals tightly aligned to the consensus goals of the astronomy education community.

Identifying Key Features of Student Performance in Educational Video Games and Simulations through Cluster Analysis

ERIC Educational Resources Information Center

Kerr, Deirdre; Chung, Gregory K. W. K.

2012-01-01

The assessment cycle of "evidence-centered design" (ECD) provides a framework for treating an educational video game or simulation as an assessment. One of the main steps in the assessment cycle of ECD is the identification of the key features of student performance. While this process is relatively simple for multiple choice tests, when…

A Combination of Teacher-Led Assessment and Self-Assessment Drives the Learning Process in Online Master Degree in Transplantation

ERIC Educational Resources Information Center

Halawa, Ahmed; Sharma, Ajay; Bridson, Julie M.; Lyon, Sarah; Prescott, Denise; Guha, Arpan; Taylor, David

2017-01-01

Background: Good performance in a summative assessment does not always equate to educational gain following a course. An educational programme may focus on improving student's performance on a particular test instrument. For example, practicing multiple choice questions may lead to mastery of the instrument itself rather than testing the knowledge…

Using Visual Assessments and Tutorials to Teach Solar System Concepts in Introductory Astronomy

ERIC Educational Resources Information Center

LoPresto, Michael C.

2010-01-01

Visual assessments and tutorials are instruments that rely on student construction and/or examination of pictures and/or diagrams rather than multiple choice and/or short answer questions. Being a very visual subject, astronomy lends itself to assessments and tutorials of this type. What follows is a report on the results of the use of visual…

Predicting Assessment Outcomes: The Effect of Full-Time and Part-Time Faculty

ERIC Educational Resources Information Center

Gerlich, R. Nicholas; Sollosy, Marc

2010-01-01

Assessments have risen in prominence in colleges of business, in response to requirements of accrediting agencies. Among the forms of assessment are embedded exams within courses, often in the form of multiple-choice tests near the end of the semester. These tests can be stand-alone comprehensive exercises, or comprise a small portion of a larger…

The Effects of Images on Multiple-Choice Questions in Computer-Based Formative Assessment

ERIC Educational Resources Information Center

Martín-SanJosé, Juan Fernando; Juan, M.-Carmen; Vivó, Roberto; Abad, Francisco

2015-01-01

Current learning and assessment are evolving into digital systems that can be used, stored, and processed online. In this paper, three different types of questionnaires for assessment are presented. All the questionnaires were filled out online on a web-based format. A study was carried out to determine whether the use of images related to each…

Reading Comprehension and the Assessment and Acquisition of Word Knowledge. Technical Report No. 249.

ERIC Educational Resources Information Center

Anderson, Richard C.; Freebody, Peter

The "yes/no" method of vocabulary assessment requires students to indicate words they know from among a list of words and nonwords. Preliminary evidence gained from a study involving fifth grade students indicates that the method is superior in many ways to the multiple choice method of assessment. Analysis of "false alarms," cases in which…

A Diagnostic Assessment for Introductory Molecular and Cell Biology

PubMed Central

Wood, William B.; Martin, Jennifer M.; Guild, Nancy A.; Vicens, Quentin; Knight, Jennifer K.

2010-01-01

We have developed and validated a tool for assessing understanding of a selection of fundamental concepts and basic knowledge in undergraduate introductory molecular and cell biology, focusing on areas in which students often have misconceptions. This multiple-choice Introductory Molecular and Cell Biology Assessment (IMCA) instrument is designed for use as a pre- and posttest to measure student learning gains. To develop the assessment, we first worked with faculty to create a set of learning goals that targeted important concepts in the field and seemed likely to be emphasized by most instructors teaching these subjects. We interviewed students using open-ended questions to identify commonly held misconceptions, formulated multiple-choice questions that included these ideas as distracters, and reinterviewed students to establish validity of the instrument. The assessment was then evaluated by 25 biology experts and modified based on their suggestions. The complete revised assessment was administered to more than 1300 students at three institutions. Analysis of statistical parameters including item difficulty, item discrimination, and reliability provides evidence that the IMCA is a valid and reliable instrument with several potential uses in gauging student learning of key concepts in molecular and cell biology. PMID:21123692

What Kinds of Numbers Do Students Assign to Literal Symbols? Aspects of the Transition from Arithmetic to Algebra

ERIC Educational Resources Information Center

Christou, Konstantinos P.; Vosniadou, Stella

2012-01-01

Three experiments used multiple methods--open-ended assessments, multiple-choice questionnaires, and interviews--to investigate the hypothesis that the development of students' understanding of the concept of real variable in algebra may be influenced in fundamental ways by their initial concept of number, which seems to be organized around the…

A Study of Students' Readiness to Learn Calculus

ERIC Educational Resources Information Center

Carlson, Marilyn P.; Madison, Bernard; West, Richard D.

2015-01-01

The Calculus Concept Readiness (CCR) instrument assesses foundational understandings and reasoning abilities that have been documented to be essential for learning calculus. The CCR Taxonomy describes the understandings and reasoning abilities assessed by CCR. The CCR is a 25-item multiple-choice instrument that can be used as a placement test for…

The Applicability of Interactive Item Templates in Varied Knowledge Types

ERIC Educational Resources Information Center

Koong, Chorng-Shiuh; Wu, Chi-Ying

2011-01-01

A well-edited assessment can enhance student's learning motives. Applicability of items, which includes item content and template, plays a crucial role in authoring a good assessment. Templates in discussion contain not only conventional true & false, multiple choice, completion item and short answer but also of those interactive ones. Methods…

Automatically Scoring Short Essays for Content. CRESST Report 836

ERIC Educational Resources Information Center

Kerr, Deirdre; Mousavi, Hamid; Iseli, Markus R.

2013-01-01

The Common Core assessments emphasize short essay constructed response items over multiple choice items because they are more precise measures of understanding. However, such items are too costly and time consuming to be used in national assessments unless a way is found to score them automatically. Current automatic essay scoring techniques are…

Assessing Logo Programming among Jordanian Seventh Grade Students through Turtle Geometry

ERIC Educational Resources Information Center

Khasawneh, Amal A.

2009-01-01

The present study is concerned with assessing Logo programming experiences among seventh grade students. A formal multiple-choice test and five performance tasks were used to collect data. The results provided that students' performance was better than the expected score by the probabilistic laws, and a very low correlation between their Logo…

Instructor Perspectives of Multiple-Choice Questions in Summative Assessment for Novice Programmers

ERIC Educational Resources Information Center

Shuhidan, Shuhaida; Hamilton, Margaret; D'Souza, Daryl

2010-01-01

Learning to program is known to be difficult for novices. High attrition and high failure rates in foundation-level programming courses undertaken at tertiary level in Computer Science programs, are commonly reported. A common approach to evaluating novice programming ability is through a combination of formative and summative assessments, with…

Assessing Community College Student Knowledge in the Liberal Arts.

ERIC Educational Resources Information Center

Cohen, Arthur M.; Schuetz, Pam; Chang, June C.; Plecha, Michelle D.

This paper describes an assessment of community college student knowledge in the liberal arts at two-year colleges in Southern California. A survey instrument with multiple choice questions covering five liberal arts subject areas was distributed to 4,200 students in randomly selected classes at ten colleges. More than 2,500 questionnaires were…

Assessing Community College Student Knowledge in the Liberal Arts

ERIC Educational Resources Information Center

Cohen, Arthur M.; Schuetz, Pam; Chang, June C.; Plecha, Michelle

2003-01-01

The General Academic Learning Experience (GALE) is an assessment of community college student knowledge in the liberal arts. The study involved the design and administration of an instrument, which included a demographic survey and a multiple-choice content test. In total, over 2,500 students from 10 colleges in Southern California participated.…

Influence of Type of Assessment and Stress on the Learning Outcome

ERIC Educational Resources Information Center

Tetteh, Godson Ayertei; Sarpong, Frederick Asafo-Adjei

2015-01-01

Purpose: The purpose of this paper is to explore the influence of constructivism on assessment approach, where the type of question (true or false, multiple-choice, calculation or essay) is used productively. Although the student's approach to learning and the teacher's approach to teaching are concepts that have been widely researched, few…

Missouri Assessment Program (MAP), Spring 2000: Elementary Health/Physical Education, Released Items, Grade 5.

ERIC Educational Resources Information Center

Missouri State Dept. of Elementary and Secondary Education, Jefferson City.

This document presents 10 released items from the Health/Physical Education Missouri Assessment Program (MAP) test given in the spring of 2000 to fifth graders. Items from the test sessions include: selected-response (multiple choice), constructed-response, and a performance event. The selected-response items consist of individual questions…

Beyond the Bubble in History/Social Studies Assessments

ERIC Educational Resources Information Center

Breakstone, Joel; Smith, Mark; Wineburg, Sam

2013-01-01

Teachers need tools and assessments that will prepare students to meet the ambitious goals laid out by the Common Core State Standards. The multiple-choice tests that dominate in history will not prepare students to analyze primary and secondary sources, cite textual evidence to support arguments, consider the influence of an author's perspective,…

Confidence-Based Assessments within an Adult Learning Environment

ERIC Educational Resources Information Center

Novacek, Paul

2013-01-01

Traditional knowledge assessments rely on multiple-choice type questions that only report a right or wrong answer. The reliance within the education system on this technique infers that a student who provides a correct answer purely through guesswork possesses knowledge equivalent to a student who actually knows the correct answer. A more complete…

Towards a Better Understanding of the Legibility Bias in Performance Assessments: The Case of Gender-Based Inferences

ERIC Educational Resources Information Center

Greifeneder, Rainer; Zelt, Sarah; Seele, Tim; Bottenberg, Konstantin; Alt, Alexander

2012-01-01

Background: Handwriting legibility systematically biases evaluations in that highly legible handwriting results in more positive evaluations than less legible handwriting. Because performance assessments in educational contexts are not only based on computerized or multiple choice tests but often include the evaluation of handwritten work samples,…

Integrated Testlets: A New Form of Expert-Student Collaborative Testing

ERIC Educational Resources Information Center

Shiell, Ralph C.; Slepkov, Aaron D.

2015-01-01

Integrated testlets are a new assessment tool that encompass the procedural benefits of multiple-choice testing, the pedagogical advantages of free-response-based tests, and the collaborative aspects of a viva voce or defence examination format. The result is a robust assessment tool that provides a significant formative aspect for students.…

Child Abuse and Neglect: Training Needs of Student Teachers

ERIC Educational Resources Information Center

McKee, Bronagh E.; Dillenburger, Karola

2009-01-01

Increasing awareness of child abuse and neglect (CAN) raises questions about how well teachers are prepared for their role in child protection. This paper assesses and differentiates training needs of first-year students (n = 216) in Northern Ireland. Multiple-choice tests were used to assess knowledge of CAN statistics; recognising and reporting;…

Advanced Marketing Core Curriculum. Test Items and Assessment Techniques.

ERIC Educational Resources Information Center

Smith, Clifton L.; And Others

This document contains duties and tasks, multiple-choice test items, and other assessment techniques for Missouri's advanced marketing core curriculum. The core curriculum begins with a list of 13 suggested textbook resources. Next, nine duties with their associated tasks are given. Under each task appears one or more citations to appropriate…

An Evaluation Framework and Instrument for Evaluating e-Assessment Tools

ERIC Educational Resources Information Center

Singh, Upasana Gitanjali; de Villiers, Mary Ruth

2017-01-01

e-Assessment, in the form of tools and systems that deliver and administer multiple choice questions (MCQs), is used increasingly, raising the need for evaluation and validation of such systems. This research uses literature and a series of six empirical action research studies to develop an evaluation framework of categories and criteria called…

Multiple-Stimulus without Replacement Preference Assessment for Students at Risk for Emotional Disturbance

ERIC Educational Resources Information Center

King, Seth A.

2016-01-01

The ability of educators to identify consequences that act as reinforcers may predict the success of behavior change strategies predicated on the use of reinforcement. Supported for individuals with severe disabilities, research concerning the effectiveness of choice-stimulus assessment for students with emotional disturbance (ED) remains limited.…

Comparison of traditional and interactive teaching methods in a UK emergency department.

PubMed

Armstrong, Peter; Elliott, Tim; Ronald, Julie; Paterson, Brodie

2009-12-01

Didactic teaching remains a core component of undergraduate education, but developing computer assisted learning (CAL) packages may provide useful alternatives. We compared the effectiveness of interactive multimedia-based tutorials with traditional, lecture-based models for teaching arterial blood gas interpretation to fourth year medical students. Participants were randomized to complete a tutorial in either lecture or multimedia format containing identical content. Upon completion, students answered five multiple choice questions assessing post-tutorial knowledge, and provided feedback on their allocated learning method. Marks revealed no significant difference between either group. All lecture candidates rated their teaching as good, compared with 89% of the CAL group. All CAL users found multiple choice questions assessment useful, compared with 83% of lecture participants. Both groups highlighted the importance of interaction. CAL complements other teaching methods, but should be seen as an adjunct to, rather than a replacement for, traditional methods, thus offering students a blended learning environment.

The impact of two multiple-choice question formats on the problem-solving strategies used by novices and experts.

PubMed

Coderre, Sylvain P; Harasym, Peter; Mandin, Henry; Fick, Gordon

2004-11-05

Pencil-and-paper examination formats, and specifically the standard, five-option multiple-choice question, have often been questioned as a means for assessing higher-order clinical reasoning or problem solving. This study firstly investigated whether two paper formats with differing number of alternatives (standard five-option and extended-matching questions) can test problem-solving abilities. Secondly, the impact of the alternatives number on psychometrics and problem-solving strategies was examined. Think-aloud protocols were collected to determine the problem-solving strategy used by experts and non-experts in answering Gastroenterology questions, across the two pencil-and-paper formats. The two formats demonstrated equal ability in testing problem-solving abilities, while the number of alternatives did not significantly impact psychometrics or problem-solving strategies utilized. These results support the notion that well-constructed multiple-choice questions can in fact test higher order clinical reasoning. Furthermore, it can be concluded that in testing clinical reasoning, the question stem, or content, remains more important than the number of alternatives.

Inquiry-based Instruction with Archived, Online Data: An Intervention Study with Preservice Teachers

NASA Astrophysics Data System (ADS)

Ucar, Sedat; Trundle, Kathy Cabe; Krissek, Lawrence

2011-03-01

This mixed methods study described preservice teachers' conceptions of tides and explored the efficacy of integrating online data into inquiry-based instruction. Data sources included a multiple-choice assessment and in-depth interviews. A total of 79 participants in secondary, middle, and early childhood teacher education programs completed the multiple-choice assessment of their baseline knowledge of tides-related concepts. A sub-group of 29 participants also was interviewed to explore their understanding of tides in more detail before instruction. Eighteen of those 29 teachers participated in the instruction, were interviewed again after the instruction, and completed the multiple-choice assessment as a posttest. The interview data sets were analyzed via a constant comparative method in order to produce profiles of each participant's pre- and post-instruction conceptual understandings of tides. Additional quantitative analysis consisted of a paired-sample t-test, which investigated the changes in scores before and after the instructional intervention. Before instruction, all participants held alternative or alternative fragments as their conceptual understandings of tides. After completing the inquiry-based instruction that integrated online tidal data, participants were more likely to hold a scientific conceptual understanding. After instruction, some preservice teachers continued to hold on to the conception that the rotation of the moon around the Earth during one 24-hour period causes the tides to move with the moon. The quantitative results, however, indicated that pre- to post-instruction gains were significant. The findings of this study provide evidence that integrating Web-based archived data into inquiry-based instruction can be used to effectively promote conceptual change among preservice teachers.

Standard setting: comparison of two methods.

PubMed

George, Sanju; Haque, M Sayeed; Oyebode, Femi

2006-09-14

The outcome of assessments is determined by the standard-setting method used. There is a wide range of standard-setting methods and the two used most extensively in undergraduate medical education in the UK are the norm-reference and the criterion-reference methods. The aims of the study were to compare these two standard-setting methods for a multiple-choice question examination and to estimate the test-retest and inter-rater reliability of the modified Angoff method. The norm-reference method of standard-setting (mean minus 1 SD) was applied to the 'raw' scores of 78 4th-year medical students on a multiple-choice examination (MCQ). Two panels of raters also set the standard using the modified Angoff method for the same multiple-choice question paper on two occasions (6 months apart). We compared the pass/fail rates derived from the norm reference and the Angoff methods and also assessed the test-retest and inter-rater reliability of the modified Angoff method. The pass rate with the norm-reference method was 85% (66/78) and that by the Angoff method was 100% (78 out of 78). The percentage agreement between Angoff method and norm-reference was 78% (95% CI 69% - 87%). The modified Angoff method had an inter-rater reliability of 0.81-0.82 and a test-retest reliability of 0.59-0.74. There were significant differences in the outcomes of these two standard-setting methods, as shown by the difference in the proportion of candidates that passed and failed the assessment. The modified Angoff method was found to have good inter-rater reliability and moderate test-retest reliability.

What makes a successful home birth service: an examination of the influential elements by review of one service.

PubMed

Brintworth, Kate; Sandall, Jane

2013-06-01

to evaluate and gain understanding of the service factors that contribute to the relatively high home birth rate found in one inner city NHS Trust providing maternity services in England. a multi-faceted approach encompassing narrative, historical, structural, demographic and cultural elements. an inner city maternity service provided in a large metropolis in England. stakeholders including clinical staff and managers in the service. a review of service provision using secondary quantative data analysis 2005-2009, structural review of the service and semi-structured interviews with staff. the structure of a service with multiple self-managed midwifery practices, mainly operating caseload models strongly supported by senior midwifery leaders, and senior obstetricians enabled the delivery of a responsive, flexible service that was able to deliver choice to women. One element of interest was home assessment in early labour, which kept open the choice around place of birth for women until they were in labour. the organisation of care into multiple small midwifery group practices, providing care using a caseload model, appears to support home birth as a choice for women. In addition the offer of home assessment in early labour whilst poorly researched may be relevant to a flexible woman centred service that can respond to women's choices in realistic way. Copyright © 2012 Elsevier Ltd. All rights reserved.

A Quantitative Assessment of the Cultural Knowledge, Attitudes, and Experiences of Junior and Senior Dietetics Students

ERIC Educational Resources Information Center

McArthur, Laura H.; Greathouse, Karen R.; Smith, Erskine R.; Holbert, Donald

2011-01-01

Objective: To assess the cultural competence of dietetics majors. Design: Self-administered questionnaire. Setting: Classrooms at 7 universities. Participants: Two hundred eighty-three students--98 juniors (34.6%) and 185 seniors (65.4%)--recruited during class time. Main Outcome Measures: Knowledge was measured using a multiple-choice test,…

Missouri Assessment Program (MAP), Spring 2000: High School Health/Physical Education, Released Items, Grade 9.

ERIC Educational Resources Information Center

Missouri State Dept. of Elementary and Secondary Education, Jefferson City.

This document presents 10 released items from the Health/Physical Education Missouri Assessment Program (MAP) test given in the spring of 2000 to ninth graders. Items from the test sessions include: selected-response (multiple choice), constructed-response, and a performance event. The selected-response items consist of individual questions…

Using Multigroup Confirmatory Factor Analysis to Test Measurement Invariance in Raters: A Clinical Skills Examination Application

ERIC Educational Resources Information Center

Kahraman, Nilufer; Brown, Crystal B.

2015-01-01

Psychometric models based on structural equation modeling framework are commonly used in many multiple-choice test settings to assess measurement invariance of test items across examinee subpopulations. The premise of the current article is that they may also be useful in the context of performance assessment tests to test measurement invariance…

Closing the Loop: Involving Faculty in the Assessment of Scientific and Quantitative Reasoning Skills of Biology Majors

ERIC Educational Resources Information Center

Hurney, Carol A.; Brown, Justin; Griscom, Heather Peckham; Kancler, Erika; Wigtil, Clifton J.; Sundre, Donna

2011-01-01

The development of scientific and quantitative reasoning skills in undergraduates majoring in science, technology, engineering, and mathematics (STEM) is an objective of many courses and curricula. The Biology Department at James Madison University (JMU) assesses these essential skills in graduating biology majors by using a multiple-choice exam…

Multiple data sets and modelling choices in a comparative LCA of disposable beverage cups.

PubMed

van der Harst, Eugenie; Potting, José; Kroeze, Carolien

2014-10-01

This study used multiple data sets and modelling choices in an environmental life cycle assessment (LCA) to compare typical disposable beverage cups made from polystyrene (PS), polylactic acid (PLA; bioplastic) and paper lined with bioplastic (biopaper). Incineration and recycling were considered as waste processing options, and for the PLA and biopaper cup also composting and anaerobic digestion. Multiple data sets and modelling choices were systematically used to calculate average results and the spread in results for each disposable cup in eleven impact categories. The LCA results of all combinations of data sets and modelling choices consistently identify three processes that dominate the environmental impact: (1) production of the cup's basic material (PS, PLA, biopaper), (2) cup manufacturing, and (3) waste processing. The large spread in results for impact categories strongly overlaps among the cups, however, and therefore does not allow a preference for one type of cup material. Comparison of the individual waste treatment options suggests some cautious preferences. The average waste treatment results indicate that recycling is the preferred option for PLA cups, followed by anaerobic digestion and incineration. Recycling is slightly preferred over incineration for the biopaper cups. There is no preferred waste treatment option for the PS cups. Taking into account the spread in waste treatment results for all cups, however, none of these preferences for waste processing options can be justified. The only exception is composting, which is least preferred for both PLA and biopaper cups. Our study illustrates that using multiple data sets and modelling choices can lead to considerable spread in LCA results. This makes comparing products more complex, but the outcomes more robust. Copyright © 2014 Elsevier B.V. All rights reserved.

Comparison of a video-based assessment and a multiple stimulus assessment to identify preferred jobs for individuals with significant intellectual disabilities.

PubMed

Horrocks, Erin L; Morgan, Robert L

2009-01-01

The authors compare two methods of identifying job preferences for individuals with significant intellectual disabilities. Three individuals with intellectual disabilities between the ages of 19 and 21 participated in a video-based preference assessment and a multiple stimulus without replacement (MSWO) assessment. Stimulus preference assessment procedures typically involve giving participants access to the selected stimuli to increase the probability that participants will associate the selected choice with the actual stimuli. Although individuals did not have access to the selected stimuli in the video-based assessment, results indicated that both assessments identified the same highest preference job for all participants. Results are discussed in terms of using a video-based assessment to accurately identify job preferences for individuals with developmental disabilities.

A Comparison between Students' Performance in Multiple Choice and Modified Essay Questions in the MBBS Pediatrics Examination at the College of Medicine, King Khalid University, KSA

ERIC Educational Resources Information Center

Salih, Karimeldin M. A.; Alshehri, Mohamed Abdullah Al-Gosadi; Elfaki, Omer Abdelgadir

2016-01-01

Objectives: To investigate the relation between the students' scores in MCQs and MEQs of the summative assessment in pediatrics at the College of medicine KKU. Introduction: Student assessment is the most difficult task in medicine since it is ultimately related to human life and safety. Assessment can take different types of formats with…

An Assessment of Pharmacy Student Confidence in Learning.

ERIC Educational Resources Information Center

Popovich, Nicholas G.; Rogers, Wallace J.

1987-01-01

A study to determine student knowledge and confidence in that knowledge when answering multiple-choice examination questions in a nonprescription drug course is described. An alternate approach to methods of confidence testing was investigated. The knowledge and experience survey is appended. (Author/MLW)

Development of the Newtonian Gravity Concept Inventory

ERIC Educational Resources Information Center

Williamson, Kathryn E.; Willoughby, Shannon; Prather, Edward E.

2013-01-01

We introduce the Newtonian Gravity Concept Inventory (NGCI), a 26-item multiple-choice instrument to assess introductory general education college astronomy ("Astro 101") student understanding of Newtonian gravity. This paper describes the development of the NGCI through four phases: Planning, Construction, Quantitative Analysis, and…

Line Up, Line Up: Using Technology to Align and Enhance Peer Learning and Assessment in a Student Centred Foundation Organic Chemistry Module

ERIC Educational Resources Information Center

Ryan, Barry J.

2013-01-01

This paper describes how three technologies were utilised in combination to align student learning and assessment as part of a case study. Multiple choice questions (MCQs) were central to all these technologies. The peer learning technologies; Personal Response Devices (a.k.a. "Clickers") and "PeerWise"…

The Validity of the Major Field Test in Psychology as a Programme Assessment Tool

ERIC Educational Resources Information Center

Gallagher, Shawn P.; Cook, Shaun P.

2013-01-01

The Major Field Test in Psychology (MFT) is a standardised test designed to assess subject mastery at the conclusion of an undergraduate career. Eighty-one graduating majors completed the MFT and 56 of them also took a multiple-choice exam of questions drawn randomly from an introductory psychology test bank. Like the MFT, the constructed exam was…

Scientific Process Flowchart Assessment (SPFA): A Method for Evaluating Changes in Understanding and Visualization of the Scientific Process in a Multidisciplinary Student Population

ERIC Educational Resources Information Center

Wilson, Kristy J.; Rigakos, Bessie

2016-01-01

The scientific process is nonlinear, unpredictable, and ongoing. Assessing the nature of science is difficult with methods that rely on Likert-scale or multiple-choice questions. This study evaluated conceptions about the scientific process using student-created visual representations that we term "flowcharts." The methodology,…

Using Diagnostic Assessment to Help Teachers Understand the Chemistry of the Lead-Acid Battery

ERIC Educational Resources Information Center

Cheung, Derek

2011-01-01

Nineteen pre-service and in-service teachers taking a chemistry teaching methods course at a university in Hong Kong were asked to take a diagnostic assessment. It consisted of seven multiple-choice questions about the chemistry of the lead-acid battery. Analysis of the teachers' responses to the questions indicated that they had difficulty in…

Exploring Different Types of Assessment Items to Measure Linguistically Diverse Students' Understanding of Energy and Matter in Chemistry

ERIC Educational Resources Information Center

Ryoo, Kihyun; Toutkoushian, Emily; Bedell, Kristin

2018-01-01

Energy and matter are fundamental, yet challenging concepts in middle school chemistry due to their abstract, unobservable nature. Although it is important for science teachers to elicit a range of students' ideas to design and revise their instruction, capturing such varied ideas using traditional assessments consisting of multiple-choice items…

Development and Application of a Four-Tier Test to Assess Pre-Service Physics Teachers' Misconceptions about Geometrical Optics

ERIC Educational Resources Information Center

Kaltakci-Gurel, Derya; Eryilmaz, Ali; McDermott, Lillian Christie

2017-01-01

Background: Correct identification of misconceptions is an important first step in order to gain an understanding of student learning. More recently, four-tier multiple choice tests have been found to be effective in assessing misconceptions. Purpose: The purposes of this study are (1) to develop and validate a four-tier misconception test to…

Small Changes: Using Assessment to Direct Instructional Practices in Large-Enrollment Biochemistry Courses

PubMed Central

Xu, Xiaoying; Lewis, Jennifer E.; Loertscher, Jennifer; Minderhout, Vicky; Tienson, Heather L.

2017-01-01

Multiple-choice assessments provide a straightforward way for instructors of large classes to collect data related to student understanding of key concepts at the beginning and end of a course. By tracking student performance over time, instructors receive formative feedback about their teaching and can assess the impact of instructional changes. The evidence of instructional effectiveness can in turn inform future instruction, and vice versa. In this study, we analyzed student responses on an optimized pretest and posttest administered during four different quarters in a large-enrollment biochemistry course. Student performance and the effect of instructional interventions related to three fundamental concepts—hydrogen bonding, bond energy, and pKa—were analyzed. After instructional interventions, a larger proportion of students demonstrated knowledge of these concepts compared with data collected before instructional interventions. Student responses trended from inconsistent to consistent and from incorrect to correct. The instructional effect was particularly remarkable for the later three quarters related to hydrogen bonding and bond energy. This study supports the use of multiple-choice instruments to assess the effectiveness of instructional interventions, especially in large classes, by providing instructors with quick and reliable feedback on student knowledge of each specific fundamental concept. PMID:28188280

Nested Logit Models for Multiple-Choice Item Response Data

ERIC Educational Resources Information Center

Suh, Youngsuk; Bolt, Daniel M.

2010-01-01

Nested logit item response models for multiple-choice data are presented. Relative to previous models, the new models are suggested to provide a better approximation to multiple-choice items where the application of a solution strategy precedes consideration of response options. In practice, the models also accommodate collapsibility across all…

Comparing comprehension measured by multiple-choice and open-ended questions.

PubMed

Ozuru, Yasuhiro; Briner, Stephen; Kurby, Christopher A; McNamara, Danielle S

2013-09-01

This study compared the nature of text comprehension as measured by multiple-choice format and open-ended format questions. Participants read a short text while explaining preselected sentences. After reading the text, participants answered open-ended and multiple-choice versions of the same questions based on their memory of the text content. The results indicated that performance on open-ended questions was correlated with the quality of self-explanations, but performance on multiple-choice questions was correlated with the level of prior knowledge related to the text. These results suggest that open-ended and multiple-choice format questions measure different aspects of comprehension processes. The results are discussed in terms of dual process theories of text comprehension. PsycINFO Database Record (c) 2013 APA, all rights reserved

Open problem-based instruction impacts understanding of physiological concepts differently in undergraduate students

PubMed Central

Franklin, Brandon M.; Xiang, Lin; Collett, Jason A.; Rhoads, Megan K.

2015-01-01

Student populations are diverse such that different types of learners struggle with traditional didactic instruction. Problem-based learning has existed for several decades, but there is still controversy regarding the optimal mode of instruction to ensure success at all levels of students' past achievement. The present study addressed this problem by dividing students into the following three instructional groups for an upper-level course in animal physiology: traditional lecture-style instruction (LI), guided problem-based instruction (GPBI), and open problem-based instruction (OPBI). Student performance was measured by three summative assessments consisting of 50% multiple-choice questions and 50% short-answer questions as well as a final overall course assessment. The present study also examined how students of different academic achievement histories performed under each instructional method. When student achievement levels were not considered, the effects of instructional methods on student outcomes were modest; OPBI students performed moderately better on short-answer exam questions than both LI and GPBI groups. High-achieving students showed no difference in performance for any of the instructional methods on any metric examined. In students with low-achieving academic histories, OPBI students largely outperformed LI students on all metrics (short-answer exam: P < 0.05, d = 1.865; multiple-choice question exam: P < 0.05, d = 1.166; and final score: P < 0.05, d = 1.265). They also outperformed GPBI students on short-answer exam questions (P < 0.05, d = 1.109) but not multiple-choice exam questions (P = 0.071, d = 0.716) or final course outcome (P = 0.328, d = 0.513). These findings strongly suggest that typically low-achieving students perform at a higher level under OPBI as long as the proper support systems (formative assessment and scaffolding) are provided to encourage student success. PMID:26628656

Development and Preliminary Testing of the Food Choice Priorities Survey (FCPS): Assessing the Importance of Multiple Factors on College Students' Food Choices.

PubMed

Vilaro, Melissa J; Zhou, Wenjun; Colby, Sarah E; Byrd-Bredbenner, Carol; Riggsbee, Kristin; Olfert, Melissa D; Barnett, Tracey E; Mathews, Anne E

2017-12-01

Understanding factors that influence food choice may help improve diet quality. Factors that commonly affect adults' food choices have been described, but measures that identify and assess food choice factors specific to college students are lacking. This study developed and tested the Food Choice Priorities Survey (FCPS) among college students. Thirty-seven undergraduates participated in two focus groups ( n = 19; 11 in the male-only group, 8 in the female-only group) and interviews ( n = 18) regarding typical influences on food choice. Qualitative data informed the development of survey items with a 5-point Likert-type scale (1 = not important, 5 = extremely important). An expert panel rated FCPS items for clarity, relevance, representativeness, and coverage using a content validity form. To establish test-retest reliability, 109 first-year college students completed the 14-item FCPS at two time points, 0-48 days apart ( M = 13.99, SD = 7.44). Using Cohen's weighted κ for responses within 20 days, 11 items demonstrated moderate agreement and 3 items had substantial agreement. Factor analysis revealed a three-factor structure (9 items). The FCPS is designed for college students and provides a way to determine the factors of greatest importance regarding food choices among this population. From a public health perspective, practical applications include using the FCPS to tailor health communications and behavior change interventions to factors most salient for food choices of college students.

[Utility of conceptual schemes and mental maps on the teaching-learning process of residents in pediatrics].

PubMed

Cruza, Norberto Sotelo; Fierros, Luis E

2006-01-01

The present study was done at the internal medicine service oft he Hospital lnfantil in the State of Sonora, Mexico. We tried to address the question of the use of conceptual schemes and mind maps and its impact on the teaching-learning-evaluation process among medical residents. Analyze the effects of conceptual schemes, and mind maps as a teaching and evaluation tool and compare them with multiple choice exams among Pediatric residents. Twenty two residents (RI, RII, RIII)on service rotation during six months were assessed initially, followed by a lecture on a medical subject. Conceptual schemes and mind maps were then introduced as a teaching-learning-evaluation instrument. Comprehension impact and comparison with a standard multiple choice evaluation was done. The statistical package (JMP version 5, SAS inst. 2004) was used. We noted that when we used conceptual schemes and mind mapping, learning improvement was noticeable among the three groups of residents (P < 0.001) and constitutes a better evaluation tool when compared with multiple choice exams (P < 0.0005). Based on our experience we recommend the use of this educational technique for medical residents in training.

Using Tests as Learning Opportunities.

ERIC Educational Resources Information Center

Foos, Paul W.; Fisher, Ronald P.

1988-01-01

A study involving 105 undergraduates assessed the value of testing as a means of increasing, rather than simply monitoring, learning. Results indicate that fill-in-the-blank and items requiring student inferences were more effective, respectively, than multiple-choice tests and verbatim items in furthering student learning. (TJH)

Wesleyan University Student Questionnaire.

ERIC Educational Resources Information Center

Haagen, C. Hess

This questionnaire assesses marijuana use practices in college students. The 30 items (multiple choice or free response) are concerned with personal and demographic data, marijuana smoking practices, use history, effects from smoking marijuana, present attitude toward the substance, and use of other drugs. The Questionnaire is untimed and…

Learning to Write about Mathematics

ERIC Educational Resources Information Center

Parker, Renee; Breyfogle, M. Lynn

2011-01-01

Beginning in third grade, Pennsylvania students are required to take the Pennsylvania State Standardized Assessment (PSSA), which presents multiple-choice mathematics questions and open-ended mathematics problems. Consistent with the Communication Standard of the National Council of Teachers of Mathematics, while solving the open-ended problems,…

Study preferences for exemplar variability in self-regulated category learning.

PubMed

Wahlheim, Christopher N; DeSoto, K Andrew

2017-02-01

Increasing exemplar variability during category learning can enhance classification of novel exemplars from studied categories. Four experiments examined whether participants preferred variability when making study choices with the goal of later classifying novel exemplars. In Experiments 1-3, participants were familiarised with exemplars of birds from multiple categories prior to making category-level assessments of learning and subsequent choices about whether to receive more variability or repetitions of exemplars during study. After study, participants classified novel exemplars from studied categories. The majority of participants showed a consistent preference for variability in their study, but choices were not related to category-level assessments of learning. Experiment 4 provided evidence that study preferences were based primarily on theoretical beliefs in that most participants indicated a preference for variability on questionnaires that did not include prior experience with exemplars. Potential directions for theoretical development and applications to education are discussed.

Measures of Partial Knowledge and Unexpected Responses in Multiple-Choice Tests

ERIC Educational Resources Information Center

Chang, Shao-Hua; Lin, Pei-Chun; Lin, Zih-Chuan

2007-01-01

This study investigates differences in the partial scoring performance of examinees in elimination testing and conventional dichotomous scoring of multiple-choice tests implemented on a computer-based system. Elimination testing that uses the same set of multiple-choice items rewards examinees with partial knowledge over those who are simply…

Validating Measurement of Knowledge Integration in Science Using Multiple-Choice and Explanation Items

ERIC Educational Resources Information Center

Lee, Hee-Sun; Liu, Ou Lydia; Linn, Marcia C.

2011-01-01

This study explores measurement of a construct called knowledge integration in science using multiple-choice and explanation items. We use construct and instructional validity evidence to examine the role multiple-choice and explanation items plays in measuring students' knowledge integration ability. For construct validity, we analyze item…

Making the Most of Multiple Choice

ERIC Educational Resources Information Center

Brookhart, Susan M.

2015-01-01

Multiple-choice questions draw criticism because many people perceive they test only recall or atomistic, surface-level objectives and do not require students to think. Although this can be the case, it does not have to be that way. Susan M. Brookhart suggests that multiple-choice questions are a useful part of any teacher's questioning repertoire…

The Positive and Negative Consequences of Multiple-Choice Testing

ERIC Educational Resources Information Center

Roediger, Henry L.; Marsh, Elizabeth J.

2005-01-01

Multiple-choice tests are commonly used in educational settings but with unknown effects on students' knowledge. The authors examined the consequences of taking a multiple-choice test on a later general knowledge test in which students were warned not to guess. A large positive testing effect was obtained: Prior testing of facts aided final…

Validity and Realibility of Chemistry Systemic Multiple Choices Questions (CSMCQs)

ERIC Educational Resources Information Center

Priyambodo, Erfan; Marfuatun

2016-01-01

Nowadays, Rasch model analysis is used widely in social research, moreover in educational research. In this research, Rasch model is used to determine the validation and the reliability of systemic multiple choices question in chemistry teaching and learning. There were 30 multiple choices question with systemic approach for high school student…

Multiple Choice Items: How to Gain the Most out of Them.

ERIC Educational Resources Information Center

Talmir, Pinchas

1991-01-01

Describes how multiple-choice items can be designed and used as an effective diagnostic tool by avoiding their pitfalls and by taking advantage of their potential benefits. The following issues are discussed: correct' versus best answers; construction of diagnostic multiple-choice items; the problem of guessing; the use of justifications of…

Comparison of Difficulties and Reliabilities of Math-Completion and Multiple-Choice Item Formats.

ERIC Educational Resources Information Center

Oosterhof, Albert C.; Coats, Pamela K.

Instructors who develop classroom examinations that require students to provide a numerical response to a mathematical problem are often very concerned about the appropriateness of the multiple-choice format. The present study augments previous research relevant to this concern by comparing the difficulty and reliability of multiple-choice and…

Factors influencing tests of auditory processing: a perspective on current issues and relevant concerns.

PubMed

Cacace, Anthony T; McFarland, Dennis J

2013-01-01

Tests of auditory perception, such as those used in the assessment of central auditory processing disorders ([C]APDs), represent a domain in audiological assessment where measurement of this theoretical construct is often confounded by nonauditory abilities due to methodological shortcomings. These confounds include the effects of cognitive variables such as memory and attention and suboptimal testing paradigms, including the use of verbal reproduction as a form of response selection. We argue that these factors need to be controlled more carefully and/or modified so that their impact on tests of auditory and visual perception is only minimal. To advocate for a stronger theoretical framework than currently exists and to suggest better methodological strategies to improve assessment of auditory processing disorders (APDs). Emphasis is placed on adaptive forced-choice psychophysical methods and the use of matched tasks in multiple sensory modalities to achieve these goals. Together, this approach has potential to improve the construct validity of the diagnosis, enhance and develop theory, and evolve into a preferred method of testing. Examination of methods commonly used in studies of APDs. Where possible, currently used methodology is compared to contemporary psychophysical methods that emphasize computer-controlled forced-choice paradigms. In many cases, the procedures used in studies of APD introduce confounding factors that could be minimized if computer-controlled forced-choice psychophysical methods were utilized. Ambiguities of interpretation, indeterminate diagnoses, and unwanted confounds can be avoided by minimizing memory and attentional demands on the input end and precluding the use of response-selection strategies that use complex motor processes on the output end. Advocated are the use of computer-controlled forced-choice psychophysical paradigms in combination with matched tasks in multiple sensory modalities to enhance the prospect of obtaining a valid diagnosis. American Academy of Audiology.

[Continuing medical education: how to write multiple choice questions].

PubMed

Soler Fernández, R; Méndez Díaz, C; Rodríguez García, E

2013-06-01

Evaluating professional competence in medicine is a difficult but indispensable task because it makes it possible to evaluate, at different times and from different perspectives, the extent to which the knowledge, skills, and values required for exercising the profession have been acquired. Tests based on multiple choice questions have been and continue to be among the most useful tools for objectively evaluating learning in medicine. When these tests are well designed and correctly used, they can stimulate learning and even measure higher cognitive skills. Designing a multiple choice test is a difficult task that requires knowledge of the material to be tested and of the methodology of test preparation as well as time to prepare the test. The aim of this article is to review what can be evaluated through multiple choice tests, the rules and guidelines that should be taken into account when writing multiple choice questions, the different formats that can be used, the most common errors in elaborating multiple choice tests, and how to analyze the results of the test to verify its quality. Copyright © 2012 SERAM. Published by Elsevier Espana. All rights reserved.

"None of the above" as a correct and incorrect alternative on a multiple-choice test: implications for the testing effect.

PubMed

Odegard, Timothy N; Koen, Joshua D

2007-11-01

Both positive and negative testing effects have been demonstrated with a variety of materials and paradigms (Roediger & Karpicke, 2006b). The present series of experiments replicate and extend the research of Roediger and Marsh (2005) with the addition of a "none-of-the-above" response option. Participants (n=32 in both experiments) read a set of passages, took an initial multiple-choice test, completed a filler task, and then completed a final cued-recall test (Experiment 1) or multiple-choice test (Experiment 2). Questions were manipulated on the initial multiple-choice test by adding a "none-of-the-above" response alternative (choice "E") that was incorrect ("E" Incorrect) or correct ("E" Correct). The results from both experiments demonstrated that the positive testing effect was negated when the "none-of-the-above" alternative was the correct response on the initial multiple-choice test, but was still present when the "none-of-the-above" alternative was an incorrect response.

A comparative analysis of multiple-choice and student performance-task assessment in the high school biology classroom

NASA Astrophysics Data System (ADS)

Cushing, Patrick Ryan

This study compared the performance of high school students on laboratory assessments. Thirty-four high school students who were enrolled in the second semester of a regular biology class or had completed the biology course the previous semester participated in this study. They were randomly assigned to examinations of two formats, performance-task and traditional multiple-choice, from two content areas, using a compound light microscope and diffusion. Students were directed to think-aloud as they performed the assessments. Additional verbal data were obtained during interviews following the assessment. The tape-recorded narrative data were analyzed for type and diversity of knowledge and skill categories, and percentage of in-depth processing demonstrated. While overall mean scores on the assessments were low, elicited statements provided additional insight into student cognition. Results indicated that a greater diversity of knowledge and skill categories was elicited by the two microscope assessments and by the two performance-task assessments. In addition, statements demonstrating in-depth processing were coded most frequently in narratives elicited during clinical interviews following the diffusion performance-task assessment. This study calls for individual teachers to design authentic assessment practices and apply them to daily classroom routines. Authentic assessment should be an integral part of the learning process and not merely an end result. In addition, teachers are encouraged to explicitly identify and model, through think-aloud methods, desired cognitive behaviors in the classroom.

Strategies for Coping in a Complex World: Adherence Behavior Among Older Adults with Chronic Illness

PubMed Central

Ross-Degnan, Dennis; Adams, Alyce S.; Safran, Dana Gelb; Soumerai, Stephen B.

2007-01-01

Background Increasing numbers of medicines increase nonadherence. Little is known about how older adults manage multiple medicines for multiple illnesses. Objectives To explore how older adults with multiple illnesses make choices about medicines. Design Semistructured interviews with older adults taking several medications. Accounts of respondents’ medicine-taking behavior were collected. Participants Twenty community-dwelling seniors with health insurance, in Eastern Massachusetts, aged 67–90, (4–12 medicines, 3–9 comorbidities). Approach Qualitative analysis using constant comparison to explain real choices made about medicines in the past (“historical”) and hypothetical (“future”) choices. Results Respondents reported both past (“historical”) choices and hypothetical (“future”) choices between medicines. Although people discussed effectiveness and future risk of the disease when prompted to prioritize their medicines (future choices), key factors leading to nonadherence (historical choices) were costs and side effects. Specific choices were generally dominated by 1 factor, and respondents rarely reported making explicit trade-offs between different factors. Factors affecting 1 choice were not necessarily the same as those affecting another choice in the same person. There was no evidence of “adherent” personalities. Conclusion Prescribing a new medicine, a change in provider or copayment can provoke new choices about both new and existing medications in older adults with multiple morbidities. PMID:17406952

Predicting social and communicative ability in school-age children with autism spectrum disorder: A pilot study of the Social Attribution Task, Multiple Choice.

PubMed

Burger-Caplan, Rebecca; Saulnier, Celine; Jones, Warren; Klin, Ami

2016-11-01

The Social Attribution Task, Multiple Choice is introduced as a measure of implicit social cognitive ability in children, addressing a key challenge in quantification of social cognitive function in autism spectrum disorder, whereby individuals can often be successful in explicit social scenarios, despite marked social adaptive deficits. The 19-question Social Attribution Task, Multiple Choice, which presents ambiguous stimuli meant to elicit social attribution, was administered to children with autism spectrum disorder (N = 23) and to age-matched and verbal IQ-matched typically developing children (N = 57). The Social Attribution Task, Multiple Choice performance differed between autism spectrum disorder and typically developing groups, with typically developing children performing significantly better than children with autism spectrum disorder. The Social Attribution Task, Multiple Choice scores were positively correlated with age (r = 0.474) while being independent from verbal IQ (r = 0.236). The Social Attribution Task, Multiple Choice was strongly correlated with Vineland Adaptive Behavior Scales Communication (r = 0.464) and Socialization (r = 0.482) scores, but not with Daily Living Skills scores (r = 0.116), suggesting that the implicit social cognitive ability underlying performance on the Social Attribution Task, Multiple Choice is associated with real-life social adaptive function. © The Author(s) 2016.

Automatic Short Essay Scoring Using Natural Language Processing to Extract Semantic Information in the Form of Propositions. CRESST Report 831

ERIC Educational Resources Information Center

Kerr, Deirdre; Mousavi, Hamid; Iseli, Markus R.

2013-01-01

The Common Core assessments emphasize short essay constructed-response items over multiple-choice items because they are more precise measures of understanding. However, such items are too costly and time consuming to be used in national assessments unless a way to score them automatically can be found. Current automatic essay-scoring techniques…

The Design and Development of a Context-Rich, Photo-Based Online Testing to Assess Students' Science Learning

ERIC Educational Resources Information Center

Lin, Min-Jin; Guo, Chorng-Jee; Hsu, Chia-Er

2011-01-01

This study designed and developed a CP-MCT (content-rich, photo-based multiple choice online test) to assess whether college students can apply the basic light concept to interpret daily light phenomena. One hundred college students volunteered to take the CP-MCT, and the results were statistically analyzed by applying t-test or ANOVA (Analysis of…

Teacher Perceptions toward the Interpretation of Results from the New Norm-Referenced Portion of the Mississippi Assessment System. A Pilot Study.

ERIC Educational Resources Information Center

Howe, Mary E.; And Others

Standardized testing, usually in the form of a multiple choice test, has dominated educational reform throughout Mississippi for the past 2 decades. Because of the minimal impact that standardized testing has traditionally had on curriculum decisions and classroom instruction, a paradigm shift in assessment format was adopted in the State from a…

Investigating Students' Understanding of the Dissolving Process

ERIC Educational Resources Information Center

Naah, Basil M.; Sanger, Michael J.

2013-01-01

In a previous study, the authors identified several student misconceptions regarding the process of dissolving ionic compounds in water. The present study used multiple-choice questions whose distractors were derived from these misconceptions to assess students' understanding of the dissolving process at the symbolic and particulate levels. The…

Testing Collective Memory: Representing the Soviet Union on Multiple-Choice Questions

ERIC Educational Resources Information Center

Reich, Gabriel A.

2011-01-01

This article tests the assumption that state-mandated multiple-choice history exams are a cultural tool for disseminating an "official" collective memory. Findings from a qualitative study of a collection of multiple-choice questions that relate to the history of the Soviet Union are presented. The 263 questions all come from New York…

Multiple-Choice and Short-Answer Exam Performance in a College Classroom

ERIC Educational Resources Information Center

Funk, Steven C.; Dickson, K. Laurie

2011-01-01

The authors experimentally investigated the effects of multiple-choice and short-answer format exam items on exam performance in a college classroom. They randomly assigned 50 students to take a 10-item short-answer pretest or posttest on two 50-item multiple-choice exams in an introduction to personality course. Students performed significantly…

Using a Classroom Response System to Improve Multiple-Choice Performance in AP[R] Physics

ERIC Educational Resources Information Center

Bertrand, Peggy

2009-01-01

Participation in rigorous high school courses such as Advanced Placement (AP[R]) Physics increases the likelihood of college success, especially for students who are traditionally underserved. Tackling difficult multiple-choice exams should be part of any AP program because well-constructed multiple-choice questions, such as those on AP exams and…

Teaching Critical Thinking without (Much) Writing: Multiple-Choice and Metacognition

ERIC Educational Resources Information Center

Bassett, Molly H.

2016-01-01

In this essay, I explore an exam format that pairs multiple-choice questions with required rationales. In a space adjacent to each multiple-choice question, students explain why or how they arrived at the answer they selected. This exercise builds the critical thinking skill known as metacognition, thinking about thinking, into an exam that also…

The Answering Process for Multiple-Choice Questions in Collaborative Learning: A Mathematical Learning Model Analysis

ERIC Educational Resources Information Center

Nakamura, Yasuyuki; Nishi, Shinnosuke; Muramatsu, Yuta; Yasutake, Koichi; Yamakawa, Osamu; Tagawa, Takahiro

2014-01-01

In this paper, we introduce a mathematical model for collaborative learning and the answering process for multiple-choice questions. The collaborative learning model is inspired by the Ising spin model and the model for answering multiple-choice questions is based on their difficulty level. An intensive simulation study predicts the possibility of…

Are Multiple Choice Tests Fair to Medical Students with Specific Learning Disabilities?

ERIC Educational Resources Information Center

Ricketts, Chris; Brice, Julie; Coombes, Lee

2010-01-01

The purpose of multiple choice tests of medical knowledge is to estimate as accurately as possible a candidate's level of knowledge. However, concern is sometimes expressed that multiple choice tests may also discriminate in undesirable and irrelevant ways, such as between minority ethnic groups or by sex of candidates. There is little literature…

Multiple Choice Testing and the Retrieval Hypothesis of the Testing Effect

ERIC Educational Resources Information Center

Sensenig, Amanda E.

2010-01-01

Taking a test often leads to enhanced later memory for the tested information, a phenomenon known as the "testing effect". This memory advantage has been reliably demonstrated with recall tests but not multiple choice tests. One potential explanation for this finding is that multiple choice tests do not rely on retrieval processes to the same…

Do Streaks Matter in Multiple-Choice Tests?

ERIC Educational Resources Information Center

Kiss, Hubert János; Selei, Adrienn

2018-01-01

Success in life is determined to a large extent by school performance, which in turn depends heavily on grades obtained in exams. In this study, we investigate a particular type of exam: multiple-choice tests. More concretely, we study if patterns of correct answers in multiple-choice tests affect performance. We design an experiment to study if…

The Effects of Clinically Relevant Multiple-Choice Items on the Statistical Discrimination of Physician Clinical Competence.

ERIC Educational Resources Information Center

Downing, Steven M.; Maatsch, Jack L.

To test the effect of clinically relevant multiple-choice item content on the validity of statistical discriminations of physicians' clinical competence, data were collected from a field test of the Emergency Medicine Examination, test items for the certification of specialists in emergency medicine. Two 91-item multiple-choice subscales were…

The Relationship of Deep and Surface Study Approaches on Factual and Applied Test-Bank Multiple-Choice Question Performance

ERIC Educational Resources Information Center

Yonker, Julie E.

2011-01-01

With the advent of online test banks and large introductory classes, instructors have often turned to textbook publisher-generated multiple-choice question (MCQ) exams in their courses. Multiple-choice questions are often divided into categories of factual or applied, thereby implicating levels of cognitive processing. This investigation examined…

A Diagnostic Study of Pre-Service Teachers' Competency in Multiple-Choice Item Development

ERIC Educational Resources Information Center

Asim, Alice E.; Ekuri, Emmanuel E.; Eni, Eni I.

2013-01-01

Large class size is an issue in testing at all levels of Education. As a panacea to this, multiple choice test formats has become very popular. This case study was designed to diagnose pre-service teachers' competency in constructing questions (IQT); direct questions (DQT); and best answer (BAT) varieties of multiple choice items. Subjects were 88…

The Impact of Escape Alternative Position Change in Multiple-Choice Test on the Psychometric Properties of a Test and Its Items Parameters

ERIC Educational Resources Information Center

Hamadneh, Iyad Mohammed

2015-01-01

This study aimed at investigating the impact changing of escape alternative position in multiple-choice test on the psychometric properties of a test and it's items parameters (difficulty, discrimination & guessing), and estimation of examinee ability. To achieve the study objectives, a 4-alternative multiple choice type achievement test…

Effects of Mayfield's Four Questions (M4Q) on Nursing Students' Self-Efficacy and Multiple-Choice Test Scores

ERIC Educational Resources Information Center

Mayfield, Linda Riggs

2010-01-01

This study examined the effects of being taught the Mayfield's Four Questions multiple-choice test-taking strategy on the perceived self-efficacy and multiple-choice test scores of nursing students in a two-year associate degree program. Experimental and control groups were chosen by stratified random sampling. Subjects completed the 10-statement…

Multiple-choice pretesting potentiates learning of related information.

PubMed

Little, Jeri L; Bjork, Elizabeth Ligon

2016-10-01

Although the testing effect has received a substantial amount of empirical attention, such research has largely focused on the effects of tests given after study. The present research examines the effect of using tests prior to study (i.e., as pretests), focusing particularly on how pretesting influences the subsequent learning of information that is not itself pretested but that is related to the pretested information. In Experiment 1, we found that multiple-choice pretesting was better for the learning of such related information than was cued-recall pretesting or a pre-fact-study control condition. In Experiment 2, we found that the increased learning of non-pretested related information following multiple-choice testing could not be attributed to increased time allocated to that information during subsequent study. Last, in Experiment 3, we showed that the benefits of multiple-choice pretesting over cued-recall pretesting for the learning of related information persist over 48 hours, thus demonstrating the promise of multiple-choice pretesting to potentiate learning in educational contexts. A possible explanation for the observed benefits of multiple-choice pretesting for enhancing the effectiveness with which related nontested information is learned during subsequent study is discussed.

Behavioral economic analysis of drug preference using multiple choice procedure data.

PubMed

Greenwald, Mark K

2008-01-11

The multiple choice procedure has been used to evaluate preference for psychoactive drugs, relative to money amounts (price), in human subjects. The present re-analysis shows that MCP data are compatible with behavioral economic analysis of drug choices. Demand curves were constructed from studies with intravenous fentanyl, intramuscular hydromorphone and oral methadone in opioid-dependent individuals; oral d-amphetamine, oral MDMA alone and during fluoxetine treatment, and smoked marijuana alone or following naltrexone pretreatment in recreational drug users. For each participant and dose, the MCP crossover point was converted into unit price (UP) by dividing the money value ($) by the drug dose (mg/70kg). At the crossover value, the dose ceases to function as a reinforcer, so "0" was entered for this and higher UPs to reflect lack of drug choice. At lower UPs, the dose functions as a reinforcer and "1" was entered to reflect drug choice. Data for UP vs. average percent choice were plotted in log-log space to generate demand functions. Rank of order of opioid inelasticity (slope of non-linear regression) was: fentanyl>hydromorphone (continuing heroin users)>methadone>hydromorphone (heroin abstainers). Rank order of psychostimulant inelasticity was d-amphetamine>MDMA>MDMA+fluoxetine. Smoked marijuana was more inelastic with high-dose naltrexone. These findings show this method translates individuals' drug preferences into estimates of population demand, which has the potential to yield insights into pharmacotherapy efficacy, abuse liability assessment, and individual differences in susceptibility to drug abuse.

Behavioral Economic Analysis of Drug Preference Using Multiple Choice Procedure Data

PubMed Central

Greenwald, Mark K.

2008-01-01

The Multiple Choice Procedure has been used to evaluate preference for psychoactive drugs, relative to money amounts (price), in human subjects. The present re-analysis shows that MCP data are compatible with behavioral economic analysis of drug choices. Demand curves were constructed from studies with intravenous fentanyl, intramuscular hydromorphone and oral methadone in opioid-dependent individuals; oral d-amphetamine, oral MDMA alone and during fluoxetine treatment, and smoked marijuana alone or following naltrexone pretreatment in recreational drug users. For each participant and dose, the MCP crossover point was converted into unit price (UP) by dividing the money value ($) by the drug dose (mg/70 kg). At the crossover value, the dose ceases to function as a reinforcer, so “0” was entered for this and higher UPs to reflect lack of drug choice. At lower UPs, the dose functions as a reinforcer and “1” was entered to reflect drug choice. Data for UP vs. average percent choice were plotted in log-log space to generate demand functions. Rank of order of opioid inelasticity (slope of non-linear regression) was: fentanyl > hydromorphone (continuing heroin users) > methadone > hydromorphone (heroin abstainers). Rank order of psychostimulant inelasticity was d-amphetamine > MDMA > MDMA + fluoxetine. Smoked marijuana was more inelastic with high-dose naltrexone. These findings show this method translates individuals’ drug preferences into estimates of population demand, which has the potential to yield insights into pharmacotherapy efficacy, abuse liability assessment, and individual differences in susceptibility to drug abuse. PMID:17949924

Ability Level Estimation of Students on Probability Unit via Computerized Adaptive Testing

ERIC Educational Resources Information Center

Özyurt, Hacer; Özyurt, Özcan

2015-01-01

Problem Statement: Learning-teaching activities bring along the need to determine whether they achieve their goals. Thus, multiple choice tests addressing the same set of questions to all are frequently used. However, this traditional assessment and evaluation form contrasts with modern education, where individual learning characteristics are…

Supporting Students' Learning: The Use of Formative Online Assessments

ERIC Educational Resources Information Center

Einig, Sandra

2013-01-01

This paper investigates the impact of online multiple choice questions (MCQs) on students' learning in an undergraduate Accounting module at a British university. The impact is considered from three perspectives: an analysis of how students use the MCQs; students' perceptions expressed in a questionnaire survey; and an investigation of the…

Constructing Multiple-Choice Items to Measure Higher-Order Thinking

ERIC Educational Resources Information Center

Scully, Darina

2017-01-01

Across education, certification and licensure, there are repeated calls for the development of assessments that target "higher-order thinking," as opposed to mere recall of facts. A common assumption is that this necessitates the use of constructed response or essay-style test questions; however, empirical evidence suggests that this may…

Utah Drug Use Questionnaire.

ERIC Educational Resources Information Center

Governor's Citizen Advisory Committee on Drugs, Salt Lake City, UT.

This questionnaire assesses drug use practices in junior and senior high school students. The 21 multiple choice items pertain to drug use practices, use history, available of drugs, main reason for drug use, and demographic data. The questionnaire is untimed, group administered, and may be given by the classroom teacher in about 10 minutes. Item…

Utah Drop-Out Drug Use Questionnaire.

ERIC Educational Resources Information Center

Governor's Citizen Advisory Committee on Drugs, Salt Lake City, UT.

This questionnaire assesses drug use practices in high school drop-outs. The 79 items (multiple choice or apply/not apply) are concerned with demographic data and use, use history, reasons for use/nonuse, attitudes toward drugs, availability of drugs, and drug information with respect to narcotics, amphetamines, LSD, Marijuana, and barbiturates.…

New York Community Environment Study Questionnaire.

ERIC Educational Resources Information Center

Glaser, Daniel; Snow, Mary

This questionnaire assesses neighborhood drug problem concern, drug use practices, knowledge of drugs and agencies dealing with drugs, and views on drug education in persons aged 13 or older. The questionnaire has 31 items (multiple-choice or free response), most with several parts. The items deal with demographic and personal data, problems in…

Heubach Smoking Habits and Attitudes Questionnaire.

ERIC Educational Resources Information Center

Heubach, Philip Gilbert

This Questionnaire, consisting of 74 yes/no, multiple choice, and completion items, is designed to assess smoking practices and attitudes toward smoking in high school students. Questions pertain to personal data, family smoking practices and attitudes, personal smoking habits, reasons for smoking or not smoking, and opinions on smoking. Detailed…

Mediating Relationship of Differential Products in Understanding Integration in Introductory Physics

ERIC Educational Resources Information Center

Amos, Nathaniel; Heckler, Andrew F.

2018-01-01

In the context of introductory physics, we study student conceptual understanding of differentials, differential products, and integrals and possible pathways to understanding these quantities. We developed a multiple choice conceptual assessment employing a variety of physical contexts probing physical understanding of these three quantities and…

Michigan High School Student Drug Attitudes and Behavior Questionnaire.

ERIC Educational Resources Information Center

Bogg, Richard A.; And Others

This questionnaire assesses drug use practices and attitudes toward drugs in high school students. The instrument has 59 items (multiple choice or completion), some with several parts. The question pertain to aspirations for the future, general attitudes and opinions, biographic and demographic data, family background and relationships, alcohol…

The Multiple-Choice Concept Map (MCCM): An Interactive Computer-Based Assessment Method

ERIC Educational Resources Information Center

Sas, Ioan Ciprian

2010-01-01

This research attempted to bridge the gap between cognitive psychology and educational measurement (Mislevy, 2008; Leighton & Gierl, 2007; Nichols, 1994; Messick, 1989; Snow & Lohman, 1989) by using cognitive theories from working memory (Baddeley, 1986; Miyake & Shah, 1999; Grimley & Banner, 2008), multimedia learning (Mayer, 2001), and cognitive…

Investigating Urban Eighth-Grade Students' Knowledge of Energy Resources

ERIC Educational Resources Information Center

Bodzin, Alec

2012-01-01

This study investigated urban eighth-grade students' knowledge of energy resources and associated issues including energy acquisition, energy generation, storage and transport, and energy consumption and conservation. A 39 multiple-choice-item energy resources knowledge assessment was completed by 1043 eighth-grade students in urban schools in two…

Building the BIKE: Development and Testing of the Biotechnology Instrument for Knowledge Elicitation (BIKE)

ERIC Educational Resources Information Center

Witzig, Stephen B.; Rebello, Carina M.; Siegel, Marcelle A.; Freyermuth, Sharyn K.; Izci, Kemal; McClure, Bruce

2014-01-01

Identifying students' conceptual scientific understanding is difficult if the appropriate tools are not available for educators. Concept inventories have become a popular tool to assess student understanding; however, traditionally, they are multiple choice tests. International science education standard documents advocate that assessments…

Development and Validation of the Homeostasis Concept Inventory

ERIC Educational Resources Information Center

McFarland, Jenny L.; Price, Rebecca M.; Wenderoth, Mary Pat; Martinková, Patrícia; Cliff, William; Michael, Joel; Modell, Harold; Wright, Ann

2017-01-01

We present the Homeostasis Concept Inventory (HCI), a 20-item multiple-choice instrument that assesses how well undergraduates understand this critical physiological concept. We used an iterative process to develop a set of questions based on elements in the Homeostasis Concept Framework. This process involved faculty experts and undergraduate…

Modeling Errors in Daily Precipitation Measurements: Additive or Multiplicative?

NASA Technical Reports Server (NTRS)

Tian, Yudong; Huffman, George J.; Adler, Robert F.; Tang, Ling; Sapiano, Matthew; Maggioni, Viviana; Wu, Huan

2013-01-01

The definition and quantification of uncertainty depend on the error model used. For uncertainties in precipitation measurements, two types of error models have been widely adopted: the additive error model and the multiplicative error model. This leads to incompatible specifications of uncertainties and impedes intercomparison and application.In this letter, we assess the suitability of both models for satellite-based daily precipitation measurements in an effort to clarify the uncertainty representation. Three criteria were employed to evaluate the applicability of either model: (1) better separation of the systematic and random errors; (2) applicability to the large range of variability in daily precipitation; and (3) better predictive skills. It is found that the multiplicative error model is a much better choice under all three criteria. It extracted the systematic errors more cleanly, was more consistent with the large variability of precipitation measurements, and produced superior predictions of the error characteristics. The additive error model had several weaknesses, such as non constant variance resulting from systematic errors leaking into random errors, and the lack of prediction capability. Therefore, the multiplicative error model is a better choice.

What influences the choice of assessment methods in health technology assessments? Statistical analysis of international health technology assessments from 1989 to 2002.

PubMed

Draborg, Eva; Andersen, Christian Kronborg

2006-01-01

Health technology assessment (HTA) has been used as input in decision making worldwide for more than 25 years. However, no uniform definition of HTA or agreement on assessment methods exists, leaving open the question of what influences the choice of assessment methods in HTAs. The objective of this study is to analyze statistically a possible relationship between methods of assessment used in practical HTAs, type of assessed technology, type of assessors, and year of publication. A sample of 433 HTAs published by eleven leading institutions or agencies in nine countries was reviewed and analyzed by multiple logistic regression. The study shows that outsourcing of HTA reports to external partners is associated with a higher likelihood of using assessment methods, such as meta-analysis, surveys, economic evaluations, and randomized controlled trials; and with a lower likelihood of using assessment methods, such as literature reviews and "other methods". The year of publication was statistically related to the inclusion of economic evaluations and shows a decreasing likelihood during the year span. The type of assessed technology was related to economic evaluations with a decreasing likelihood, to surveys, and to "other methods" with a decreasing likelihood when pharmaceuticals were the assessed type of technology. During the period from 1989 to 2002, no major developments in assessment methods used in practical HTAs were shown statistically in a sample of 433 HTAs worldwide. Outsourcing to external assessors has a statistically significant influence on choice of assessment methods.

A multiple choice testing program coupled with a year-long elective experience is associated with improved performance on the internal medicine in-training examination.

PubMed

Mathis, Bradley R; Warm, Eric J; Schauer, Daniel P; Holmboe, Eric; Rouan, Gregory W

2011-11-01

The Internal Medicine In-Training Exam (IM-ITE) assesses the content knowledge of internal medicine trainees. Many programs use the IM-ITE to counsel residents, to create individual remediation plans, and to make fundamental programmatic and curricular modifications. To assess the association between a multiple-choice testing program administered during 12 consecutive months of ambulatory and inpatient elective experience and IM-ITE percentile scores in third post-graduate year (PGY-3) categorical residents. Retrospective cohort study. One hundred and four categorical internal medicine residents. Forty-five residents in the 2008 and 2009 classes participated in the study group, and the 59 residents in the three classes that preceded the use of the testing program, 2005-2007, served as controls. A comprehensive, elective rotation specific, multiple-choice testing program and a separate board review program, both administered during a continuous long-block elective experience during the twelve months between the second post-graduate year (PGY-2) and PGY-3 in-training examinations. We analyzed the change in median individual percent correct and percentile scores between the PGY-1 and PGY-2 IM-ITE and between the PGY-2 and PGY-3 IM-ITE in both control and study cohorts. For our main outcome measure, we compared the change in median individual percentile rank between the control and study cohorts between the PGY-2 and the PGY-3 IM-ITE testing opportunities. After experiencing the educational intervention, the study group demonstrated a significant increase in median individual IM-ITE percentile score between PGY-2 and PGY-3 examinations of 8.5 percentile points (p < 0.01). This is significantly better than the increase of 1.0 percentile point seen in the control group between its PGY-2 and PGY-3 examination (p < 0.01). A comprehensive multiple-choice testing program aimed at PGY-2 residents during a 12-month continuous long-block elective experience is associated with improved PGY-3 IM-ITE performance.

The Impact of High School Science Teachers' Beliefs, Curricular Enactments and Experience on Student Learning During an Inquiry-based Urban Ecology Curriculum

NASA Astrophysics Data System (ADS)

McNeill, Katherine L.; Silva Pimentel, Diane; Strauss, Eric G.

2013-10-01

Inquiry-based curricula are an essential tool for reforming science education yet the role of the teacher is often overlooked in terms of the impact of the curriculum on student achievement. Our research focuses on 22 teachers' use of a year-long high school urban ecology curriculum and how teachers' self-efficacy, instructional practices, curricular enactments and previous experience impacted student learning. Data sources included teacher belief surveys, teacher enactment surveys, a student multiple-choice assessment focused on defining and identifying science concepts and a student open-ended assessment focused on scientific inquiry. Results from the two hierarchical linear models indicate that there was significant variation between teachers in terms of student achievement. For the multiple-choice assessment, teachers who spent a larger percentage of time on group work and a smaller percentage of time lecturing had greater student learning. For the open-ended assessment, teachers who reported a higher frequency of students engaging in argument and sharing ideas had greater student learning while teachers who adapted the curriculum more had lower student learning. These results suggest the importance of supporting the active role of students in instruction, emphasising argumentation, and considering the types of adaptations teachers make to curriculum.

Experiencing new forms of genetic choice: findings from an ethnographic study of preimplantation genetic diagnosis.

PubMed

Roberts, Celia; Franklin, Sarah

2004-12-01

Contemporary scientific and clinical knowledges and practices continue to make available new forms of genetic information, and to create new forms of reproductive choice. For example, couples at high risk of passing on a serious genetic condition to their offspring in Britain today have the opportunity to use Preimplantation Genetic Diagnosis (PGD) to select embryos that are unaffected by serious genetic disease. This information assists these couples in making reproductive choices. This article presents an analysis of patients' experiences of making the decision to undertake PGD treatment and of making reproductive choices based on genetic information. We present qualitative interview data from an ethnographic study of PGD based in two British clinics which indicate how these new forms of genetic choice are experienced by patients. Our data suggest that PGD patients make decisions about treatment in a complex way, taking multiple variables into account, and maintaining ongoing assessments of the multiple costs of engaging with PGD. Patients are aware of broader implications of their decisions, at personal, familial, and societal levels, as well as clinical ones. Based on these findings we argue that the ethical and social aspects of PGD are often as innovative as the scientific and medical aspects of this technique, and that in this sense, science cannot be described as "racing ahead" of society.

Effect of response format on cognitive reflection: Validating a two- and four-option multiple choice question version of the Cognitive Reflection Test.

PubMed

Sirota, Miroslav; Juanchich, Marie

2018-03-27

The Cognitive Reflection Test, measuring intuition inhibition and cognitive reflection, has become extremely popular because it reliably predicts reasoning performance, decision-making, and beliefs. Across studies, the response format of CRT items sometimes differs, based on the assumed construct equivalence of tests with open-ended versus multiple-choice items (the equivalence hypothesis). Evidence and theoretical reasons, however, suggest that the cognitive processes measured by these response formats and their associated performances might differ (the nonequivalence hypothesis). We tested the two hypotheses experimentally by assessing the performance in tests with different response formats and by comparing their predictive and construct validity. In a between-subjects experiment (n = 452), participants answered stem-equivalent CRT items in an open-ended, a two-option, or a four-option response format and then completed tasks on belief bias, denominator neglect, and paranormal beliefs (benchmark indicators of predictive validity), as well as on actively open-minded thinking and numeracy (benchmark indicators of construct validity). We found no significant differences between the three response formats in the numbers of correct responses, the numbers of intuitive responses (with the exception of the two-option version, which had a higher number than the other tests), and the correlational patterns of the indicators of predictive and construct validity. All three test versions were similarly reliable, but the multiple-choice formats were completed more quickly. We speculate that the specific nature of the CRT items helps build construct equivalence among the different response formats. We recommend using the validated multiple-choice version of the CRT presented here, particularly the four-option CRT, for practical and methodological reasons. Supplementary materials and data are available at https://osf.io/mzhyc/ .

Multiple-Choice Exams and Guessing: Results from a One-Year Study of General Chemistry Tests Designed to Discourage Guessing

ERIC Educational Resources Information Center

Campbell, Mark L.

2015-01-01

Multiple-choice exams, while widely used, are necessarily imprecise due to the contribution of the final student score due to guessing. This past year at the United States Naval Academy the construction and grading scheme for the department-wide general chemistry multiple-choice exams were revised with the goal of decreasing the contribution of…

Resistance of Collard Green Genotypes to Bemisia tabaci Biotype B: Characterization of Antixenosis.

PubMed

Domingos, G M; Baldin, E L L; Canassa, V F; Silva, I F; Lourenção, A L

2018-08-01

Bemisia tabaci (Genn.) biotype B (Hemiptera: Aleyrodidae) is an important pest of vegetable crops, including collard greens Brassica oleracea var. acephala (Brassicaceae). The use of resistant genotypes is an interesting option to reduce insect populations and can be used as an important tool for integrated pest management (IPM). This study evaluated 32 genotypes of collard greens against the attack of silver leaf whitefly, with the aim to characterize antixenosis. Initially, a multiple-choice trial was conducted using all genotypes, in which the adult attractiveness was assessed on two leaves per genotype at 24 and 48 h after infestation. After 48 h, one leaf of each genotype was randomly selected for the determination of the number of eggs per square centimeter. From the results of the multiple-choice trial, 13 genotypes were selected for a no-choice oviposition test, following the same method of the previous test. Colorimetric analyses were also performed to establish possible correlations between leaf color and insect colonization. Genotypes HS-20, OE, and VA were less attractive, demonstrating antixenosis. Genotypes LG, VE, J, MG, MOP, HS-20, VA, and MT had less oviposition in the multiple-choice test, which indicated expression of antixenosis. In the no-choice test, genotypes VE, P1C, CCB, RI-919, H, and J had less oviposition, which also characterized antixenosis. Therefore, genotypes VE and J showed the highest resistance stability because both had less oviposition in both test modalities. Thus, the resistance to B. tabaci biotype B indicates the genotypes HS-20, OE, VA, VE, and J are promising for use in breeding programs to develop resistance to whitefly.

Step by Step: Biology Undergraduates’ Problem-Solving Procedures during Multiple-Choice Assessment

PubMed Central

Prevost, Luanna B.; Lemons, Paula P.

2016-01-01

This study uses the theoretical framework of domain-specific problem solving to explore the procedures students use to solve multiple-choice problems about biology concepts. We designed several multiple-choice problems and administered them on four exams. We trained students to produce written descriptions of how they solved the problem, and this allowed us to systematically investigate their problem-solving procedures. We identified a range of procedures and organized them as domain general, domain specific, or hybrid. We also identified domain-general and domain-specific errors made by students during problem solving. We found that students use domain-general and hybrid procedures more frequently when solving lower-order problems than higher-order problems, while they use domain-specific procedures more frequently when solving higher-order problems. Additionally, the more domain-specific procedures students used, the higher the likelihood that they would answer the problem correctly, up to five procedures. However, if students used just one domain-general procedure, they were as likely to answer the problem correctly as if they had used two to five domain-general procedures. Our findings provide a categorization scheme and framework for additional research on biology problem solving and suggest several important implications for researchers and instructors. PMID:27909021

Comparison of paragraph comprehension test scores with reading versus listening-reading and multiple-choice versus nominal recall administration techniques: justification for the bypass approach.

PubMed

Weinberg, W A; McLean, A; Snider, R L; Rintelmann, J W; Brumback, R A

1989-12-01

Eight groups of learning disabled children (N = 100), categorized by the clinical Lexical Paradigm as good readers or poor readers, were individually administered the Gilmore Oral Reading Test, Form D, by one of four input/retrieval methods: (1) the standardized method of administration in which the child reads each paragraph aloud and then answers five questions relating to the paragraph [read/recall method]; (2) the child reads each paragraph aloud and then for each question selects the correct answer from among three choices read by the examiner [read/choice method]; (3) the examiner reads each paragraph aloud and reads each of the five questions to the child to answer [listen/recall method]; and (4) the examiner reads each paragraph aloud and then for each question reads three multiple-choice answers from which the child selects the correct answer [listen/choice method]. The major difference in scores was between the groups tested by the recall versus the orally read multiple-choice methods. This study indicated that poor readers who listened to the material and were tested by orally read multiple-choice format could perform as well as good readers. The performance of good readers was not affected by listening or by the method of testing. The multiple-choice testing improved the performance of poor readers independent of the input method. This supports the arguments made previously that a "bypass approach" to education of poor readers in which testing is accomplished using an orally read multiple-choice format can enhance the child's school performance on reading-related tasks. Using a listening while reading input method may further enhance performance.

Using a Classroom Response System to Improve Multiple-Choice Performance in AP® Physics

NASA Astrophysics Data System (ADS)

Bertrand, Peggy

2009-04-01

Participation in rigorous high school courses such as Advanced Placement (AP®) Physics increases the likelihood of college success, especially for students who are traditionally underserved. Tackling difficult multiple-choice exams should be part of any AP program because well-constructed multiple-choice questions, such as those on AP exams and on the Force Concept Inventory,2 are particularly good at rooting out common and persisting student misconceptions. Additionally, there are barriers to multiple-choice performance that have little to do with content mastery. For example, a student might fail to read the question thoroughly, forget to apply a reasonableness test to the answer, or simply work too slowly.

Using concept maps in a modified team-based learning exercise.

PubMed

Knollmann-Ritschel, Barbara E C; Durning, Steven J

2015-04-01

Medical school education has traditionally been driven by single discipline teaching and assessment. Newer medical school curricula often implement an organ-based approach that fosters integration of basic science and clinical disciplines. Concept maps are widely used in education. Through diagrammatic depiction of a variety of concepts and their specific connections with other ideas, concept maps provide a unique perspective into learning and performance that can complement other assessment methods commonly used in medical schools. In this innovation, we describe using concepts maps as a vehicle for a modified a classic Team-Based Learning (TBL) exercise. Modifications to traditional TBL in our innovation included replacing an individual assessment using multiple-choice questions with concept maps as well as combining the group assessment and application exercise whereby teams created concept maps. These modifications were made to further assess understanding of content across the Fundamentals module (the introductory module of the preclerkship curriculum). While preliminary, student performance and feedback from faculty and students support the use of concept maps in TBL. Our findings suggest concept maps can provide a unique means of determining assessment of learning and generating feedback to students. Concept maps can also demonstrate knowledge acquisition, organization of prior and new knowledge, and synthesis of that knowledge across disciplines in a unique way providing an additional means of assessment in addition to traditional multiple-choice questions. Reprint & Copyright © 2015 Association of Military Surgeons of the U.S.

Delayed, but not immediate, feedback after multiple-choice questions increases performance on a subsequent short-answer, but not multiple-choice, exam: evidence for the dual-process theory of memory.

PubMed

Sinha, Neha; Glass, Arnold Lewis

2015-01-01

Three experiments, two performed in the laboratory and one embedded in a college psychology lecture course, investigated the effects of immediate versus delayed feedback following a multiple-choice exam on subsequent short answer and multiple-choice exams. Performance on the subsequent multiple-choice exam was not affected by the timing of the feedback on the prior exam; however, performance on the subsequent short answer exam was better following delayed than following immediate feedback. This was true regardless of the order in which immediate versus delayed feedback was given. Furthermore, delayed feedback only had a greater effect than immediate feedback on subsequent short answer performance following correct, confident responses on the prior exam. These results indicate that delayed feedback cues a student's prior response and increases subsequent recollection of that response. The practical implication is that delayed feedback is better than immediate feedback during academic testing.

Can Multiple-Choice Testing Induce Desirable Difficulties? Evidence from the Laboratory and the Classroom.

PubMed

Bjork, Elizabeth Ligon; Soderstrom, Nicholas C; Little, Jeri L

2015-01-01

The term desirable difficulties (Bjork, 1994) refers to conditions of learning that, though often appearing to cause difficulties for the learner and to slow down the process of acquisition, actually improve long-term retention and transfer. One known desirable difficulty is testing (as compared with restudy), although typically it is tests that clearly involve retrieval--such as free and cued recall tests--that are thought to induce these learning benefits and not multiple-choice tests. Nonetheless, multiple-choice testing is ubiquitous in educational settings and many other high-stakes situations. In this article, we discuss research, in both the laboratory and the classroom, exploring whether multiple-choice testing can also be fashioned to promote the type of retrieval processes known to improve learning, and we speculate about the necessary properties that multiple-choice questions must possess, as well as the metacognitive strategy students need to use in answering such questions, to achieve this goal.

Student Opinion Inventory. Instructions for Use. Part A. Part B.

ERIC Educational Resources Information Center

National Study of School Evaluation, Arlington, VA.

An important part of any school's self-evaluation is student input or feedback. This inventory was developed in order to accomplish two goals: assessing student attitudes toward many facets of the school, and providing an opportunity for students to make recommendations for improvement. Thirty-four multiple choice items collect information on…

Critical Reading Comprehension in an Era of Accountability

ERIC Educational Resources Information Center

Comber, Barbara; Nixon, Helen

2011-01-01

This paper argues for the need for critical reading comprehension in an era of accountability that often promotes reading comprehension as readily assessable through students answering multiple choice questions of unseen texts. Based upon a 1 year study investigating literacy in Years 4-9 the ways strong-performing primary schools develop serious…

Testing to the Top: Everything But the Kitchen Sink?

ERIC Educational Resources Information Center

Dietel, Ron

2011-01-01

Two tests intended to measure student achievement of the Common Core State Standards will face intense scrutiny, but the test makers say they will include performance assessments and other items that are not multiple-choice questions. Incorporating performance items on this tests will bring up issues over scoring, costs, and validity.

Exploring Secondary Students' Understanding of Chemical Kinetics through Inquiry-Based Learning Activities

ERIC Educational Resources Information Center

Chairam, Sanoe; Klahan, Nutsuda; Coll, Richard K.

2015-01-01

This research is trying to evaluate the feedback of Thai secondary school students to inquiry-based teaching and learning methods, exemplified by the study of chemical kinetics. This work used the multiple-choice questions, scientifically practical diagram and questionnaire to assess students' understanding of chemical kinetics. The findings…

The Influence of Distractor Strength and Response Order on MCQ Responding

ERIC Educational Resources Information Center

Kiat, John Emmanuel; Ong, Ai Rene; Ganesan, Asha

2018-01-01

Multiple-choice questions (MCQs) play a key role in standardised testing and in-class assessment. Research into the influence of within-item response order on MCQ characteristics has been mixed. While some researchers have shown preferential selection of response options presented earlier in the answer list, others have failed to replicate these…

Development and Use of a Conceptual Survey in Introductory Quantum Physics

ERIC Educational Resources Information Center

Wuttiprom, Sura; Sharma, Manjula Devi; Johnston, Ian D.; Chitaree, Ratchapak; Soankwan, Chernchok

2009-01-01

Conceptual surveys have become increasingly popular at many levels to probe various aspects of science education research such as measuring student understanding of basic concepts and assessing the effectiveness of pedagogical material. The aim of this study was to construct a valid and reliable multiple-choice conceptual survey to investigate…

WordBytes: Exploring an Intermediate Constraint Format for Rapid Classification of Student Answers on Constructed Response Assessments

ERIC Educational Resources Information Center

Kim, Kerry J.; Meir, Eli; Pope, Denise S.; Wendel, Daniel

2017-01-01

Computerized classification of student answers offers the possibility of instant feedback and improved learning. Open response (OR) questions provide greater insight into student thinking and understanding than more constrained multiple choice (MC) questions, but development of automated classifiers is more difficult, often requiring training a…

University of Michigan Drug Education Questionnaire.

ERIC Educational Resources Information Center

Francis, John Bruce; Patch, David J.

This questionnaire assesses attitudes toward potential drug education programs and drug use practices in college students. The 87 items (multiple choice or free response) pertain to the history and extent of usage of 27 different drugs, including two non-existent drugs which may be utilized as a validity check; attitude toward the content, format,…

Observed Hierarchy of Student Proficiency with Period, Frequency, and Angular Frequency

ERIC Educational Resources Information Center

Young, Nicholas T.; Heckler, Andrew F.

2018-01-01

In the context of a generic harmonic oscillator, we investigated students' accuracy in determining the period, frequency, and angular frequency from mathematical and graphical representations. In a series of studies including interviews, free response tests, and multiple-choice tests developed in an iterative process, we assessed students in both…

A Quantum Chemistry Concept Inventory for Physical Chemistry Classes

ERIC Educational Resources Information Center

Dick-Perez, Marilu; Luxford, Cynthia J.; Windus, Theresa L.; Holme, Thomas

2016-01-01

A 14-item, multiple-choice diagnostic assessment tool, the quantum chemistry concept inventory or QCCI, is presented. Items were developed based on published student misconceptions and content coverage and then piloted and used in advanced physical chemistry undergraduate courses. In addition to the instrument itself, data from both a pretest,…

Improving Formative Assessment in Language Classrooms Using "GradeCam Go!"

ERIC Educational Resources Information Center

Kiliçkaya, Ferit

2017-01-01

This study aimed to determine EFL (English as a Foreign Language) teachers' perceptions and experience regarding their use of "GradeCam Go!" to grade multiple choice tests. The results of the study indicated that the participants overwhelmingly valued "GradeCam Go!" due to its features such as grading printed forms for…

Components of Answers to Multiple-Choice Questions on a Published Reading Comprehension Test: An Application of the Hanna-Oaster Approach.

ERIC Educational Resources Information Center

Entin, Eileen B.; Klare, George B.

1980-01-01

An approach to assessing context dependence was applied to data from the Nelson-Denny Reading Test. The results suggest that scores on the difficult passages are inflated because the examinees can answer the questions without having to comprehend the passage. (MKM)

A Multiple-Choice Mushroom: Schools, Colleges Rely More than Ever on Standardized Tests.

ERIC Educational Resources Information Center

Hawkins, B. Denise

1995-01-01

This discussion of college entrance examinations reviews differences between the Scholastic Assessment Test (SAT) and the American College Test. It then focuses on the SAT, discussing numbers of students taking the tests, changes in test construction to recognize contributions of women and minorities, involvement of African Americans in…

Understanding Misconceptions: Teaching and Learning in Middle School Physical Science

ERIC Educational Resources Information Center

Sadler, Philip M.; Sonnert, Gerhard

2016-01-01

In this study the authors set out to better understand the relationship between teacher knowledge of science and student learning. The authors administered identical multiple-choice assessment items both to teachers of middle school physical science and to their students throughout the school year. The authors found that teachers who have strong…

Assessment of Electrochemical Concepts: A Comparative Study Involving Senior High-School Students in Indonesia and Japan

ERIC Educational Resources Information Center

Rahayu, Sri; Treagust, David F.; Chandrasegaran, A. L.; Kita, Masakazu; Ibnu, Suhadi

2011-01-01

Background and purpose: This study investigated Indonesian and Japanese senior high-school students' understanding of electrochemistry concepts. Sample: The questionnaire was administered to 244 Indonesian and 189 Japanese public senior high-school students. Design and methods: An 18-item multiple-choice questionnaire relating to five conceptual…

Single-Word Intelligibility in Speakers with Repaired Cleft Palate

ERIC Educational Resources Information Center

Whitehill, Tara; Chau, Cynthia

2004-01-01

Many speakers with repaired cleft palate have reduced intelligibility, but there are limitations with current procedures for assessing intelligibility. The aim of this study was to construct a single-word intelligibility test for speakers with cleft palate. The test used a multiple-choice identification format, and was based on phonetic contrasts…

Assessment of Foundation Knowledge: Are Students Confident in Their Ability?

ERIC Educational Resources Information Center

Fenna, Doug S.

2004-01-01

Multiple-choice testing (MCT) has several advantages which are becoming more relevant in the current financial climate. In particular, they can be machine marked. As an objective testing method it is particularly relevant to engineering and other factual courses, but MCTs are not widely used in engineering because students can benefit from…

Developing and Validating Proof Comprehension Tests in Undergraduate Mathematics

ERIC Educational Resources Information Center

Mejía-Ramos, Juan Pablo; Lew, Kristen; de la Torre, Jimmy; Weber, Keith

2017-01-01

In this article, we describe and illustrate the process by which we developed and validated short, multiple-choice, reliable tests to assess undergraduate students' comprehension of three mathematical proofs. We discuss the purpose for each stage and how it benefited the design of our instruments. We also suggest ways in which this process could…

Ability, Demography, Learning Style, and Personality Trait Correlates of Student Preference for Assessment Method

ERIC Educational Resources Information Center

Furnham, Adrian; Christopher, Andrew; Garwood, Jeanette; Martin, Neil G.

2008-01-01

More than 400 students from four universities in America and Britain completed measures of learning style preference, general knowledge (as a proxy for intelligence), and preference for examination method. Learning style was consistently associated with preferences: surface learners preferred multiple choice and group work options, and viewed…

Progress Monitoring in Grade 5 Science for Low Achievers

ERIC Educational Resources Information Center

Vannest, Kimberly J.; Parker, Richard; Dyer, Nicole

2011-01-01

This article presents procedures and results from a 2-year project developing science key vocabulary (KV) short tests suitable for progress monitoring Grade 5 science in Texas public schools using computer-generated, -administered, and -scored assessments. KV items included KV definitions and important usages in a multiple-choice cloze format. A…

The Effect and Implications of a "Self-Correcting" Assessment Procedure

ERIC Educational Resources Information Center

Francis, Alisha L.; Barnett, Jerrold

2012-01-01

We investigated Montepare's (2005, 2007) self-correcting procedure for multiple-choice exams. Findings related to memory suggest this procedure should lead to improved retention by encouraging students to distribute the time spent reviewing the material. Results from a general psychology class (n = 98) indicate that the benefits are not as…

Decision making under internal uncertainty: the case of multiple-choice tests with different scoring rules.

PubMed

Bereby-Meyer, Yoella; Meyer, Joachim; Budescu, David V

2003-02-01

This paper assesses framing effects on decision making with internal uncertainty, i.e., partial knowledge, by focusing on examinees' behavior in multiple-choice (MC) tests with different scoring rules. In two experiments participants answered a general-knowledge MC test that consisted of 34 solvable and 6 unsolvable items. Experiment 1 studied two scoring rules involving Positive (only gains) and Negative (only losses) scores. Although answering all items was the dominating strategy for both rules, the results revealed a greater tendency to answer under the Negative scoring rule. These results are in line with the predictions derived from Prospect Theory (PT) [Econometrica 47 (1979) 263]. The second experiment studied two scoring rules, which allowed respondents to exhibit partial knowledge. Under the Inclusion-scoring rule the respondents mark all answers that could be correct, and under the Exclusion-scoring rule they exclude all answers that might be incorrect. As predicted by PT, respondents took more risks under the Inclusion rule than under the Exclusion rule. The results illustrate that the basic process that underlies choice behavior under internal uncertainty and especially the effect of framing is similar to the process of choice under external uncertainty and can be described quite accurately by PT. Copyright 2002 Elsevier Science B.V.

A discrete-choice experiment to determine patient preferences for injectable multiple sclerosis treatments in Germany.

PubMed

Poulos, Christine; Kinter, Elizabeth; Yang, Jui-Chen; Bridges, John F P; Posner, Joshua; Gleißner, Erika; Mühlbacher, Axel; Kieseier, Bernd

2016-03-01

The aim of this study was to assess the relative importance of features of a hypothetical injectable disease-modifying treatment for patients with multiple sclerosis using a discrete-choice experiment. German residents at least 18 years of age with a self-reported physician diagnosis of multiple sclerosis completed a 25-30 minute online discrete-choice experiment. Patients were asked to choose one of two hypothetical injectable treatments for multiple sclerosis, defined by different levels of six attributes (disability progression, the number of relapses in the next 4 years, injection time, frequency of injections, presence of flu-like symptoms, and presence of injection-site reactions). The data were analyzed using a random-parameters logit model. Of 202 adults who completed the survey, results from 189 were used in the analysis. Approximately 50% of all patients reported a diagnosis of relapsing-remitting multiple sclerosis, and 31% reported secondary progressive multiple sclerosis. Approximately 71% of patients had current or prior experience with injectable multiple sclerosis medication. Approximately 53% had experienced flu-like symptoms caused by their medication, and 47% had experienced mild injection-site reactions. At least one significant difference was seen between levels in all attributes, except injection time. The greatest change in relative importance between levels of an attribute was years until symptoms get worse from 1 to 4 years. The magnitude of this difference was about twice that of relapses in the next 4 years, frequency of injections, and flu-like symptoms. Most attributes examined in this experiment had an influence on patient preference. Patients placed a significant value on improvements in the frequency of dosing and disability progression. Results suggest that changes in injection frequency can be as important as changes in efficacy and safety attributes. Understanding which attributes of injectable therapies influence patient preference could potentially improve outcomes and adherence in patients with multiple sclerosis.

Evaluation of the flipped classroom approach in a veterinary professional skills course

PubMed Central

Moffett, Jenny; Mill, Aileen C

2014-01-01

Background The flipped classroom is an educational approach that has had much recent coverage in the literature. Relatively few studies, however, use objective assessment of student performance to measure the impact of the flipped classroom on learning. The purpose of this study was to evaluate the use of a flipped classroom approach within a medical education setting to the first two levels of Kirkpatrick and Kirkpatrick’s effectiveness of training framework. Methods This study examined the use of a flipped classroom approach within a professional skills course offered to postgraduate veterinary students. A questionnaire was administered to two cohorts of students: those who had completed a traditional, lecture-based version of the course (Introduction to Veterinary Medicine [IVM]) and those who had completed a flipped classroom version (Veterinary Professional Foundations I [VPF I]). The academic performance of students within both cohorts was assessed using a set of multiple-choice items (n=24) nested within a written examination. Data obtained from the questionnaire were analyzed using Cronbach’s alpha, Kruskal–Wallis tests, and factor analysis. Data obtained from student performance in the written examination were analyzed using the nonparametric Wilcoxon rank sum test. Results A total of 133 IVM students and 64 VPF I students (n=197) agreed to take part in the study. Overall, study participants favored the flipped classroom approach over the traditional classroom approach. With respect to student academic performance, the traditional classroom students outperformed the flipped classroom students on a series of multiple-choice items (IVM mean =21.4±1.48 standard deviation; VPF I mean =20.25±2.20 standard deviation; Wilcoxon test, w=7,578; P<0.001). Conclusion This study demonstrates that learners seem to prefer a flipped classroom approach. The flipped classroom was rated more positively than the traditional classroom on many different characteristics. This preference, however, did not translate into improved student performance, as assessed by a series of multiple-choice items delivered during a written examination. PMID:25419164

Evaluation of the flipped classroom approach in a veterinary professional skills course.

PubMed

Moffett, Jenny; Mill, Aileen C

2014-01-01

The flipped classroom is an educational approach that has had much recent coverage in the literature. Relatively few studies, however, use objective assessment of student performance to measure the impact of the flipped classroom on learning. The purpose of this study was to evaluate the use of a flipped classroom approach within a medical education setting to the first two levels of Kirkpatrick and Kirkpatrick's effectiveness of training framework. This study examined the use of a flipped classroom approach within a professional skills course offered to postgraduate veterinary students. A questionnaire was administered to two cohorts of students: those who had completed a traditional, lecture-based version of the course (Introduction to Veterinary Medicine [IVM]) and those who had completed a flipped classroom version (Veterinary Professional Foundations I [VPF I]). The academic performance of students within both cohorts was assessed using a set of multiple-choice items (n=24) nested within a written examination. Data obtained from the questionnaire were analyzed using Cronbach's alpha, Kruskal-Wallis tests, and factor analysis. Data obtained from student performance in the written examination were analyzed using the nonparametric Wilcoxon rank sum test. A total of 133 IVM students and 64 VPF I students (n=197) agreed to take part in the study. Overall, study participants favored the flipped classroom approach over the traditional classroom approach. With respect to student academic performance, the traditional classroom students outperformed the flipped classroom students on a series of multiple-choice items (IVM mean =21.4±1.48 standard deviation; VPF I mean =20.25±2.20 standard deviation; Wilcoxon test, w=7,578; P<0.001). This study demonstrates that learners seem to prefer a flipped classroom approach. The flipped classroom was rated more positively than the traditional classroom on many different characteristics. This preference, however, did not translate into improved student performance, as assessed by a series of multiple-choice items delivered during a written examination.

Comparison of the didactic lecture with the simulation/model approach for the teaching of a novel perioperative ultrasound curriculum to anesthesiology residents.

PubMed

Ramsingh, Davinder; Alexander, Brenton; Le, Khanhvan; Williams, Wendell; Canales, Cecilia; Cannesson, Maxime

2014-09-01

To expose residents to two methods of education for point-of-care ultrasound, a traditional didactic lecture and a model/simulation-based lecture, which focus on concepts of cardiopulmonary function, volume status, and evaluation of severe thoracic/abdominal injuries; and to assess which method is more effective. Single-center, prospective, blinded trial. University hospital. Anesthesiology residents who were assigned to an educational day during the two-month research study period. Residents were allocated to two groups to receive either a 90-minute, one-on-one didactic lecture or a 90-minute lecture in a simulation center, during which they practiced on a human model and simulation mannequin (normal pathology). Data points included a pre-lecture multiple-choice test, post-lecture multiple-choice test, and post-lecture, human model-based examination. Post-lecture tests were performed within three weeks of the lecture. An experienced sonographer who was blinded to the education modality graded the model-based skill assessment examinations. Participants completed a follow-up survey to assess the perceptions of the quality of their instruction between the two groups. 20 residents completed the study. No differences were noted between the two groups in pre-lecture test scores (P = 0.97), but significantly higher scores for the model/simulation group occurred on both the post-lecture multiple choice (P = 0.038) and post-lecture model (P = 0.041) examinations. Follow-up resident surveys showed significantly higher scores in the model/simulation group regarding overall interest in perioperative ultrasound (P = 0.047) as well understanding of the physiologic concepts (P = 0.021). A model/simulation-based based lecture series may be more effective in teaching the skills needed to perform a point-of-care ultrasound examination to anesthesiology residents. Copyright © 2014 Elsevier Inc. All rights reserved.

Piloting a Geoscience Literacy Exam for Assessing Students' Understanding of Earth, Climate, Atmospheric and Ocean Science Concepts

NASA Astrophysics Data System (ADS)

Steer, D. N.; Iverson, E. A.; Manduca, C. A.

2013-12-01

This research seeks to develop valid and reliable questions that faculty can use to assess geoscience literacy across the curriculum. We are particularly interested on effects of curricula developed to teach Earth, Climate, Atmospheric, and Ocean Science concepts in the context of societal issues across the disciplines. This effort is part of the InTeGrate project designed to create a population of college graduates who are poised to use geoscience knowledge in developing solutions to current and future environmental and resource challenges. Details concerning the project are found at http://serc.carleton.edu/integrate/index.html. The Geoscience Literacy Exam (GLE) under development presently includes 90 questions. Each big idea from each literacy document can be probed using one or more of three independent questions: 1) a single answer, multiple choice question aimed at basic understanding or application of key concepts, 2) a multiple correct answer, multiple choice question targeting the analyzing to analysis levels and 3) a short essay question that tests analysis or evaluation cognitive levels. We anticipate multiple-choice scores and the detail and sophistication of essay responses will increase as students engage with the curriculum. As part of the field testing of InTeGrate curricula, faculty collected student responses from classes that involved over 700 students. These responses included eight pre- and post-test multiple-choice questions that covered various concepts across the four literacies. Discrimination indices calculated from the data suggest that the eight tested questions provide a valid measure of literacy within the scope of the concepts covered. Student normalized gains across an academic term with limited InTeGrate exposure (typically two or fewer weeks of InTeGrate curriculum out of 14 weeks) were found to average 16% gain. A small set of control data (250 students in classes from one institution where no InTeGrate curricula were used) was also collected from a larger bank of test questions. Discrimination indices across the full bank showed variation and additional work is underway to refine and field test in other settings these questions in the absence of InTeGrate curricula. When complete, faculty will be able to assemble sets of questions to track progress toward meeting literacy goals. In addition to covering geoscience content knowledge and understanding, a complementary attitudinal pre/post survey was also developed with the intent to probe InTeGrate students' ability and motivation to use their geoscience expertise to address problems of environmental sustainability. The final instruments will be made available to the geoscience education community as an assessment to be used in conjunction with InTeGrate teaching materials or as a stand-alone tool for departments to measure student learning and attitudinal gains across the major.

Fixed or mixed: a comparison of three, four and mixed-option multiple-choice tests in a Fetal Surveillance Education Program

PubMed Central

2013-01-01

Background Despite the widespread use of multiple-choice assessments in medical education assessment, current practice and published advice concerning the number of response options remains equivocal. This article describes an empirical study contrasting the quality of three 60 item multiple-choice test forms within the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG) Fetal Surveillance Education Program (FSEP). The three forms are described below. Methods The first form featured four response options per item. The second form featured three response options, having removed the least functioning option from each item in the four-option counterpart. The third test form was constructed by retaining the best performing version of each item from the first two test forms. It contained both three and four option items. Results Psychometric and educational factors were taken into account in formulating an approach to test construction for the FSEP. The four-option test performed better than the three-option test overall, but some items were improved by the removal of options. The mixed-option test demonstrated better measurement properties than the fixed-option tests, and has become the preferred test format in the FSEP program. The criteria used were reliability, errors of measurement and fit to the item response model. Conclusions The position taken is that decisions about the number of response options be made at the item level, with plausible options being added to complete each item on both psychometric and educational grounds rather than complying with a uniform policy. The point is to construct the better performing item in providing the best psychometric and educational information. PMID:23453056

Fixed or mixed: a comparison of three, four and mixed-option multiple-choice tests in a Fetal Surveillance Education Program.

PubMed

Zoanetti, Nathan; Beaves, Mark; Griffin, Patrick; Wallace, Euan M

2013-03-04

Despite the widespread use of multiple-choice assessments in medical education assessment, current practice and published advice concerning the number of response options remains equivocal. This article describes an empirical study contrasting the quality of three 60 item multiple-choice test forms within the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG) Fetal Surveillance Education Program (FSEP). The three forms are described below. The first form featured four response options per item. The second form featured three response options, having removed the least functioning option from each item in the four-option counterpart. The third test form was constructed by retaining the best performing version of each item from the first two test forms. It contained both three and four option items. Psychometric and educational factors were taken into account in formulating an approach to test construction for the FSEP. The four-option test performed better than the three-option test overall, but some items were improved by the removal of options. The mixed-option test demonstrated better measurement properties than the fixed-option tests, and has become the preferred test format in the FSEP program. The criteria used were reliability, errors of measurement and fit to the item response model. The position taken is that decisions about the number of response options be made at the item level, with plausible options being added to complete each item on both psychometric and educational grounds rather than complying with a uniform policy. The point is to construct the better performing item in providing the best psychometric and educational information.

Comparison between three option, four option and five option multiple choice question tests for quality parameters: A randomized study.

PubMed

Vegada, Bhavisha; Shukla, Apexa; Khilnani, Ajeetkumar; Charan, Jaykaran; Desai, Chetna

2016-01-01

Most of the academic teachers use four or five options per item of multiple choice question (MCQ) test as formative and summative assessment. Optimal number of options in MCQ item is a matter of considerable debate among academic teachers of various educational fields. There is a scarcity of the published literature regarding the optimum number of option in each item of MCQ in the field of medical education. To compare three options, four options, and five options MCQs test for the quality parameters - reliability, validity, item analysis, distracter analysis, and time analysis. Participants were 3 rd semester M.B.B.S. students. Students were divided randomly into three groups. Each group was given one set of MCQ test out of three options, four options, and five option randomly. Following the marking of the multiple choice tests, the participants' option selections were analyzed and comparisons were conducted of the mean marks, mean time, validity, reliability and facility value, discrimination index, point biserial value, distracter analysis of three different option formats. Students score more ( P = 0.000) and took less time ( P = 0.009) for the completion of three options as compared to four options and five options groups. Facility value was more ( P = 0.004) in three options group as compared to four and five options groups. There was no significant difference between three groups for the validity, reliability, and item discrimination. Nonfunctioning distracters were more in the four and five options group as compared to three option group. Assessment based on three option MCQs is can be preferred over four option and five option MCQs.

All of the above: When multiple correct response options enhance the testing effect.

PubMed

Bishara, Anthony J; Lanzo, Lauren A

2015-01-01

Previous research has shown that multiple choice tests often improve memory retention. However, the presence of incorrect lures often attenuates this memory benefit. The current research examined the effects of "all of the above" (AOTA) options. When such options are correct, no incorrect lures are present. In the first three experiments, a correct AOTA option on an initial test led to a larger memory benefit than no test and standard multiple choice test conditions. The benefits of a correct AOTA option occurred even without feedback on the initial test; for both 5-minute and 48-hour retention delays; and for both cued recall and multiple choice final test formats. In the final experiment, an AOTA question led to better memory retention than did a control condition that had identical timing and exposure to response options. However, the benefits relative to this control condition were similar regardless of the type of multiple choice test (AOTA or not). Results suggest that retrieval contributes to multiple choice testing effects. However, the extra testing effect from a correct AOTA option, rather than being due to more retrieval, might be due simply to more exposure to correct information.

A Two-Tier Multiple Choice Questions to Diagnose Thermodynamic Misconception of Thai and Laos Students

NASA Astrophysics Data System (ADS)

Kamcharean, Chanwit; Wattanakasiwich, Pornrat

The objective of this study was to diagnose misconceptions of Thai and Lao students in thermodynamics by using a two-tier multiple-choice test. Two-tier multiple choice questions consist of the first tier, a content-based question and the second tier, a reasoning-based question. Data of student understanding was collected by using 10 two-tier multiple-choice questions. Thai participants were the first-year students (N = 57) taking a fundamental physics course at Chiang Mai University in 2012. Lao participants were high school students in Grade 11 (N = 57) and Grade 12 (N = 83) at Muengnern high school in Xayaboury province, Lao PDR. As results, most students answered content-tier questions correctly but chose incorrect answers for reason-tier questions. When further investigating their incorrect reasons, we found similar misconceptions as reported in previous studies such as incorrectly relating pressure with temperature when presenting with multiple variables.

Analysis of strength-of-preference measures in dichotomous choice models

Treesearch

Donald F. Dennis; Peter Newman; Robert Manning

2008-01-01

Choice models are becoming increasingly useful for soliciting and analyzing multiple objective decisions faced by recreation managers and others interested in decisions involving natural resources. Choice models are used to estimate relative values for multiple aspects of natural resource management, not individually but within the context of other relevant decision...

The Role of Linguistic Modification in Nursing Education.

PubMed

Moore, Brenda S; Clark, Michele C

2016-06-01

English-as-a-second-language (ESL) nursing students fail to graduate from programs at alarming rates. For many of these students, academic failure results from poor performance on multiple choice examinations, which frequently contain linguistic errors. A remedy for these errors is to linguistically modify examination questions. This study assessed the effects of linguistic modification on examination scores. Scores of ESL and non-ESL nursing students were compared on an experimental multiple choice examination and a control examination. After exclusion, 67 ESL and 252 non-ESL students completed the experimental examination; 68 ESL and 257 non-ESL students completed the control examination. Both ESL and non-ESL students scored higher on the experimental examination than on the control examination. For ESL students, the increase in observed means between the experimental and control examination was 0.6%; for non-ESL students, the increase was 0.48%. [J Nurs Educ. 2016;55(6):309-315.]. Copyright 2016, SLACK Incorporated.

The effect of podcast lectures on nursing students' knowledge retention and application.

PubMed

Abate, Karen S

2013-01-01

This pilot study sought to evaluate the effectiveness of academic podcasts in promoting knowledge retention and application in nursing students. Nursing education no longer simply occurs in a fixed location or time. Computer-enhanced mobile learning technologies, such as academic podcasts, must be grounded in pedagogically sound characteristics to ensure effective implementation and learning in nursing education. A convenience sample of 35 female undergraduate nursing students was randomized into three groups: a traditional face-to-face lecture group, an unsegmented (non-stop) podcast lecture group, and a segmented podcast lecture group. Retention and application of information were measured through a multiple-choice quiz and a case study based on lecture content. Students in the segmented podcast lecture group demonstrated higher scores on multiple-choice and case-study assessments than those in the other two groups. Nurse educators should be aware of this finding when seeking to employ podcast lectures in nursing education.

Comedy workshop: an enjoyable way to develop multiple-choice questions.

PubMed

Droegemueller, William; Gant, Norman; Brekken, Alvin; Webb, Lynn

2005-01-01

To describe an innovative method of developing multiple-choice items for a board certification examination. The development of appropriate multiple-choice items is definitely more of an art, rather than a science. The comedy workshop format for developing questions for a certification examination is similar to the process used by comedy writers composing scripts for television shows. This group format dramatically diminishes the frustrations faced by an individual question writer attempting to create items. The vast majority of our comedy workshop participants enjoy and prefer the comedy workshop format. It provides an ideal environment in which to teach and blend the talents of inexperienced and experienced question writers. This is a descriptive article, in which we suggest an innovative process in the art of creating multiple-choice items for a high-stakes examination.

Peerwise Provides Significant Academic Benefits to Biological Science Students across Diverse Learning Tasks, but with Minimal Instructor Intervention

ERIC Educational Resources Information Center

McQueen, H. A.; Shields, C.; Finnegan, D. J.; Higham, J.; Simmen, M. W.

2014-01-01

We demonstrate that student engagement with PeerWise, an online tool that allows students to author and answer multiple-choice questions (MCQs), is associated with enhanced academic performance across diverse assessment types on a second year Genetics course. Benefits were consistent over three course deliveries, with differential benefits…

Knowledge about Aging and Alzheimer Disease: A Comparison of Professional Caregivers and Noncaregivers

ERIC Educational Resources Information Center

Rust, Tiana B.; See, Sheree Kwong

2007-01-01

This study assessed professional caregivers of persons with Alzheimer disease (AD) and non-caregivers' knowledge about aging and AD. Participants completed modified versions of the Alzheimer Disease Knowledge Test and the multiple-choice version of the Facts on Aging Quiz #1. Overall, knowledge levels about AD and aging were low. Caregivers were…

Construction of Valid and Reliable Test for Assessment of Students

ERIC Educational Resources Information Center

Osadebe, P. U.

2015-01-01

The study was carried out to construct a valid and reliable test in Economics for secondary school students. Two research questions were drawn to guide the establishment of validity and reliability for the Economics Achievement Test (EAT). It is a multiple choice objective test of five options with 100 items. A sample of 1000 students was randomly…

Construction of Economics Achievement Test for Assessment of Students

ERIC Educational Resources Information Center

Osadebe, P. U.

2014-01-01

The study was carried out to construct a valid and reliable test in Economics for secondary school students. Two research questions were drawn to guide the establishment of validity and reliability for the Economics Achievement Test (EAT). It is a multiple choice objective test of five options with 100 items. A sample of 1000 students was randomly…

Validation of a Standardized Multiple-Choice Multicultural Competence Test: Implications for Training, Assessment, and Practice

ERIC Educational Resources Information Center

Gillem, Angela R.; Bartoli, Eleonora; Bertsch, Kristin N.; McCarthy, Maureen A.; Constant, Kerra; Marrero-Meisky, Sheila; Robbins, Steven J.; Bellamy, Scarlett

2016-01-01

The Multicultural Counseling and Psychotherapy Test (MCPT), a measure of multicultural counseling competence (MCC), was validated in 2 phases. In Phase 1, the authors administered 451 test items derived from multicultural guidelines in counseling and psychology to 32 multicultural experts and 30 nonexperts. In Phase 2, the authors administered the…

Correction for Guessing in the Framework of the 3PL Item Response Theory

ERIC Educational Resources Information Center

Chiu, Ting-Wei

2010-01-01

Guessing behavior is an important topic with regard to assessing proficiency on multiple choice tests, particularly for examinees at lower levels of proficiency due to greater the potential for systematic error or bias which that inflates observed test scores. Methods that incorporate a correction for guessing on high-stakes tests generally rely…

Development and Analysis of an Instrument to Assess Student Understanding of Foundational Concepts before Biochemistry Coursework

ERIC Educational Resources Information Center

Villafane, Sachel M.; Bailey, Cheryl P.; Loertscher, Jennifer; Minderhout, Vicky; Lewis, Jennifer E.

2011-01-01

Biochemistry is a challenging subject because student learning depends on the application of previously learned concepts from general chemistry and biology to new, biological contexts. This article describes the development of a multiple-choice instrument intended to measure five concepts from general chemistry and three from biology that are…

Longitudinal Assessment of Progress in Reasoning Capacity and Relation with Self-Estimation of Knowledge Base

ERIC Educational Resources Information Center

Collard, Anne; Mélot, France; Bourguignon, Jean-Pierre

2015-01-01

The aim of the study was to investigate progress in reasoning capacity and knowledge base appraisal in a longitudinal analysis of data from summative evaluation throughout a medical problem-based learning curriculum. The scores in multidisciplinary discussion of a clinical case and multiple choice questionnaires (MCQs) were studied longitudinally…

Student Assessment System. Domain Referenced Tests. Transportation/Automotive Mechanics. Volume II: Theory. Georgia Vocational Education Program Articulation.

ERIC Educational Resources Information Center

Watkins, James F., Comp.

These written domain referenced tests (DRTs) for the area of transportation/automotive mechanics test cognitive abilities or knowledge of theory. Introductory materials describe domain referenced testing and test development. Each multiple choice test includes a domain statement, describing the behavior and content of the domain, and a test item…

Knowledge Retention among Graduates of Basic Electricity and Electronics Schools.

ERIC Educational Resources Information Center

Hall, Eugene R.; And Others

The extent of knowledge decay during the interval between graduation from a basic electricity and electronics (BE/E) school and entry into a construction electrician (CE) "A" school was assessed. A sample consisting of 307 BE/E graduates was retested using a multiple choice test identical to the final examination taken at BE/E school.…

An Examination of the Effects of Flow on Learning in a Graduate-Level Introductory Operations Management Course

ERIC Educational Resources Information Center

Klein, Barbara D.; Rossin, Don; Guo, Yi Maggie; Ro, Young K.

2010-01-01

The authors investigated the effects of flow on learning outcomes in a graduate-level operations management course. Flow was assessed through an overall flow score, four dimensions of flow, and three characteristics of flow activities. Learning outcomes were measured objectively through multiple-choice quiz scores and subjectively using measures…

Peer Review Improves the Quality of MCQ Examinations

ERIC Educational Resources Information Center

Malau-Aduli, Bunmi S.; Zimitat, Craig

2012-01-01

The aim of this study was to assess the effect of the introduction of peer review processes on the quality of multiple-choice examinations in the first three years of an Australian medical course. The impact of the peer review process and overall quality assurance (QA) processes were evaluated by comparing the examination data generated in earlier…

The Positive and Negative Effects of Science Concept Tests on Student Conceptual Understanding

ERIC Educational Resources Information Center

Chang, Chun-Yen; Yeh, Ting-Kuang; Barufaldi, James P.

2010-01-01

This study explored the phenomenon of testing effect during science concept assessments, including the mechanism behind it and its impact upon a learner's conceptual understanding. The participants consisted of 208 high school students, in either the 11th or 12th grade. Three types of tests (traditional multiple-choice test, correct concept test,…

A Critical Analysis of the Body of Work Method for Setting Cut-Scores

ERIC Educational Resources Information Center

Radwan, Nizam; Rogers, W. Todd

2006-01-01

The recent increase in the use of constructed-response items in educational assessment and the dissatisfaction with the nature of the decision that the judges must make using traditional standard-setting methods created a need to develop new and effective standard-setting procedures for tests that include both multiple-choice and…

What Are They Thinking? Automated Analysis of Student Writing about Acid-Base Chemistry in Introductory Biology

ERIC Educational Resources Information Center

Haudek, Kevin C.; Prevost, Luanna B.; Moscarella, Rosa A.; Merrill, John; Urban-Lurain, Mark

2012-01-01

Students' writing can provide better insight into their thinking than can multiple-choice questions. However, resource constraints often prevent faculty from using writing assessments in large undergraduate science courses. We investigated the use of computer software to analyze student writing and to uncover student ideas about chemistry in an…

On the Optimality of Answer-Copying Indices: Theory and Practice

ERIC Educational Resources Information Center

Romero, Mauricio; Riascos, Álvaro; Jara, Diego

2015-01-01

Multiple-choice exams are frequently used as an efficient and objective method to assess learning, but they are more vulnerable to answer copying than tests based on open questions. Several statistical tests (known as indices in the literature) have been proposed to detect cheating; however, to the best of our knowledge, they all lack mathematical…

Examining Gender DIF on a Multiple-Choice Test of Mathematics: A Confirmatory Approach.

ERIC Educational Resources Information Center

Ryan, Katherine E.; Fan, Meichu

1996-01-01

Results for 3,244 female and 3,033 male junior high school students from the Second International Mathematics Study show that applied items in algebra, geometry, and computation were easier for males but arithmetic items were differentially easier for females. Implications of these findings for assessment and instruction are discussed. (SLD)

Uncovering Students' Incorrect Ideas about Foundational Concepts for Biochemistry

ERIC Educational Resources Information Center

Villafane, Sachel M.; Loertscher, Jennifer; Minderhout, Vicky; Lewis, Jennifer E.

2011-01-01

This paper presents preliminary data on how an assessment instrument with a unique structure can be used to identify common incorrect ideas from prior coursework at the beginning of a biochemistry course, and to determine whether these ideas have changed by the end of the course. The twenty-one multiple-choice items address seven different…

Using Word Clouds for Fast, Formative Assessment of Students' Short Written Responses

ERIC Educational Resources Information Center

Brooks, Bill J.; Gilbuena, Debra M.; Krause, Stephen J.; Koretsky, Milo D.

2014-01-01

Active learning in class helps students develop deeper understanding of chemical engineering principles. While the use of multiple-choice ConcepTests is clearly effective, we advocate for including student writing in learning activities as well. In this article, we demonstrate that word clouds can provide a quick analytical technique to assess…

Effects of Partner's Ability on the Achievement and Conceptual Organization of High-Achieving Fifth-Grade Students.

ERIC Educational Resources Information Center

Carter, Glenda; Jones, M. Gail; Rua, Melissa

2003-01-01

Investigates high-achieving fifth-grade students' achievement gains and conceptual reorganization on convection. Features an instructional sequence of three dyadic inquiry investigations related to convection currents as well as pre- and post-assessment consisting of a multiple-choice test, a card sorting task, construction of a concept map, and…

A Dialogue about MCQs, Reliability, and Item Response Modelling

ERIC Educational Resources Information Center

Wright, Daniel B.; Skagerberg, Elin M.

2006-01-01

Multiple choice questions (MCQs) are becoming more common in UK psychology departments and the need to assess their reliability is apparent. Having examined the reliability of MCQs in our department we faced many questions from colleagues about why we were examining reliability, what it was that we were doing, and what should be reported when…

Differential Competencies Contributing to Children's Comprehension of Narrative and Expository Texts

ERIC Educational Resources Information Center

Best, Rachel M.; Floyd, Randy G.; Mcnamara, Danielle S.

2008-01-01

This study examined the influences of reading decoding skills and world knowledge on third graders' comprehension of narrative and expository texts. Children read a narrative text and an expository text. Comprehension of each text was assessed with a free recall prompt, three cued recall prompts, and 12 multiple-choice questions. Tests from the…

Leveraging Innovation in Science Education: Using Writing and Assessment to Decode the Class Size Conundrum

ERIC Educational Resources Information Center

Camfield, Eileen Kogl; McFall, Eileen Eckert; Land, Kirkwood M.

2016-01-01

Introductory biology courses are supposed to serve as gateways for many majors, but too often they serve instead as gatekeepers. Reliance on lectures, large classes, and multiple-choice tests results in high drop and failure rates. Critiques of undergraduate science education are clear about the problems with conventional introductory science…

The Impact of Kentucky's Educational Reform Act on Writing throughout the Commonwealth.

ERIC Educational Resources Information Center

Harnack, Andrew; And Others

1994-01-01

The central role of writing in Kentucky's Education Reform Act is most evident in Kentucky's new assessment system, which employs writing on all levels. Even tests that have recently included multiple-choice items may be replaced by response items that require students to apply knowledge, concepts, and skills in a writing format. Writing itself is…

Using Computer-Based Technology to Improve Feedback to Staff and Students on MCQ Assessments

ERIC Educational Resources Information Center

Malau-Aduli, Bunmi S.; Assenheimer, Dwight; Choi-Lundberg, Derek; Zimitat, Craig

2014-01-01

The massification of higher education (HE) has led to an unprecedented increase in the number of students in the classrooms, resulting in increased workload for teaching staff, sometimes leading to a great reliance on Multiple Choice Questions (MCQs) examinations with limited feedback provided to students. The central role of feedback in student…

The Disaggregation of Value-Added Test Scores to Assess Learning Outcomes in Economics Courses

ERIC Educational Resources Information Center

Walstad, William B.; Wagner, Jamie

2016-01-01

This study disaggregates posttest, pretest, and value-added or difference scores in economics into four types of economic learning: positive, retained, negative, and zero. The types are derived from patterns of student responses to individual items on a multiple-choice test. The micro and macro data from the "Test of Understanding in College…

Fifth graders' science inquiry abilities: A comparative study of students in hands-on and textbook curricula

NASA Astrophysics Data System (ADS)

Pine, Jerome; Aschbacher, Pamela; Roth, Ellen; Jones, Melanie; McPhee, Cameron; Martin, Catherine; Phelps, Scott; Kyle, Tara; Foley, Brian

2006-05-01

A large number of American elementary school students are now studying science using the hands-on inquiry curricula developed in the 1990s: Insights; Full Option Science System (FOSS); and Science and Technology for Children (STC). A goal of these programs, echoed in the National Science Education Standards, is that children should gain abilities to do scientific inquiry and understanding about scientific inquiry. We have studied the degree to which students can do inquiries by using four hands-on performance assessments, which required one or three class periods. To be fair, the assessments avoided content that is studied in depth in the hands-on programs. For a sample of about 1000 fifth grade students, we compared the performance of students in hands-on curricula with an equal number of students with textbook curricula. The students were from 41 classrooms in nine school districts. The results show little or no curricular effect. There was a strong dependence on students' cognitive ability, as measured with a standard multiple-choice instrument. There was no significant difference between boys and girls. Also, there was no difference on a multiple-choice test, which used items released from the Trends in International Mathematics and Science Study (TIMSS). It is not completely clear whether the lack of difference on the performance assessments was a consequence of the assessments, the curricula, and/or the teaching.

Advances in the management of multiple sclerosis symptoms: pathophysiology and assessment of spasticity in multiple sclerosis.

PubMed

Tintoré, Mar

2015-01-01

Spasticity is a prevalent and troublesome symptom for people with multiple sclerosis (MS). Common instruments to measure MS spasticity include the clinician-rated (modified) Ashworth scale and the patient-rated 0-10 spasticity Numerical Rating Scale (NRS). Current opinion is that measurement of MS spasticity should incorporate the patient's perspective. Other instruments to assess spasticity-associated symptoms such as the Penn spasms frequency scale, sleep quality NRS and pain NRS can assist in tracking MS spasticity evolution and inform management choices. Worsening spasticity reduces patient autonomy, impacts negatively on quality of life and increases health resource utilization and costs. Despite the wide range of issues associated with MS spasticity, undertreatment is common and standard treatment options (physiotherapy and classical oral therapies) often fail to provide adequate symptomatic control.

Decision making and preferences for acoustic signals in choice situations by female crickets.

PubMed

Gabel, Eileen; Kuntze, Janine; Hennig, R Matthias

2015-08-01

Multiple attributes usually have to be assessed when choosing a mate. Efficient choice of the best mate is complicated if the available cues are not positively correlated, as is often the case during acoustic communication. Because of varying distances of signalers, a female may be confronted with signals of diverse quality at different intensities. Here, we examined how available cues are weighted for a decision by female crickets. Two songs with different temporal patterns and/or sound intensities were presented in a choice paradigm and compared with female responses from a no-choice test. When both patterns were presented at equal intensity, preference functions became wider in choice situations compared with a no-choice paradigm. When the stimuli in two-choice tests were presented at different intensities, this effect was counteracted as preference functions became narrower compared with choice tests using stimuli of equal intensity. The weighting of intensity differences depended on pattern quality and was therefore non-linear. A simple computational model based on pattern and intensity cues reliably predicted female decisions. A comparison of processing schemes suggested that the computations for pattern recognition and directionality are performed in a network with parallel topology. However, the computational flow of information corresponded to serial processing. © 2015. Published by The Company of Biologists Ltd.

Variation in Estimated Ozone-Related Health Impacts of Climate Change due to Modeling Choices and Assumptions

PubMed Central

Post, Ellen S.; Grambsch, Anne; Weaver, Chris; Morefield, Philip; Leung, Lai-Yung; Nolte, Christopher G.; Adams, Peter; Liang, Xin-Zhong; Zhu, Jin-Hong; Mahoney, Hardee

2012-01-01

Background: Future climate change may cause air quality degradation via climate-induced changes in meteorology, atmospheric chemistry, and emissions into the air. Few studies have explicitly modeled the potential relationships between climate change, air quality, and human health, and fewer still have investigated the sensitivity of estimates to the underlying modeling choices. Objectives: Our goal was to assess the sensitivity of estimated ozone-related human health impacts of climate change to key modeling choices. Methods: Our analysis included seven modeling systems in which a climate change model is linked to an air quality model, five population projections, and multiple concentration–response functions. Using the U.S. Environmental Protection Agency’s (EPA’s) Environmental Benefits Mapping and Analysis Program (BenMAP), we estimated future ozone (O3)-related health effects in the United States attributable to simulated climate change between the years 2000 and approximately 2050, given each combination of modeling choices. Health effects and concentration–response functions were chosen to match those used in the U.S. EPA’s 2008 Regulatory Impact Analysis of the National Ambient Air Quality Standards for O3. Results: Different combinations of methodological choices produced a range of estimates of national O3-related mortality from roughly 600 deaths avoided as a result of climate change to 2,500 deaths attributable to climate change (although the large majority produced increases in mortality). The choice of the climate change and the air quality model reflected the greatest source of uncertainty, with the other modeling choices having lesser but still substantial effects. Conclusions: Our results highlight the need to use an ensemble approach, instead of relying on any one set of modeling choices, to assess the potential risks associated with O3-related human health effects resulting from climate change. PMID:22796531

Academic performance of day scholars versus boarders in pharmacology examinations of a medical school in Pakistan.

PubMed

Faisal, Rizwan; Shinwari, Laiyla; Izzat, Saadia

2016-09-01

To compare the academic performance of day scholar and boarder students in Pharmacology examinations. This comparative study was conducted at Rehman Medical College, Peshawar, Pakistan, from June to September, 2015. It comprised third-year medical students of the sessions 2013-14 and 2014-15.The record of the results of examinations, which had already been conducted, were assessed. All the exams had two components, i.e. multiple-choice questions and short-essay questions. Students were categorised into 4 groups according to their academic performance: those who got <50% marks (Group 1); 51-69% marks (Group 2); 70-80% marks (Group 3); and >80% marks (Group 4). SPSS 20 was used for data analysis. Of the 200 students, 159(79.5%) were day scholars and 41(20.5%) were boarders. In multiple-choice questions, 29(70.7%) boarder students were in Group 2, while none of them was in Group 4. In short-essay questions, 11(26.8%) of them were in Group 1 and 17(41.5%) in Group 2. Results of day scholars' multiple-choice questions exams showed 93(58.5%) were in Group 2 and 2(1.3%) in Group 4. In short-essay questions, 63(39.6%) were in Group 2 (p>o.o5 each). No significant difference was found between the academic performance of boarders and day scholars.

Does Correct Answer Distribution Influence Student Choices When Writing Multiple Choice Examinations?

ERIC Educational Resources Information Center

Carnegie, Jacqueline A.

2017-01-01

Summative evaluation for large classes of first- and second-year undergraduate courses often involves the use of multiple choice question (MCQ) exams in order to provide timely feedback. Several versions of those exams are often prepared via computer-based question scrambling in an effort to deter cheating. An important parameter to consider when…

The role of action representations in thematic object relations

PubMed Central

Tsagkaridis, Konstantinos; Watson, Christine E.; Jax, Steven A.; Buxbaum, Laurel J.

2014-01-01

A number of studies have explored the role of associative/event-based (thematic) and categorical (taxonomic) relations in the organization of object representations. Recent evidence suggests that thematic information may be particularly important in determining relationships between manipulable artifacts. However, although sensorimotor information is on many accounts an important component of manipulable artifact representations, little is known about the role that action may play during the processing of semantic relationships (particularly thematic relationships) between multiple objects. In this study, we assessed healthy and left hemisphere stroke participants to explore three questions relevant to object relationship processing. First, we assessed whether participants tended to favor thematic relations including action (Th+A, e.g., wine bottle—corkscrew), thematic relationships without action (Th-A, e.g., wine bottle—cheese), or taxonomic relationships (Tax, e.g., wine bottle—water bottle) when choosing between them in an association judgment task with manipulable artifacts. Second, we assessed whether the underlying constructs of event relatedness, action relatedness, and categorical relatedness determined the choices that participants made. Third, we assessed the hypothesis that degraded action knowledge and/or damage to temporo-parietal cortex, a region of the brain associated with the representation of action knowledge, would reduce the influence of action on the choice task. Experiment 1 showed that explicit ratings of event, action, and categorical relatedness were differentially predictive of healthy participants' choices, with action relatedness determining choices between Th+A and Th-A associations above and beyond event and categorical ratings. Experiment 2 focused more specifically on these Th+A vs. Th-A choices and demonstrated that participants with left temporo-parietal lesions, a brain region known to be involved in sensorimotor processing, were less likely than controls and tended to be less likely than patients with lesions sparing that region to use action relatedness in determining their choices. These data indicate that action knowledge plays a critical role in processing of thematic relations for manipulable artifacts. PMID:24672461

The role of action representations in thematic object relations.

PubMed

Tsagkaridis, Konstantinos; Watson, Christine E; Jax, Steven A; Buxbaum, Laurel J

2014-01-01

A number of studies have explored the role of associative/event-based (thematic) and categorical (taxonomic) relations in the organization of object representations. Recent evidence suggests that thematic information may be particularly important in determining relationships between manipulable artifacts. However, although sensorimotor information is on many accounts an important component of manipulable artifact representations, little is known about the role that action may play during the processing of semantic relationships (particularly thematic relationships) between multiple objects. In this study, we assessed healthy and left hemisphere stroke participants to explore three questions relevant to object relationship processing. First, we assessed whether participants tended to favor thematic relations including action (Th+A, e.g., wine bottle-corkscrew), thematic relationships without action (Th-A, e.g., wine bottle-cheese), or taxonomic relationships (Tax, e.g., wine bottle-water bottle) when choosing between them in an association judgment task with manipulable artifacts. Second, we assessed whether the underlying constructs of event relatedness, action relatedness, and categorical relatedness determined the choices that participants made. Third, we assessed the hypothesis that degraded action knowledge and/or damage to temporo-parietal cortex, a region of the brain associated with the representation of action knowledge, would reduce the influence of action on the choice task. Experiment 1 showed that explicit ratings of event, action, and categorical relatedness were differentially predictive of healthy participants' choices, with action relatedness determining choices between Th+A and Th-A associations above and beyond event and categorical ratings. Experiment 2 focused more specifically on these Th+A vs. Th-A choices and demonstrated that participants with left temporo-parietal lesions, a brain region known to be involved in sensorimotor processing, were less likely than controls and tended to be less likely than patients with lesions sparing that region to use action relatedness in determining their choices. These data indicate that action knowledge plays a critical role in processing of thematic relations for manipulable artifacts.

A Multiple Choice Version of the Sentence Completion Method

ERIC Educational Resources Information Center

Shouval, Ron; And Others

1975-01-01

It was concluded that a multiple choice form corresponding to a sentence completion measure, test clearly defined personality areas (such as autonomy) could be a reasonable alternative for many purposes. (Author/DEP)

Which form of assessment provides the best information about student performance in chemistry examinations?

NASA Astrophysics Data System (ADS)

Hudson, Ross D.; Treagust, David F.

2013-04-01

Background . This study developed from observations of apparent achievement differences between male and female chemistry performances in a state university entrance examination. Male students performed more strongly than female students, especially in higher scores. Apart from the gender of the students, two other important factors that might influence student performance were format of questions (short-answer or multiple-choice) and type of questions (recall or application). Purpose The research question addressed in this study was: Is there a relationship between performance in state university entrance examinations in chemistry and school chemistry examinations and student gender, format of questions - multiple-choice or short-answer, and conceptual level - recall or application? Sample The two sources of data were: (1) secondary analyses of five consecutive years' data published by the examining authority of chemistry examinations, and (2) tests conducted with 192 students which provided information about all aspects of the three variables (question format, question type and gender) under consideration. Design and methods Both sources of data were analysed using ANOVA to compare means for the variables under consideration and the statistical significance of any differences. The data from the tests were also analysed using Rasch analysis to determine differences in gender performance. Results When overall mean data are considered, both male and female students performed better on multiple-choice questions and recall questions than on short-answer questions and application questions, respectively. When overall mean data are considered, male students outperformed female students in both the university entrance and school tests, particularly in the higher scores. When data were analysed with Rasch, there was no statistically significant difference in performance between males and females of equal ability. Conclusions Both male and female students generally perform better on multiple-choice questions than they do on short-answer questions. However, when the questions are matched in terms of difficulty (using Rasch analysis), the differences in performance between multiple-choice and short-answer are quite small. Rasch analysis showed that there was little difference in performance between males and females of equal ability. This study shows that a simple face-value score analysis of relative student performance - in this case, in chemistry - can be deceptive unless the actual abilities of the students concerned, as measured by a tool such as Rasch, are taken into consideration before reaching any conclusion.

The Latent Structure of Impulsivity: Impulsive Choice, Impulsive Action, and Impulsive Personality Traits

PubMed Central

MacKillop, James; Weafer, Jessica; Gray, Joshua; Oshri, Assaf; Palmer, Abraham; de Wit, Harriet

2016-01-01

Rationale Impulsivity has been strongly linked to addictive behaviors, but can be operationalized in a number of ways that vary considerably in overlap, suggesting multidimensionality. Objective This study tested the hypothesis that the latent structure among multiple measures of impulsivity would reflect three broad categories: impulsive choice, reflecting discounting of delayed rewards; impulsive action, reflecting ability to inhibit a prepotent motor response; and impulsive personality traits, reflecting self-reported attributions of self-regulatory capacity. Methods The study used a cross-sectional confirmatory factor analysis of multiple impulsivity assessments. Participants were 1252 young adults (62% female) with low levels of addictive behavior who were assessed in individual laboratory rooms at the University of Chicago and the University of Georgia. The battery comprised a delay discounting task, Monetary Choice Questionnaire, Conners Continuous Performance Test, Go/NoGo Task, Stop Signal Task, Barratt Impulsivity Scale, and the UPPS-P Impulsive Behavior Scale. Results The hypothesized three-factor model provided the best fit to the data, although Sensation Seeking was excluded from the final model. The three latent factors were largely unrelated to each other and were variably associated with substance use. Conclusions These findings support the hypothesis that diverse measures of impulsivity can broadly be organized into three categories that are largely distinct from one another. These findings warrant investigation among individuals with clinical levels of addictive behavior and may be applied to understanding the underlying biological mechanisms of these categories. PMID:27449350

Olive cultivar and maturation process on the oviposition preference of Bactrocera oleae (Rossi) (Diptera: Tephritidae).

PubMed

Malheiro, R; Casal, S; Pinheiro, L; Baptista, P; Pereira, J A

2018-02-21

The olive fly, Bactrocera oleae (Rossi) (Diptera: Tephritidae), is a key-pest in the main olives producing areas worldwide, and displays distinct preference to different olive cultivars. The present work intended to study oviposition preference towards three Portuguese cultivars (Cobrançosa, Madural, and Verdeal Transmontana) at different maturation indexes. Multiple oviposition bioassays (multiple-choice and no-choice) were conducted to assess cultivar preference. No-choice bioassays were conducted to assess the influence of different maturation indexes (MI 2; MI 3, and MI 4) in single cultivars. The longevity of olive fly adults according to the cultivar in which its larvae developed was also evaluated through survival assays. Cultivar and maturation are crucial aspects in olive fly preference. Field and laboratory assays revealed a preference towards cv. Verdeal Transmontana olives and a lower susceptibility to cv. Cobrançosa olives. A higher preference was observed for olives at MI 2 and MI 3. The slower maturation process in cv. Verdeal Transmontana (still green while the other cultivars are reddish or at black stage) seems to have an attractive effect on olive fly females, thus increasing its infestation levels. Olive fly adults from both sexes live longer if emerged from pupae developed from cv. Verdeal Transmontana fruits and live less if emerged from cv. Cobrançosa. Therefore, olive cultivar and maturation process are crucial aspects in olive fly preference, also influencing the longevity of adults.

Wrong Answers on Multiple-Choice Achievement Tests: Blind Guesses or Systematic Choices?.

ERIC Educational Resources Information Center

Powell, J. C.

A multi-faceted model for the selection of answers for multiple-choice tests was developed from the findings of a series of exploratory studies. This model implies that answer selection should be curvilinear. A series of models were tested for fit using the chi square procedure. Data were collected from 359 elementary school students ages 9-12.…

Analyzing Student Confidence in Classroom Voting with Multiple Choice Questions

ERIC Educational Resources Information Center

Stewart, Ann; Storm, Christopher; VonEpps, Lahna

2013-01-01

The purpose of this paper is to present results of a recent study in which students voted on multiple choice questions in mathematics courses of varying levels. Students used clickers to select the best answer among the choices given; in addition, they were also asked whether they were confident in their answer. In this paper we analyze data…

Guide to the assessment of physical activity: Clinical and research applications: a scientific statement from the American Heart Association.

PubMed

Strath, Scott J; Kaminsky, Leonard A; Ainsworth, Barbara E; Ekelund, Ulf; Freedson, Patty S; Gary, Rebecca A; Richardson, Caroline R; Smith, Derek T; Swartz, Ann M

2013-11-12

The deleterious health consequences of physical inactivity are vast, and they are of paramount clinical and research importance. Risk identification, benchmarks, efficacy, and evaluation of physical activity behavior change initiatives for clinicians and researchers all require a clear understanding of how to assess physical activity. In the present report, we have provided a clear rationale for the importance of assessing physical activity levels, and we have documented key concepts in understanding the different dimensions, domains, and terminology associated with physical activity measurement. The assessment methods presented allow for a greater understanding of the vast number of options available to clinicians and researchers when trying to assess physical activity levels in their patients or participants. The primary outcome desired is the main determining factor in the choice of physical activity assessment method. In combination with issues of feasibility/practicality, the availability of resources, and administration considerations, the desired outcome guides the choice of an appropriate assessment tool. The decision matrix, along with the accompanying tables, provides a mechanism for this selection that takes all of these factors into account. Clearly, the assessment method adopted and implemented will vary depending on circumstances, because there is no single best instrument appropriate for every situation. In summary, physical activity assessment should be considered a vital health measure that is tracked regularly over time. All other major modifiable cardiovascular risk factors (diabetes mellitus, hypertension, hypercholesterolemia, obesity, and smoking) are assessed routinely. Physical activity status should also be assessed regularly. Multiple physical activity assessment methods provide reasonably accurate outcome measures, with choices dependent on setting-specific resources and constraints. The present scientific statement provides a guide to allow professionals to make a goal-specific selection of a meaningful physical activity assessment method.

The Relationship Between College Zoology Students' Religious Beliefs and Their Ability to Objectively View the Scientific Evidence Supporting Evolutionary Theory.

ERIC Educational Resources Information Center

Sinclair, Anne; Baldwin, Beatrice

An anonymous 12-item, multiple-choice questionnaire was administered to 218 southern college, introductory zoology students prior to and following a study of evolutionary theory to assess their understanding and acceptance of the credibility of the evidence supporting the theory. Key topics addressed were the history of evolutionary thought, basic…

The Ability of Beginning University Chemistry Students to Use ICT (Information and Communication Technology) in Their Learning in 2002

ERIC Educational Resources Information Center

Lim, Kieran F.

2003-01-01

There is an assumption that high-school students are becoming more computer literate, but published studies of specific skill level are lacking. An anonymous multiple-choice survey self-assessed the ICT (information and communication technology) skills of first-year chemistry students at the beginning of 2002. The general level of ICT skill…

High School Students' Proficiency and Confidence Levels in Displaying Their Understanding of Basic Electrolysis Concepts

ERIC Educational Resources Information Center

Sia, Ding Teng; Treagust, David F.; Chandrasegaran, A. L.

2012-01-01

This study was conducted with 330 Form 4 (grade 10) students (aged 15-16 years) who were involved in a course of instruction on electrolysis concepts. The main purposes of this study were (1) to assess high school chemistry students' understanding of 19 major principles of electrolysis using a recently developed 2-tier multiple-choice diagnostic…

A Mixed Exam Format Closes the Gap for Students with a Conflict between Their Religious Belief & the Theory of Evolution

ERIC Educational Resources Information Center

Stanger-Hall, Kathrin F.; Wenner, Julianne A.

2014-01-01

We assessed the performance of students with a self-reported conflict between their religious belief and the theory of evolution in two sections of a large introductory biology course (N = 373 students). Student performance was measured through pretest and posttest evolution essays and multiple- choice (MC) questions (evolution-related and…

The Role of Professional Identity in Patterns of Use of Multiple-Choice Assessment Tools

ERIC Educational Resources Information Center

Johannesen, Monica; Habib, Laurence

2010-01-01

This article uses the notion of professional identity within the framework of actor network theory to understand didactic practices within three faculties in an institution of higher education. The study is based on a series of interviews with lecturers in each faculty and diaries of their didactic practices. The article focuses on the use of a…

Criterion Referenced Assessment Bank. Grade 6 Skill Clusters, Objectives, and Illustrations.

ERIC Educational Resources Information Center

Montgomery County Public Schools, Rockville, MD.

Part of a series of competency-based test materials for grades six through ten, this set of nine test booklets for sixth graders contains multiple-choice questions designed to aid in the evaluation of the pupils' library skills. Accompanied by a separate, tenth booklet of illustrations which are to be used in conjunction with the questions, the…

Reliability and Validity of a Procedure to Measure Diagnostic Reasoning and Problem-Solving Skills Taught in Predoctoral Orthodontic Education.

ERIC Educational Resources Information Center

Albanese, Mark A.; Jacobs, Richard M.

1990-01-01

The reliability and validity of a procedure to measure diagnostic-reasoning and problem-solving skills taught in predoctoral orthodontic education were studied using 68 second year dental students. The procedure includes stimulus material and 33 multiple-choice items. It is a feasible way of assessing problem-solving skills in dentistry education…

Formulation of Multiple Choice Questions as a Revision Exercise at the End of a Teaching Module in Biochemistry

ERIC Educational Resources Information Center

Bobby, Zachariah; Radhika, M. R.; Nandeesha, H.; Balasubramanian, A.; Prerna, Singh; Archana, Nimesh; Thippeswamy, D. N.

2012-01-01

The graduate medical students often get less opportunity for clarifying their doubts and to reinforce their concepts after lecture classes. Assessment of the effect of MCQ preparation by graduate medical students as a revision exercise on the topic "Mineral metabolism." At the end of regular teaching module on the topic "Mineral metabolism,"…

Dividing the Force Concept Inventory into Two Equivalent Half-Length Tests

ERIC Educational Resources Information Center

Han, Jing; Bao, Lei; Chen, Li; Cai, Tianfang; Pi, Yuan; Zhou, Shaona; Tu, Yan; Koenig, Kathleen

2015-01-01

The Force Concept Inventory (FCI) is a 30-question multiple-choice assessment that has been a building block for much of the physics education research done today. In practice, there are often concerns regarding the length of the test and possible test-retest effects. Since many studies in the literature use the mean score of the FCI as the…

The Presence of Gender Disparity on the Force Concept Inventory in a Sample of Canadian Undergraduate Students

ERIC Educational Resources Information Center

Normandeau, Magdalen; Iyengar, Seshu; Newling, Benedict

2017-01-01

Concept inventories (CI) are validated, research-based, multiple-choice tests, which are widely used to assess the effectiveness of pedagogical practices in bringing about conceptual change. In order to be a useful diagnostic tool, a CI must reflect only the student understanding of the conceptual material. The Force Concept Inventory (FCI) is…

On the Use of the Immediate Recall Task as a Measure of Second Language Reading Comprehension

ERIC Educational Resources Information Center

Chang, Yuh-Fang

2006-01-01

The immediate written recall task, a widely used measure of both first language (L1) and second language (L2) reading comprehension, has been advocated over traditional test methods such as multiple choice, cloze tests and open-ended questions because it is a direct and integrative assessment task. It has been, however, criticized as requiring…

The Effects of Pictorial Aids on Inferentially-produced Interference in Younger and Older Children's Sentence Learning.

ERIC Educational Resources Information Center

Truman, Diane L.

As part of a series of studies dealing with varieties of interference in sentence learning as assessed by multiple choice tests, a study was undertaken to explore the effects of pictures on inferentially produced interference in recognition memory for sentence information. The subjects were 104 first grade students and 104 fourth, fifth, and sixth…

Automatic Generation of Analogy Questions for Student Assessment: An Ontology-Based Approach

ERIC Educational Resources Information Center

Alsubait, Tahani; Parsia, Bijan; Sattler, Uli

2012-01-01

Different computational models for generating analogies of the form "A is to B as C is to D" have been proposed over the past 35 years. However, analogy generation is a challenging problem that requires further research. In this article, we present a new approach for generating analogies in Multiple Choice Question (MCQ) format that can be used…

Assessing the Life Science Knowledge of Students and Teachers Represented by the K-8 National Science Standards

ERIC Educational Resources Information Center

Sadler, Philip M.; Coyle, Harold; Cook Smith, Nancy; Miller, Jaimie; Mintzes, Joel; Tanner, Kimberly; Murray, John

2013-01-01

We report on the development of an item test bank and associated instruments based on the National Research Council (NRC) K-8 life sciences content standards. Utilizing hundreds of studies in the science education research literature on student misconceptions, we constructed 476 unique multiple-choice items that measure the degree to which test…

Assessment of Numeracy in Sports and Exercise Science Students at an Australian University

ERIC Educational Resources Information Center

Green, Simon; McGlynn, Susan; Stuart, Deidre; Fahey, Paul; Pettigrew, Jim; Clothier, Peter

2018-01-01

The effect of high school study of mathematics on numeracy performance of sports and exercise science (SES) students is not clear. To investigate this further, we tested the numeracy skills of 401 students enrolled in a Bachelor of Health Sciences degree in SES using a multiple-choice survey consisting of four background questions and 39 numeracy…

Medical Students' vs. Family Physicians' Assessment of Practical and Logical Values of Pathophysiology Multiple-Choice Questions

ERIC Educational Resources Information Center

Secic, Damir; Husremovic, Dzenana; Kapur, Eldan; Jatic, Zaim; Hadziahmetovic, Nina; Vojnikovic, Benjamin; Fajkic, Almir; Meholjic, Amir; Bradic, Lejla; Hadzic, Amila

2017-01-01

Testing strategies can either have a very positive or negative effect on the learning process. The aim of this study was to examine the degree of consistency in evaluating the practicality and logic of questions from a medical school pathophysiology test, between students and family medicine doctors. The study engaged 77 family medicine doctors…

The Development and Evaluation of Software to Foster Professional Development in Educational Assessment

ERIC Educational Resources Information Center

Benton, Morgan C.

2008-01-01

This dissertation sought to answer the question: Is it possible to build a software tool that will allow teachers to write better multiple-choice questions? The thesis proceeded from the finding that the quality of teaching is very influential in the amount that students learn. A basic premise of this research, then, is that improving teachers…

Comparing Two Types of Diagnostic Items to Evaluate Understanding of Heat and Temperature Concepts

ERIC Educational Resources Information Center

Chu, Hye-Eun; Chandrasegaran, A. L.; Treagust, David F.

2018-01-01

The purpose of this research was to investigate an efficient method to assess year 8 (age 13-14) students' conceptual understanding of heat and temperature concepts. Two different types of instruments were used in this study: Type 1, consisting of multiple-choice items with open-ended justifications; and Type 2, consisting of two-tier…

Interpreting Secondary Students' Performance on a Timed, Multiple-Choice Reading Comprehension Assessment: The Prevalence and Impact of Non-Attempted Items

ERIC Educational Resources Information Center

Clemens, Nathan H.; Davis, John L.; Simmons, Leslie E.; Oslund, Eric L.; Simmons, Deborah C.

2015-01-01

Standardized measures are often used as an index of students' reading comprehension and scores have important implications, particularly for students who perform below expectations. This study examined secondary-level students' patterns of responding and the prevalence and impact of non-attempted items on a timed, group-administered,…

Can Item Analysis of MCQs Accomplish the Need of a Proper Assessment Strategy for Curriculum Improvement in Medical Education?

ERIC Educational Resources Information Center

Pawade, Yogesh R.; Diwase, Dipti S.

2016-01-01

Item analysis of Multiple Choice Questions (MCQs) is the process of collecting, summarizing and utilizing information from students' responses to evaluate the quality of test items. Difficulty Index (p-value), Discrimination Index (DI) and Distractor Efficiency (DE) are the parameters which help to evaluate the quality of MCQs used in an…

An Instrument to Predict Job Performance of Home Health Aides--Testing the Reliability and Validity.

ERIC Educational Resources Information Center

Sturges, Jack; Quina, Patricia

The development of four paper-and-pencil tests, useful in assessing the effectiveness of inservice training provided to either nurses aides or home health aides, was described. These tests were designed for utilization in employment selection and case assignment. Two tests of 37 multiple-choice items and two tests of 10 matching items were…

Evaluation of geriatrics education at a Chinese University: a survey of attitudes and knowledge among undergraduate medical students.

PubMed

Shi, Sandra; Lio, Jonathan; Dong, Hongmei; Jiang, Ivy; Cooper, Brian; Sherer, Renslow

2018-05-08

Despite widespread reforms in medical education across China, nationally there has been no mandate or movement toward systemically incorporating geriatrics into curricula. To what degree medical students are trained and have exposure to geriatric topics remains unclear. We surveyed 190 medical students during their final year of medical school at a Chinese medical university, graduating from reformed and also traditional curricula. The survey was comprised of a subjective assessment of attitudes and reported knowledge, as well as an objective assessment of knowledge via a multiple choice test. Student attitudes were favorable toward geriatrics, with 91% supporting the addition of specialized clinical experiences to the curriculum. Students generally reported low exposure to geriatrics, with no statistically significant differences between reform and traditional curricula. There was a statistically significant difference in performance on the multiple choice test between curricula but at a degree unlikely to be practically significant. Students had very favorable attitudes toward geriatrics as a field and specialty; however scored poorly on competency exams, with the lowest performance around diagnosis and treatment of specific geriatric conditions. Our results suggest that there is a need and desire for increased geriatric-oriented learning at Chinese medical schools.

Practices in habilitation of pediatric recipients of cochlear implants in India: A survey.

PubMed

Jeyaraman, Janani

2013-01-01

Cochlear implant (CI) (re)habilitation programs are long-term processes, with many factors contributing to the overall success. The clinics in India that are working toward pediatric CI habilitation vary in their team philosophy, clinical practices, and service delivery. It is important to explore their clinical perspectives and practices to appreciate their current state and suggest directions for improvement in the future. The objective of the study was to characterize the current status and clinical practices of the pediatric CI programs in India. Twenty-two clinics involved in the pediatric CI habilitation program across India participated in the survey. The heads of the CI teams of the participant clinics completed a validated survey questionnaire containing multiple-choice and open-ended questions on the details of the CI habilitation team, assessment and therapy protocols used, and other related clinical services. The categorical data obtained were analyzed using descriptive statistical measures. The interpretation of results indicated a need to focus future discussions on early identification and management of hearing impairment, funding for CIs, continuing education programs for professionals, decision processes for providing CIs for children with multiple concerns, choice of language(s) of instruction, assessment protocols used, and outreach/consultation services.

Impediments to the success of management actions for species recovery.

PubMed

Ng, Chooi Fei; Possingham, Hugh P; McAlpine, Clive A; de Villiers, Deidré L; Preece, Harriet J; Rhodes, Jonathan R

2014-01-01

Finding cost-effective management strategies to recover species declining due to multiple threats is challenging, especially when there are limited resources. Recent studies offer insights into how costs and threats can influence the best choice of management actions. However, when implementing management actions in the real-world, a range of impediments to management success often exist that can be driven by social, technological and land-use factors. These impediments may limit the extent to which we can achieve recovery objectives and influence the optimal choice of management actions. Nonetheless, the implications of these impediments are not well understood, especially for recovery planning involving multiple actions. We used decision theory to assess the impact of these types of impediments for allocating resources among recovery actions to mitigate multiple threats. We applied this to a declining koala (Phascolarctos cinereus) population threatened by habitat loss, vehicle collisions, dog attacks and disease. We found that the unwillingness of dog owners to restrain their dogs at night (a social impediment), the effectiveness of wildlife crossings to reduce vehicle collisions (a technological impediment) and the unavailability of areas for restoration (a land-use impediment) significantly reduced the effectiveness of our actions. In the presence of these impediments, achieving successful recovery may be unlikely. Further, these impediments influenced the optimal choice of recovery actions, but the extent to which this was true depended on the target koala population growth rate. Given that species recovery is an important strategy for preserving biodiversity, it is critical that we consider how impediments to the success of recovery actions modify our choice of actions. In some cases, it may also be worth considering whether investing in reducing or removing impediments may be a cost-effective course of action.

Searching for the right fit: development of applicant person-organization fit perceptions during the recruitment process.

PubMed

Swider, Brian W; Zimmerman, Ryan D; Barrick, Murray R

2015-05-01

Numerous studies link applicant fit perceptions measured at a single point in time to recruitment outcomes. Expanding upon this prior research by incorporating decision-making theory, this study examines how applicants develop these fit perceptions over the duration of the recruitment process, showing meaningful changes in fit perceptions across and within organizations overtime. To assess the development of applicant fit perceptions, eight assessments of person-organization (PO) fit with up to four different organizations across 169 applicants for 403 job choice decisions were analyzed. Results showed the presence of initial levels and changes in differentiation of applicant PO fit perceptions across organizations, which significantly predicted future job choice. In addition, changes in within-organizational PO fit perceptions across two stages of recruitment predicted applicant job choices among multiple employers. The implications of these results for accurately understanding the development of fit perceptions, relationships between fit perceptions and key recruiting outcomes, and possible limitations of past meta-analytically derived estimates of these relationships are discussed. (c) 2015 APA, all rights reserved.

Does the MCAT predict medical school and PGY-1 performance?

PubMed

Saguil, Aaron; Dong, Ting; Gingerich, Robert J; Swygert, Kimberly; LaRochelle, Jeffrey S; Artino, Anthony R; Cruess, David F; Durning, Steven J

2015-04-01

The Medical College Admissions Test (MCAT) is a high-stakes test required for entry to most U. S. medical schools; admissions committees use this test to predict future accomplishment. Although there is evidence that the MCAT predicts success on multiple choice-based assessments, there is little information on whether the MCAT predicts clinical-based assessments of undergraduate and graduate medical education performance. This study looked at associations between the MCAT and medical school grade point average (GPA), Medical Licensing Examination (USMLE) scores, observed patient care encounters, and residency performance assessments. This study used data collected as part of the Long-Term Career Outcome Study to determine associations between MCAT scores, USMLE Step 1, Step 2 clinical knowledge and clinical skill, and Step 3 scores, Objective Structured Clinical Examination performance, medical school GPA, and PGY-1 program director (PD) assessment of physician performance for students graduating 2010 and 2011. MCAT data were available for all students, and the PGY PD evaluation response rate was 86.2% (N = 340). All permutations of MCAT scores (first, last, highest, average) were weakly associated with GPA, Step 2 clinical knowledge scores, and Step 3 scores. MCAT scores were weakly to moderately associated with Step 1 scores. MCAT scores were not significantly associated with Step 2 clinical skills Integrated Clinical Encounter and Communication and Interpersonal Skills subscores, Objective Structured Clinical Examination performance or PGY-1 PD evaluations. MCAT scores were weakly to moderately associated with assessments that rely on multiple choice testing. The association is somewhat stronger for assessments occurring earlier in medical school, such as USMLE Step 1. The MCAT was not able to predict assessments relying on direct clinical observation, nor was it able to predict PD assessment of PGY-1 performance. Reprint & Copyright © 2015 Association of Military Surgeons of the U.S.

Developing multiple-choices test items as tools for measuring the scientific-generic skills on solar system

NASA Astrophysics Data System (ADS)

Bhakti, Satria Seto; Samsudin, Achmad; Chandra, Didi Teguh; Siahaan, Parsaoran

2017-05-01

The aim of research is developing multiple-choices test items as tools for measuring the scientific of generic skills on solar system. To achieve the aim that the researchers used the ADDIE model consisting Of: Analyzing, Design, Development, Implementation, dan Evaluation, all of this as a method research. While The scientific of generic skills limited research to five indicator including: (1) indirect observation, (2) awareness of the scale, (3) inference logic, (4) a causal relation, and (5) mathematical modeling. The participants are 32 students at one of junior high schools in Bandung. The result shown that multiple-choices that are constructed test items have been declared valid by the expert validator, and after the tests show that the matter of developing multiple-choices test items be able to measuring the scientific of generic skills on solar system.

MCDA swing weighting and discrete choice experiments for elicitation of patient benefit-risk preferences: a critical assessment.

PubMed

Tervonen, Tommi; Gelhorn, Heather; Sri Bhashyam, Sumitra; Poon, Jiat-Ling; Gries, Katharine S; Rentz, Anne; Marsh, Kevin

2017-12-01

Multiple criteria decision analysis swing weighting (SW) and discrete choice experiments (DCE) are appropriate methods for capturing patient preferences on treatment benefit-risk trade-offs. This paper presents a qualitative comparison of the 2 methods. We review and critically assess similarities and differences of SW and DCE based on 6 aspects: comprehension by study participants, cognitive biases, sample representativeness, ability to capture heterogeneity in preferences, reliability and validity, and robustness of the results. The SW choice task can be more difficult, but the workshop context in which SW is conducted may provide more support to patients who are unfamiliar with the end points being evaluated or who have cognitive impairments. Both methods are similarly prone to a number of biases associated with preference elicitation, and DCE is prone to simplifying heuristics, which limits its application with large number of attributes. The low cost per patient of the DCE means that it can be better at achieving a representative sample, though SW does not require such large sample sizes due to exact nature of the collected preference data. This also means that internal validity is automatically enforced with SW, while the internal validity of DCE results needs to be assessed manually. Choice between the 2 methods depends on characteristics of the benefit-risk assessment, especially on how difficult the trade-offs are for the patients to make and how many patients are available. Although there exist some empirical studies on many of the evaluation aspects, critical evidence gaps remain. Copyright © 2017 John Wiley & Sons, Ltd.

Pick-N Multiple Choice-Exams: A Comparison of Scoring Algorithms

ERIC Educational Resources Information Center

Bauer, Daniel; Holzer, Matthias; Kopp, Veronika; Fischer, Martin R.

2011-01-01

To compare different scoring algorithms for Pick-N multiple correct answer multiple-choice (MC) exams regarding test reliability, student performance, total item discrimination and item difficulty. Data from six 3rd year medical students' end of term exams in internal medicine from 2005 to 2008 at Munich University were analysed (1,255 students,…

A Randomized Educational Intervention Trial to Determine the Effect of Online Education on the Quality of Resident-Delivered Care.

PubMed

Dolan, Brigid M; Yialamas, Maria A; McMahon, Graham T

2015-09-01

There is limited research on whether online formative self-assessment and learning can change the behavior of medical professionals. We sought to determine if an adaptive longitudinal online curriculum in bone health would improve resident physicians' knowledge, and change their behavior regarding prevention of fragility fractures in women. We used a randomized control trial design in which 50 internal medicine resident physicians at a large academic practice were randomized to either receive a standard curriculum in bone health care alone, or to receive it augmented with an adaptive, longitudinal, online formative self-assessment curriculum delivered via multiple-choice questions. Outcomes were assessed 10 months after the start of the intervention. Knowledge outcomes were measured by a multiple-choice question examination. Clinical outcomes were measured by chart review, including bone density screening rate, calculation of the fracture risk assessment tool (FRAX) score, and rate of appropriate bisphosphonate prescription. Compared to the control group, residents participating in the intervention had higher scores on the knowledge test at the end of the study. Bone density screening rates and appropriate use of bisphosphonates were significantly higher in the intervention group compared with the control group. FRAX score reporting did not differ between the groups. Residents participating in a novel adaptive online curriculum outperformed peers in knowledge of fragility fracture prevention and care practices to prevent fracture. Online adaptive education can change behavior to improve patient care.

A Randomized Educational Intervention Trial to Determine the Effect of Online Education on the Quality of Resident-Delivered Care

PubMed Central

Dolan, Brigid M.; Yialamas, Maria A.; McMahon, Graham T.

2015-01-01

Background There is limited research on whether online formative self-assessment and learning can change the behavior of medical professionals. Objective We sought to determine if an adaptive longitudinal online curriculum in bone health would improve resident physicians' knowledge, and change their behavior regarding prevention of fragility fractures in women. Methods We used a randomized control trial design in which 50 internal medicine resident physicians at a large academic practice were randomized to either receive a standard curriculum in bone health care alone, or to receive it augmented with an adaptive, longitudinal, online formative self-assessment curriculum delivered via multiple-choice questions. Outcomes were assessed 10 months after the start of the intervention. Knowledge outcomes were measured by a multiple-choice question examination. Clinical outcomes were measured by chart review, including bone density screening rate, calculation of the fracture risk assessment tool (FRAX) score, and rate of appropriate bisphosphonate prescription. Results Compared to the control group, residents participating in the intervention had higher scores on the knowledge test at the end of the study. Bone density screening rates and appropriate use of bisphosphonates were significantly higher in the intervention group compared with the control group. FRAX score reporting did not differ between the groups. Conclusions Residents participating in a novel adaptive online curriculum outperformed peers in knowledge of fragility fracture prevention and care practices to prevent fracture. Online adaptive education can change behavior to improve patient care. PMID:26457142

To Show or Not to Show: The Effects of Item Stems and Answer Options on Performance on a Multiple-Choice Listening Comprehension Test

ERIC Educational Resources Information Center

Yanagawa, Kozo; Green, Anthony

2008-01-01

The purpose of this study is to examine whether the choice between three multiple-choice listening comprehension test formats results in any difference in listening comprehension test performance. The three formats entail (a) allowing test takers to preview both the question stem and answer options prior to listening; (b) allowing test takers to…

Pushing Critical Thinking Skills With Multiple-Choice Questions: Does Bloom's Taxonomy Work?

PubMed

Zaidi, Nikki L Bibler; Grob, Karri L; Monrad, Seetha M; Kurtz, Joshua B; Tai, Andrew; Ahmed, Asra Z; Gruppen, Larry D; Santen, Sally A

2018-06-01

Medical school assessments should foster the development of higher-order thinking skills to support clinical reasoning and a solid foundation of knowledge. Multiple-choice questions (MCQs) are commonly used to assess student learning, and well-written MCQs can support learner engagement in higher levels of cognitive reasoning such as application or synthesis of knowledge. Bloom's taxonomy has been used to identify MCQs that assess students' critical thinking skills, with evidence suggesting that higher-order MCQs support a deeper conceptual understanding of scientific process skills. Similarly, clinical practice also requires learners to develop higher-order thinking skills that include all of Bloom's levels. Faculty question writers and examinees may approach the same material differently based on varying levels of knowledge and expertise, and these differences can influence the cognitive levels being measured by MCQs. Consequently, faculty question writers may perceive that certain MCQs require higher-order thinking skills to process the question, whereas examinees may only need to employ lower-order thinking skills to render a correct response. Likewise, seemingly lower-order questions may actually require higher-order thinking skills to respond correctly. In this Perspective, the authors describe some of the cognitive processes examinees use to respond to MCQs. The authors propose that various factors affect both the question writer and examinee's interaction with test material and subsequent cognitive processes necessary to answer a question.

An analysis of four ways of assessing student beliefs about sts topics

NASA Astrophysics Data System (ADS)

Aikenhead, Glen S.

The study investigated the degree of ambiguity harbored by four different response modes used to monitor student beliefs about science-technology-society topics: Likert-type, written paragraph, semistrue tured interview, and empirically developed multiple choice. The study also explored the sources of those beliefs. Grade-12 students in a Canadian urban setting responded, in each of the four modes, to statements from Views on Science-Technology-Society. It was discovered that TV had far more influence on what students believed about science and its social, technological context than did numerous science courses. The challenge to science educators is to use the media effectively in combating naive views about science. Regarding ambiguity in student assessment, the Likert-type responses were the most inaccurate, offering only a guess at student beliefs. Such guesswork calls into question the use of Likert-type standardized tests that claim to assess student views about science. Student paragraph responses contained significant ambiguities in about 50% of the cases. The empirically developed multiple choices, however, reduced the ambiguity to the 20% level. Predictably, the semistructured interview was the least ambiguous of all four response modes, but it required the most time to administer. These findings encourage researchers to develop instruments grounded in the empirical data of student viewpoints, rather than relying solely on instruments structured by the philosophical stances of science educators.

Measuring ability to assess claims about treatment effects: a latent trait analysis of items from the ‘Claim Evaluation Tools’ database using Rasch modelling

PubMed Central

Austvoll-Dahlgren, Astrid; Guttersrud, Øystein; Nsangi, Allen; Semakula, Daniel; Oxman, Andrew D

2017-01-01

Background The Claim Evaluation Tools database contains multiple-choice items for measuring people’s ability to apply the key concepts they need to know to be able to assess treatment claims. We assessed items from the database using Rasch analysis to develop an outcome measure to be used in two randomised trials in Uganda. Rasch analysis is a form of psychometric testing relying on Item Response Theory. It is a dynamic way of developing outcome measures that are valid and reliable. Objectives To assess the validity, reliability and responsiveness of 88 items addressing 22 key concepts using Rasch analysis. Participants We administrated four sets of multiple-choice items in English to 1114 people in Uganda and Norway, of which 685 were children and 429 were adults (including 171 health professionals). We scored all items dichotomously. We explored summary and individual fit statistics using the RUMM2030 analysis package. We used SPSS to perform distractor analysis. Results Most items conformed well to the Rasch model, but some items needed revision. Overall, the four item sets had satisfactory reliability. We did not identify significant response dependence between any pairs of items and, overall, the magnitude of multidimensionality in the data was acceptable. The items had a high level of difficulty. Conclusion Most of the items conformed well to the Rasch model’s expectations. Following revision of some items, we concluded that most of the items were suitable for use in an outcome measure for evaluating the ability of children or adults to assess treatment claims. PMID:28550019

Assessment of musculoskeletal physical examination skills and attitudes of orthopaedic residents.

PubMed

Beran, Matthew C; Awan, Hisham; Rowley, David; Samora, Julie Balch; Griesser, Michael J; Bishop, Julie Y

2012-03-21

Although the musculoskeletal physical examination is an essential part of patient encounters, we believe that it is underemphasized in residency education and that residents' physical examination skills may be lacking. We sought to assess attitudes regarding teaching of the physical examination in orthopaedic residencies, to assess physical examination knowledge and skills among residents, and to develop a method to track the skill level of residents in order to improve our physical examination curriculum. We created a thirty-question multiple-choice musculoskeletal physical examination test and administered it to our residents. We created a five-question survey assessing attitudes toward physical examination teaching in orthopaedic residencies and distributed it to U.S. orthopaedic department chairs We developed an Objective Structured Clinical Examination (OSCE), in which standardized patients enact four clinical scenarios, to observe and assess physical examination skills. The mean score on the multiple-choice physical examination test was 76% despite the fact that our residents consistently scored above 90% on the Orthopaedic In-Training Examination. Department chairs and residents agreed that, although learning to perform the physical examination is important, there is not enough time in the clinical setting to observe and critique a resident's patient examination. The overall score of our residents on the OSCE was 66%. We have exposed a deficiency in the physical examination knowledge and skills of our residents. Although the musculoskeletal physical examination is a vital practice component, our data indicate that it is likely underemphasized in training. Clinic time alone is likely insufficient for the teaching and learning of the musculoskeletal physical examination.

A Three-Tier Diagnostic Test to Assess Pre-Service Teachers' Misconceptions about Global Warming, Greenhouse Effect, Ozone Layer Depletion, and Acid Rain

ERIC Educational Resources Information Center

Arslan, Harika Ozge; Cigdemoglu, Ceyhan; Moseley, Christine

2012-01-01

This study describes the development and validation of a three-tier multiple-choice diagnostic test, the atmosphere-related environmental problems diagnostic test (AREPDiT), to reveal common misconceptions of global warming (GW), greenhouse effect (GE), ozone layer depletion (OLD), and acid rain (AR). The development of a two-tier diagnostic test…

Perceptions of Student-Teacher Relationships, Self-Efficacy, and Subject Matter Retention in a Secondary Chemistry Course

ERIC Educational Resources Information Center

Bechtel, Michael Dean

2012-01-01

This was a study of students who had completed a chemistry course taught by one instructor in a large urban high school during 2009-2010. It was conducted in two phases: Phase One assessed self-efficacy, teaching practices, and subject matter retention taken 16 months after course completion. Phase Two consisted of a multiple-choice final exam…

Use of the NBME Comprehensive Basic Science Examination as a Progress Test in the Preclerkship Curriculum of a New Medical School

ERIC Educational Resources Information Center

Johnson, Teresa R.; Khalil, Mohammed K.; Peppler, Richard D.; Davey, Diane D.; Kibble, Jonathan D.

2014-01-01

In the present study, we describe the innovative use of the National Board of Medical Examiners (NBME) Comprehensive Basic Science Examination (CBSE) as a progress test during the preclerkship medical curriculum. The main aim of this study was to provide external validation of internally developed multiple-choice assessments in a new medical…

An Analysis of 16-17-Year-Old Students' Understanding of Solution Chemistry Concepts Using a Two-Tier Diagnostic Instrument

ERIC Educational Resources Information Center

Adadan, Emine; Savasci, Funda

2012-01-01

This study focused on the development of a two-tier multiple-choice diagnostic instrument, which was designed and then progressively modified, and implemented to assess students' understanding of solution chemistry concepts. The results of the study are derived from the responses of 756 Grade 11 students (age 16-17) from 14 different high schools…

Science Library of Test Items. Volume Nineteen. A Collection of Multiple Choice Test Items Relating Mainly to Geology.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Large-Scale Assessment of Language Proficiency: Theoretical and Pedagogical Reflections on the Use of Multiple-Choice Tests

ERIC Educational Resources Information Center

Argüelles Álvarez, Irina

2013-01-01

The new requirement placed on students in tertiary settings in Spain to demonstrate a B1 or a B2 proficiency level of English, in accordance with the Common European Framework of Reference for Languages (CEFRL), has led most Spanish universities to develop a program of certification or accreditation of the required level. The first part of this…

An Investigation into the Relationship between Students' Conceptions of the Particulate Nature of Matter and Their Understanding of Chemical Bonding

ERIC Educational Resources Information Center

Othman, Jazilah; Treagust, David F.; Chandrasegaran, A. L.

2008-01-01

A thorough understanding of chemical bonding requires familiarity with the particulate nature of matter. In this study, a two-tier multiple-choice diagnostic instrument consisting of ten items (five items involving each of the two concepts) was developed to assess students' understanding of the particulate nature of matter and chemical bonding so…

Using Student Writing and Lexical Analysis to Reveal Student Thinking about the Role of Stop Codons in the Central Dogma

ERIC Educational Resources Information Center

Prevost, Luanna B.; Smith, Michelle K.; Knight, Jennifer K.

2016-01-01

Previous work has shown that students have persistent difficulties in understanding how central dogma processes can be affected by a stop codon mutation. To explore these difficulties, we modified two multiple-choice questions from the Genetics Concept Assessment into three open-ended questions that asked students to write about how a stop codon…

Science Library of Test Items. Volume Seventeen. A Collection of Multiple Choice Test Items Relating Mainly to Biology.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Science Library of Test Items. Volume Eighteen. A Collection of Multiple Choice Test Items Relating Mainly to Chemistry.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Health-Related Fitness Knowledge and Its Relation to Student Physical Activity Patterns at a Large U.S. Southern State University

ERIC Educational Resources Information Center

Keating, Xiaofen D.; Castro-Pinero, Jose; Centeio, Erin; Harrison, Louis, Jr.; Ramirez, Tere; Chen, Li

2010-01-01

This study examined student health-related fitness (HRF) knowledge and its relationship to physical activity (PA). The participants were undergraduate students from a large U.S. state university. HRF knowledge was assessed using a test consisting of 150 multiple choice items. Differences in HRF knowledge scores by sex, ethnicity, and years in…

Writing a Writing Assessment: Saying What You Want to Say Isn't as Simple as It Seems.

ERIC Educational Resources Information Center

Escoe, Adrienne

Since acceptable writing is essential to success in job training programs and in many entry-level jobs, a writing sample was included in the Training and Employment Prerequisites Survey, a multiple-choice test about skills like mechanics, usage, and spelling. The two writing prompts asked students to give directions for finding a location in a…

Program of Research on Legal Writing: Phase II: Research on a Writing Exercise. LSAC Research Report Series.

ERIC Educational Resources Information Center

Breland, Hunter M.; Carlton, Sydell T.; Taylor, Susan

Based on the results of a Phase 1 investigation into the nature of legal writing, a prototype writing assessment, the Diagnostic Writing Skills Test (DWST) for entering law students was developed. The DWST is composed of two multiple-choice testlets based on prompts and responses to the Law School Admission Test (LSAT) Writing Sample. It contains…

How Well Do Engineering Students Retain Core Mathematical Knowledge after a Series of High Threshold Online Mathematics Tests?

ERIC Educational Resources Information Center

Carr, Michael; Prendergast, Mark; Breen, Cormac; Faulkner, Fiona

2017-01-01

In the Dublin Institute of Technology, high threshold core skills assessments are run in mathematics for third-year engineering students. Such tests require students to reach a threshold of 90% on a multiple choice test based on a randomized question bank. The material covered by the test consists of the more important aspects of undergraduate…

Spasticity management in multiple sclerosis.

PubMed

Hughes, Christina; Howard, Ileana M

2013-11-01

Spasticity is a prevalent and potentially disabling symptom common in individuals with multiple sclerosis. Adequate evaluation and management of spasticity requires a careful assessment of the patient's history to determine functional impact of spasticity and potential exacerbating factors, and physical examination to determine the extent of the condition and culpable muscles. A host of options for spasticity management are available: therapeutic exercise, physical modalities, complementary/alternative medicine interventions, oral medications, chemodenervation, and implantation of an intrathecal baclofen pump. Choice of treatment hinges on a combination of the extent of symptoms, patient preference, and availability of services. Copyright © 2013 Elsevier Inc. All rights reserved.

Effects of the Informed Health Choices primary school intervention on the ability of children in Uganda to assess the reliability of claims about treatment effects: a cluster-randomised controlled trial.

PubMed

Nsangi, Allen; Semakula, Daniel; Oxman, Andrew D; Austvoll-Dahlgren, Astrid; Oxman, Matt; Rosenbaum, Sarah; Morelli, Angela; Glenton, Claire; Lewin, Simon; Kaseje, Margaret; Chalmers, Iain; Fretheim, Atle; Ding, Yunpeng; Sewankambo, Nelson K

2017-07-22

Claims about what improves or harms our health are ubiquitous. People need to be able to assess the reliability of these claims. We aimed to evaluate an intervention designed to teach primary school children to assess claims about the effects of treatments (ie, any action intended to maintain or improve health). In this cluster-randomised controlled trial, we included primary schools in the central region of Uganda that taught year-5 children (aged 10-12 years). We excluded international schools, special needs schools for children with auditory and visual impairments, schools that had participated in user-testing and piloting of the resources, infant and nursery schools, adult education schools, and schools that were difficult for us to access in terms of travel time. We randomly allocated a representative sample of eligible schools to either an intervention or control group. Intervention schools received the Informed Health Choices primary school resources (textbooks, exercise books, and a teachers' guide). Teachers attended a 2 day introductory workshop and gave nine 80 min lessons during one school term. The lessons addressed 12 concepts essential to assessing claims about treatment effects and making informed health choices. We did not intervene in the control schools. The primary outcome, measured at the end of the school term, was the mean score on a test with two multiple-choice questions for each of the 12 concepts and the proportion of children with passing scores on the same test. This trial is registered with the Pan African Clinical Trial Registry, number PACTR201606001679337. Between April 11, 2016, and June 8, 2016, 2960 schools were assessed for eligibility; 2029 were eligible, and a random sample of 170 were invited to recruitment meetings. After recruitment meetings, 120 eligible schools consented and were randomly assigned to either the intervention group (n=60, 76 teachers and 6383 children) or control group (n=60, 67 teachers and 4430 children). The mean score in the multiple-choice test for the intervention schools was 62·4% (SD 18·8) compared with 43·1% (15·2) for the control schools (adjusted mean difference 20·0%, 95% CI 17·3-22·7; p<0·00001). In the intervention schools, 3967 (69%) of 5753 children achieved a predetermined passing score (≥13 of 24 correct answers) compared with 1186 (27%) of 4430 children in the control schools (adjusted difference 50%, 95% CI 44-55). The intervention was effective for children with different levels of reading skills, but was more effective for children with better reading skills. The use of the Informed Health Choices primary school learning resources, after an introductory workshop for the teachers, led to a large improvement in the ability of children to assess claims about the effects of treatments. The results show that it is possible to teach primary school children to think critically in schools with large student to teacher ratios and few resources. Future studies should address how to scale up use of the resources, long-term effects, including effects on actual health choices, transferability to other countries, and how to build on this programme with additional primary and secondary school learning resources. Research Council of Norway. Copyright © 2017 Elsevier Ltd. All rights reserved.

Quality Multiple-Choice Test Questions: Item-Writing Guidelines and an Analysis of Auditing Testbanks.

ERIC Educational Resources Information Center

Hansen, James D.; Dexter, Lee

1997-01-01

Analysis of test item banks in 10 auditing textbooks found that 75% of questions violated one or more guidelines for multiple-choice items. In comparison, 70% of a certified public accounting exam bank had no violations. (SK)

Anonymity and Electronics: Adapting Preparation for Radiology Resident Examination.

PubMed

Chapman, Teresa; Reid, Janet R; O'Conner, Erin E

2017-06-01

Diagnostic radiology resident assessment has evolved from a traditional oral examination to computerized testing. Teaching faculty struggle to reconcile the differences between traditional teaching methods and residents' new preferences for computerized testing models generated by new examination styles. We aim to summarize the collective experiences of senior residents at three different teaching hospitals who participated in case review sessions using a computer-based, interactive, anonymous teaching tool, rather than the Socratic method. Feedback was collected from radiology residents following participation in a senior resident case review session using Nearpod, which allows residents to anonymously respond to the teaching material. Subjective resident feedback was uniformly enthusiastic. Ninety percent of residents favor a case-based board review incorporating multiple-choice questions, and 94% favor an anonymous response system. Nearpod allows for inclusion of multiple-choice questions while also providing direct feedback to the teaching faculty, helping to direct the instruction and clarify residents' gaps in knowledge before the Core Examination. Copyright © 2017 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved.

Male satin bowerbirds (Ptilonorhynchus violaceus) compensate for sexual signal loss by enhancing multiple display features

NASA Astrophysics Data System (ADS)

Bravery, Benjamin D.; Goldizen, Anne W.

2007-06-01

Numerous studies have focussed on the relationship between female choice and the multiple exaggerated sexual traits of males. However, little is known about the ability of males to actively enhance specific components of their display in response to the loss of one component. We investigated the capacity of male satin bowerbirds (Ptilonorhynchus violaceus) to respond to the loss of one of their sexual signals by performing an experiment in which we removed decorations at their bowers. We found that males compensated for decoration loss by increasing bower construction behaviour and decreasing their latency to bower painting. These results are novel because they suggest that males can assess the quality of their own display and make decisions about how to augment their displays. We discuss these results in the context of previous studies of mate choice in satin bowerbirds, as both of the supplementary behaviours we observed are known correlates of male mating success.

Students’ Conception on Heat and Temperature toward Science Process Skill

NASA Astrophysics Data System (ADS)

Ratnasari, D.; Sukarmin, S.; Suparmi, S.; Aminah, N. S.

2017-09-01

This research is aimed to analyze the effect of students’ conception toward science process skill. This is a descriptive research with subjects of the research were 10th-grade students in Surakarta from high, medium and low categorized school. The sample selection uses purposive sampling technique based on physics score in national examination four latest years. Data in this research collecting from essay test, two-tier multiple choice test, and interview. Two-tier multiple choice test consists of 30 question that contains an indicator of science process skill. Based on the result of the research and analysis, it shows that students’ conception of heat and temperature affect science process skill of students. The students’ conception that still contains the wrong concept can emerge misconception. For the future research, it is suggested to improve students’ conceptual understanding and students’ science process skill with appropriate learning method and assessment instrument because heat and temperature is one of physics material that closely related with students’ daily life.

Test of understanding of vectors: A reliable multiple-choice vector concept test

NASA Astrophysics Data System (ADS)

Barniol, Pablo; Zavala, Genaro

2014-06-01

In this article we discuss the findings of our research on students' understanding of vector concepts in problems without physical context. First, we develop a complete taxonomy of the most frequent errors made by university students when learning vector concepts. This study is based on the results of several test administrations of open-ended problems in which a total of 2067 students participated. Using this taxonomy, we then designed a 20-item multiple-choice test [Test of understanding of vectors (TUV)] and administered it in English to 423 students who were completing the required sequence of introductory physics courses at a large private Mexican university. We evaluated the test's content validity, reliability, and discriminatory power. The results indicate that the TUV is a reliable assessment tool. We also conducted a detailed analysis of the students' understanding of the vector concepts evaluated in the test. The TUV is included in the Supplemental Material as a resource for other researchers studying vector learning, as well as instructors teaching the material.

The Prevalence of Multiple-Choice Testing in Registered Nurse Licensure-Qualifying Nursing Education Programs in New York State.

PubMed

Birkhead, Susan; Kelman, Glenda; Zittel, Barbara; Jatulis, Linnea

The aim of this study was to describe nurse educators' use of multiple-choice questions (MCQs) in testing in registered nurse licensure-qualifying nursing education programs in New York State. This study was a descriptive correlational analysis of data obtained from surveying 1,559 nurse educators; 297 educators from 61 institutions responded (response rate [RR] = 19 percent), yielding a final cohort of 200. MCQs were reported to comprise a mean of 81 percent of questions on a typical test. Baccalaureate program respondents were equally likely to use MCQs as associate degree program respondents (p > .05) but were more likely to report using other methods of assessing student achievement to construct course grades (p < .01). Both groups reported little use of alternate format-type questions. Respondent educators reported substantial reliance upon the use of MCQs, corroborating the limited data quantifying the prevalence of use of MCQ tests in licensure-qualifying nursing education programs.

How we tackled the problem of assessing humanities, social and behavioural sciences in medical education.

PubMed

Goodwin, Dawn; Machin, Laura

2016-01-01

Assessment serves as an important motivation for learning. However, multiple choice and short answer question formats are often considered unsatisfactory for assessment of medical humanities, and the social and behavioural sciences. Little consensus exists as to what might constitute 'best' assessment practice. What we did: We designed an assessment format closely aligned to the curricular approach of problem-based learning which allows for greater assessment of students' understanding, depth of knowledge and interpretation, rather than recall of rote learning. The educational impact of scenario-based assessment has been profound. Students reported changing their approach to PBL, independent learning and exam preparation by taking a less reductionist, more interpretative approach to the topics studied.

Treatment optimization in MS: Canadian MS Working Group updated recommendations.

PubMed

Freedman, Mark S; Selchen, Daniel; Arnold, Douglas L; Prat, Alexandre; Banwell, Brenda; Yeung, Michael; Morgenthau, David; Lapierre, Yves

2013-05-01

The Canadian Multiple Sclerosis Working Group (CMSWG) developed practical recommendations in 2004 to assist clinicians in optimizing the use of disease-modifying therapies (DMT) in patients with relapsing multiple sclerosis. The CMSWG convened to review how disease activity is assessed, propose a more current approach for assessing suboptimal response, and to suggest a scheme for switching or escalating treatment. Practical criteria for relapses, Expanded Disability Status Scale (EDSS) progression and MRI were developed to classify the clinical level of concern as Low, Medium and High. The group concluded that a change in treatment may be considered in any RRMS patient if there is a high level of concern in any one domain (relapses, progression or MRI), a medium level of concern in any two domains, or a low level of concern in all three domains. These recommendations for assessing treatment response should assist clinicians in making more rational choices in their management of relapsing MS patients.

A multiple indicator solution approach to endogeneity in discrete-choice models for environmental valuation.

PubMed

Mariel, Petr; Hoyos, David; Artabe, Alaitz; Guevara, C Angelo

2018-08-15

Endogeneity is an often neglected issue in empirical applications of discrete choice modelling despite its severe consequences in terms of inconsistent parameter estimation and biased welfare measures. This article analyses the performance of the multiple indicator solution method to deal with endogeneity arising from omitted explanatory variables in discrete choice models for environmental valuation. We also propose and illustrate a factor analysis procedure for the selection of the indicators in practice. Additionally, the performance of this method is compared with the recently proposed hybrid choice modelling framework. In an empirical application we find that the multiple indicator solution method and the hybrid model approach provide similar results in terms of welfare estimates, although the multiple indicator solution method is more parsimonious and notably easier to implement. The empirical results open a path to explore the performance of this method when endogeneity is thought to have a different cause or under a different set of indicators. Copyright © 2018 Elsevier B.V. All rights reserved.

Assessment of representational competence in kinematics

NASA Astrophysics Data System (ADS)

Klein, P.; Müller, A.; Kuhn, J.

2017-06-01

A two-tier instrument for representational competence in the field of kinematics (KiRC) is presented, designed for a standard (1st year) calculus-based introductory mechanics course. It comprises 11 multiple choice (MC) and 7 multiple true-false (MTF) questions involving multiple representational formats, such as graphs, pictures, and formal (mathematical) expressions (1st tier). Furthermore, students express their answer confidence for selected items, providing additional information (2nd tier). Measurement characteristics of KiRC were assessed in a validation sample (pre- and post-test, N =83 and N =46 , respectively), including usefulness for measuring learning gain. Validity is checked by interviews and by benchmarking KiRC against related measures. Values for item difficulty, discrimination, and consistency are in the desired ranges; in particular, a good reliability was obtained (KR 20 =0.86 ). Confidence intervals were computed and a replication study yielded values within the latter. For practical and research purposes, KiRC as a diagnostic tool goes beyond related extant instruments both for the representational formats (e.g., mathematical expressions) and for the scope of content covered (e.g., choice of coordinate systems). Together with the satisfactory psychometric properties it appears a versatile and reliable tool for assessing students' representational competency in kinematics (and of its potential change). Confidence judgments add further information to the diagnostic potential of the test, in particular for representational misconceptions. Moreover, we present an analytic result for the question—arising from guessing correction or educational considerations—of how the total effect size (Cohen's d ) varies upon combination of two test components with known individual effect sizes, and then discuss the results in the case of KiRC (MC and MTF combination). The introduced method of test combination analysis can be applied to any test comprising two components for the purpose of finding effect size ranges.

Developing Multiple Choice Tests: Tips & Techniques

ERIC Educational Resources Information Center

McCowan, Richard J.

1999-01-01

Item writing is a major responsibility of trainers. Too often, qualified staff who prepare lessons carefully and teach conscientiously use inadequate tests that do not validly reflect the true level of trainee achievement. This monograph describes techniques for constructing multiple-choice items that measure student performance accurately. It…

No Computer Left Behind

ERIC Educational Resources Information Center

Cohen, Daniel J.; Rosenzweig, Roy

2006-01-01

The combination of the Web and the cell phone forecasts the end of the inexpensive technologies of multiple-choice tests and grading machines. These technological developments are likely to bring the multiple-choice test to the verge of obsolescence, mounting a substantial challenge to the presentation of history and other disciplines.

Can an educational podcast improve the ability of parents of primary school children to assess the reliability of claims made about the benefits and harms of treatments: study protocol for a randomised controlled trial.

PubMed

Semakula, Daniel; Nsangi, Allen; Oxman, Matt; Austvoll-Dahlgren, Astrid; Rosenbaum, Sarah; Kaseje, Margaret; Nyirazinyoye, Laetitia; Fretheim, Atle; Chalmers, Iain; Oxman, Andrew D; Sewankambo, Nelson K

2017-01-21

Claims made about the effects of treatments are very common in the media and in the population more generally. The ability of individuals to understand and assess such claims can affect their decisions and health outcomes. Many people in both low- and high-income countries have inadequate aptitude to assess information about the effects of treatments. As part of the Informed Healthcare Choices project, we have prepared a series of podcast episodes to help improve people's ability to assess claims made about treatment effects. We will evaluate the effect of the Informed Healthcare Choices podcast on people's ability to assess claims made about the benefits and harms of treatments. Our study population will be parents of primary school children in schools with limited educational and financial resources in Uganda. This will be a two-arm, parallel-group, individual-randomised trial. We will randomly allocate consenting participants who meet the inclusion criteria for the trial to either listen to nine episodes of the Informed Healthcare Choices podcast (intervention) or to listen to nine typical public service announcements about health issues (control). Each podcast includes a story about a treatment claim, a message about one key concept that we believe is important for people to be able to understand to assess treatment claims, an explanation of how that concept applies to the claim, and a second example illustrating the concept. We designed the Claim Evaluation Tools to measure people's ability to apply key concepts related to assessing claims made about the effects of treatments and making informed health care choices. The Claim Evaluation Tools that we will use include multiple-choice questions addressing each of the nine concepts covered by the podcast. Using the Claim Evaluation Tools, we will measure two primary outcomes: (1) the proportion that 'pass', based on an absolute standard and (2) the average score. As far as we are aware this is the first randomised trial to assess the use of mass media to promote understanding of the key concepts needed to judge claims made about the effects of treatments. Pan African Clinical Trials Registry, PACTR201606001676150. Registered on 12 June 2016. http://www.pactr.org/ATMWeb/appmanager/atm/atmregistry?dar=true&tNo=PACTR201606001676150 .

Science Library of Test Items. Volume Twenty. A Collection of Multiple Choice Test Items Relating Mainly to Physics, 1.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Science Library of Test Items. Volume Twenty-One. A Collection of Multiple Choice Test Items Relating Mainly to Physics, 2.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Science Library of Test Items. Volume Twenty-Two. A Collection of Multiple Choice Test Items Relating Mainly to Skills.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

The Test of Basic Mechanics Conceptual Understanding (bMCU): Using Rasch Analysis to Develop and Evaluate an Efficient Multiple Choice Test on Newton's Mechanics

ERIC Educational Resources Information Center

Hofer, Sarah I.; Schumacher, Ralph; Rubin, Herbert

2017-01-01

Background: Valid assessment of the understanding of Newton's mechanics is highly relevant to both physics classrooms and research. Several tests have been developed. What remains missing, however, is an efficient and fair test of conceptual understanding that is adapted to the content taught to secondary school students and that can be validly…

Comparing Students With and Without Reading Difficulties on Reading Comprehension Assessments: A Meta-Analysis.

PubMed

Collins, Alyson A; Lindström, Esther R; Compton, Donald L

Researchers have increasingly investigated sources of variance in reading comprehension test scores, particularly with students with reading difficulties (RD). The purpose of this meta-analysis was to determine if the achievement gap between students with RD and typically developing (TD) students varies as a function of different reading comprehension response formats (e.g., multiple choice, cloze). A systematic literature review identified 82 eligible studies. All studies administered reading comprehension assessments to students with RD and TD students in Grades K-12. Hedge's g standardized mean difference effect sizes were calculated, and random effects robust variance estimation techniques were used to aggregate average weighted effect sizes for each response format. Results indicated that the achievement gap between students with RD and TD students was larger for some response formats (e.g., picture selection ES g = -1.80) than others (e.g., retell ES g = -0.60). Moreover, for multiple-choice, cloze, and open-ended question response formats, single-predictor metaregression models explored potential moderators of heterogeneity in effect sizes. No clear patterns, however, emerged in regard to moderators of heterogeneity in effect sizes across response formats. Findings suggest that the use of different response formats may lead to variability in the achievement gap between students with RD and TD students.

What are they thinking? Automated analysis of student writing about acid-base chemistry in introductory biology.

PubMed

Haudek, Kevin C; Prevost, Luanna B; Moscarella, Rosa A; Merrill, John; Urban-Lurain, Mark

2012-01-01

Students' writing can provide better insight into their thinking than can multiple-choice questions. However, resource constraints often prevent faculty from using writing assessments in large undergraduate science courses. We investigated the use of computer software to analyze student writing and to uncover student ideas about chemistry in an introductory biology course. Students were asked to predict acid-base behavior of biological functional groups and to explain their answers. Student explanations were rated by two independent raters. Responses were also analyzed using SPSS Text Analysis for Surveys and a custom library of science-related terms and lexical categories relevant to the assessment item. These analyses revealed conceptual connections made by students, student difficulties explaining these topics, and the heterogeneity of student ideas. We validated the lexical analysis by correlating student interviews with the lexical analysis. We used discriminant analysis to create classification functions that identified seven key lexical categories that predict expert scoring (interrater reliability with experts = 0.899). This study suggests that computerized lexical analysis may be useful for automatically categorizing large numbers of student open-ended responses. Lexical analysis provides instructors unique insights into student thinking and a whole-class perspective that are difficult to obtain from multiple-choice questions or reading individual responses.

What Are They Thinking? Automated Analysis of Student Writing about Acid–Base Chemistry in Introductory Biology

PubMed Central

Haudek, Kevin C.; Prevost, Luanna B.; Moscarella, Rosa A.; Merrill, John; Urban-Lurain, Mark

2012-01-01

Students’ writing can provide better insight into their thinking than can multiple-choice questions. However, resource constraints often prevent faculty from using writing assessments in large undergraduate science courses. We investigated the use of computer software to analyze student writing and to uncover student ideas about chemistry in an introductory biology course. Students were asked to predict acid–base behavior of biological functional groups and to explain their answers. Student explanations were rated by two independent raters. Responses were also analyzed using SPSS Text Analysis for Surveys and a custom library of science-related terms and lexical categories relevant to the assessment item. These analyses revealed conceptual connections made by students, student difficulties explaining these topics, and the heterogeneity of student ideas. We validated the lexical analysis by correlating student interviews with the lexical analysis. We used discriminant analysis to create classification functions that identified seven key lexical categories that predict expert scoring (interrater reliability with experts = 0.899). This study suggests that computerized lexical analysis may be useful for automatically categorizing large numbers of student open-ended responses. Lexical analysis provides instructors unique insights into student thinking and a whole-class perspective that are difficult to obtain from multiple-choice questions or reading individual responses. PMID:22949425

Multiple-Choice Exams: An Obstacle for Higher-Level Thinking in Introductory Science Classes

PubMed Central

Stanger-Hall, Kathrin F.

2012-01-01

Learning science requires higher-level (critical) thinking skills that need to be practiced in science classes. This study tested the effect of exam format on critical-thinking skills. Multiple-choice (MC) testing is common in introductory science courses, and students in these classes tend to associate memorization with MC questions and may not see the need to modify their study strategies for critical thinking, because the MC exam format has not changed. To test the effect of exam format, I used two sections of an introductory biology class. One section was assessed with exams in the traditional MC format, the other section was assessed with both MC and constructed-response (CR) questions. The mixed exam format was correlated with significantly more cognitively active study behaviors and a significantly better performance on the cumulative final exam (after accounting for grade point average and gender). There was also less gender-bias in the CR answers. This suggests that the MC-only exam format indeed hinders critical thinking in introductory science classes. Introducing CR questions encouraged students to learn more and to be better critical thinkers and reduced gender bias. However, student resistance increased as students adjusted their perceptions of their own critical-thinking abilities. PMID:22949426

Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables

PubMed Central

Kennerley, Steven W.; Wallis, Jonathan D.

2009-01-01

Damage to the frontal lobe can cause severe decision-making impairments. A mechanism that may underlie this is that neurons in the frontal cortex encode many variables that contribute to the valuation of a choice, such as its costs, benefits and probability of success. However, optimal decision-making requires that one considers these variables, not only when faced with the choice, but also when evaluating the outcome of the choice, in order to adapt future behaviour appropriately. To examine the role of the frontal cortex in encoding the value of different choice outcomes, we simultaneously recorded the activity of multiple single neurons in the anterior cingulate cortex (ACC), orbitofrontal cortex (OFC) and lateral prefrontal cortex (LPFC) while subjects evaluated the outcome of choices involving manipulations of probability, payoff and cost. Frontal neurons encoded many of the parameters that enabled the calculation of the value of these variables, including the onset and offset of reward and the amount of work performed, and often encoded the value of outcomes across multiple decision variables. In addition, many neurons encoded both the predicted outcome during the choice phase of the task as well as the experienced outcome in the outcome phase of the task. These patterns of selectivity were more prevalent in ACC relative to OFC and LPFC. These results support a role for the frontal cortex, principally ACC, in selecting between choice alternatives and evaluating the outcome of that selection thereby ensuring that choices are optimal and adaptive. PMID:19453638

Sources of Inspiration: The role of significant persons in young people's choice of science in higher education

NASA Astrophysics Data System (ADS)

Sjaastad, Jørgen

2012-07-01

The objectives of this article were to investigate to which extent and in what ways persons influence students' choice of science, technology, engineering, and mathematics (STEM) in tertiary education, and to assess the suitability of an analytical framework for describing this influence. In total, 5,007 Norwegian STEM students completed a questionnaire including multiple-choice as well as open-ended questions about sources of inspiration for their educational choice. Using the conceptualisation of significant persons suggested by Woelfel and Haller, the respondents' descriptions of parents and teachers are presented in order to elaborate on the different ways these significant persons influence a STEM-related educational choice. Parents engaged in STEM themselves are models, making the choice of STEM familiar, and they help youngsters define themselves through conversation and support, thus being definers. Teachers are models by displaying how STEM might bring fulfilment in someone's life and by giving pupils a positive experience with the subjects. They help young people discover their STEM abilities, thus being definers. Celebrities are reported to have minor influence on STEM-related educational choices. Both qualitative and quantitative analyses indicate that interpersonal relationships are key factors in order to inspire and motivate a choice of STEM education. Implications for recruitment issues and for research on interpersonal influence are discussed. It is suggested that initiatives to increase recruitment to STEM might be aimed at parents and other persons in interpersonal relationships with youth as a target group.

A Comparison of Alternate-Choice and True-False Item Forms Used in Classroom Examinations.

ERIC Educational Resources Information Center

Maihoff, N. A.; Mehrens, Wm. A.

A comparison is presented of alternate-choice and true-false item forms used in an undergraduate natural science course. The alternate-choice item is a modified two-choice multiple-choice item in which the two responses are included within the question stem. This study (1) compared the difficulty level, discrimination level, reliability, and…

Modeling Incorrect Responses to Multiple-Choice Items with Multilinear Formula Score Theory.

ERIC Educational Resources Information Center

Drasgow, Fritz; And Others

This paper addresses the information revealed in incorrect option selection on multiple choice items. Multilinear Formula Scoring (MFS), a theory providing methods for solving psychological measurement problems of long standing, is first used to estimate option characteristic curves for the Armed Services Vocational Aptitude Battery Arithmetic…

Introducing Standardized EFL/ESL Exams

ERIC Educational Resources Information Center

Laborda, Jesus Garcia

2007-01-01

This article presents the features, and a brief comparison, of some of the most well-known high-stakes exams. They are classified in the following fashion: tests that only include multiple-choice questions, tests that include writing and multiple-choice questions, and tests that include speaking questions. The tests reviewed are: BULATS, IELTS,…

Further Support for Changing Multiple-Choice Answers.

ERIC Educational Resources Information Center

Fabrey, Lawrence J.; Case, Susan M.

1985-01-01

The effect on test scores of changing answers to multiple-choice questions was studied and compared to earlier research. The current setting was a nationally administered, in-training, specialty examination for medical residents in obstetrics and gynecology. Both low and high scorers improved their scores when they changed answers. (SW)

Cognitive Diagnostic Models for Tests with Multiple-Choice and Constructed-Response Items

ERIC Educational Resources Information Center

Kuo, Bor-Chen; Chen, Chun-Hua; Yang, Chih-Wei; Mok, Magdalena Mo Ching

2016-01-01

Traditionally, teachers evaluate students' abilities via their total test scores. Recently, cognitive diagnostic models (CDMs) have begun to provide information about the presence or absence of students' skills or misconceptions. Nevertheless, CDMs are typically applied to tests with multiple-choice (MC) items, which provide less diagnostic…

High School Students' Concepts of Acids and Bases.

ERIC Educational Resources Information Center

Ross, Bertram H. B.

An investigation of Ontario high school students' understanding of acids and bases with quantitative and qualitative methods revealed misconceptions. A concept map, based on the objectives of the Chemistry Curriculum Guideline, generated multiple-choice items and interview questions. The multiple-choice test was administered to 34 grade 12…

Samejima Items in Multiple-Choice Tests: Identification and Implications

ERIC Educational Resources Information Center

Rahman, Nazia

2013-01-01

Samejima hypothesized that non-monotonically increasing item response functions (IRFs) of ability might occur for multiple-choice items (referred to here as "Samejima items") if low ability test takers with some, though incomplete, knowledge or skill are drawn to a particularly attractive distractor, while very low ability test takers…

Difficulty and Discriminability of Introductory Psychology Test Items.

ERIC Educational Resources Information Center

Scialfa, Charles; Legare, Connie; Wenger, Larry; Dingley, Louis

2001-01-01

Analyzes multiple-choice questions provided in test banks for introductory psychology textbooks. Study 1 offered a consistent picture of the objective difficulty of multiple-choice tests for introductory psychology students, while both studies 1 and 2 indicated that test items taken from commercial test banks have poor psychometric properties.…

Diagnosing Secondary Students' Misconceptions of Photosynthesis and Respiration in Plants Using a Two-Tier Multiple Choice Instrument.

ERIC Educational Resources Information Center

Haslam, Filocha; Treagust, David F.

1987-01-01

Describes a multiple-choice instrument that reliably and validly diagnoses secondary students' understanding of photosynthesis and respiration in plants. Highlights the consistency of students' misconceptions across secondary levels and indicates a high percentage of students have misconceptions regarding plant physiology. (CW)

DOE Office of Scientific and Technical Information (OSTI.GOV)

Pollesch, N.; Dale, V. H.

In order to aid in transition towards operations that promote sustainability goals, researchers and stakeholders use sustainability assessments. Although assessments take various forms, many utilize diverse sets of indicators that can number anywhere from two to over 2000. Indices, composite indicators, or aggregate values are used to simplify high dimensional and complex data sets and to clarify assessment results. Although the choice of aggregation function is a key component in the development of the assessment, there are few examples to be found in literature to guide appropriate aggregation function selection. This paper develops a connection between the mathematical study ofmore » aggregation functions and sustainability assessment in order to aid in providing criteria for aggregation function selection. Relevant mathematical properties of aggregation functions are presented and interpreted. Lastly, we provide cases of these properties and their relation to previous sustainability assessment research. Examples show that mathematical aggregation properties can be used to address the topics of compensatory behavior and weak versus strong sustainability, aggregation of data under varying units of measurements, multiple site multiple indicator aggregation, and the determination of error bounds in aggregate output for normalized and non-normalized indicator measures.« less

Anticipation and Choice Heuristics in the Dynamic Consumption of Pain Relief

PubMed Central

Story, Giles W.; Vlaev, Ivo; Dayan, Peter; Seymour, Ben; Darzi, Ara; Dolan, Raymond J.

2015-01-01

Humans frequently need to allocate resources across multiple time-steps. Economic theory proposes that subjects do so according to a stable set of intertemporal preferences, but the computational demands of such decisions encourage the use of formally less competent heuristics. Few empirical studies have examined dynamic resource allocation decisions systematically. Here we conducted an experiment involving the dynamic consumption over approximately 15 minutes of a limited budget of relief from moderately painful stimuli. We had previously elicited the participants’ time preferences for the same painful stimuli in one-off choices, allowing us to assess self-consistency. Participants exhibited three characteristic behaviors: saving relief until the end, spreading relief across time, and early spending, of which the last was markedly less prominent. The likelihood that behavior was heuristic rather than normative is suggested by the weak correspondence between one-off and dynamic choices. We show that the consumption choices are consistent with a combination of simple heuristics involving early-spending, spreading or saving of relief until the end, with subjects predominantly exhibiting the last two. PMID:25793302

Anticipation and choice heuristics in the dynamic consumption of pain relief.

PubMed

Story, Giles W; Vlaev, Ivo; Dayan, Peter; Seymour, Ben; Darzi, Ara; Dolan, Raymond J

2015-03-01

Humans frequently need to allocate resources across multiple time-steps. Economic theory proposes that subjects do so according to a stable set of intertemporal preferences, but the computational demands of such decisions encourage the use of formally less competent heuristics. Few empirical studies have examined dynamic resource allocation decisions systematically. Here we conducted an experiment involving the dynamic consumption over approximately 15 minutes of a limited budget of relief from moderately painful stimuli. We had previously elicited the participants' time preferences for the same painful stimuli in one-off choices, allowing us to assess self-consistency. Participants exhibited three characteristic behaviors: saving relief until the end, spreading relief across time, and early spending, of which the last was markedly less prominent. The likelihood that behavior was heuristic rather than normative is suggested by the weak correspondence between one-off and dynamic choices. We show that the consumption choices are consistent with a combination of simple heuristics involving early-spending, spreading or saving of relief until the end, with subjects predominantly exhibiting the last two.

The Association among Emotions and Food Choices in First-Year College Students Using mobile-Ecological Momentary Assessments.

PubMed

Ashurst, Jessica; van Woerden, Irene; Dunton, Genevieve; Todd, Michael; Ohri-Vachaspati, Punam; Swan, Pamela; Bruening, Meg

2018-05-02

Studies have examined the associations between emotions and overeating but have only rarely considered associations between emotions and specific food choices. The purpose of this secondary data analysis was to use mobile ecological momentary assessments (mEMAs) to examine associations between emotions and food choices among first-year college students living in residence halls. Using an intensive repeated-measures design, mEMAs were used to assess concurrent emotions and food choices in a racially/ethnically diverse sample of first-year college students (n = 663). Emotions were categorized as negative (sad, stressed, tired), positive (happy, energized, relaxed), and apathetic (bored, meh). Assessments were completed multiple times per day on four quasi-randomly selected days (three random weekdays and one random weekend day) during a 7-day period using random prompt times. Generalized estimating equations (GEE) were used to examine between- and within-person associations of emotional status with a variety of healthy and unhealthy food choices (sweets, salty snacks/fried foods, fruits/vegetables, pizza/fast food, sandwiches/wraps, meats/proteins, pasta/rice, cereals), adjusting for gender, day of week, and time of day, accounting for within-person dependencies among repeated measurements of eating behavior. At the between-person level, participants who reported positive emotions more frequently compared to others consumed meats/proteins more often (OR = 1.8; 99% CI = 1.2, 2.8). At the within-person level, on occasions when any negative emotion was reported (versus no negative emotion reported) participants were more likely to consume meats/proteins (OR = 1.5, 99% CI = 1.0, 2.1); on occasions when any positive emotion was reported as compared to occasions with no positive emotions, participants were more likely to consume sweets (OR = 1.7, 99% CI = 1.1, 2.6), but less likely to consume pizza/fast food (OR = 0.6, 99% CI = 0.4, 1.0). Negative and positive emotions were significantly associated with food choices. mEMA methodology provides a unique opportunity to examine these associations within and between people, providing insights for individual and population-level interventions. These findings can be used to guide future longitudinal studies and to develop and test interventions that encourage healthy food choices among first-year college students and ultimately reduce the risk of weight gain.

Delay and probability discounting of multiple commodities in smokers and never-smokers using multiple-choice tasks.

PubMed

Poltavski, Dmitri V; Weatherly, Jeffrey N

2013-12-01

The purpose of the present study was to investigate temporal and probabilistic discounting in smokers and never-smokers, across a number of commodities, using a multiple-choice method. One hundred and eighty-two undergraduate university students, of whom 90 had never smoked, 73 were self-reported light smokers (<10 cigarettes/day), and 17 were heavy smokers (10+cigarettes/day), completed computerized batteries of delay and probability discounting questions pertaining to a total of eight commodities and administered in a multiple-choice format. In addition to cigarettes, monetary rewards, and health outcomes, the tasks included novel commodities such as ideal dating partner and retirement income. The results showed that heavy smokers probability discounted commodities at a significantly shallower rate than never-smokers, suggesting greater risk-taking. No effect of smoking status was observed for delay discounting questions. The only commodity that was probability discounted significantly less than others was 'finding an ideal dating partner'. The results suggest that probability discounting tasks using the multiple-choice format can discriminate between non-abstaining smokers and never-smokers and could be further explored in the context of behavioral and drug addictions.

Developing a Web-Based Mechanism for Assessing Teacher Science Content Knowledge

NASA Astrophysics Data System (ADS)

Byers, Al; Koba, Susan; Sherman, Greg; Scheppke, Joan; Bolus, Roger

2011-04-01

The National Science Teachers Association (NSTA) recently launched a comprehensive electronic professional development (e-PD) online portal, the NSTA Learning Center. This support site for educators currently includes over 6,000 e-PD resources and opportunities available on-demand, as well as various tools designed to help educators maximize the effectiveness of using NSTA resources. One tool, the PD Indexer, helps teachers identify their own areas of content strengths and weaknesses by selecting content-specific assessments. Individual NSTA resources are recommended based on assessment outcomes. This paper presents a detailed description of the procedures employed by NSTA to develop valid and reliable PD Indexer content-specific multiple-choice assessment items.

Small Changes: Using Assessment to Direct Instructional Practices in Large-Enrollment Biochemistry Courses.

PubMed

Xu, Xiaoying; Lewis, Jennifer E; Loertscher, Jennifer; Minderhout, Vicky; Tienson, Heather L

2017-01-01

Multiple-choice assessments provide a straightforward way for instructors of large classes to collect data related to student understanding of key concepts at the beginning and end of a course. By tracking student performance over time, instructors receive formative feedback about their teaching and can assess the impact of instructional changes. The evidence of instructional effectiveness can in turn inform future instruction, and vice versa. In this study, we analyzed student responses on an optimized pretest and posttest administered during four different quarters in a large-enrollment biochemistry course. Student performance and the effect of instructional interventions related to three fundamental concepts-hydrogen bonding, bond energy, and pK a -were analyzed. After instructional interventions, a larger proportion of students demonstrated knowledge of these concepts compared with data collected before instructional interventions. Student responses trended from inconsistent to consistent and from incorrect to correct. The instructional effect was particularly remarkable for the later three quarters related to hydrogen bonding and bond energy. This study supports the use of multiple-choice instruments to assess the effectiveness of instructional interventions, especially in large classes, by providing instructors with quick and reliable feedback on student knowledge of each specific fundamental concept. © 2017 X. Xu et al. CBE—Life Sciences Education © 2017 The American Society for Cell Biology. This article is distributed by The American Society for Cell Biology under license from the author(s). It is available to the public under an Attribution–Noncommercial–Share Alike 3.0 Unported Creative Commons License (http://creativecommons.org/licenses/by-nc-sa/3.0).

First Results from the Test Of Astronomy STandards (TOAST) Assessment Instrument

NASA Astrophysics Data System (ADS)

Slater, Stephanie

2009-01-01

Considerable effort in the astronomy education research over the past several years has focused on developing assessment tools in the form of multiple-choice conceptual diagnostics and content knowledge surveys. This has been critically important in advancing astronomy as a sub-discipline of physics education research, allowing researchers to establish the initial knowledge state of students as well as to attempt to measure some of the impacts of innovative instructional interventions. Before now, few of the existing instruments were constructed upon a solid list of clearly articulated and widely agreed upon learning objectives. Moving beyond the 10-year old Astronomy Diagnostics Test, we have developed and validated a new assessment instrument that is tightly aligned to the consensus learning goals stated by the American Astronomical Society - Chair's Conference on ASTRO 101, the American Association of the Advancement of Science's Project 2061 Benchmarks, and the National Research Council's National Science Education Standards. Researchers from the Cognition in Astronomy, Physics and Earth sciences Research (CAPER) Team at the University of Wyoming's Science and Math Teaching Center (UWYO SMTC) designed a criterion-referenced assessment tool, called the Test Of Astronomy STandards (TOAST). Through iterative development, this multiple-choice instrument has a high degree of reliability and validity for instructors and researchers needing information on students’ initial knowledge state at the beginning of a course and can be used, in aggregate, to help measure the impact of course-length duration instructional strategies for undergraduate science survey courses with learning goals tightly aligned to the consensus goals of the astronomy education community.

Approaches to learning for the ANZCA Final Examination and validation of the revised Study Process Questionnaire in specialist medical training.

PubMed

Weller, J M; Henning, M; Civil, N; Lavery, L; Boyd, M J; Jolly, B

2013-09-01

When evaluating assessments, the impact on learning is often overlooked. Approaches to learning can be deep, surface and strategic. To provide insights into exam quality, we investigated the learning approaches taken by trainees preparing for the Australian and New Zealand College of Anaesthetists (ANZCA) Final Exam. The revised two-factor Study Process Questionnaire (R-SPQ-2F) was modified and validated for this context and was administered to ANZCA advanced trainees. Additional questions were asked about perceived value for anaesthetic practice, study time and approaches to learning for each exam component. Overall, 236 of 690 trainees responded (34%). Responses indicated both deep and surface approaches to learning with a clear preponderance of deep approaches. The anaesthetic viva was valued most highly and the multiple choice question component the least. Despite this, respondents spent the most time studying for the multiple choice questions. The traditionally low short answer questions pass rate could not be explained by limited study time, perceived lack of value or study approaches. Written responses suggested that preparation for multiple choice questions was characterised by a surface approach, with rote memorisation of past questions. Minimal reference was made to the ANZCA syllabus as a guide for learning. These findings indicate that, although trainees found the exam generally relevant to practice and adopted predominantly deep learning approaches, there was considerable variation between the four components. These results provide data with which to review the existing ANZCA Final Exam and comparative data for future studies of the revisions to the ANZCA curriculum and exam process.

Multiple choice questions can be designed or revised to challenge learners' critical thinking.

PubMed

Tractenberg, Rochelle E; Gushta, Matthew M; Mulroney, Susan E; Weissinger, Peggy A

2013-12-01

Multiple choice (MC) questions from a graduate physiology course were evaluated by cognitive-psychology (but not physiology) experts, and analyzed statistically, in order to test the independence of content expertise and cognitive complexity ratings of MC items. Integration of higher order thinking into MC exams is important, but widely known to be challenging-perhaps especially when content experts must think like novices. Expertise in the domain (content) may actually impede the creation of higher-complexity items. Three cognitive psychology experts independently rated cognitive complexity for 252 multiple-choice physiology items using a six-level cognitive complexity matrix that was synthesized from the literature. Rasch modeling estimated item difficulties. The complexity ratings and difficulty estimates were then analyzed together to determine the relative contributions (and independence) of complexity and difficulty to the likelihood of correct answers on each item. Cognitive complexity was found to be statistically independent of difficulty estimates for 88 % of items. Using the complexity matrix, modifications were identified to increase some item complexities by one level, without affecting the item's difficulty. Cognitive complexity can effectively be rated by non-content experts. The six-level complexity matrix, if applied by faculty peer groups trained in cognitive complexity and without domain-specific expertise, could lead to improvements in the complexity targeted with item writing and revision. Targeting higher order thinking with MC questions can be achieved without changing item difficulties or other test characteristics, but this may be less likely if the content expert is left to assess items within their domain of expertise.

A surgical simulation curriculum for senior medical students based on TeamSTEPPS.

PubMed

Meier, Andreas H; Boehler, Maggie L; McDowell, Chris M; Schwind, Cathy; Markwell, Steve; Roberts, Nicole K; Sanfey, Hilary

2012-08-01

To investigate whether the existing Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) curriculum can effectively teach senior medical students team skills. DESIGN Single-group preintervention and postintervention study. We integrated a TeamSTEPPS module into our existing resident readiness elective. The curriculum included interactive didactic sessions, discussion groups, role-plays, and videotaped immersive simulation scenarios. Improvement of self-assessment scores, multiple-choice examination scores, and performance ratings of videotaped simulation scenarios before and after intervention. The videos were rated by masked reviewers on the basis of a global rating instrument (TeamSTEPPS) and a more detailed nontechnical skills evaluation tool(NOTECHS). Seventeen students participated and completed the study. The self-evaluation scores improved from 12.76 to 16.06 (P < .001). The increase was significant for all of the TeamSTEPPS competencies and highest for leadership skills (from 2.2 to 3.2; P < .001). The multiple-choice score rose from 84.9% to 94.1% (P < .01). The postintervention video ratings were significantly higher for both instruments (TeamSTEPPS, from 2.99 to 3.56; P < .01; and NOTECHS, from 4.07 to 4.59; P < .001). The curriculum led to improved self-evaluation and multiple-choice scores as well as improved team skills during simulated immersive patient encounters. The TeamSTEPPS framework may be suitable for teaching medical students teamwork concepts and improving their competencies. Larger studies using this framework should be considered to further evaluate the generalizability of our results and the effectiveness of TeamSTEPPS for medical students.

Step by Step: Biology Undergraduates' Problem-Solving Procedures during Multiple-Choice Assessment.

PubMed

Prevost, Luanna B; Lemons, Paula P

2016-01-01

This study uses the theoretical framework of domain-specific problem solving to explore the procedures students use to solve multiple-choice problems about biology concepts. We designed several multiple-choice problems and administered them on four exams. We trained students to produce written descriptions of how they solved the problem, and this allowed us to systematically investigate their problem-solving procedures. We identified a range of procedures and organized them as domain general, domain specific, or hybrid. We also identified domain-general and domain-specific errors made by students during problem solving. We found that students use domain-general and hybrid procedures more frequently when solving lower-order problems than higher-order problems, while they use domain-specific procedures more frequently when solving higher-order problems. Additionally, the more domain-specific procedures students used, the higher the likelihood that they would answer the problem correctly, up to five procedures. However, if students used just one domain-general procedure, they were as likely to answer the problem correctly as if they had used two to five domain-general procedures. Our findings provide a categorization scheme and framework for additional research on biology problem solving and suggest several important implications for researchers and instructors. © 2016 L. B. Prevost and P. P. Lemons. CBE—Life Sciences Education © 2016 The American Society for Cell Biology. This article is distributed by The American Society for Cell Biology under license from the author(s). It is available to the public under an Attribution–Noncommercial–Share Alike 3.0 Unported Creative Commons License (http://creativecommons.org/licenses/by-nc-sa/3.0).

Managing Disease Risks from Trade: Strategic Behavior with Many Choices and Price Effects.

PubMed

Chitchumnong, Piyayut; Horan, Richard D

2018-03-16

An individual's infectious disease risks, and hence the individual's incentives for risk mitigation, may be influenced by others' risk management choices. If so, then there will be strategic interactions among individuals, whereby each makes his or her own risk management decisions based, at least in part, on the expected decisions of others. Prior work has shown that multiple equilibria could arise in this setting, with one equilibrium being a coordination failure in which individuals make too few investments in protection. However, these results are largely based on simplified models involving a single management choice and fixed prices that may influence risk management incentives. Relaxing these assumptions, we find strategic interactions influence, and are influenced by, choices involving multiple management options and market price effects. In particular, we find these features can reduce or eliminate concerns about multiple equilibria and coordination failure. This has important policy implications relative to simpler models.

A One-Day Dental Faculty Workshop in Writing Multiple-Choice Questions: An Impact Evaluation.

PubMed

AlFaris, Eiad; Naeem, Naghma; Irfan, Farhana; Qureshi, Riaz; Saad, Hussain; Al Sadhan, Ra'ed; Abdulghani, Hamza Mohammad; Van der Vleuten, Cees

2015-11-01

Long training workshops on the writing of exam questions have been shown to be effective; however, the effectiveness of short workshops needs to be demonstrated. The aim of this study was to evaluate the impact of a one-day, seven-hour faculty development workshop at the College of Dentistry, King Saud University, Saudi Arabia, on the quality of multiple-choice questions (MCQs). Kirkpatrick's four-level evaluation model was used. Participants' satisfaction (Kirkpatrick's Level 1) was evaluated with a post-workshop questionnaire. A quasi-experimental, randomized separate sample, pretest-posttest design was used to assess the learning effect (Kirkpatrick's Level 2). To evaluate transfer of learning to practice (Kirkpatrick's Level 3), MCQs created by ten faculty members as a result of the training were assessed. To assess Kirkpatrick's Level 4 regarding institutional change, interviews with three key leaders of the school were conducted, coded, and analyzed. A total of 72 course directors were invited to and attended some part of the workshop; all 52 who attended the entire workshop completed the satisfaction form; and 22 of the 36 participants in the experimental group completed the posttest. The results showed that all 52 participants were highly satisfied with the workshop, and significant positive changes were found in the faculty members' knowledge and the quality of their MCQs with effect sizes of 0.7 and 0.28, respectively. At the institutional level, the interviews demonstrated positive structural changes in the school's assessment system. Overall, this one-day item-writing faculty workshop resulted in positive changes at all four of Kirkpatrick's levels; these effects suggest that even a short training session can improve a dental school's assessment of its students.

Measuring ability to assess claims about treatment effects: a latent trait analysis of items from the 'Claim Evaluation Tools' database using Rasch modelling.

PubMed

Austvoll-Dahlgren, Astrid; Guttersrud, Øystein; Nsangi, Allen; Semakula, Daniel; Oxman, Andrew D

2017-05-25

The Claim Evaluation Tools database contains multiple-choice items for measuring people's ability to apply the key concepts they need to know to be able to assess treatment claims. We assessed items from the database using Rasch analysis to develop an outcome measure to be used in two randomised trials in Uganda. Rasch analysis is a form of psychometric testing relying on Item Response Theory. It is a dynamic way of developing outcome measures that are valid and reliable. To assess the validity, reliability and responsiveness of 88 items addressing 22 key concepts using Rasch analysis. We administrated four sets of multiple-choice items in English to 1114 people in Uganda and Norway, of which 685 were children and 429 were adults (including 171 health professionals). We scored all items dichotomously. We explored summary and individual fit statistics using the RUMM2030 analysis package. We used SPSS to perform distractor analysis. Most items conformed well to the Rasch model, but some items needed revision. Overall, the four item sets had satisfactory reliability. We did not identify significant response dependence between any pairs of items and, overall, the magnitude of multidimensionality in the data was acceptable. The items had a high level of difficulty. Most of the items conformed well to the Rasch model's expectations. Following revision of some items, we concluded that most of the items were suitable for use in an outcome measure for evaluating the ability of children or adults to assess treatment claims. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Assessing learning outcomes and cost effectiveness of an online sleep curriculum for medical students.

PubMed

Bandla, Hari; Franco, Rose A; Simpson, Deborah; Brennan, Kimberly; McKanry, Jennifer; Bragg, Dawn

2012-08-15

Sleep disorders are highly prevalent across all age groups but often remain undiagnosed and untreated, resulting in significant health consequences. To overcome an inadequacy of available curricula and learner and instructor time constraints, this study sought to determine if an online sleep medicine curriculum would achieve equivalent learner outcomes when compared with traditional, classroom-based, face-to-face instruction at equivalent costs. Medical students rotating on a required clinical clerkship received instruction in 4 core clinical sleep-medicine competency domains in 1 of 2 delivery formats: a single 2.5-hour face-to-face workshop or 4 asynchronous e-learning modules. Immediate learning outcomes were assessed in a subsequent clerkship using a multiple-choice examination and standardized patient station, with long-term outcomes assessed through analysis of students' patient write-ups for inclusion of sleep complaints and diagnoses before and after the intervention. Instructional costs by delivery format were tracked. Descriptive and inferential statistical analyses compared learning outcomes and costs by instructional delivery method (face-to-face versus e-learning). Face-to-face learners, compared with online learners, were more satisfied with instruction. Learning outcomes (i.e., multiple-choice examination, standardized patient encounter, patient write-up), as measured by short-term and long-term assessments, were roughly equivalent. Design, delivery, and learner-assessment costs by format were equivalent at the end of 1 year, due to higher ongoing teaching costs associated with face-to-face learning offsetting online development and delivery costs. Because short-term and long-term learner performance outcomes were roughly equivalent, based on delivery method, the cost effectiveness of online learning is an economically and educationally viable instruction platform for clinical clerkships.

Evaluation of a preschool nutrition education program based on the theory of multiple intelligences.

PubMed

Cason, K L

2001-01-01

This report describes the evaluation of a preschool nutrition education program based on the theory of multiple intelligences. Forty-six nutrition educators provided a series of 12 lessons to 6102 preschool-age children. The program was evaluated using a pretest/post-test design to assess differences in fruit and vegetable identification, healthy snack choices, willingness to taste foods, and eating behaviors. Subjects showed significant improvement in food identification and recognition, healthy snack identification, willingness to taste foods, and frequency of fruit, vegetable, meat, and dairy consumption. The evaluation indicates that the program was an effective approach for educating preschool children about nutrition.

"iBIM"--internet-based interactive modules: an easy and interesting learning tool for general surgery residents.

PubMed

Azer, Nader; Shi, Xinzhe; de Gara, Chris; Karmali, Shahzeer; Birch, Daniel W

2014-04-01

The increased use of information technology supports a resident- centred educational approach that promotes autonomy, flexibility and time management and helps residents to assess their competence, promoting self-awareness. We established a web-based e-learning tool to introduce general surgery residents to bariatric surgery and evaluate them to determine the most appropriate implementation strategy for Internet-based interactive modules (iBIM) in surgical teaching. Usernames and passwords were assigned to general surgery residents at the University of Alberta. They were directed to the Obesity101 website and prompted to complete a multiple-choice precourse test. Afterwards, they were able to access the interactive modules. Residents could review the course material as often as they wanted before completing a multiple-choice postcourse test and exit survey. We used paired t tests to assess the difference between pre- and postcourse scores. Out of 34 residents who agreed to participate in the project, 12 completed the project (35.3%). For these 12 residents, the precourse mean score was 50 ± 17.3 and the postcourse mean score was 67 ± 14 (p = 0.020). Most residents who participated in this study recommended using the iBIMs as a study tool for bariatric surgery. Course evaluation scores suggest this novel approach was successful in transferring knowledge to surgical trainees. Further development of this tool and assessment of implementation strategies will determine how iBIM in bariatric surgery may be integrated into the curriculum.

Demand Characteristics of Multiple-Choice Items.

ERIC Educational Resources Information Center

Diamond, James J.; Williams, David V.

Thirteen graduate students were asked to indicate for each of 24 multiple-choice items whether the item tested "recall of specific information," a "higher order skill," or "don't know." The students were also asked to state their general basis for judging the items. The 24 items had been previously classified according to Bloom's cognitive-skills…

Examining the Prediction of Reading Comprehension on Different Multiple-Choice Tests

ERIC Educational Resources Information Center

Andreassen, Rune; Braten, Ivar

2010-01-01

In this study, 180 Norwegian fifth-grade students with a mean age of 10.5 years were administered measures of word recognition skills, strategic text processing, reading motivation and working memory. Six months later, the same students were given three different multiple-choice reading comprehension measures. Based on three forced-order…

Written Justifications to Multiple-Choice Concept Questions during Active Learning in Class

ERIC Educational Resources Information Center

Koretsky, Milo D.; Brooks, Bill J.; Higgins, Adam Z.

2016-01-01

Increasingly, instructors of large, introductory STEM courses are having students actively engage during class by answering multiple-choice concept questions individually and in groups. This study investigates the use of a technology-based tool that allows students to answer such questions during class. The tool also allows the instructor to…

Violating Conventional Wisdom in Multiple Choice Test Construction

ERIC Educational Resources Information Center

Taylor, Annette Kujawski

2005-01-01

This research examined 2 elements of multiple-choice test construction, balancing the key and optimal number of options. In Experiment 1 the 3 conditions included a balanced key, overrepresentation of a and b responses, and overrepresentation of c and d responses. The results showed that error-patterns were independent of the key, reflecting…

Multiple-Choice Tests with Correction Allowed in Autism: An Excel Applet

ERIC Educational Resources Information Center

Martinez, Elisabetta Monari

2010-01-01

The valuation of academic achievements in students with severe language impairment is problematic if they also have difficulties in sustaining attention and in praxic skills. In severe autism all of these difficulties may occur together. Multiple-choice tests offer the advantage that simple praxic skills are required, allowing the tasks to be…

Automatic Scoring of Paper-and-Pencil Figural Responses. Research Report.

ERIC Educational Resources Information Center

Martinez, Michael E.; And Others

Large-scale testing is dominated by the multiple-choice question format. Widespread use of the format is due, in part, to the ease with which multiple-choice items can be scored automatically. This paper examines automatic scoring procedures for an alternative item type: figural response. Figural response items call for the completion or…

FormScanner: Open-Source Solution for Grading Multiple-Choice Exams

ERIC Educational Resources Information Center

Young, Chadwick; Lo, Glenn; Young, Kaisa; Borsetta, Alberto

2016-01-01

The multiple-choice exam remains a staple for many introductory physics courses. In the past, people have graded these by hand or even flaming needles. Today, one usually grades the exams with a form scanner that utilizes optical mark recognition (OMR). Several companies provide these scanners and particular forms, such as the eponymous…

Application of a Multidimensional Nested Logit Model to Multiple-Choice Test Items

ERIC Educational Resources Information Center

Bolt, Daniel M.; Wollack, James A.; Suh, Youngsuk

2012-01-01

Nested logit models have been presented as an alternative to multinomial logistic models for multiple-choice test items (Suh and Bolt in "Psychometrika" 75:454-473, 2010) and possess a mathematical structure that naturally lends itself to evaluating the incremental information provided by attending to distractor selection in scoring. One potential…

Semantic Similarity Measures for the Generation of Science Tests in Basque

ERIC Educational Resources Information Center

Aldabe, Itziar; Maritxalar, Montse

2014-01-01

The work we present in this paper aims to help teachers create multiple-choice science tests. We focus on a scientific vocabulary-learning scenario taking place in a Basque-language educational environment. In this particular scenario, we explore the option of automatically generating Multiple-Choice Questions (MCQ) by means of Natural Language…

Negatively-Worded Multiple Choice Questions: An Avoidable Threat to Validity

ERIC Educational Resources Information Center

Chiavaroli, Neville

2017-01-01

Despite the majority of MCQ writing guides discouraging the use of negatively-worded multiple choice questions (NWQs), they continue to be regularly used both in locally produced examinations and commercially available questions. There are several reasons why the use of NWQs may prove resistant to sound pedagogical advice. Nevertheless, systematic…

Instrument Formatting with Computer Data Entry in Mind.

ERIC Educational Resources Information Center

Boser, Judith A.; And Others

Different formats for four types of research items were studied for ease of computer data entry. The types were: (1) numeric response items; (2) individual multiple choice items; (3) multiple choice items with the same response items; and (4) card column indicator placement. Each of the 13 experienced staff members of a major university's Data…

Validation and Structural Analysis of the Kinematics Concept Test

ERIC Educational Resources Information Center

Lichtenberger, A.; Wagner, C.; Hofer, S. I.; Stem, E.; Vaterlaus, A.

2017-01-01

The kinematics concept test (KCT) is a multiple-choice test designed to evaluate students' conceptual understanding of kinematics at the high school level. The test comprises 49 multiple-choice items about velocity and acceleration, which are based on seven kinematic concepts and which make use of three different representations. In the first part…

A Practical Methodology for the Systematic Development of Multiple Choice Tests.

ERIC Educational Resources Information Center

Blumberg, Phyllis; Felner, Joel

Using Guttman's facet design analysis, four parallel forms of a multiple-choice test were developed. A mapping sentence, logically representing the universe of content of a basic cardiology course, specified the facets of the course and the semantic structural units linking them. The facets were: cognitive processes, disease priority, specific…

Delayed Instructional Feedback May Be More Effective, but Is This Contrary to Learners' Preferences?

ERIC Educational Resources Information Center

Lefevre, David; Cox, Benita

2017-01-01

This research investigates learners' preferences for the timing of feedback provided to multiple-choice questions within technology-based instruction, hitherto an area of little empirical attention. Digital materials are undergoing a period of renewed prominence within online learning and multiple-choice questions remain a common component. There…

Multiple-Choice Test Bias Due to Answering Strategy Variation.

ERIC Educational Resources Information Center

Frary, Robert B.; Giles, Mary B.

This paper describes the development and investigation of a new approach to determining the existence of bias in multiple-choice test scores. Previous work in this area has concentrated almost exclusively on bias attributable to specific test items or to differences in test score distributions across racial or ethnic groups. In contrast, the…

The Use of Management and Marketing Textbook Multiple-Choice Questions: A Case Study.

ERIC Educational Resources Information Center

Hampton, David R.; And Others

1993-01-01

Four management and four marketing professors classified multiple-choice questions in four widely adopted introductory textbooks according to the two levels of Bloom's taxonomy of educational objectives: knowledge and intellectual ability and skill. Inaccuracies may cause instructors to select questions that require less thinking than they intend.…

Visual Attention for Solving Multiple-Choice Science Problem: An Eye-Tracking Analysis

ERIC Educational Resources Information Center

Tsai, Meng-Jung; Hou, Huei-Tse; Lai, Meng-Lung; Liu, Wan-Yi; Yang, Fang-Ying

2012-01-01

This study employed an eye-tracking technique to examine students' visual attention when solving a multiple-choice science problem. Six university students participated in a problem-solving task to predict occurrences of landslide hazards from four images representing four combinations of four factors. Participants' responses and visual attention…

Piloting a Polychotomous Partial-Credit Scoring Procedure in a Multiple-Choice Test

ERIC Educational Resources Information Center

Tsopanoglou, Antonios; Ypsilandis, George S.; Mouti, Anna

2014-01-01

Multiple-choice (MC) tests are frequently used to measure language competence because they are quick, economical and straightforward to score. While degrees of correctness have been investigated for partially correct responses in combined-response MC tests, degrees of incorrectness in distractors and the role they play in determining the…

English 30, Part B: Reading. Questions Booklet. Grade 12 Diploma Examination, January 1997.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton. Student Evaluation Branch.

Intended for students taking the Grade 12 Diploma Examinations in English 30, this "questions booklet" presents 70 multiple choice test items based on 8 reading selections in the accompanying readings booklet. After instructions for students, the booklet presents the multiple choice items which test students' comprehension of the poetry,…

The Relationship of Expert-System Scored Constrained Free-Response Items to Multiple-Choice and Open-Ended Items.

ERIC Educational Resources Information Center

Bennett, Randy Elliot; And Others

1990-01-01

The relationship of an expert-system-scored constrained free-response item type to multiple-choice and free-response items was studied using data for 614 students on the College Board's Advanced Placement Computer Science (APCS) Examination. Implications for testing and the APCS test are discussed. (SLD)

Climbing Bloom's Taxonomy Pyramid: Lessons from a Graduate Histology Course

ERIC Educational Resources Information Center

Zaidi, Nikki B.; Hwang, Charles; Scott, Sara; Stallard, Stefanie; Purkiss, Joel; Hortsch, Michael

2017-01-01

Bloom's taxonomy was adopted to create a subject-specific scoring tool for histology multiple-choice questions (MCQs). This Bloom's Taxonomy Histology Tool (BTHT) was used to analyze teacher- and student-generated quiz and examination questions from a graduate level histology course. Multiple-choice questions using histological images were…

Computational Precision of Mental Inference as Critical Source of Human Choice Suboptimality.

PubMed

Drugowitsch, Jan; Wyart, Valentin; Devauchelle, Anne-Dominique; Koechlin, Etienne

2016-12-21

Making decisions in uncertain environments often requires combining multiple pieces of ambiguous information from external cues. In such conditions, human choices resemble optimal Bayesian inference, but typically show a large suboptimal variability whose origin remains poorly understood. In particular, this choice suboptimality might arise from imperfections in mental inference rather than in peripheral stages, such as sensory processing and response selection. Here, we dissociate these three sources of suboptimality in human choices based on combining multiple ambiguous cues. Using a novel quantitative approach for identifying the origin and structure of choice variability, we show that imperfections in inference alone cause a dominant fraction of suboptimal choices. Furthermore, two-thirds of this suboptimality appear to derive from the limited precision of neural computations implementing inference rather than from systematic deviations from Bayes-optimal inference. These findings set an upper bound on the accuracy and ultimate predictability of human choices in uncertain environments. Copyright © 2016 Elsevier Inc. All rights reserved.

Instructor perspectives of multiple-choice questions in summative assessment for novice programmers

NASA Astrophysics Data System (ADS)

Shuhidan, Shuhaida; Hamilton, Margaret; D'Souza, Daryl

2010-09-01

Learning to program is known to be difficult for novices. High attrition and high failure rates in foundation-level programming courses undertaken at tertiary level in Computer Science programs, are commonly reported. A common approach to evaluating novice programming ability is through a combination of formative and summative assessments, with the latter typically represented by a final examination. Preparation of such assessment is driven by instructor perceptions of student learning of programming concepts. This in turn may yield instructor perspectives of summative assessment that do not necessarily correlate with student expectations or abilities. In this article, we present results of our study around instructor perspectives of summative assessment for novice programmers. Both quantitative and qualitative data have been obtained via survey responses from programming instructors with varying teaching experience, and from novice student responses to targeted examination questions. Our findings highlight that most of the instructors believed that summative assessment is, and is meant to be, a valid measure of a student's ability to program. Most instructors further believed that Multiple-choice Questions (MCQs) provide a means of testing a low level of understanding, and a few added qualitative comments to suggest that MCQs are easy questions, and others refused to use them at all. There was no agreement around the proposition that if a question was designed to test a low level of skill, or a low level in a hierarchy of a body of knowledge, that such a question should or would be found to be easy by the student. To aid our analysis of assessment questions, we introduced four measures: Syntax Knowledge; Semantic Knowledge; Problem Solving Skill and the Level of Difficulty of the Problem. We applied these measures to selected examination questions, and have identified gaps between the instructor perspectives of what is considered to be an easy question and also in what is required to be assessed to determine whether students have achieved the goals of their course.

Survey of Current Doctrine, Training, and Special Considerations for Military Operations on Urbanized Terrain (MOUT).

DTIC Science & Technology

1983-11-01

Securing and fortifying (a) Doors (b) Hallways (c) Stairs (d) Windows (e) Floors (f) Ceilings 3 (g) Unoccupied rooms (h) Basements (i) Upper floors...observed, the instructors were interviewed, and training K was assessed through administration of a multiple-choice test and a Perception of Training...instructing clearing procedures. It would provide the opportunity to both critique and practice, using one structure. A Perception of Training

An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis.

PubMed

Tarrant, Marie; Ware, James; Mohammed, Ahmed M

2009-07-07

Four- or five-option multiple choice questions (MCQs) are the standard in health-science disciplines, both on certification-level examinations and on in-house developed tests. Previous research has shown, however, that few MCQs have three or four functioning distractors. The purpose of this study was to investigate non-functioning distractors in teacher-developed tests in one nursing program in an English-language university in Hong Kong. Using item-analysis data, we assessed the proportion of non-functioning distractors on a sample of seven test papers administered to undergraduate nursing students. A total of 514 items were reviewed, including 2056 options (1542 distractors and 514 correct responses). Non-functioning options were defined as ones that were chosen by fewer than 5% of examinees and those with a positive option discrimination statistic. The proportion of items containing 0, 1, 2, and 3 functioning distractors was 12.3%, 34.8%, 39.1%, and 13.8% respectively. Overall, items contained an average of 1.54 (SD = 0.88) functioning distractors. Only 52.2% (n = 805) of all distractors were functioning effectively and 10.2% (n = 158) had a choice frequency of 0. Items with more functioning distractors were more difficult and more discriminating. The low frequency of items with three functioning distractors in the four-option items in this study suggests that teachers have difficulty developing plausible distractors for most MCQs. Test items should consist of as many options as is feasible given the item content and the number of plausible distractors; in most cases this would be three. Item analysis results can be used to identify and remove non-functioning distractors from MCQs that have been used in previous tests.

Modeling Confidence Judgments, Response Times, and Multiple Choices in Decision Making: Recognition Memory and Motion Discrimination

PubMed Central

Ratcliff, Roger; Starns, Jeffrey J.

2014-01-01

Confidence in judgments is a fundamental aspect of decision making, and tasks that collect confidence judgments are an instantiation of multiple-choice decision making. We present a model for confidence judgments in recognition memory tasks that uses a multiple-choice diffusion decision process with separate accumulators of evidence for the different confidence choices. The accumulator that first reaches its decision boundary determines which choice is made. Five algorithms for accumulating evidence were compared, and one of them produced proportions of responses for each of the choices and full response time distributions for each choice that closely matched empirical data. With this algorithm, an increase in the evidence in one accumulator is accompanied by a decrease in the others so that the total amount of evidence in the system is constant. Application of the model to the data from an earlier experiment (Ratcliff, McKoon, & Tindall, 1994) uncovered a relationship between the shapes of z-transformed receiver operating characteristics and the behavior of response time distributions. Both are explained in the model by the behavior of the decision boundaries. For generality, we also applied the decision model to a 3-choice motion discrimination task and found it accounted for data better than a competing class of models. The confidence model presents a coherent account of confidence judgments and response time that cannot be explained with currently popular signal detection theory analyses or dual-process models of recognition. PMID:23915088

Increasing Choice Making in Students with Intellectual Disability

ERIC Educational Resources Information Center

Sparks, Shannon Lynn; Pierce, Tom; Higgins, Kyle; Miller, Susan; Tandy, Richard

2016-01-01

The purpose of this study was to examine the effectiveness of choice-making training with six high school students with intellectual disability. A multiple probe design with one replication was used to evaluate the efficacy of the choice-making training. The results suggest participants increased and maintained their choice-making abilities.…

Regulatory Fit and Systematic Exploration in a Dynamic Decision-Making Environment

ERIC Educational Resources Information Center

Otto, A. Ross; Markman, Arthur B.; Gureckis, Todd M.; Love, Bradley C.

2010-01-01

This work explores the influence of motivation on choice behavior in a dynamic decision-making environment, where the payoffs from each choice depend on one's recent choice history. Previous research reveals that participants in a regulatory fit exhibit increased levels of exploratory choice and flexible use of multiple strategies over the course…

Assessing the Impact of Student Learning Style Preferences

NASA Astrophysics Data System (ADS)

Davis, Stacey M.; Franklin, Scott V.

2004-09-01

Students express a wide range of preferences for learning environments. We are trying to measure the manifestation of learning styles in various learning environments. In particular, we are interested in performance in an environment that disagrees with the expressed learning style preference, paying close attention to social (group vs. individual) and auditory (those who prefer to learn by listening) environments. These are particularly relevant to activity-based curricula which typically emphasize group-work and de-emphasize lectures. Our methods include multiple-choice assessments, individual student interviews, and a study in which we attempt to isolate the learning environment.

A Healthy Eating Identity is Associated with Healthier Food Choice Behaviors Among U.S. Army Soldiers.

PubMed

Jayne, Julianna M; Frongillo, Edward A; Torres-McGehee, Toni M; Emerson, Dawn M; Glover, Saundra H; Blake, Christine E

2018-04-04

Promoting healthy eating among Soldiers is a priority to the Army due to the link between nutrition and performance. The Army typically uses nutrition education to encourage Soldiers to make healthier food choices with low emphasis on other psychosocial determinants of food choice behaviors. Drill Sergeant Candidates (n = 575) completed surveys assessing nutrition knowledge, eating identity type, and food choice behaviors including fruit and vegetable intake, skipping meals, and eating out frequency. In multiple linear regression models using full-information maximum likelihood estimation while controlling for race/ethnicity, education, and marital status, we examined relationships between nutrition knowledge, a healthy eating identity, and Soldiers' food choice behaviors. The study was approved by the Department of Defense and University of South Carolina's Institutional Review Boards. A healthy eating identity was positively associated with greater fruit and vegetable consumption (p < 0.05), and negatively associated with skipping meals and eating out frequency (p < 0.05). Nutrition knowledge was negatively associated with skipping meals (p < 0.05). Findings suggest that fostering a healthy eating identity may be more effective for promoting healthy food choice behaviors than nutrition education alone. Determining if various points in a Soldier's career could be leveraged to influence a healthy eating identity and behaviors could be an important strategy to improve compliance with health promotion programs.

Tailoring auditory training to patient needs with single and multiple talkers: transfer-appropriate gains on a four-choice discrimination test.

PubMed

Barcroft, Joe; Sommers, Mitchell S; Tye-Murray, Nancy; Mauzé, Elizabeth; Schroy, Catherine; Spehar, Brent

2011-11-01

Our long-term objective is to develop an auditory training program that will enhance speech recognition in those situations where patients most want improvement. As a first step, the current investigation trained participants using either a single talker or multiple talkers to determine if auditory training leads to transfer-appropriate gains. The experiment implemented a 2 × 2 × 2 mixed design, with training condition as a between-participants variable and testing interval and test version as repeated-measures variables. Participants completed a computerized six-week auditory training program wherein they heard either the speech of a single talker or the speech of six talkers. Training gains were assessed with single-talker and multi-talker versions of the Four-choice discrimination test. Participants in both groups were tested on both versions. Sixty-nine adult hearing-aid users were randomly assigned to either single-talker or multi-talker auditory training. Both groups showed significant gains on both test versions. Participants who trained with multiple talkers showed greater improvement on the multi-talker version whereas participants who trained with a single talker showed greater improvement on the single-talker version. Transfer-appropriate gains occurred following auditory training, suggesting that auditory training can be designed to target specific patient needs.

A Study of the Homogeneity of Items Produced From Item Forms Across Different Taxonomic Levels.

ERIC Educational Resources Information Center

Weber, Margaret B.; Argo, Jana K.

This study determined whether item forms ( rules for constructing items related to a domain or set of tasks) would enable naive item writers to generate multiple-choice items at three taxonomic levels--knowledge, comprehension, and application. Students wrote 120 multiple-choice items from 20 item forms, corresponding to educational objectives…

Effectiveness of Guided Multiple Choice Objective Questions Test on Students' Academic Achievement in Senior School Mathematics by School Location

ERIC Educational Resources Information Center

Igbojinwaekwu, Patrick Chukwuemeka

2015-01-01

This study investigated, using pretest-posttest quasi-experimental research design, the effectiveness of guided multiple choice objective questions test on students' academic achievement in Senior School Mathematics, by school location, in Delta State Capital Territory, Nigeria. The sample comprised 640 Students from four coeducation secondary…

An Empirical Comparison of DDF Detection Methods for Understanding the Causes of DIF in Multiple-Choice Items

ERIC Educational Resources Information Center

Suh, Youngsuk; Talley, Anna E.

2015-01-01

This study compared and illustrated four differential distractor functioning (DDF) detection methods for analyzing multiple-choice items. The log-linear approach, two item response theory-model-based approaches with likelihood ratio tests, and the odds ratio approach were compared to examine the congruence among the four DDF detection methods.…

Format of Options in Multiple Choice Test vis-a-vis Test Performance

ERIC Educational Resources Information Center

Bendulo, Hermabeth O.; Tibus, Erlinda D.; Bande, Rhodora A.; Oyzon, Voltaire Q.; Milla, Norberto E.; Macalinao, Myrna L.

2017-01-01

Testing or evaluation in an educational context is primarily used to measure or evaluate and authenticate the academic readiness, learning advancement, acquisition of skills, or instructional needs of learners. This study tried to determine whether the varied combinations of arrangements of options and letter cases in a Multiple-Choice Test (MCT)…

Some Effects of Changes in Question Structure and Sequence on Performance in a Multiple Choice Chemistry Test.

ERIC Educational Resources Information Center

Hodson, D.

1984-01-01

Investigated the effect on student performance of changes in question structure and sequence on a GCE 0-level multiple-choice chemistry test. One finding noted is that there was virtually no change in test reliability on reducing the number of options (from five to per test item). (JN)

Gender and Performance in Accounting Examinations: Exploring the Impact of Examination Format

ERIC Educational Resources Information Center

Arthur, Neal; Everaert, Patricia

2012-01-01

This paper addresses the question of whether the increasing use of multiple-choice questions will favour particular student groups, i.e. male or female students. Using data from Belgium, this paper empirically examines the existence of a gender effect by comparing the relative performance of male and female students in both multiple-choice and…

Multiple-Choice Exams: An Obstacle for Higher-Level Thinking in Introductory Science Classes

ERIC Educational Resources Information Center

Stanger-Hall, Kathrin F.

2012-01-01

Learning science requires higher-level (critical) thinking skills that need to be practiced in science classes. This study tested the effect of exam format on critical-thinking skills. Multiple-choice (MC) testing is common in introductory science courses, and students in these classes tend to associate memorization with MC questions and may not…

Grading Multiple Choice Exams with Low-Cost and Portable Computer-Vision Techniques

ERIC Educational Resources Information Center

Fisteus, Jesus Arias; Pardo, Abelardo; García, Norberto Fernández

2013-01-01

Although technology for automatic grading of multiple choice exams has existed for several decades, it is not yet as widely available or affordable as it should be. The main reasons preventing this adoption are the cost and the complexity of the setup procedures. In this paper, "Eyegrade," a system for automatic grading of multiple…

The Influence of a Response Format Test Accommodation for College Students with and without Disabilities

ERIC Educational Resources Information Center

Potter, Kyle; Lewandowski, Lawrence; Spenceley, Laura

2016-01-01

Standardised and other multiple-choice examinations often require the use of an answer sheet with fill-in bubbles (i.e. "bubble" or Scantron sheet). Students with disabilities causing impairments in attention, learning and/or visual-motor skill may have difficulties with multiple-choice examinations that employ such a response style.…

Multiple Choice Questions Can Be Designed or Revised to Challenge Learners' Critical Thinking

ERIC Educational Resources Information Center

Tractenberg, Rochelle E.; Gushta, Matthew M.; Mulroney, Susan E.; Weissinger, Peggy A.

2013-01-01

Multiple choice (MC) questions from a graduate physiology course were evaluated by cognitive-psychology (but not physiology) experts, and analyzed statistically, in order to test the independence of content expertise and cognitive complexity ratings of MC items. Integration of higher order thinking into MC exams is important, but widely known to…

Sex Differences in the Tendency to Omit Items on Multiple-Choice Tests: 1980-2000

ERIC Educational Resources Information Center

von Schrader, Sarah; Ansley, Timothy

2006-01-01

Much has been written concerning the potential group differences in responding to multiple-choice achievement test items. This discussion has included references to possible disparities in tendency to omit such test items. When test scores are used for high-stakes decision making, even small differences in scores and rankings that arise from male…

The Development and Validation of a Two-Tiered Multiple-Choice Instrument to Identify Alternative Conceptions in Earth Science

ERIC Educational Resources Information Center

Mangione, Katherine Anna

2010-01-01

This study was to determine reliability and validity for a two-tiered, multiple- choice instrument designed to identify alternative conceptions in earth science. Additionally, this study sought to identify alternative conceptions in earth science held by preservice teachers, to investigate relationships between self-reported confidence scores and…

The Display of Multiple Choice Question Bank on Microfilm

ERIC Educational Resources Information Center

Stevens, J. M.; Harris, F. T. C.

1977-01-01

An automated question bank maintained by the Department of Research and Services in Education at the Middlesex Hospital Medical School provides a printed copy of each of 25,000 multiple choice questions (95 percent relating to the whole spectrum of the medical curriculum). Problems with this procedure led to experimental work storing the data on…

Equal Opportunity in the Classroom: Test Construction in a Diversity-Sensitive Environment.

ERIC Educational Resources Information Center

Ghorpade, Jai; Lackritz, James R.

1998-01-01

Two multiple-choice tests and one essay test were taken by 231 students (50/50 male/female, 192 White, 39 East Asian, Black, Mexican American, or Middle Eastern). Multiple-choice tests showed no significant differences in equal employment opportunity terms; women and men scored about the same on essays, but minority students had significantly…

The Effect of Images on Item Statistics in Multiple Choice Anatomy Examinations

ERIC Educational Resources Information Center

Notebaert, Andrew J.

2017-01-01

Although multiple choice examinations are often used to test anatomical knowledge, these often forgo the use of images in favor of text-based questions and answers. Because anatomy is reliant on visual resources, examinations using images should be used when appropriate. This study was a retrospective analysis of examination items that were text…

Free-Response and Multiple-Choice Items: Measures of the Same Ability?

ERIC Educational Resources Information Center

Bennett, Randy Elliot; And Others

This study examined the relationship of multiple-choice and free-response items contained on the College Board's Advanced Placement Computer Science (APCS) examination. Subjects were two samples of 1,000 randomly drawn from the population of 7,372 high school students taking the 1988 examination of the APCS "AB" form. Most were high…

Validity and Reliability of Scores Obtained on Multiple-Choice Questions: Why Functioning Distractors Matter

ERIC Educational Resources Information Center

Ali, Syed Haris; Carr, Patrick A.; Ruit, Kenneth G.

2016-01-01

Plausible distractors are important for accurate measurement of knowledge via multiple-choice questions (MCQs). This study demonstrates the impact of higher distractor functioning on validity and reliability of scores obtained on MCQs. Freeresponse (FR) and MCQ versions of a neurohistology practice exam were given to four cohorts of Year 1 medical…