A Two-Tier Multiple Choice Questions to Diagnose Thermodynamic Misconception of Thai and Laos Students

NASA Astrophysics Data System (ADS)

Kamcharean, Chanwit; Wattanakasiwich, Pornrat

The objective of this study was to diagnose misconceptions of Thai and Lao students in thermodynamics by using a two-tier multiple-choice test. Two-tier multiple choice questions consist of the first tier, a content-based question and the second tier, a reasoning-based question. Data of student understanding was collected by using 10 two-tier multiple-choice questions. Thai participants were the first-year students (N = 57) taking a fundamental physics course at Chiang Mai University in 2012. Lao participants were high school students in Grade 11 (N = 57) and Grade 12 (N = 83) at Muengnern high school in Xayaboury province, Lao PDR. As results, most students answered content-tier questions correctly but chose incorrect answers for reason-tier questions. When further investigating their incorrect reasons, we found similar misconceptions as reported in previous studies such as incorrectly relating pressure with temperature when presenting with multiple variables.

English 30, Part B: Reading. Questions Booklet. Grade 12 Diploma Examination, January 1997.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton. Student Evaluation Branch.

Intended for students taking the Grade 12 Diploma Examinations in English 30, this "questions booklet" presents 70 multiple choice test items based on 8 reading selections in the accompanying readings booklet. After instructions for students, the booklet presents the multiple choice items which test students' comprehension of the poetry,…

Effectiveness of Guided Multiple Choice Objective Questions Test on Students' Academic Achievement in Senior School Mathematics by School Location

ERIC Educational Resources Information Center

Igbojinwaekwu, Patrick Chukwuemeka

2015-01-01

This study investigated, using pretest-posttest quasi-experimental research design, the effectiveness of guided multiple choice objective questions test on students' academic achievement in Senior School Mathematics, by school location, in Delta State Capital Territory, Nigeria. The sample comprised 640 Students from four coeducation secondary…

The impact of two multiple-choice question formats on the problem-solving strategies used by novices and experts.

PubMed

Coderre, Sylvain P; Harasym, Peter; Mandin, Henry; Fick, Gordon

2004-11-05

Pencil-and-paper examination formats, and specifically the standard, five-option multiple-choice question, have often been questioned as a means for assessing higher-order clinical reasoning or problem solving. This study firstly investigated whether two paper formats with differing number of alternatives (standard five-option and extended-matching questions) can test problem-solving abilities. Secondly, the impact of the alternatives number on psychometrics and problem-solving strategies was examined. Think-aloud protocols were collected to determine the problem-solving strategy used by experts and non-experts in answering Gastroenterology questions, across the two pencil-and-paper formats. The two formats demonstrated equal ability in testing problem-solving abilities, while the number of alternatives did not significantly impact psychometrics or problem-solving strategies utilized. These results support the notion that well-constructed multiple-choice questions can in fact test higher order clinical reasoning. Furthermore, it can be concluded that in testing clinical reasoning, the question stem, or content, remains more important than the number of alternatives.

Difficulty and Discriminability of Introductory Psychology Test Items.

ERIC Educational Resources Information Center

Scialfa, Charles; Legare, Connie; Wenger, Larry; Dingley, Louis

2001-01-01

Analyzes multiple-choice questions provided in test banks for introductory psychology textbooks. Study 1 offered a consistent picture of the objective difficulty of multiple-choice tests for introductory psychology students, while both studies 1 and 2 indicated that test items taken from commercial test banks have poor psychometric properties.…

Retrieval practice with short-answer, multiple-choice, and hybrid tests.

PubMed

Smith, Megan A; Karpicke, Jeffrey D

2014-01-01

Retrieval practice improves meaningful learning, and the most frequent way of implementing retrieval practice in classrooms is to have students answer questions. In four experiments (N=372) we investigated the effects of different question formats on learning. Students read educational texts and practised retrieval by answering short-answer, multiple-choice, or hybrid questions. In hybrid conditions students first attempted to recall answers in short-answer format, then identified answers in multiple-choice format. We measured learning 1 week later using a final assessment with two types of questions: those that could be answered by recalling information verbatim from the texts and those that required inferences. Practising retrieval in all format conditions enhanced retention, relative to a study-only control condition, on both verbatim and inference questions. However, there were little or no advantages of answering short-answer or hybrid format questions over multiple-choice questions in three experiments. In Experiment 4, when retrieval success was improved under initial short-answer conditions, there was an advantage of answering short-answer or hybrid questions over multiple-choice questions. The results challenge the simple conclusion that short-answer questions always produce the best learning, due to increased retrieval effort or difficulty, and demonstrate the importance of retrieval success for retrieval-based learning activities.

Assessment of item-writing flaws in multiple-choice questions.

PubMed

Nedeau-Cayo, Rosemarie; Laughlin, Deborah; Rus, Linda; Hall, John

2013-01-01

This study evaluated the quality of multiple-choice questions used in a hospital's e-learning system. Constructing well-written questions is fraught with difficulty, and item-writing flaws are common. Study results revealed that most items contained flaws and were written at the knowledge/comprehension level. Few items had linked objectives, and no association was found between the presence of objectives and flaws. Recommendations include education for writing test questions.

Application of a Utility Analysis to Evaluate a Novel Assessment Tool for Clinically Oriented Physiology and Pharmacology

ERIC Educational Resources Information Center

Cramer, Nicholas; Asmar, Abdo; Gorman, Laurel; Gros, Bernard; Harris, David; Howard, Thomas; Hussain, Mujtaba; Salazar, Sergio; Kibble, Jonathan D.

2016-01-01

Multiple-choice questions are a gold-standard tool in medical school for assessment of knowledge and are the mainstay of licensing examinations. However, multiple-choice questions items can be criticized for lacking the ability to test higher-order learning or integrative thinking across multiple disciplines. Our objective was to develop a novel…

To Show or Not to Show: The Effects of Item Stems and Answer Options on Performance on a Multiple-Choice Listening Comprehension Test

ERIC Educational Resources Information Center

Yanagawa, Kozo; Green, Anthony

2008-01-01

The purpose of this study is to examine whether the choice between three multiple-choice listening comprehension test formats results in any difference in listening comprehension test performance. The three formats entail (a) allowing test takers to preview both the question stem and answer options prior to listening; (b) allowing test takers to…

Investigating the potential influence of established multiple-choice test-taking cues on item response in a pharmacotherapy board certification examination preparatory manual: a pilot study.

PubMed

Gettig, Jacob P

2006-04-01

To determine the prevalence of established multiple-choice test-taking correct and incorrect answer cues in the American College of Clinical Pharmacy's Updates in Therapeutics: The Pharmacotherapy Preparatory Course, 2005 Edition, as an equal or lesser surrogate indication of the prevalence of such cues in the Pharmacotherapy board certification examination. All self-assessment and patient case question-and-answer sets were assessed individually to determine if they were subject to selected correct and incorrect answer cues commonly seen in multiple-choice question writing. If the question was considered evaluable, correct answer cues-longest answer, mid-range number, one of two similar choices, and one of two opposite choices-were tallied. In addition, incorrect answer cues- inclusionary language and grammatical mismatch-were also tallied. Each cue was counted if it did what was expected or did the opposite of what was expected. Multiple cues could be identified in each question. A total of 237 (47.7%) of 497 questions in the manual were deemed evaluable. A total of 325 correct answer cues and 35 incorrect answer cues were identified in the 237 evaluable questions. Most evaluable questions contained one to two correct and/or incorrect answer cue(s). Longest answer was the most frequently identified correct answer cue; however, it was the least likely to identify the correct answer. Inclusionary language was the most frequently identified incorrect answer cue. Incorrect answer cues were considerably more likely to identify incorrect answer choices than correct answer cues were able to identify correct answer choices. The use of established multiple-choice test-taking cues is unlikely to be of significant help when taking the Pharmacotherapy board certification examination, primarily because of the lack of questions subject to such cues and the inability of correct answer cues to accurately identify correct answers. Incorrect answer cues, especially the use of inclusionary language, almost always will accurately identify an incorrect answer choice. Assuming that questions in the preparatory course manual were equal or lesser surrogates of those in the board certification examination, it is unlikely that intuition alone can replace adequate preparation and studying as the sole determinant of examination success.

Examen en Vue du Diplome Douzieme Annee. Langue et Litterature 30. Partie B: Lecture (Choix Multiples). Livret de Questions (Examination for the Twelfth Grade Diploma, Language and Literature 30. Part B: Reading--Multiple Choice. Questions Booklet).

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

As part of an examination required by the Alberta (Canada) Department of Education in order for 12th grade students to receive a diploma in French, this booklet contains the 80 multiple choice questions portion of Part B, the language and literature component of the January 1987 tests. Representing the genres of poetry, short story, the novel, and…

Examen en Vue du Diplome Douzieme Annee. Langue et Litterature 30. Partie B: Lecture (Choix Multiples). Livret de Questions (Examination for the Twelfth Grade Diploma, Language and Literature 30. Part B: Reading--Multiple Choice. Questions Booklet.)

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

As part of an examination required by the Alberta (Canada) Department of Education in order for 12th grade students to receive a diploma in French, this booklet contains the 80 multiple choice questions portion of Part B, the language and literature component of the January 1988 tests. Representing the genres of poetry, short story, novel, and…

Multiple Choice Questions Can Be Designed or Revised to Challenge Learners' Critical Thinking

ERIC Educational Resources Information Center

Tractenberg, Rochelle E.; Gushta, Matthew M.; Mulroney, Susan E.; Weissinger, Peggy A.

2013-01-01

Multiple choice (MC) questions from a graduate physiology course were evaluated by cognitive-psychology (but not physiology) experts, and analyzed statistically, in order to test the independence of content expertise and cognitive complexity ratings of MC items. Integration of higher order thinking into MC exams is important, but widely known to…

Fast Assessments with Digital Tools Using Multiple-Choice Questions

ERIC Educational Resources Information Center

Howell, Dusti D.; Tseng, Daphne ChingYu; Colorado-Resa, Jozenia T.

2017-01-01

Multiple Choice Questions (MCQs) have come a long way since they were used in "The Kansas Silent Reading Test" in 1915. After over 100 years of MCQs, new innovative digital tools using this form of assessment can help foster interactivity in today's classrooms. This article describes three free online MCQ tools that are relatively quick…

"None of the above" as a correct and incorrect alternative on a multiple-choice test: implications for the testing effect.

PubMed

Odegard, Timothy N; Koen, Joshua D

2007-11-01

Both positive and negative testing effects have been demonstrated with a variety of materials and paradigms (Roediger & Karpicke, 2006b). The present series of experiments replicate and extend the research of Roediger and Marsh (2005) with the addition of a "none-of-the-above" response option. Participants (n=32 in both experiments) read a set of passages, took an initial multiple-choice test, completed a filler task, and then completed a final cued-recall test (Experiment 1) or multiple-choice test (Experiment 2). Questions were manipulated on the initial multiple-choice test by adding a "none-of-the-above" response alternative (choice "E") that was incorrect ("E" Incorrect) or correct ("E" Correct). The results from both experiments demonstrated that the positive testing effect was negated when the "none-of-the-above" alternative was the correct response on the initial multiple-choice test, but was still present when the "none-of-the-above" alternative was an incorrect response.

Using a Classroom Response System to Improve Multiple-Choice Performance in AP® Physics

NASA Astrophysics Data System (ADS)

Bertrand, Peggy

2009-04-01

Participation in rigorous high school courses such as Advanced Placement (AP®) Physics increases the likelihood of college success, especially for students who are traditionally underserved. Tackling difficult multiple-choice exams should be part of any AP program because well-constructed multiple-choice questions, such as those on AP exams and on the Force Concept Inventory,2 are particularly good at rooting out common and persisting student misconceptions. Additionally, there are barriers to multiple-choice performance that have little to do with content mastery. For example, a student might fail to read the question thoroughly, forget to apply a reasonableness test to the answer, or simply work too slowly.

A Case Study on Multiple-Choice Testing in Anatomical Sciences

ERIC Educational Resources Information Center

Golda, Stephanie DuPont

2011-01-01

Objective testing techniques, such as multiple-choice examinations, are a widely accepted method of assessment in gross anatomy. In order to deter cheating on these types of examinations, instructors often design several versions of an examination to distribute. These versions usually involve the rearrangement of questions and their corresponding…

Second Language Reading Topic Familiarity and Test Score: Test-Taking Strategies for Multiple-Choice Comprehension Questions

ERIC Educational Resources Information Center

Lee, Jia-Ying

2011-01-01

The main purpose of this study was to compare the strategies used by Chinese-speaking students when confronted with familiar versus unfamiliar topics in a multiple-choice format reading comprehension test. The focus was on describing what students do when they are taking reading comprehension tests by asking students to verbalize their thoughts.…

Test Pool Questions, Area III.

ERIC Educational Resources Information Center

Sloan, Jamee Reid

This manual contains multiple choice questions to be used in testing students on nurse training objectives. Each test includes several questions covering each concept. The concepts in section A, medical surgical nursing, are diseases of the following systems: musculoskeletal; central nervous; cardiovascular; gastrointestinal; urinary and male…

Semantic Similarity Measures for the Generation of Science Tests in Basque

ERIC Educational Resources Information Center

Aldabe, Itziar; Maritxalar, Montse

2014-01-01

The work we present in this paper aims to help teachers create multiple-choice science tests. We focus on a scientific vocabulary-learning scenario taking place in a Basque-language educational environment. In this particular scenario, we explore the option of automatically generating Multiple-Choice Questions (MCQ) by means of Natural Language…

"I Don't Know" and Multiple Choice Analysis of Pre- and Post-Tests

ERIC Educational Resources Information Center

Spears, Karen; Wilson, Mary

2010-01-01

Evaluation is an essential component of any Extension education program. One tool, the pre- and post-test, provides measurable evaluation data. Yet often the answer "I don't know" or all possible answers to a multiple choice question are not included in the repeated measure analysis. Because more than two answers are offered, the test of marginal…

Psychometrics of Multiple Choice Questions with Non-Functioning Distracters: Implications to Medical Education.

PubMed

Deepak, Kishore K; Al-Umran, Khalid Umran; AI-Sheikh, Mona H; Dkoli, B V; Al-Rubaish, Abdullah

2015-01-01

The functionality of distracters in a multiple choice question plays a very important role. We examined the frequency and impact of functioning and non-functioning distracters on psychometric properties of 5-option items in clinical disciplines. We analyzed item statistics of 1115 multiple choice questions from 15 summative assessments of undergraduate medical students and classified the items into five groups by their number of non-functioning distracters. We analyzed the effect of varying degree of non-functionality ranging from 0 to 4, on test reliability, difficulty index, discrimination index and point biserial correlation. The non-functionality of distracters inversely affected the test reliability and quality of items in a predictable manner. The non-functioning distracters made the items easier and lowered the discrimination index significantly. Three non-functional distracters in a 5-option MCQ significantly affected all psychometric properties (p < 0.5). The corrected point biserial correlation revealed that the items with 3 functional options were psychometrically as effective as 5-option items. Our study reveals that a multiple choice question with 3 functional options provides lower most limit of item format that has adequate psychometric property. The test containing items with less number of functioning options have significantly lower reliability. The distracter function analysis and revision of nonfunctioning distracters can serve as important methods to improve the psychometrics and reliability of assessment.

Which form of assessment provides the best information about student performance in chemistry examinations?

NASA Astrophysics Data System (ADS)

Hudson, Ross D.; Treagust, David F.

2013-04-01

Background . This study developed from observations of apparent achievement differences between male and female chemistry performances in a state university entrance examination. Male students performed more strongly than female students, especially in higher scores. Apart from the gender of the students, two other important factors that might influence student performance were format of questions (short-answer or multiple-choice) and type of questions (recall or application). Purpose The research question addressed in this study was: Is there a relationship between performance in state university entrance examinations in chemistry and school chemistry examinations and student gender, format of questions - multiple-choice or short-answer, and conceptual level - recall or application? Sample The two sources of data were: (1) secondary analyses of five consecutive years' data published by the examining authority of chemistry examinations, and (2) tests conducted with 192 students which provided information about all aspects of the three variables (question format, question type and gender) under consideration. Design and methods Both sources of data were analysed using ANOVA to compare means for the variables under consideration and the statistical significance of any differences. The data from the tests were also analysed using Rasch analysis to determine differences in gender performance. Results When overall mean data are considered, both male and female students performed better on multiple-choice questions and recall questions than on short-answer questions and application questions, respectively. When overall mean data are considered, male students outperformed female students in both the university entrance and school tests, particularly in the higher scores. When data were analysed with Rasch, there was no statistically significant difference in performance between males and females of equal ability. Conclusions Both male and female students generally perform better on multiple-choice questions than they do on short-answer questions. However, when the questions are matched in terms of difficulty (using Rasch analysis), the differences in performance between multiple-choice and short-answer are quite small. Rasch analysis showed that there was little difference in performance between males and females of equal ability. This study shows that a simple face-value score analysis of relative student performance - in this case, in chemistry - can be deceptive unless the actual abilities of the students concerned, as measured by a tool such as Rasch, are taken into consideration before reaching any conclusion.

The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments.

PubMed

Tarrant, Marie; Knierim, Aimee; Hayes, Sasha K; Ware, James

2006-12-01

Multiple-choice questions are a common assessment method in nursing examinations. Few nurse educators, however, have formal preparation in constructing multiple-choice questions. Consequently, questions used in baccalaureate nursing assessments often contain item-writing flaws, or violations to accepted item-writing guidelines. In one nursing department, 2770 MCQs were collected from tests and examinations administered over a five-year period from 2001 to 2005. Questions were evaluated for 19 frequently occurring item-writing flaws, for cognitive level, for question source, and for the distribution of correct answers. Results show that almost half (46.2%) of the questions contained violations of item-writing guidelines and over 90% were written at low cognitive levels. Only a small proportion of questions were teacher generated (14.1%), while 36.2% were taken from testbanks and almost half (49.4%) had no source identified. MCQs written at a lower cognitive level were significantly more likely to contain item-writing flaws. While there was no relationship between the source of the question and item-writing flaws, teacher-generated questions were more likely to be written at higher cognitive levels (p<0.001). Correct answers were evenly distributed across all four options and no bias was noted in the placement of correct options. Further training in item-writing is recommended for all faculty members who are responsible for developing tests. Pre-test review and quality assessment is also recommended to reduce the occurrence of item-writing flaws and to improve the quality of test questions.

The "None of the Above" Option in Multiple-Choice Testing: An Experimental Study

ERIC Educational Resources Information Center

DiBattista, David; Sinnige-Egger, Jo-Anne; Fortuna, Glenda

2014-01-01

The authors assessed the effects of using "none of the above" as an option in a 40-item, general-knowledge multiple-choice test administered to undergraduate students. Examinees who selected "none of the above" were given an incentive to write the correct answer to the question posed. Using "none of the above" as the…

Multiple-Choice Exams: An Obstacle for Higher-Level Thinking in Introductory Science Classes

ERIC Educational Resources Information Center

Stanger-Hall, Kathrin F.

2012-01-01

Learning science requires higher-level (critical) thinking skills that need to be practiced in science classes. This study tested the effect of exam format on critical-thinking skills. Multiple-choice (MC) testing is common in introductory science courses, and students in these classes tend to associate memorization with MC questions and may not…

Multiple-Choice Question Tests: A Convenient, Flexible and Effective Learning Tool? A Case Study

ERIC Educational Resources Information Center

Douglas, Mercedes; Wilson, Juliette; Ennis, Sean

2012-01-01

The research presented in this paper is part of a project investigating assessment practices, funded by the Scottish Funding Council. Using established principles of good assessment and feedback, the use of online formative and summative multiple choice tests (MCT's) was piloted to support independent and self-directed learning and improve…

Solving Geometric Problems by Using Algebraic Representation for Junior High School Level 3 in Van Hiele at Geometric Thinking Level

ERIC Educational Resources Information Center

Suwito, Abi; Yuwono, Ipung; Parta, I. Nengah; Irawati, Santi; Oktavianingtyas, Ervin

2016-01-01

This study aims to determine the ability of algebra students who have 3 levels van Hiele levels. Follow its framework Dindyal framework (2007). Students are required to do 10 algebra shaped multiple choice, then students work 15 about the geometry of the van Hiele level in the form of multiple choice questions. The question has been tested levels…

All of the above: When multiple correct response options enhance the testing effect.

PubMed

Bishara, Anthony J; Lanzo, Lauren A

2015-01-01

Previous research has shown that multiple choice tests often improve memory retention. However, the presence of incorrect lures often attenuates this memory benefit. The current research examined the effects of "all of the above" (AOTA) options. When such options are correct, no incorrect lures are present. In the first three experiments, a correct AOTA option on an initial test led to a larger memory benefit than no test and standard multiple choice test conditions. The benefits of a correct AOTA option occurred even without feedback on the initial test; for both 5-minute and 48-hour retention delays; and for both cued recall and multiple choice final test formats. In the final experiment, an AOTA question led to better memory retention than did a control condition that had identical timing and exposure to response options. However, the benefits relative to this control condition were similar regardless of the type of multiple choice test (AOTA or not). Results suggest that retrieval contributes to multiple choice testing effects. However, the extra testing effect from a correct AOTA option, rather than being due to more retrieval, might be due simply to more exposure to correct information.

Memorial Consequences of Answering SAT II Questions

ERIC Educational Resources Information Center

Marsh, Elizabeth J.; Agarwal, Pooja K.; Roediger, Henry L., III

2009-01-01

Many thousands of students take standardized tests every year. In the current research, we asked whether answering standardized test questions affects students' later test performance. Prior research has shown both positive and negative effects of multiple-choice testing on later tests, with negative effects arising from students selecting…

The Effects of Question Generation, Question Answering and Reading on Prose Learning.

ERIC Educational Resources Information Center

Owens, Anthony Mathew

Eighty-seven students took part in a study (using three experimental conditions) of the effect of question generation, question answering, and rereading on an immediate posttest and on a delayed test of knowledge of a text passage. In condition G, subjects generated multiple-choice questions that would test knowledge of each of the six content…

A Comparison of Methods for Transforming Sentences into Test Questions for Instructional Materials. Technical Report #1.

ERIC Educational Resources Information Center

Roid, Gale; And Others

Several measurement theorists have convincingly argued that methods of writing test questions, particularly for criterion-referenced tests, should be based on operationally defined rules. This study was designed to examine and further refine a method for objectively generating multiple-choice questions for prose instructional materials. Important…

Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper

PubMed Central

Palmer, Edward J; Devitt, Peter G

2007-01-01

Background Reliable and valid written tests of higher cognitive function are difficult to produce, particularly for the assessment of clinical problem solving. Modified Essay Questions (MEQs) are often used to assess these higher order abilities in preference to other forms of assessment, including multiple-choice questions (MCQs). MEQs often form a vital component of end-of-course assessments in higher education. It is not clear how effectively these questions assess higher order cognitive skills. This study was designed to assess the effectiveness of the MEQ to measure higher-order cognitive skills in an undergraduate institution. Methods An analysis of multiple-choice questions and modified essay questions (MEQs) used for summative assessment in a clinical undergraduate curriculum was undertaken. A total of 50 MCQs and 139 stages of MEQs were examined, which came from three exams run over two years. The effectiveness of the questions was determined by two assessors and was defined by the questions ability to measure higher cognitive skills, as determined by a modification of Bloom's taxonomy, and its quality as determined by the presence of item writing flaws. Results Over 50% of all of the MEQs tested factual recall. This was similar to the percentage of MCQs testing factual recall. The modified essay question failed in its role of consistently assessing higher cognitive skills whereas the MCQ frequently tested more than mere recall of knowledge. Conclusion Construction of MEQs, which will assess higher order cognitive skills cannot be assumed to be a simple task. Well-constructed MCQs should be considered a satisfactory replacement for MEQs if the MEQs cannot be designed to adequately test higher order skills. Such MCQs are capable of withstanding the intellectual and statistical scrutiny imposed by a high stakes exit examination. PMID:18045500

Further Support for Changing Multiple-Choice Answers.

ERIC Educational Resources Information Center

Fabrey, Lawrence J.; Case, Susan M.

1985-01-01

The effect on test scores of changing answers to multiple-choice questions was studied and compared to earlier research. The current setting was a nationally administered, in-training, specialty examination for medical residents in obstetrics and gynecology. Both low and high scorers improved their scores when they changed answers. (SW)

High School Students' Concepts of Acids and Bases.

ERIC Educational Resources Information Center

Ross, Bertram H. B.

An investigation of Ontario high school students' understanding of acids and bases with quantitative and qualitative methods revealed misconceptions. A concept map, based on the objectives of the Chemistry Curriculum Guideline, generated multiple-choice items and interview questions. The multiple-choice test was administered to 34 grade 12…

How did you guess? Or, what do multiple-choice questions measure?

PubMed

Cox, K R

1976-06-05

Multiple-choice questions classified as requiring problem-solving skills have been interpreted as measuring problem-solving skills within students, with the implicit hypothesis that questions needing an increasingly complex intellectual process should present increasing difficulty to the student. This hypothesis was tested in a 150-question paper taken by 721 students in seven Australian medical schools. No correlation was observed between difficulty and assigned process. Consequently, the question-answering process was explored with a group of final-year students. Anecdotal recall by students gave heavy weight to knowledge rather than problem solving in answering these questions. Assignment of the 150 questions to the classification by three teachers and six students showed their congruence to be a little above random probability.

Comparing comprehension measured by multiple-choice and open-ended questions.

PubMed

Ozuru, Yasuhiro; Briner, Stephen; Kurby, Christopher A; McNamara, Danielle S

2013-09-01

This study compared the nature of text comprehension as measured by multiple-choice format and open-ended format questions. Participants read a short text while explaining preselected sentences. After reading the text, participants answered open-ended and multiple-choice versions of the same questions based on their memory of the text content. The results indicated that performance on open-ended questions was correlated with the quality of self-explanations, but performance on multiple-choice questions was correlated with the level of prior knowledge related to the text. These results suggest that open-ended and multiple-choice format questions measure different aspects of comprehension processes. The results are discussed in terms of dual process theories of text comprehension. PsycINFO Database Record (c) 2013 APA, all rights reserved

Automatic Scoring of Paper-and-Pencil Figural Responses. Research Report.

ERIC Educational Resources Information Center

Martinez, Michael E.; And Others

Large-scale testing is dominated by the multiple-choice question format. Widespread use of the format is due, in part, to the ease with which multiple-choice items can be scored automatically. This paper examines automatic scoring procedures for an alternative item type: figural response. Figural response items call for the completion or…

What is an Objective Structured Practical Examination in Anatomy?

ERIC Educational Resources Information Center

Yaqinuddin, Ahmed; Zafar, Muhammad; Ikram, Muhammad Faisal; Ganguly, Paul

2013-01-01

Assessing teaching-learning outcomes in anatomical knowledge is a complex task that requires the evaluation of multiple domains: theoretical, practical, and clinical knowledge. In general, theoretical knowledge is tested by a written examination system constituted by multiple choice questions (MCQs) and/or short answer questions (SAQ). The…

Student certainty answering misconception question: study of Three-Tier Multiple-Choice Diagnostic Test in Acid-Base and Solubility Equilibrium

NASA Astrophysics Data System (ADS)

Ardiansah; Masykuri, M.; Rahardjo, S. B.

2018-04-01

Students’ concept comprehension in three-tier multiple-choice diagnostic test related to student confidence level. The confidence level related to certainty and student’s self-efficacy. The purpose of this research was to find out students’ certainty in misconception test. This research was quantitative-qualitative research method counting students’ confidence level. The research participants were 484 students that were studying acid-base and equilibrium solubility subject. Data was collected using three-tier multiple-choice (3TMC) with thirty questions and students’ questionnaire. The findings showed that #6 item gives the highest misconception percentage and high student confidence about the counting of ultra-dilute solution’s pH. Other findings were that 1) the student tendency chosen the misconception answer is to increase over item number, 2) student certainty decreased in terms of answering the 3TMC, and 3) student self-efficacy and achievement were related each other in the research. The findings suggest some implications and limitations for further research.

Piloting a Geoscience Literacy Exam for Assessing Students' Understanding of Earth, Climate, Atmospheric and Ocean Science Concepts

NASA Astrophysics Data System (ADS)

Steer, D. N.; Iverson, E. A.; Manduca, C. A.

2013-12-01

This research seeks to develop valid and reliable questions that faculty can use to assess geoscience literacy across the curriculum. We are particularly interested on effects of curricula developed to teach Earth, Climate, Atmospheric, and Ocean Science concepts in the context of societal issues across the disciplines. This effort is part of the InTeGrate project designed to create a population of college graduates who are poised to use geoscience knowledge in developing solutions to current and future environmental and resource challenges. Details concerning the project are found at http://serc.carleton.edu/integrate/index.html. The Geoscience Literacy Exam (GLE) under development presently includes 90 questions. Each big idea from each literacy document can be probed using one or more of three independent questions: 1) a single answer, multiple choice question aimed at basic understanding or application of key concepts, 2) a multiple correct answer, multiple choice question targeting the analyzing to analysis levels and 3) a short essay question that tests analysis or evaluation cognitive levels. We anticipate multiple-choice scores and the detail and sophistication of essay responses will increase as students engage with the curriculum. As part of the field testing of InTeGrate curricula, faculty collected student responses from classes that involved over 700 students. These responses included eight pre- and post-test multiple-choice questions that covered various concepts across the four literacies. Discrimination indices calculated from the data suggest that the eight tested questions provide a valid measure of literacy within the scope of the concepts covered. Student normalized gains across an academic term with limited InTeGrate exposure (typically two or fewer weeks of InTeGrate curriculum out of 14 weeks) were found to average 16% gain. A small set of control data (250 students in classes from one institution where no InTeGrate curricula were used) was also collected from a larger bank of test questions. Discrimination indices across the full bank showed variation and additional work is underway to refine and field test in other settings these questions in the absence of InTeGrate curricula. When complete, faculty will be able to assemble sets of questions to track progress toward meeting literacy goals. In addition to covering geoscience content knowledge and understanding, a complementary attitudinal pre/post survey was also developed with the intent to probe InTeGrate students' ability and motivation to use their geoscience expertise to address problems of environmental sustainability. The final instruments will be made available to the geoscience education community as an assessment to be used in conjunction with InTeGrate teaching materials or as a stand-alone tool for departments to measure student learning and attitudinal gains across the major.

Computer Analysis in HSC German

ERIC Educational Resources Information Center

Clutterbuck, Michael; Mowchanuk, Timothy

1977-01-01

In October, 1976, a new type of question was introduced into the Victorian HSC German test: A listening comprehension question with multiple-choice answers has replaced the written reproduction question. Results of a computer analysis of the answers given in the exam are reported. (SW)

The Effect of Images on Item Statistics in Multiple Choice Anatomy Examinations

ERIC Educational Resources Information Center

Notebaert, Andrew J.

2017-01-01

Although multiple choice examinations are often used to test anatomical knowledge, these often forgo the use of images in favor of text-based questions and answers. Because anatomy is reliant on visual resources, examinations using images should be used when appropriate. This study was a retrospective analysis of examination items that were text…

Measuring University students' understanding of the greenhouse effect - a comparison of multiple-choice, short answer and concept sketch assessment tools with respect to students' mental models

NASA Astrophysics Data System (ADS)

Gold, A. U.; Harris, S. E.

2013-12-01

The greenhouse effect comes up in most discussions about climate and is a key concept related to climate change. Existing studies have shown that students and adults alike lack a detailed understanding of this important concept or might hold misconceptions. We studied the effectiveness of different interventions on University-level students' understanding of the greenhouse effect. Introductory level science students were tested for their pre-knowledge of the greenhouse effect using validated multiple-choice questions, short answers and concept sketches. All students participated in a common lesson about the greenhouse effect and were then randomly assigned to one of two lab groups. One group explored an existing simulation about the greenhouse effect (PhET-lesson) and the other group worked with absorption spectra of different greenhouse gases (Data-lesson) to deepen the understanding of the greenhouse effect. All students completed the same assessment including multiple choice, short answers and concept sketches after participation in their lab lesson. 164 students completed all the assessments, 76 completed the PhET lesson and 77 completed the data lesson. 11 students missed the contrasting lesson. In this presentation we show the comparison between the multiple-choice questions, short answer questions and the concept sketches of students. We explore how well each of these assessment types represents student's knowledge. We also identify items that are indicators of the level of understanding of the greenhouse effect as measured in correspondence of student answers to an expert mental model and expert responses. Preliminary data analysis shows that student who produce concept sketch drawings that come close to expert drawings also choose correct multiple-choice answers. However, correct multiple-choice answers are not necessarily an indicator that a student produces an expert-like correlating concept sketch items. Multiple-choice questions that require detailed knowledge of the greenhouse effect (e.g. direction of re-emission of infrared energy from greenhouse gas) are significantly more likely to be answered correctly by students who also produce expert-like concept sketch items than by students who don't include this aspect in their sketch and don't answer the multiple choice questions correctly. This difference is not as apparent for less technical multiple-choice questions (e.g. type of radiation emitted by Sun). Our findings explore the formation of student's mental models throughout different interventions and how well the different assessment techniques used in this study represent the student understanding of the overall concept.

Validity and Realibility of Chemistry Systemic Multiple Choices Questions (CSMCQs)

ERIC Educational Resources Information Center

Priyambodo, Erfan; Marfuatun

2016-01-01

Nowadays, Rasch model analysis is used widely in social research, moreover in educational research. In this research, Rasch model is used to determine the validation and the reliability of systemic multiple choices question in chemistry teaching and learning. There were 30 multiple choices question with systemic approach for high school student…

Mechanical waves conceptual survey: Its modification and conversion to a standard multiple-choice test

NASA Astrophysics Data System (ADS)

Barniol, Pablo; Zavala, Genaro

2016-06-01

In this article we present several modifications of the mechanical waves conceptual survey, the most important test to date that has been designed to evaluate university students' understanding of four main topics in mechanical waves: propagation, superposition, reflection, and standing waves. The most significant changes are (i) modification of several test questions that had some problems in their original design, (ii) standardization of the number of options for each question to five, (iii) conversion of the two-tier questions to multiple-choice questions, and (iv) modification of some questions to make them independent of others. To obtain a final version of the test, we administered both the original and modified versions several times to students at a large private university in Mexico. These students were completing a course that covers the topics tested by the survey. The final modified version of the test was administered to 234 students. In this study we present the modifications for each question, and discuss the reasons behind them. We also analyze the results obtained by the final modified version and offer a comparison between the original and modified versions. In the Supplemental Material we present the final modified version of the test. It can be used by teachers and researchers to assess students' understanding of, and learning about, mechanical waves.

Testing with feedback improves recall of information in informed consent: A proof of concept study.

PubMed

Roberts, Katherine J; Revenson, Tracey A; Urken, Mark L; Fleszar, Sara; Cipollina, Rebecca; Rowe, Meghan E; Reis, Laura L Dos; Lepore, Stephen J

2016-08-01

This study investigates whether applying educational testing approaches to an informed consent video for a medical procedure can lead to greater recall of the information presented. Undergraduate students (n=120) were randomly assigned to watch a 20-min video on informed consent under one of three conditions: 1) tested using multiple-choice knowledge questions and provided with feedback on their answers after each 5-min segment; 2) tested with multiple choice knowledge questions but not provided feedback after each segment; or 3) watched the video without knowledge testing. Participants who were tested and provided feedback had significantly greater information recall compared to those who were tested but not provided feedback and to those not tested. The effect of condition was stronger for moderately difficult questions versus easy questions. Inserting knowledge tests and providing feedback about the responses at timed intervals in videos can be effective in improving recall of information. Providing informed consent information through a video not only standardizes the material, but using testing with feedback inserted within the video has the potential to increase recall and retention of this material. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

The Answering Process for Multiple-Choice Questions in Collaborative Learning: A Mathematical Learning Model Analysis

ERIC Educational Resources Information Center

Nakamura, Yasuyuki; Nishi, Shinnosuke; Muramatsu, Yuta; Yasutake, Koichi; Yamakawa, Osamu; Tagawa, Takahiro

2014-01-01

In this paper, we introduce a mathematical model for collaborative learning and the answering process for multiple-choice questions. The collaborative learning model is inspired by the Ising spin model and the model for answering multiple-choice questions is based on their difficulty level. An intensive simulation study predicts the possibility of…

Asymmetry in Student Achievement on Multiple-Choice and Constructed-Response Items in Reversible Mathematics Processes

ERIC Educational Resources Information Center

Sangwin, Christopher J.; Jones, Ian

2017-01-01

In this paper we report the results of an experiment designed to test the hypothesis that when faced with a question involving the inverse direction of a reversible mathematical process, students solve a multiple-choice version by verifying the answers presented to them by the direct method, not by undertaking the actual inverse calculation.…

The Development of Multiple-Choice Items Consistent with the AP Chemistry Curriculum Framework to More Accurately Assess Deeper Understanding

ERIC Educational Resources Information Center

Domyancich, John M.

2014-01-01

Multiple-choice questions are an important part of large-scale summative assessments, such as the advanced placement (AP) chemistry exam. However, past AP chemistry exam items often lacked the ability to test conceptual understanding and higher-order cognitive skills. The redesigned AP chemistry exam shows a distinctive shift in item types toward…

Impression formation of tests: retrospective judgments of performance are higher when easier questions come first.

PubMed

Jackson, Abigail; Greene, Robert L

2014-11-01

Four experiments are reported on the importance of retrospective judgments of performance (postdictions) on tests. Participants answered general knowledge questions and estimated how many questions they answered correctly. They gave higher postdictions when easy questions preceded difficult questions. This was true when time to answer each question was equalized and constrained, when participants were instructed not to write answers, and when questions were presented in a multiple-choice format. Results are consistent with the notion that first impressions predominate in overall perception of test difficulty.

Duchenne Muscular Dystrophy: a Survey of Perspectives on Carrier Testing and Communication Within the Family.

PubMed

Hayes, Brenna; Hassed, Susan; Chaloner, Jae Lindsay; Aston, Christopher E; Guy, Carrie

2016-06-01

Carrier testing is widely available for multiple genetic conditions, and several professional organizations have created practice guidelines regarding appropriate clinical application and the testing of minors. Previous research has focused on carrier screening, predictive testing, and testing for X-linked conditions. However, family perspectives on carrier testing for X-linked lethal diseases have yet to be described. In this study, we explored communication within the family about carrier testing and the perspectives of mothers of sons with an X-linked lethal disease, Duchenne muscular dystrophy (DMD). Twenty-five mothers of sons with DMD participated in an anonymous online survey. Survey questions included multiple choice, Likert scale, and open ended, short answer questions. Analysis of the multiple choice and Likert scale questions revealed that most mothers preferred a gradual style of communication with their daughters regarding risk status. In addition, most participants reported having consulted with a genetic counselor and found it helpful. Comparisons between groups, analyzed using Fisher's exact tests, found no differences in preferred style due to mother's carrier status or having a daughter. Thematic analysis was conducted on responses to open ended questions. Themes identified included the impact of family implications, age and maturity, and a desire for autonomy regarding the decision to discuss and undergo carrier testing with at-risk daughters, particularly timing of these discussions. Implications for genetic counseling practice are discussed.

Criterion Referenced Inventory. Grade 7 Skill Clusters, Objectives, and Illustrations.

ERIC Educational Resources Information Center

Montgomery County Public Schools, Rockville, MD.

Part of a series of competency-based test materials for grades six through ten, this test booklet for seventh graders contains multiple-choice questions designed to aid in the evaluation of the pupils' library skills. Accompanied by a separate booklet of illustrations which are to be used in conjunction with the questions, the test covers the…

Development of Achievement Test: Validity and Reliability Study for Achievement Test on Matter Changing

ERIC Educational Resources Information Center

Kara, Filiz; Celikler, Dilek

2015-01-01

For "Matter Changing" unit included in the Secondary School 5th Grade Science Program, it is intended to develop a test conforming the gains described in the program, and that can determine students' achievements. For this purpose, a multiple-choice test of 48 questions is arranged, consisting of 8 questions for each gain included in the…

Components of Answers to Multiple-Choice Questions on a Published Reading Comprehension Test: An Application of the Hanna-Oaster Approach.

ERIC Educational Resources Information Center

Entin, Eileen B.; Klare, George B.

1980-01-01

An approach to assessing context dependence was applied to data from the Nelson-Denny Reading Test. The results suggest that scores on the difficult passages are inflated because the examinees can answer the questions without having to comprehend the passage. (MKM)

Selected Test Items in American History. Bulletin Number 6, Fifth Edition.

ERIC Educational Resources Information Center

Anderson, Howard R.; Lindquist, E. F.

Designed for high school students, this bulletin provides an extensive file of 1,062 multiple-choice questions in American history. Taken largely from the Iowa Every-Pupil Program and the Cooperative Test Service standardized examinations, the questions are chronologically divided into 16 topic areas. They include exploration and discovery;…

Potential Values of Incorporating a Multiple-Choice Question Construction in Physics Experimentation Instruction

NASA Astrophysics Data System (ADS)

Yu, Fu-Yun; Liu, Yu-Hsin

2005-09-01

The potential value of a multiple-choice question-construction instructional strategy for the support of students’ learning of physics experiments was examined in the study. Forty-two university freshmen participated in the study for a whole semester. A constant comparison method adopted to categorize students’ qualitative data indicated that the influences of multiple-choice question construction were evident in several significant ways (promoting constructive and productive studying habits; reflecting and previewing course-related materials; increasing in-group communication and interaction; breaking passive learning style and habits, etc.), which, worked together, not only enhanced students’ comprehension and retention of the obtained knowledge, but also helped distil a sense of empowerment and learning community within the participants. Analysis with one-group t-tests, using 3 as the expected mean, on quantitative data further found that students’ satisfaction toward past learning experience, and perceptions toward this strategy’s potentials for promoting learning were statistically significant at the 0.0005 level, while learning anxiety was not statistically significant. Suggestions for incorporating question-generation activities within classroom and topics for future studies were rendered.

The Prevalence of Multiple-Choice Testing in Registered Nurse Licensure-Qualifying Nursing Education Programs in New York State.

PubMed

Birkhead, Susan; Kelman, Glenda; Zittel, Barbara; Jatulis, Linnea

The aim of this study was to describe nurse educators' use of multiple-choice questions (MCQs) in testing in registered nurse licensure-qualifying nursing education programs in New York State. This study was a descriptive correlational analysis of data obtained from surveying 1,559 nurse educators; 297 educators from 61 institutions responded (response rate [RR] = 19 percent), yielding a final cohort of 200. MCQs were reported to comprise a mean of 81 percent of questions on a typical test. Baccalaureate program respondents were equally likely to use MCQs as associate degree program respondents (p > .05) but were more likely to report using other methods of assessing student achievement to construct course grades (p < .01). Both groups reported little use of alternate format-type questions. Respondent educators reported substantial reliance upon the use of MCQs, corroborating the limited data quantifying the prevalence of use of MCQ tests in licensure-qualifying nursing education programs.

Developing an Array Binary Code Assessment Rubric for Multiple- Choice Questions Using Item Arrays and Binary-Coded Responses

ERIC Educational Resources Information Center

Haro, Elizabeth K.; Haro, Luis S.

2014-01-01

The multiple-choice question (MCQ) is the foundation of knowledge assessment in K-12, higher education, and standardized entrance exams (including the GRE, MCAT, and DAT). However, standard MCQ exams are limited with respect to the types of questions that can be asked when there are only five choices. MCQs offering additional choices more…

Modeling Polytomous Item Responses Using Simultaneously Estimated Multinomial Logistic Regression Models

ERIC Educational Resources Information Center

Anderson, Carolyn J.; Verkuilen, Jay; Peyton, Buddy L.

2010-01-01

Survey items with multiple response categories and multiple-choice test questions are ubiquitous in psychological and educational research. We illustrate the use of log-multiplicative association (LMA) models that are extensions of the well-known multinomial logistic regression model for multiple dependent outcome variables to reanalyze a set of…

The Development of the Planet Formation Concept Inventory: A Preliminary Analysis of Version 1

NASA Astrophysics Data System (ADS)

Simon, Molly; Impey, Chris David; Buxner, Sanlyn

2018-01-01

The topic of planet formation is poorly represented in the educational literature, especially at the college level. As recently as 2014, when developing the Test of Astronomy Standards (TOAST), Slater (2014) noted that for two topics (formation of the Solar System and cosmology), “high quality test items that reflect our current understanding of students’ conceptions were not available [in the literature]” (Slater,2014, p. 8). Furthermore, nearly half of ASTR 101 enrollments are at 2 year/community colleges where both instructors and students have little access to current research and models of planet formation. In response, we administered six student replied response (SSR) short answer questions on the topic of planet formation to n = 1,050 students enrolled in introductory astronomy and planetary science courses at The University of Arizona in the Fall 2016 and Spring 2017 semesters. After analyzing and coding the data from the SSR questions, we developed a preliminary version of the Planet Formation Concept Inventory (PFCI). The PFCI is a multiple-choice instrument with 20 planet formation-related questions, and 4 demographic-related questions. We administered version 1 of the PFCI to six introductory astronomy and planetary science courses (n ~ 700 students) during the Fall 2017 semester. We provided students with 7-8 multiple-choice with explanation of reasoning (MCER) questions from the PFCI. Students selected an answer (similar to a traditional multiple-choice test), and then briefly explained why they chose the answer they did. We also conducted interviews with ~15 students to receive feedback on the quality of the questions and clarity of the instrument. We will present an analysis of the MCER responses and student interviews, and discuss any modifications that will be made to the instrument as a result.

Biology Question Generation from a Semantic Network

NASA Astrophysics Data System (ADS)

Zhang, Lishan

Science instructors need questions for use in exams, homework assignments, class discussions, reviews, and other instructional activities. Textbooks never have enough questions, so instructors must find them from other sources or generate their own questions. In order to supply instructors with biology questions, a semantic network approach was developed for generating open response biology questions. The generated questions were compared to professional authorized questions. To boost students' learning experience, adaptive selection was built on the generated questions. Bayesian Knowledge Tracing was used as embedded assessment of the student's current competence so that a suitable question could be selected based on the student's previous performance. A between-subjects experiment with 42 participants was performed, where half of the participants studied with adaptive selected questions and the rest studied with mal-adaptive order of questions. Both groups significantly improved their test scores, and the participants in adaptive group registered larger learning gains than participants in the control group. To explore the possibility of generating rich instructional feedback for machine-generated questions, a question-paragraph mapping task was identified. Given a set of questions and a list of paragraphs for a textbook, the goal of the task was to map the related paragraphs to each question. An algorithm was developed whose performance was comparable to human annotators. A multiple-choice question with high quality distractors (incorrect answers) can be pedagogically valuable as well as being much easier to grade than open-response questions. Thus, an algorithm was developed to generate good distractors for multiple-choice questions. The machine-generated multiple-choice questions were compared to human-generated questions in terms of three measures: question difficulty, question discrimination and distractor usefulness. By recruiting 200 participants from Amazon Mechanical Turk, it turned out that the two types of questions performed very closely on all the three measures.

Insights into Students' Conceptual Understanding Using Textual Analysis: A Case Study in Signal Processing

ERIC Educational Resources Information Center

Goncher, Andrea M.; Jayalath, Dhammika; Boles, Wageeh

2016-01-01

Concept inventory tests are one method to evaluate conceptual understanding and identify possible misconceptions. The multiple-choice question format, offering a choice between a correct selection and common misconceptions, can provide an assessment of students' conceptual understanding in various dimensions. Misconceptions of some engineering…

Teaching Critical Thinking without (Much) Writing: Multiple-Choice and Metacognition

ERIC Educational Resources Information Center

Bassett, Molly H.

2016-01-01

In this essay, I explore an exam format that pairs multiple-choice questions with required rationales. In a space adjacent to each multiple-choice question, students explain why or how they arrived at the answer they selected. This exercise builds the critical thinking skill known as metacognition, thinking about thinking, into an exam that also…

Criterion Referenced Assessment Bank. Grade 6 Skill Clusters, Objectives, and Illustrations.

ERIC Educational Resources Information Center

Montgomery County Public Schools, Rockville, MD.

Part of a series of competency-based test materials for grades six through ten, this set of nine test booklets for sixth graders contains multiple-choice questions designed to aid in the evaluation of the pupils' library skills. Accompanied by a separate, tenth booklet of illustrations which are to be used in conjunction with the questions, the…

Medical Students' vs. Family Physicians' Assessment of Practical and Logical Values of Pathophysiology Multiple-Choice Questions

ERIC Educational Resources Information Center

Secic, Damir; Husremovic, Dzenana; Kapur, Eldan; Jatic, Zaim; Hadziahmetovic, Nina; Vojnikovic, Benjamin; Fajkic, Almir; Meholjic, Amir; Bradic, Lejla; Hadzic, Amila

2017-01-01

Testing strategies can either have a very positive or negative effect on the learning process. The aim of this study was to examine the degree of consistency in evaluating the practicality and logic of questions from a medical school pathophysiology test, between students and family medicine doctors. The study engaged 77 family medicine doctors…

Comedy workshop: an enjoyable way to develop multiple-choice questions.

PubMed

Droegemueller, William; Gant, Norman; Brekken, Alvin; Webb, Lynn

2005-01-01

To describe an innovative method of developing multiple-choice items for a board certification examination. The development of appropriate multiple-choice items is definitely more of an art, rather than a science. The comedy workshop format for developing questions for a certification examination is similar to the process used by comedy writers composing scripts for television shows. This group format dramatically diminishes the frustrations faced by an individual question writer attempting to create items. The vast majority of our comedy workshop participants enjoy and prefer the comedy workshop format. It provides an ideal environment in which to teach and blend the talents of inexperienced and experienced question writers. This is a descriptive article, in which we suggest an innovative process in the art of creating multiple-choice items for a high-stakes examination.

Estimating the Effect on Grades of Using Multiple-Choice versus Constructive-Response Questions: Data from the Classroom

ERIC Educational Resources Information Center

Hickson, Stephen; Reed, W. Robert; Sander, Nicholas

2012-01-01

This study investigates the degree to which grades based solely on constructed-response (CR) questions differ from grades based solely on multiple-choice (MC) questions. If CR questions are to justify their higher costs, they should produce different grade outcomes than MC questions. We use a data set composed of thousands of observations on…

Team-based learning to improve learning outcomes in a therapeutics course sequence.

PubMed

Bleske, Barry E; Remington, Tami L; Wells, Trisha D; Dorsch, Michael P; Guthrie, Sally K; Stumpf, Janice L; Alaniz, Marissa C; Ellingrod, Vicki L; Tingen, Jeffrey M

2014-02-12

To compare the effectiveness of team-based learning (TBL) to that of traditional lectures on learning outcomes in a therapeutics course sequence. A revised TBL curriculum was implemented in a therapeutic course sequence. Multiple choice and essay questions identical to those used to test third-year students (P3) taught using a traditional lecture format were administered to the second-year pharmacy students (P2) taught using the new TBL format. One hundred thirty-one multiple-choice questions were evaluated; 79 tested recall of knowledge and 52 tested higher level, application of knowledge. For the recall questions, students taught through traditional lectures scored significantly higher compared to the TBL students (88%±12% vs. 82%±16%, p=0.01). For the questions assessing application of knowledge, no differences were seen between teaching pedagogies (81%±16% vs. 77%±20%, p=0.24). Scores on essay questions and the number of students who achieved 100% were also similar between groups. Transition to a TBL format from a traditional lecture-based pedagogy allowed P2 students to perform at a similar level as students with an additional year of pharmacy education on application of knowledge type questions. However, P3 students outperformed P2 students regarding recall type questions and overall. Further assessment of long-term learning outcomes is needed to determine if TBL produces more persistent learning and improved application in clinical settings.

Passage Independence within Standardized Reading Comprehension Tests

ERIC Educational Resources Information Center

Roy-Charland, Annie; Colangelo, Gabrielle; Foglia, Victoria; Reguigui, Leïla

2017-01-01

In tests used to measure reading comprehension, validity is important in obtaining accurate results. Unfortunately, studies have shown that people can correctly answer some questions of these tests without reading the related passage. These findings bring forth the need to address whether this phenomenon is observed in multiple-choice only tests…

The Performance of IRT Model Selection Methods with Mixed-Format Tests

ERIC Educational Resources Information Center

Whittaker, Tiffany A.; Chang, Wanchen; Dodd, Barbara G.

2012-01-01

When tests consist of multiple-choice and constructed-response items, researchers are confronted with the question of which item response theory (IRT) model combination will appropriately represent the data collected from these mixed-format tests. This simulation study examined the performance of six model selection criteria, including the…

Middle School Students' Responses to Two-Tier Tasks

ERIC Educational Resources Information Center

Haja, Shajahan; Clarke, David

2011-01-01

The structure of two-tier testing is such that the first tier consists of a multiple-choice question and the second tier requires justifications for choices of answers made in the first tier. This study aims to evaluate two-tier tasks in "proportion" in terms of students' capacity to write and select justifications and to examine the effect of…

La Mujer Chicana. (The Chicana Woman).

ERIC Educational Resources Information Center

Herrera, Gloria; Lizcano, Jeanette

As objectives for this secondary level unit, students are to: (1) read the unit with comprehension; (2) demonstrate their comprehension of the Chicana's history by participating in an oral discussion utilizing four discussion questions; and (3) correctly answer 15 of the 20 questions on a multiple choice test. The unit consists of a brief history…

The Relationship between Homework and Performance in an Introductory Operations Management Course.

ERIC Educational Resources Information Center

Peters, Michael; Kethley, Bryan; Bullington, Kimball

2002-01-01

Homework of 142 students in an operations management was graded and performance on a multiple-choice test was compared to that of 188 without graded homework. The treatment group had lower overall scores. Grading did not affect performance on quantitative questions but had a significant effect on nonquantitative questions. (SK)

Delayed, but not immediate, feedback after multiple-choice questions increases performance on a subsequent short-answer, but not multiple-choice, exam: evidence for the dual-process theory of memory.

PubMed

Sinha, Neha; Glass, Arnold Lewis

2015-01-01

Three experiments, two performed in the laboratory and one embedded in a college psychology lecture course, investigated the effects of immediate versus delayed feedback following a multiple-choice exam on subsequent short answer and multiple-choice exams. Performance on the subsequent multiple-choice exam was not affected by the timing of the feedback on the prior exam; however, performance on the subsequent short answer exam was better following delayed than following immediate feedback. This was true regardless of the order in which immediate versus delayed feedback was given. Furthermore, delayed feedback only had a greater effect than immediate feedback on subsequent short answer performance following correct, confident responses on the prior exam. These results indicate that delayed feedback cues a student's prior response and increases subsequent recollection of that response. The practical implication is that delayed feedback is better than immediate feedback during academic testing.

The Effect of English Language on Multiple Choice Question Scores of Thai Medical Students.

PubMed

Phisalprapa, Pochamana; Muangkaew, Wayuda; Assanasen, Jintana; Kunavisarut, Tada; Thongngarm, Torpong; Ruchutrakool, Theera; Kobwanthanakun, Surapon; Dejsomritrutai, Wanchai

2016-04-01

Universities in Thailand are preparing for Thailand's integration into the ASEAN Economic Community (AEC) by increasing the number of tests in English language. English language is not the native language of Thailand Differences in English language proficiency may affect scores among test-takers, even when subject knowledge among test-takers is comparable and may falsely represent the knowledge level of the test-taker. To study the impact of English language multiple choice test questions on test scores of medical students. The final examination of fourth-year medical students completing internal medicine rotation contains 120 multiple choice questions (MCQ). The languages used on the test are Thai and English at a ratio of 3:1. Individual scores of tests taken in both languages were collected and the effect of English language on MCQ was analyzed Individual MCQ scores were then compared with individual student English language proficiency and student grade point average (GPA). Two hundred ninety five fourth-year medical students were enrolled. The mean percentage of MCQ scores in Thai and English were significantly different (65.0 ± 8.4 and 56.5 ± 12.4, respectively, p < 0.001). The correlation between MCQ scores in Thai and English was fair (Spearman's correlation coefficient = 0.41, p < 0.001). Of 295 students, only 73 (24.7%) students scored higher when being tested in English than in Thai language. Students were classified into six grade categories (A, B+, B, C+, C, and D+), which cumulatively measured total internal medicine rotation performance score plus final examination score. MCQ scores from Thai language examination were more closely correlated with total course grades than were the scores from English language examination (Spearman's correlation coefficient = 0.73 (p < 0.001) and 0.53 (p < 0.001), respectively). The gap difference between MCQ scores in both languages was higher in borderline students than in the excellent student group (11.2 ± 11.2 and 7.1 ± 8.2, respectively, p < 0.001). Overall, average student English proficiency score was very high, at 3.71 ± 0.35 from a total of 4.00. Mean student GPA was 3.40 ± 0.33 from a possible 4.00. English language MCQ examination scores were more highly associated with GPA than with English language proficiency. The use of English language multiple choice question test may decrease scores of the fourth-year internal medicine post-rotation final examination, especially those of borderline students.

Testing to the Top: Everything But the Kitchen Sink?

ERIC Educational Resources Information Center

Dietel, Ron

2011-01-01

Two tests intended to measure student achievement of the Common Core State Standards will face intense scrutiny, but the test makers say they will include performance assessments and other items that are not multiple-choice questions. Incorporating performance items on this tests will bring up issues over scoring, costs, and validity.

ACER Chemistry Test Item Collection. ACER Chemtic Year 12.

ERIC Educational Resources Information Center

Australian Council for Educational Research, Hawthorn.

The chemistry test item banks contains 225 multiple-choice questions suitable for diagnostic and achievement testing; a three-page teacher's guide; answer key with item facilities; an answer sheet; and a 45-item sample achievement test. Although written for the new grade 12 chemistry course in Victoria, Australia, the items are widely applicable.…

Grade 12 Diploma Examination: Chemistry 30. January 1989 = Examen en vue du diplome douzieme annee: Chimie 30. Janvier 1989.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Chemistry 30 is a 12th-grade science course for students in Alberta, Canada. This document is the final test for the course. Intended for administration during January 1989, it contains 56 multiple-choice questions and three written-response questions. Two and one-half hours are allowed for completing the test. No answer key is included since…

Grade 12 Diploma Examination: Chemistry 30. June 1988 = Examen en vue du Diplome Douzieme Annee: Chimie 30. Juin 1988.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Chemistry 30 is a 12th-grade chemistry course for students in Alberta, Canada. This document is the final test for the course. Intended for administration during June, 1988, it contains 56 multiple-choice questions and three written-response questions. Two-and-one-half hours are allowed for completing the test. No answer key is included since…

Grade 12 Diploma Examination: Chemistry 30. January 1988 = Examen en vue du diplome douzieme annee: Chimie 30. Janvier 1988.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Chemistry 30 is a 12th-grade science course for students in Alberta, Canada. This document is the final test for the course. Intended for administration during January 1988, it contains 56 multiple-choice questions and three written-response questions. Two-and-one-half hours are allowed for completing the test. No answer key is included since…

Grade 12 Diploma Examination: Chemistry 30. June 1989 = Examen en vue du diplome douzieme annee: Chimie 30. Juin 1989.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Chemistry 30 is a 12th-grade science course for students in Alberta, Canada. This document is the final test for the course. Intended for administration during June 1989, it contains 56 multiple-choice questions and 3 written-response questions. Two-and-one-half hours are allowed for completing the test. No answer key is included since scoring is…

Grade 12 Diploma Examination: Biology 30. June 1988 = Examen en vue du Diplome Douzieme Annee: Biologie 30. Juin 1988.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Biology 30 is a 12th-grade science course for students in Alberta, Canada. This document is a final test for the course. Intended for administration during June, 1988, it contains 80 multiple-choice questions and five written-response questions. Two-and-one-half hours are allowed for completing the test. No answer key is included since scoring is…

Grade 12 Diploma Examination: Biology 30. January 1989 = Examen en vue du diplome douzieme annee: Biologie 30. Janvier 1989.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Biology 30 is a 12th-grade science course for students in Alberta, Canada. This document is the final test for the course. Intended for administration during January 1989, it contains 70 multiple-choice questions and seven written-response questions. Two-and-one-half hours are allowed for completing the test. No answer key is included since…

Grade 12 Diploma Examination: Biology 30. January 1988 = Examen en vue du diplome douzieme annee: Biologie 30. Janvier 1988.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Biology 30 is a 12th-grade science course for students in Alberta, Canada. This document is a final test for the course. Intended for administration during January 1988, it contains 80 multiple-choice questions and four written-response questions. Two-and-one-half hours are allowed for completing the test. No answer key is included since scoring…

Written Justifications to Multiple-Choice Concept Questions during Active Learning in Class

ERIC Educational Resources Information Center

Koretsky, Milo D.; Brooks, Bill J.; Higgins, Adam Z.

2016-01-01

Increasingly, instructors of large, introductory STEM courses are having students actively engage during class by answering multiple-choice concept questions individually and in groups. This study investigates the use of a technology-based tool that allows students to answer such questions during class. The tool also allows the instructor to…

Negatively-Worded Multiple Choice Questions: An Avoidable Threat to Validity

ERIC Educational Resources Information Center

Chiavaroli, Neville

2017-01-01

Despite the majority of MCQ writing guides discouraging the use of negatively-worded multiple choice questions (NWQs), they continue to be regularly used both in locally produced examinations and commercially available questions. There are several reasons why the use of NWQs may prove resistant to sound pedagogical advice. Nevertheless, systematic…

The Use of Management and Marketing Textbook Multiple-Choice Questions: A Case Study.

ERIC Educational Resources Information Center

Hampton, David R.; And Others

1993-01-01

Four management and four marketing professors classified multiple-choice questions in four widely adopted introductory textbooks according to the two levels of Bloom's taxonomy of educational objectives: knowledge and intellectual ability and skill. Inaccuracies may cause instructors to select questions that require less thinking than they intend.…

Climbing Bloom's Taxonomy Pyramid: Lessons from a Graduate Histology Course

ERIC Educational Resources Information Center

Zaidi, Nikki B.; Hwang, Charles; Scott, Sara; Stallard, Stefanie; Purkiss, Joel; Hortsch, Michael

2017-01-01

Bloom's taxonomy was adopted to create a subject-specific scoring tool for histology multiple-choice questions (MCQs). This Bloom's Taxonomy Histology Tool (BTHT) was used to analyze teacher- and student-generated quiz and examination questions from a graduate level histology course. Multiple-choice questions using histological images were…

Does Correct Answer Distribution Influence Student Choices When Writing Multiple Choice Examinations?

ERIC Educational Resources Information Center

Carnegie, Jacqueline A.

2017-01-01

Summative evaluation for large classes of first- and second-year undergraduate courses often involves the use of multiple choice question (MCQ) exams in order to provide timely feedback. Several versions of those exams are often prepared via computer-based question scrambling in an effort to deter cheating. An important parameter to consider when…

Pattern recognition as a concept for multiple-choice questions in a national licensing exam.

PubMed

Freiwald, Tilo; Salimi, Madjid; Khaljani, Ehsan; Harendza, Sigrid

2014-11-14

Multiple-choice questions (MCQ) are still widely used in high stakes medical exams. We wanted to examine whether and to what extent a national licensing exam uses the concept of pattern recognition to test applied clinical knowledge. We categorized all 4,134 German National medical licensing exam questions between October 2006 and October 2012 by discipline, year, and type. We analyzed questions from the four largest disciplines: internal medicine (n = 931), neurology (n = 305), pediatrics (n = 281), and surgery (n = 233), with respect to the following question types: knowledge questions (KQ), pattern recognition questions (PRQ), inverse PRQ (IPRQ), and pseudo PRQ (PPRQ). A total 51.1% of all questions were of a higher taxonomical order (PRQ and IPRQ) with a significant decrease in the percentage of these questions (p <0.001) from 2006 (61.5%) to 2012 (41.6%). The proportion of PRQs and IPRQs was significantly lower (p <0.001) in internal medicine and surgery, compared to neurology and pediatrics. PRQs were mostly used in questions about diagnoses (71.7%). A significantly higher (p <0.05) percentage of PR/therapy questions was found for internal medicine compared with neurology and pediatrics. The concept of pattern recognition is used with different priorities and to various extents by the different disciplines in a high stakes exam to test applied clinical knowledge. Being aware of this concept may aid in the design and balance of MCQs in an exam with respect to testing clinical reasoning as a desired skill at the threshold of postgraduate medical education.

Effects of Differential Feedback on the Answering of Two Types of Questions by Fifth- and Sixth-Graders.

ERIC Educational Resources Information Center

Peeck, J.

1979-01-01

Fifth- and sixth-graders were tested with multiple-choice fact and inference questions about a reading passage, then given feedback either with or without the text present. Retesting occurred four days later. Results corroborated recent findings on age-related improvement in inferential processing comprehension and memory. (Author/SJL)

The Display of Multiple Choice Question Bank on Microfilm

ERIC Educational Resources Information Center

Stevens, J. M.; Harris, F. T. C.

1977-01-01

An automated question bank maintained by the Department of Research and Services in Education at the Middlesex Hospital Medical School provides a printed copy of each of 25,000 multiple choice questions (95 percent relating to the whole spectrum of the medical curriculum). Problems with this procedure led to experimental work storing the data on…

Construction of Valid and Reliable Test for Assessment of Students

ERIC Educational Resources Information Center

Osadebe, P. U.

2015-01-01

The study was carried out to construct a valid and reliable test in Economics for secondary school students. Two research questions were drawn to guide the establishment of validity and reliability for the Economics Achievement Test (EAT). It is a multiple choice objective test of five options with 100 items. A sample of 1000 students was randomly…

Construction of Economics Achievement Test for Assessment of Students

ERIC Educational Resources Information Center

Osadebe, P. U.

2014-01-01

The study was carried out to construct a valid and reliable test in Economics for secondary school students. Two research questions were drawn to guide the establishment of validity and reliability for the Economics Achievement Test (EAT). It is a multiple choice objective test of five options with 100 items. A sample of 1000 students was randomly…

Test of Achievement in Quantitative Economics for Secondary Schools: Construction and Validation Using Item Response Theory

ERIC Educational Resources Information Center

Eleje, Lydia I.; Esomonu, Nkechi P. M.

2018-01-01

A Test to measure achievement in quantitative economics among secondary school students was developed and validated in this study. The test is made up 20 multiple choice test items constructed based on quantitative economics sub-skills. Six research questions guided the study. Preliminary validation was done by two experienced teachers in…

The effectiveness of the 1996 Wood Magic Science Fair as an experiential field trip

NASA Astrophysics Data System (ADS)

Pearson, Robert Wayne

In response to the need for evaluating the effectiveness of the 1996 Wood Magic Science Fair (WMSF) at Mississippi State University's Forest and Wildlife Research Center/Forest Products Laboratory, two tests were developed. One test was designed for third-grade students and one for fourth-grade students who would attend the WMSF. Both tests have multiple choice answers and contained thirteen questions each. Five of the questions addressed general issues of the forest products industry that would be mentioned in an opening video but not stressed in the oral presentations of the WMSF. The students would have no active involvement in these presentations. These general issue questions represent passive involvement teaching. The eight remaining questions addressed specific information that would be stressed in the WMSF oral presentations and would allow students to actively participate in the presentations. The participation of the students in these presentations represents participatory teaching. The tests were given to the students (third and fourth grades) both before they attended the fair as a pre-test and after their attendance as a post-test with the only difference being that the multiple choice answers were arranged in a different order. Classroom teachers administered the tests to the students. The test results were evaluated for each class individually. Each question was evaluated and the results recorded in a table. The general information questions were grouped together as were the specific information questions. This grouping allowed a comparison to be made between passive and participatory learning. The results from the tests will help those making the science fair presentations evaluate their materials and methods. Statistical analysis of the results would indicate which questions had a significant change in the number of correct answers between the pre-test and post-test. This information will help the Fair planners and presenters in developing future programs.

Analyzing Student Confidence in Classroom Voting with Multiple Choice Questions

ERIC Educational Resources Information Center

Stewart, Ann; Storm, Christopher; VonEpps, Lahna

2013-01-01

The purpose of this paper is to present results of a recent study in which students voted on multiple choice questions in mathematics courses of varying levels. Students used clickers to select the best answer among the choices given; in addition, they were also asked whether they were confident in their answer. In this paper we analyze data…

Project Physics Tests 6, The Nucleus.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 6 are presented in this booklet. Included are 70 multiple-choice and 24 problem-and-essay questions. Nuclear physics fundamentals are examined with respect to the shell model, isotopes, neutrons, protons, nuclides, charge-to-mass ratios, alpha particles, Becquerel's discovery, gamma rays, cyclotrons,…

Ability Level Estimation of Students on Probability Unit via Computerized Adaptive Testing

ERIC Educational Resources Information Center

Özyurt, Hacer; Özyurt, Özcan

2015-01-01

Problem Statement: Learning-teaching activities bring along the need to determine whether they achieve their goals. Thus, multiple choice tests addressing the same set of questions to all are frequently used. However, this traditional assessment and evaluation form contrasts with modern education, where individual learning characteristics are…

Project Physics Tests 1, Concepts of Motion.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 1 are presented in this booklet, consisting of 70 multiple-choice and 20 problem-and-essay questions. Concepts of motion are examined with respect to velocities, acceleration, forces, vectors, Newton's laws, and circular motion. Suggestions are made for time consumption in answering some items. Besides…

Project Physics Tests 5, Models of the Atom.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 5 are presented in this booklet. Included are 70 multiple-choice and 23 problem-and-essay questions. Concepts of atomic model are examined on aspects of relativistic corrections, electron emission, photoelectric effects, Compton effect, quantum theories, electrolysis experiments, atomic number and mass,…

Project Physics Tests 4, Light and Electromagnetism.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 4 are presented in this booklet. Included are 70 multiple-choice and 22 problem-and-essay questions. Concepts of light and electromagnetism are examined on charges, reflection, electrostatic forces, electric potential, speed of light, electromagnetic waves and radiations, Oersted's and Faraday's work,…

A Database System for Course Administration.

ERIC Educational Resources Information Center

Benbasat, Izak; And Others

1982-01-01

Describes a computer-assisted testing system which produces multiple-choice examinations for a college course in business administration. The system uses SPIRES (Stanford Public Information REtrieval System) to manage a database of questions and related data, mark-sense cards for machine grading tests, and ACL (6) (Audit Command Language) to…

Project Physics Tests 3, The Triumph of Mechanics.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 3 are presented in this booklet. Included are 70 multiple-choice and 20 problem-and-essay questions. Concepts of mechanics are examined on energy, momentum, kinetic theory of gases, pulse analyses, "heat death," water waves, power, conservation laws, normal distribution, thermodynamic laws, and…

Use of Multi-Response Format Test in the Assessment of Medical Students' Critical Thinking Ability.

PubMed

Mafinejad, Mahboobeh Khabaz; Arabshahi, Seyyed Kamran Soltani; Monajemi, Alireza; Jalili, Mohammad; Soltani, Akbar; Rasouli, Javad

2017-09-01

To evaluate students critical thinking skills effectively, change in assessment practices is must. The assessment of a student's ability to think critically is a constant challenge, and yet there is considerable debate on the best assessment method. There is evidence that the intrinsic nature of open and closed-ended response questions is to measure separate cognitive abilities. To assess critical thinking ability of medical students by using multi-response format of assessment. A cross-sectional study was conducted on a group of 159 undergraduate third-year medical students. All the participants completed the California Critical Thinking Skills Test (CCTST) consisting of 34 multiple-choice questions to measure general critical thinking skills and a researcher-developed test that combines open and closed-ended questions. A researcher-developed 48-question exam, consisting of 8 short-answers and 5 essay questions, 19 Multiple-Choice Questions (MCQ), and 16 True-False (TF) questions, was used to measure critical thinking skills. Correlation analyses were performed using Pearson's coefficient to explore the association between the total scores of tests and subtests. One hundred and fifty-nine students participated in this study. The sample comprised 81 females (51%) and 78 males (49%) with an age range of 20±2.8 years (mean 21.2 years). The response rate was 64.1%. A significant positive correlation was found between types of questions and critical thinking scores, of which the correlations of MCQ (r=0.82) and essay questions (r=0.77) were strongest. The significant positive correlations between multi-response format test and CCTST's subscales were seen in analysis, evaluation, inference and inductive reasoning. Unlike CCTST subscales, multi-response format test have weak correlation with CCTST total score (r=0.45, p=0.06). This study highlights the importance of considering multi-response format test in the assessment of critical thinking abilities of medical students by using both open and closed-ended response questions.

A Comparison of the Performance on Three Multiple Choice Question Papers in Obstetrics and Gynecology Over a Period of Three Years Administered at Five London Medical Schools

ERIC Educational Resources Information Center

Stevens, J. M.; And Others

1977-01-01

Five of the medical schools in the University of London collaborated in administering one multiple choice question paper in obstetrics and gynecology, and results showed differences in performance between the five schools on questions and alternatives within questions. The rank order of the schools may result from differences in teaching methods.…

Test Performance Vis a Vis Test-Taking Strategies in Reading Comprehension of English as a Second Language.

ERIC Educational Resources Information Center

Roizen, Merle A.

A study of students' self-expressed test-taking strategies for reading comprehension tests in English as a second language (ESL) had as subjects 86 Hebrew-speaking university students in Israel. The students were given a reading comprehension test, half with open-ended questions and half with multiple choice. Half the responses were anonymous and…

Evaluation of Students' Views about the Use of SCORM (Sharable Content Object Reference Model)-Compatible Materials in Physics Teaching

ERIC Educational Resources Information Center

Gonen, Selahattin; Basaran, Bulent

2013-01-01

In the present study, a web site including instructional materials such as Whiteboard Movies (WBM), simulations and animations and testing materials such as true-false, fill-in-the-blanks, puzzles, open-ended questions and multiple-choice questions was designed. The study was carried out with 76 students attending Dicle College (DC), Diyarbakir…

The Validity of the Major Field Test in Psychology as a Programme Assessment Tool

ERIC Educational Resources Information Center

Gallagher, Shawn P.; Cook, Shaun P.

2013-01-01

The Major Field Test in Psychology (MFT) is a standardised test designed to assess subject mastery at the conclusion of an undergraduate career. Eighty-one graduating majors completed the MFT and 56 of them also took a multiple-choice exam of questions drawn randomly from an introductory psychology test bank. Like the MFT, the constructed exam was…

Understanding Test-Takers' Perceptions of Difficulty in EAP Vocabulary Tests: The Role of Experiential Factors

ERIC Educational Resources Information Center

Oruç Ertürk, Nesrin; Mumford, Simon E.

2017-01-01

This study, conducted by two researchers who were also multiple-choice question (MCQ) test item writers at a private English-medium university in an English as a foreign language (EFL) context, was designed to shed light on the factors that influence test-takers' perceptions of difficulty in English for academic purposes (EAP) vocabulary, with the…

Comprehension of confidence intervals - development and piloting of patient information materials for people with multiple sclerosis: qualitative study and pilot randomised controlled trial.

PubMed

Rahn, Anne C; Backhus, Imke; Fuest, Franz; Riemann-Lorenz, Karin; Köpke, Sascha; van de Roemer, Adrianus; Mühlhauser, Ingrid; Heesen, Christoph

2016-09-20

Presentation of confidence intervals alongside information about treatment effects can support informed treatment choices in people with multiple sclerosis. We aimed to develop and pilot-test different written patient information materials explaining confidence intervals in people with relapsing-remitting multiple sclerosis. Further, a questionnaire on comprehension of confidence intervals was developed and piloted. We developed different patient information versions aiming to explain confidence intervals. We used an illustrative example to test three different approaches: (1) short version, (2) "average weight" version and (3) "worm prophylaxis" version. Interviews were conducted using think-aloud and teach-back approaches to test feasibility and analysed using qualitative content analysis. To assess comprehension of confidence intervals, a six-item multiple choice questionnaire was developed and tested in a pilot randomised controlled trial using the online survey software UNIPARK. Here, the average weight version (intervention group) was tested against a standard patient information version on confidence intervals (control group). People with multiple sclerosis were invited to take part using existing mailing-lists of people with multiple sclerosis in Germany and were randomised using the UNIPARK algorithm. Participants were blinded towards group allocation. Primary endpoint was comprehension of confidence intervals, assessed with the six-item multiple choice questionnaire with six points representing perfect knowledge. Feasibility of the patient information versions was tested with 16 people with multiple sclerosis. For the pilot randomised controlled trial, 64 people with multiple sclerosis were randomised (intervention group: n = 36; control group: n = 28). More questions were answered correctly in the intervention group compared to the control group (mean 4.8 vs 3.8, mean difference 1.1 (95 % CI 0.42-1.69), p = 0.002). The questionnaire's internal consistency was moderate (Cronbach's alpha = 0.56). The pilot-phase shows promising results concerning acceptability and feasibility. Pilot randomised controlled trial results indicate that the patient information is well understood and that knowledge gain on confidence intervals can be assessed with a set of six questions. German Clinical Trials Register: DRKS00008561 . Registered 8th of June 2015.

Regulating Accuracy on University Tests with the Plurality Option

ERIC Educational Resources Information Center

Higham, Philip A.

2013-01-01

A single experiment is reported in which introductory psychology students were administered a multiple-choice test on psychology with either 4 (n = 78) or 5 alternatives (n = 92) prior to any lectures being delivered. Two answers were generated for each question: a small answer consisting of their favorite alternative, and a large answer…

Components of Spatial Thinking: Evidence from a Spatial Thinking Ability Test

ERIC Educational Resources Information Center

Lee, Jongwon; Bednarz, Robert

2012-01-01

This article introduces the development and validation of the spatial thinking ability test (STAT). The STAT consists of sixteen multiple-choice questions of eight types. The STAT was validated by administering it to a sample of 532 junior high, high school, and university students. Factor analysis using principal components extraction was applied…

Project Physics Tests 2, Motion in the Heavens.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 2 are presented in this booklet. Included are 70 multiple-choice and 22 problem-and-essay questions. Concepts of motion in the heavens are examined for planetary motions, heliocentric theory, forces exerted on the planets, Kepler's laws, gravitational force, Galileo's work, satellite orbits, Jupiter's…

SU-E-E-02: An Excel-Based Study Tool for ABR-Style Exams

DOE Office of Scientific and Technical Information (OSTI.GOV)

Cline, K; Stanley, D; Defoor, D

2015-06-15

Purpose: As the landscape of learning and testing shifts toward a computer-based environment, a replacement for paper-based methods of studying is desirable. Using Microsoft Excel, a study tool was developed that allows the user to populate multiple-choice questions and then generate an interactive quiz session to answer them. Methods: The code for the tool was written using Microsoft Excel Visual Basic for Applications with the intent that this tool could be implemented by any institution with Excel. The base tool is a template with a setup macro, which builds out the structure based on user’s input. Once the framework ismore » built, the user can input sets of multiple-choice questions, answer choices, and even add figures. The tool can be run in random-question or sequential-question mode for single or multiple courses of study. The interactive session allows the user to select answer choices and immediate feedback is provided. Once the user is finished studying, the tool records the day’s progress by reporting progress statistics useful for trending. Results: Six doctoral students at UTHSCSA have used this tool for the past two months to study for their qualifying exam, which is similar in format and content to the American Board of Radiology (ABR) Therapeutic Part II exam. The students collaborated to create a repository of questions, met weekly to go over these questions, and then used the tool to prepare for their exam. Conclusion: The study tool has provided an effective and efficient way for students to collaborate and be held accountable for exam preparation. The ease of use and familiarity of Excel are important factors for the tool’s use. There are software packages to create similar question banks, but this study tool has no additional cost for those that already have Excel. The study tool will be made openly available.« less

Bilingual Test as a Test Accommodation to Determine the Mathematics Achievement of Mainstream Students with Limited English Proficiency

ERIC Educational Resources Information Center

Shanmugam, S. Kanageswari Suppiah; Lan, Ong Saw

2013-01-01

Purpose: This study aims to investigate the validity of using bilingual test to measure the mathematics achievement of students who have limited English proficiency (LEP). The bilingual test and the English-only test consist of 20 computation and 20 word problem multiple-choice questions (from TIMSS 2003 and 2007 released items. The bilingual test…

Graded Multiple Choice Questions: Rewarding Understanding and Preventing Plagiarism

NASA Astrophysics Data System (ADS)

Denyer, G. S.; Hancock, D.

2002-08-01

This paper describes an easily implemented method that allows the generation and analysis of graded multiple-choice examinations. The technique, which uses standard functions in user-end software (Microsoft Excel 5+), can also produce several different versions of an examination that can be employed to prevent the reward of plagarism. The manuscript also discusses the advantages of having a graded marking system for the elimination of ambiguities, use in multi-step calculation questions, and questions that require extrapolation or reasoning. The advantages of the scrambling strategy, which maintains the same question order, is discussed with reference to student equity. The system provides a non-confrontational mechanism for dealing with cheating in large-class multiple-choice examinations, as well as providing a reward for problem solving over surface learning.

Constructing objective tests

NASA Astrophysics Data System (ADS)

Aubrecht, Gordon J.; Aubrecht, Judith D.

1983-07-01

True-false or multiple-choice tests can be useful instruments for evaluating student progress. We examine strategies for planning objective tests which serve to test the material covered in science (physics) courses. We also examine strategies for writing questions for tests within a test blueprint. The statistical basis for judging the quality of test items are discussed. Reliability, difficulty, and discrimination indices are defined and examples presented. Our recommendation are rather easily put into practice.

Confidence-Based Learning in Investment Analysis

NASA Astrophysics Data System (ADS)

Serradell-Lopez, Enric; Lara-Navarra, Pablo; Castillo-Merino, David; González-González, Inés

The aim of this study is to determine the effectiveness of using multiple choice tests in subjects related to the administration and business management. To this end we used a multiple-choice test with specific questions to verify the extent of knowledge gained and the confidence and trust in the answers. The tests were performed in a group of 200 students at the bachelor's degree in Business Administration and Management. The analysis made have been implemented in one subject of the scope of investment analysis and measured the level of knowledge gained and the degree of trust and security in the responses at two different times of the course. The measurements have been taken into account different levels of difficulty in the questions asked and the time spent by students to complete the test. The results confirm that students are generally able to obtain more knowledge along the way and get increases in the degree of trust and confidence in the answers. It is confirmed as the difficulty level of the questions set a priori by the heads of the subjects are related to levels of security and confidence in the answers. It is estimated that the improvement in the skills learned is viewed favourably by businesses and are especially important for job placement of students.

An Assessment of Pharmacy Student Confidence in Learning.

ERIC Educational Resources Information Center

Popovich, Nicholas G.; Rogers, Wallace J.

1987-01-01

A study to determine student knowledge and confidence in that knowledge when answering multiple-choice examination questions in a nonprescription drug course is described. An alternate approach to methods of confidence testing was investigated. The knowledge and experience survey is appended. (Author/MLW)

MCAID--A Generalized Text Driver.

ERIC Educational Resources Information Center

Ahmed, K.; Dickinson, C. J.

MCAID is a relatively machine-independent technique for writing computer-aided instructional material consisting of descriptive text, multiple choice questions, and the ability to call compiled subroutines to perform extensive calculations. It was specially developed to incorporate test-authoring around complex mathematical models to explore a…

Dividing the Force Concept Inventory into Two Equivalent Half-Length Tests

ERIC Educational Resources Information Center

Han, Jing; Bao, Lei; Chen, Li; Cai, Tianfang; Pi, Yuan; Zhou, Shaona; Tu, Yan; Koenig, Kathleen

2015-01-01

The Force Concept Inventory (FCI) is a 30-question multiple-choice assessment that has been a building block for much of the physics education research done today. In practice, there are often concerns regarding the length of the test and possible test-retest effects. Since many studies in the literature use the mean score of the FCI as the…

Delayed Instructional Feedback May Be More Effective, but Is This Contrary to Learners' Preferences?

ERIC Educational Resources Information Center

Lefevre, David; Cox, Benita

2017-01-01

This research investigates learners' preferences for the timing of feedback provided to multiple-choice questions within technology-based instruction, hitherto an area of little empirical attention. Digital materials are undergoing a period of renewed prominence within online learning and multiple-choice questions remain a common component. There…

FAA Pilot Knowledge Tests: Learning or Rote Memorization?

NASA Technical Reports Server (NTRS)

Casner, Stephen M.; Jones, Karen M.; Puentes, Antonio; Irani, Homi

2004-01-01

The FAA pilot knowledge test is a multiple-choice assessment tool designed to measure the extent to which applicants for FAA pilot certificates and ratings have mastered a corpus of required aeronautical knowledge. All questions that appear on the test are drawn from a database of questions that is made available to the public. The FAA and others are concerned that releasing test questions may encourage students to focus their study on memorizing test questions. To investigate this concern, we created our own database of questions that differed from FAA questions in four different ways. Our first three question types were derived by modifying existing FAA questions: (1) rewording questions and answers; (2) shuffling answers; and (3) substituting different figures for problems that used figures. Our last question type posed a question about required knowledge for which no FAA question currently exists. Forty-eight student pilots completed one of two paper-and-pencil knowledge tests that contained a mix of these experimental questions. The results indicate significantly lower scores for some question types when compared to unaltered FAA questions to which participants had prior access.

Electronic evaluation for freshmen nursing students: Advantages and disadvantages.

PubMed

González, Fidelina; Ramírez, Julia

2006-01-01

An electronic evaluation was applied to freshmen nursing students who are taking Biology and Nursing Introduction. Questions and alternatives were randomly ordered. Evaluation took place simultaneously for all the students. Students preferred multiple choice questions with no more than two lines of headings and requested about three minutes for test timing. Test scores from electronic evaluation were higher than the ones obtained with written evaluation. This result implies that information technology can be used for evaluating large group of students.

How Well Do Engineering Students Retain Core Mathematical Knowledge after a Series of High Threshold Online Mathematics Tests?

ERIC Educational Resources Information Center

Carr, Michael; Prendergast, Mark; Breen, Cormac; Faulkner, Fiona

2017-01-01

In the Dublin Institute of Technology, high threshold core skills assessments are run in mathematics for third-year engineering students. Such tests require students to reach a threshold of 90% on a multiple choice test based on a randomized question bank. The material covered by the test consists of the more important aspects of undergraduate…

Helping physics teacher-candidates develop questioning skills through innovative technology use

NASA Astrophysics Data System (ADS)

Milner-Bolotin, Marina

2015-12-01

Peer Instruction has been used successfully in undergraduate classrooms for decades. Its success depends largely on the quality of multiple-choice questions. Yet it is still rare in secondary schools because of teachers' lack of experience in designing, evaluating, and implementing conceptual questions. Research-based multiple-choice conceptual questions are also underutilized in physics teacher education. This study explores the implementation of Peer Instruction enhanced by PeerWise collaborative online system, in a physics methods course in a physics teacher education program.

Does Educator Training or Experience Affect the Quality of Multiple-Choice Questions?

PubMed

Webb, Emily M; Phuong, Jonathan S; Naeger, David M

2015-10-01

Physicians receive little training on proper multiple-choice question (MCQ) writing methods. Well-constructed MCQs follow rules, which ensure that a question tests what it is intended to test. Questions that break these are described as "flawed." We examined whether the prevalence of flawed questions differed significantly between those with or without prior training in question writing and between those with different levels of educator experience. We assessed 200 unedited MCQs from a question bank for our senior medical student radiology elective: an equal number of questions (50) were written by faculty with previous training in MCQ writing, other faculty, residents, and medical students. Questions were scored independently by two readers for the presence of 11 distinct flaws described in the literature. Questions written by faculty with MCQ writing training had significantly fewer errors: mean 0.4 errors per question compared to a mean of 1.5-1.7 errors per question for the other groups (P < .001). There were no significant differences in the total number of errors between the untrained faculty, residents, and students (P values .35-.91). Among trained faculty 17/50 questions (34%) were flawed, whereas other faculty wrote 38/50 (76%) flawed questions, residents 37/50 (74%), and students 44/50 (88%). Trained question writers' higher performance was mainly manifest in the reduced frequency of five specific errors. Faculty with training in effective MCQ writing made fewer errors in MCQ construction. Educator experience alone had no effect on the frequency of flaws; faculty without dedicated training, residents, and students performed similarly. Copyright © 2015 AUR. Published by Elsevier Inc. All rights reserved.

A multi-instructor, team-based, active-learning exercise to integrate basic and clinical sciences content.

PubMed

Kolluru, Srikanth; Roesch, Darren M; Akhtar de la Fuente, Ayesha

2012-03-12

To introduce a multiple-instructor, team-based, active-learning exercise to promote the integration of basic sciences (pathophysiology, pharmacology, and medicinal chemistry) and clinical sciences in a doctor of pharmacy curriculum. A team-based learning activity that involved pre-class reading assignments, individual-and team-answered multiple-choice questions, and evaluation and discussion of a clinical case, was designed, implemented, and moderated by 3 faculty members from the pharmaceutical sciences and pharmacy practice departments. Student performance was assessed using a multiple-choice examination, an individual readiness assurance test (IRAT), a team readiness assurance test (TRAT), and a subjective, objective, assessment, and plan (SOAP) note. Student attitudes were assessed using a pre- and post-exercise survey instrument. Students' understanding of possible correct treatment strategies for depression improved. Students were appreciative of this true integration of basic sciences knowledge in a pharmacotherapy course and to have faculty members from both disciplines present to answer questions. Mean student score on the on depression module for the examination was 80.4%, indicating mastery of the content. An exercise led by multiple instructors improved student perceptions of the importance of team-based teaching. Integrated teaching and learning may be achieved when instructors from multiple disciplines work together in the classroom using proven team-based, active-learning exercises.

Can a Two-Question Test Be Reliable and Valid for Predicting Academic Outcomes?

ERIC Educational Resources Information Center

Bridgeman, Brent

2016-01-01

Scores on essay-based assessments that are part of standardized admissions tests are typically given relatively little weight in admissions decisions compared to the weight given to scores from multiple-choice assessments. Evidence is presented to suggest that more weight should be given to these assessments. The reliability of the writing scores…

To Change or Not to Change: Investigating the Value of MCQ Answer Changing for Gulf Arab Students

ERIC Educational Resources Information Center

Al-Hamly, Mashael; Coombe, Christine

2005-01-01

This study investigates whether the practice of answer changing on multiple-choice questions (MCQs) is beneficial to Gulf Arab students' overall test performance. The proficiency exam used in this study is the Michigan English Language Institute College English Test - Grammar, Cloze, Vocabulary, Reading (MELICET-GCVR), which was developed using…

Use of MCAT Data in Admissions. A Guide for Medical School Admissions Officers and Faculty.

ERIC Educational Resources Information Center

Mitchell, Karen J.

A description of the standardized, multiple-choice Medical College Admission Test (MCAT) and how to use it is offered. Medical school admissions officers medical educators, college faculty members, and practicing physicians are active participants in selecting content, drafting test specifications, and authoring questions for the exam. The MCAT is…

On the Optimality of Answer-Copying Indices: Theory and Practice

ERIC Educational Resources Information Center

Romero, Mauricio; Riascos, Álvaro; Jara, Diego

2015-01-01

Multiple-choice exams are frequently used as an efficient and objective method to assess learning, but they are more vulnerable to answer copying than tests based on open questions. Several statistical tests (known as indices in the literature) have been proposed to detect cheating; however, to the best of our knowledge, they all lack mathematical…

Industrial Arts Test Development, Book III. Resource Items for Graphics Technology, Power Technology, Production Technology.

ERIC Educational Resources Information Center

New York State Education Dept., Albany.

This booklet is designed to assist teachers in developing examinations for classroom use. It is a collection of 955 objective test questions, mostly multiple choice, for industrial arts students in the three areas of graphics technology, power technology, and production technology. Scoring keys are provided. There are no copyright restrictions,…

Anonymity and Electronics: Adapting Preparation for Radiology Resident Examination.

PubMed

Chapman, Teresa; Reid, Janet R; O'Conner, Erin E

2017-06-01

Diagnostic radiology resident assessment has evolved from a traditional oral examination to computerized testing. Teaching faculty struggle to reconcile the differences between traditional teaching methods and residents' new preferences for computerized testing models generated by new examination styles. We aim to summarize the collective experiences of senior residents at three different teaching hospitals who participated in case review sessions using a computer-based, interactive, anonymous teaching tool, rather than the Socratic method. Feedback was collected from radiology residents following participation in a senior resident case review session using Nearpod, which allows residents to anonymously respond to the teaching material. Subjective resident feedback was uniformly enthusiastic. Ninety percent of residents favor a case-based board review incorporating multiple-choice questions, and 94% favor an anonymous response system. Nearpod allows for inclusion of multiple-choice questions while also providing direct feedback to the teaching faculty, helping to direct the instruction and clarify residents' gaps in knowledge before the Core Examination. Copyright © 2017 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved.

The Effectiveness of Problem-Based Learning Approach Based on Multiple Intelligences in Terms of Student’s Achievement, Mathematical Connection Ability, and Self-Esteem

NASA Astrophysics Data System (ADS)

Kartikasari, A.; Widjajanti, D. B.

2017-02-01

The aim of this study is to explore the effectiveness of learning approach using problem-based learning based on multiple intelligences in developing student’s achievement, mathematical connection ability, and self-esteem. This study is experimental research with research sample was 30 of Grade X students of MIA III MAN Yogyakarta III. Learning materials that were implemented consisting of trigonometry and geometry. For the purpose of this study, researchers designed an achievement test made up of 44 multiple choice questions with respectively 24 questions on the concept of trigonometry and 20 questions for geometry. The researcher also designed a connection mathematical test and self-esteem questionnaire that consisted of 7 essay questions on mathematical connection test and 30 items of self-esteem questionnaire. The learning approach said that to be effective if the proportion of students who achieved KKM on achievement test, the proportion of students who achieved a minimum score of high category on the results of both mathematical connection test and self-esteem questionnaire were greater than or equal to 70%. Based on the hypothesis testing at the significance level of 5%, it can be concluded that the learning approach using problem-based learning based on multiple intelligences was effective in terms of student’s achievement, mathematical connection ability, and self-esteem.

Teaching and evaluating point of care learning with an Internet-based clinical-question portfolio.

PubMed

Green, Michael L; Reddy, Siddharta G; Holmboe, Eric

2009-01-01

Diplomates in the American Board of Internal Medicine (ABIM) Maintenance of Certification (MOC) program satisfy the self-evaluation of medical knowledge requirement by completing open-book multiple-choice exams. However, this method remains unlikely to affect practice change and often covers content areas not relevant to diplomates' practices. We developed and evaluated an Internet-based point of care (POC) learning portfolio to serve as an alternative. Participants enter information about their clinical questions, including characteristics, information pursuit, application, and practice change. After documenting 20 questions, they reflect upon a summary report and write commitment-to-change statements about their learning strategies. They can link to help screens and medical information resources. We report on the beta test evaluation of the module, completed by 23 internists and 4 internal medicine residents. Participants found the instructions clear and navigated the module without difficulty. The majority preferred the POC portfolio to multiple-choice examinations, citing greater relevance to their practice, guidance in expanding their palette of information resources, opportunity to reflect on their learning needs, and "credit" for self-directed learning related to their patients. Participants entered a total of 543 clinical questions, of which 250 (46%) resulted in a planned practice change. After completing the module, 14 of 27 (52%) participants committed to at least 1 change in their POC learning strategies. Internists found the portfolio valuable, preferred it to multiple-choice examinations, often changed their practice after pursuing clinical questions, and productively reflected on their learning strategies. The ABIM will offer this portfolio as an elective option in MOC.

Evidence-based point-of-care tests and device designs for disaster preparedness.

PubMed

Brock, T Keith; Mecozzi, Daniel M; Sumner, Stephanie; Kost, Gerald J

2010-01-01

To define pathogen tests and device specifications needed for emerging point-of-care (POC) technologies used in disasters. Surveys included multiple-choice and ranking questions. Multiple-choice questions were analyzed with the chi2 test for goodness-of-fit and the binomial distribution test. Rankings were scored and compared using analysis of variance and Tukey's multiple comparison test. Disaster care experts on the editorial boards of the American Journal of Disaster Medicine and the Disaster Medicine and Public Health Preparedness, and the readers of the POC Journal. Vibrio cholera and Staphylococcus aureus were top-ranked pathogens for testing in disaster settings. Respondents felt that disaster response teams should be equipped with pandemic infectious disease tests for novel 2009 H1N1 and avian H5N1 influenza (disaster care, p < 0.05; POC, p < 0.01). In disaster settings, respondents preferred self-contained test cassettes (disaster care, p < 0.05; POC, p < 0.001) for direct blood sampling (POC, p < 0.01) and disposal of biological waste (disaster care, p < 0.05; POC, p < 0.001). Multiplex testing performed at the POC was preferred in urgent care and emergency room settings. Evidence-based needs assessment identifies pathogen detection priorities in disaster care scenarios, in which Vibrio cholera, methicillin-sensitive and methicillin-resistant Staphylococcus aureus, and Escherichia coli ranked the highest. POC testing should incorporate setting-specific design criteria such as safe disposable cassettes and direct blood sampling at the site of care.

The Testing Methods and Gender Differences in Multiple-Choice Assessment

NASA Astrophysics Data System (ADS)

Ng, Annie W. Y.; Chan, Alan H. S.

2009-10-01

This paper provides a comprehensive review of the multiple-choice assessment in the past two decades for facilitating people to conduct effective testing in various subject areas. It was revealed that a variety of multiple-choice test methods viz. conventional multiple-choice, liberal multiple-choice, elimination testing, confidence marking, probability testing, and order-of-preference scheme are available for use in assessing subjects' knowledge and decision ability. However, the best multiple-choice test method for use has not yet been identified. The review also indicated that the existence of gender differences in multiple-choice task performance might be due to the test area, instruction/scoring condition, and item difficulty.

Gender and Performance in Accounting Examinations: Exploring the Impact of Examination Format

ERIC Educational Resources Information Center

Arthur, Neal; Everaert, Patricia

2012-01-01

This paper addresses the question of whether the increasing use of multiple-choice questions will favour particular student groups, i.e. male or female students. Using data from Belgium, this paper empirically examines the existence of a gender effect by comparing the relative performance of male and female students in both multiple-choice and…

A Participatory Learning Approach to Biochemistry Using Student Authored and Evaluated Multiple-Choice Questions

ERIC Educational Resources Information Center

Bottomley, Steven; Denny, Paul

2011-01-01

A participatory learning approach, combined with both a traditional and a competitive assessment, was used to motivate students and promote a deep approach to learning biochemistry. Students were challenged to research, author, and explain their own multiple-choice questions (MCQs). They were also required to answer, evaluate, and discuss MCQs…

Validity and Reliability of Scores Obtained on Multiple-Choice Questions: Why Functioning Distractors Matter

ERIC Educational Resources Information Center

Ali, Syed Haris; Carr, Patrick A.; Ruit, Kenneth G.

2016-01-01

Plausible distractors are important for accurate measurement of knowledge via multiple-choice questions (MCQs). This study demonstrates the impact of higher distractor functioning on validity and reliability of scores obtained on MCQs. Freeresponse (FR) and MCQ versions of a neurohistology practice exam were given to four cohorts of Year 1 medical…

Multiple-choice tests stabilize access to marginal knowledge.

PubMed

Cantor, Allison D; Eslick, Andrea N; Marsh, Elizabeth J; Bjork, Robert A; Bjork, Elizabeth Ligon

2015-02-01

Marginal knowledge refers to knowledge that is stored in memory, but is not accessible at a given moment. For example, one might struggle to remember who wrote The Call of the Wild, even if that knowledge is stored in memory. Knowing how best to stabilize access to marginal knowledge is important, given that new learning often requires accessing and building on prior knowledge. While even a single opportunity to restudy marginal knowledge boosts its later accessibility (Berger, Hall, & Bahrick, 1999), in many situations explicit relearning opportunities are not available. Our question is whether multiple-choice tests (which by definition expose the learner to the correct answers) can also serve this function and, if so, how testing compares to restudying given that tests can be particularly powerful learning devices (Roediger & Karpicke, 2006). In four experiments, we found that multiple-choice testing had the power to stabilize access to marginal knowledge, and to do so for at least up to a week. Importantly, such tests did not need to be paired with feedback, although testing was no more powerful than studying. Overall, the results support the idea that one's knowledge base is unstable, with individual pieces of information coming in and out of reach. The present findings have implications for a key educational challenge: ensuring that students have continuing access to information they have learned.

Use of Multi-Response Format Test in the Assessment of Medical Students’ Critical Thinking Ability

PubMed Central

Mafinejad, Mahboobeh Khabaz; Monajemi, Alireza; Jalili, Mohammad; Soltani, Akbar; Rasouli, Javad

2017-01-01

Introduction To evaluate students critical thinking skills effectively, change in assessment practices is must. The assessment of a student’s ability to think critically is a constant challenge, and yet there is considerable debate on the best assessment method. There is evidence that the intrinsic nature of open and closed-ended response questions is to measure separate cognitive abilities. Aim To assess critical thinking ability of medical students by using multi-response format of assessment. Materials and Methods A cross-sectional study was conducted on a group of 159 undergraduate third-year medical students. All the participants completed the California Critical Thinking Skills Test (CCTST) consisting of 34 multiple-choice questions to measure general critical thinking skills and a researcher-developed test that combines open and closed-ended questions. A researcher-developed 48-question exam, consisting of 8 short-answers and 5 essay questions, 19 Multiple-Choice Questions (MCQ), and 16 True-False (TF) questions, was used to measure critical thinking skills. Correlation analyses were performed using Pearson’s coefficient to explore the association between the total scores of tests and subtests. Results One hundred and fifty-nine students participated in this study. The sample comprised 81 females (51%) and 78 males (49%) with an age range of 20±2.8 years (mean 21.2 years). The response rate was 64.1%. A significant positive correlation was found between types of questions and critical thinking scores, of which the correlations of MCQ (r=0.82) and essay questions (r=0.77) were strongest. The significant positive correlations between multi-response format test and CCTST’s subscales were seen in analysis, evaluation, inference and inductive reasoning. Unlike CCTST subscales, multi-response format test have weak correlation with CCTST total score (r=0.45, p=0.06). Conclusion This study highlights the importance of considering multi-response format test in the assessment of critical thinking abilities of medical students by using both open and closed-ended response questions. PMID:29207742

Multiple choice questions can be designed or revised to challenge learners' critical thinking.

PubMed

Tractenberg, Rochelle E; Gushta, Matthew M; Mulroney, Susan E; Weissinger, Peggy A

2013-12-01

Multiple choice (MC) questions from a graduate physiology course were evaluated by cognitive-psychology (but not physiology) experts, and analyzed statistically, in order to test the independence of content expertise and cognitive complexity ratings of MC items. Integration of higher order thinking into MC exams is important, but widely known to be challenging-perhaps especially when content experts must think like novices. Expertise in the domain (content) may actually impede the creation of higher-complexity items. Three cognitive psychology experts independently rated cognitive complexity for 252 multiple-choice physiology items using a six-level cognitive complexity matrix that was synthesized from the literature. Rasch modeling estimated item difficulties. The complexity ratings and difficulty estimates were then analyzed together to determine the relative contributions (and independence) of complexity and difficulty to the likelihood of correct answers on each item. Cognitive complexity was found to be statistically independent of difficulty estimates for 88 % of items. Using the complexity matrix, modifications were identified to increase some item complexities by one level, without affecting the item's difficulty. Cognitive complexity can effectively be rated by non-content experts. The six-level complexity matrix, if applied by faculty peer groups trained in cognitive complexity and without domain-specific expertise, could lead to improvements in the complexity targeted with item writing and revision. Targeting higher order thinking with MC questions can be achieved without changing item difficulties or other test characteristics, but this may be less likely if the content expert is left to assess items within their domain of expertise.

Grade 12 Diploma Examination: Biology 30. June 1989 = Examen en vue du diplome douzieme annee: Biologie 30. Juin 1989.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Biology 30 is a 12th-grade science course for students in Alberta, Canada. Intended for administration during June 1989, it contains 70 multiple-choice questions and 7 written-response questions. Two-and-one-half hours are allowed for completing the test. No answer key is included since scoring is done by the provincial education department. The…

Is It Working? Distractor Analysis Results from the Test Of Astronomy STandards (TOAST) Assessment Instrument

NASA Astrophysics Data System (ADS)

Slater, Stephanie

2009-05-01

The Test Of Astronomy STandards (TOAST) assessment instrument is a multiple-choice survey tightly aligned to the consensus learning goals stated by the American Astronomical Society - Chair's Conference on ASTRO 101, the American Association of the Advancement of Science's Project 2061 Benchmarks, and the National Research Council's National Science Education Standards. Researchers from the Cognition in Astronomy, Physics and Earth sciences Research (CAPER) Team at the University of Wyoming's Science and Math Teaching Center (UWYO SMTC) have been conducting a question-by-question distractor analysis procedure to determine the sensitivity and effectiveness of each item. In brief, the frequency each possible answer choice, known as a foil or distractor on a multiple-choice test, is determined and compared to the existing literature on the teaching and learning of astronomy. In addition to having statistical difficulty and discrimination values, a well functioning assessment item will show students selecting distractors in the relative proportions to how we expect them to respond based on known misconceptions and reasoning difficulties. In all cases, our distractor analysis suggests that all items are functioning as expected. These results add weight to the validity of the Test Of Astronomy STandards (TOAST) assessment instrument, which is designed to help instructors and researchers measure the impact of course-length duration instructional strategies for undergraduate science survey courses with learning goals tightly aligned to the consensus goals of the astronomy education community.

Standard setting: comparison of two methods.

PubMed

George, Sanju; Haque, M Sayeed; Oyebode, Femi

2006-09-14

The outcome of assessments is determined by the standard-setting method used. There is a wide range of standard-setting methods and the two used most extensively in undergraduate medical education in the UK are the norm-reference and the criterion-reference methods. The aims of the study were to compare these two standard-setting methods for a multiple-choice question examination and to estimate the test-retest and inter-rater reliability of the modified Angoff method. The norm-reference method of standard-setting (mean minus 1 SD) was applied to the 'raw' scores of 78 4th-year medical students on a multiple-choice examination (MCQ). Two panels of raters also set the standard using the modified Angoff method for the same multiple-choice question paper on two occasions (6 months apart). We compared the pass/fail rates derived from the norm reference and the Angoff methods and also assessed the test-retest and inter-rater reliability of the modified Angoff method. The pass rate with the norm-reference method was 85% (66/78) and that by the Angoff method was 100% (78 out of 78). The percentage agreement between Angoff method and norm-reference was 78% (95% CI 69% - 87%). The modified Angoff method had an inter-rater reliability of 0.81-0.82 and a test-retest reliability of 0.59-0.74. There were significant differences in the outcomes of these two standard-setting methods, as shown by the difference in the proportion of candidates that passed and failed the assessment. The modified Angoff method was found to have good inter-rater reliability and moderate test-retest reliability.

Courseware Review.

ERIC Educational Resources Information Center

Risley, John S.

1983-01-01

Describes computer program (available on diskette for Apple IIe/II-plus, Commodore PET/CBM, or Commodore 64) providing drill/practice on concepts of electric charge, electric current, and electric potential difference. A second diskette provides a test of fifteen multiple-choice questions, with option to print score and areas of weakness. (JM)

Multiple-Choice Exams: An Obstacle for Higher-Level Thinking in Introductory Science Classes

PubMed Central

Stanger-Hall, Kathrin F.

2012-01-01

Learning science requires higher-level (critical) thinking skills that need to be practiced in science classes. This study tested the effect of exam format on critical-thinking skills. Multiple-choice (MC) testing is common in introductory science courses, and students in these classes tend to associate memorization with MC questions and may not see the need to modify their study strategies for critical thinking, because the MC exam format has not changed. To test the effect of exam format, I used two sections of an introductory biology class. One section was assessed with exams in the traditional MC format, the other section was assessed with both MC and constructed-response (CR) questions. The mixed exam format was correlated with significantly more cognitively active study behaviors and a significantly better performance on the cumulative final exam (after accounting for grade point average and gender). There was also less gender-bias in the CR answers. This suggests that the MC-only exam format indeed hinders critical thinking in introductory science classes. Introducing CR questions encouraged students to learn more and to be better critical thinkers and reduced gender bias. However, student resistance increased as students adjusted their perceptions of their own critical-thinking abilities. PMID:22949426

Comparing narrative and multiple-choice formats in online communication skill assessment.

PubMed

Kim, Sara; Spielberg, Freya; Mauksch, Larry; Farber, Stu; Duong, Cuong; Fitch, Wes; Greer, Tom

2009-06-01

We compared multiple-choice and open-ended responses collected from a web-based tool designated 'Case for Change', which had been developed for assessing and teaching medical students in the skills involved in integrating sexual risk assessment and behaviour change discussions into patient-centred primary care visits. A total of 111 Year 3 students completed the web-based tool. A series of videos from one patient encounter illustrated how a clinician uses patient-centred communication and health behaviour change skills while caring for a patient presenting with a urinary tract infection. Each video clip was followed by a request for students to respond in two ways to the question: 'What would you do next?' Firstly, students typed their statements of what they would say to the patient. Secondly, students selected from a multiple-choice list the statements that most closely resembled their free text entries. These two modes of students' answers were analysed and compared. When articulating what they would say to the patient in a narrative format, students frequently used doctor-centred approaches that focused on premature diagnostic questioning or neglected to elicit patient perspectives. Despite the instruction to select a matching statement from the multiple-choice list, students tended to choose the most exemplary patient-centred statement, which was contrary to the doctor-centred approaches reflected in their narrative responses. Open-ended questions facilitate in-depth understanding of students' educational needs, although the scoring of narrative responses is time-consuming. Multiple-choice questions allow efficient scoring and individualised feedback associated with question items but do not fully elicit students' thought processes.

Using a Classroom Response System to Improve Multiple-Choice Performance in AP[R] Physics

ERIC Educational Resources Information Center

Bertrand, Peggy

2009-01-01

Participation in rigorous high school courses such as Advanced Placement (AP[R]) Physics increases the likelihood of college success, especially for students who are traditionally underserved. Tackling difficult multiple-choice exams should be part of any AP program because well-constructed multiple-choice questions, such as those on AP exams and…

Application of Item Analysis to Assess Multiple-Choice Examinations in the Mississippi Master Cattle Producer Program

ERIC Educational Resources Information Center

Parish, Jane A.; Karisch, Brandi B.

2013-01-01

Item analysis can serve as a useful tool in improving multiple-choice questions used in Extension programming. It can identify gaps between instruction and assessment. An item analysis of Mississippi Master Cattle Producer program multiple-choice examination responses was performed to determine the difficulty of individual examinations, assess the…

The Use of a Comprehensive Multiple Choice Final Exam in the Macroeconomics Principles Course: An Assessment.

ERIC Educational Resources Information Center

Petrowsky, Michael C.

This paper analyzes the results of a pilot study at Glendale Community College (Arizona) to assess the effectiveness of a comprehensive multiple choice final exam in the macroeconomic principles course. The "pilot project" involved the administration of a 50-question multiple choice exam to 71 students in three macroeconomics sections.…

Risk of error estimated from Palestine pharmacists' knowledge and certainty on the adverse effects and contraindications of active pharmaceutical ingredients and excipients.

PubMed

Shawahna, Ramzi; Al-Rjoub, Mohammed; Al-Horoub, Mohammed M; Al-Hroub, Wasif; Al-Rjoub, Bisan; Al-Nabi, Bashaaer Abd

2016-01-01

This study aimed to investigate community pharmacists' knowledge and certainty of adverse effects and contraindications of pharmaceutical products to estimate the risk of error. Factors influencing their knowledge and certainty were also investigated. The knowledge of community pharmacists was assessed in a cross-sectional design using a multiple-choice questions test on the adverse effects and contraindications of active pharmaceutical ingredients and excipients from May 2014 to March 2015. Self-rated certainty scores were also recorded for each question. Knowledge and certainty scores were combined to estimate the risk of error. Out of 315 subjects, 129 community pharmacists (41.0%) completed the 30 multiple-choice questions test on active ingredients and excipients. Knowledge on active ingredients was associated with the year of graduation and obtaining a licence to practice pharmacy. Knowledge on excipients was associated with the degree obtained. There was higher risk of error in items on excipients than those on ingredients (P<0.01). The knowledge of community pharmacists in Palestine was insufficient with high risk of errors. Knowledge of community pharmacists on the safety issues of active ingredients and excipients need to be improved.

The positive and negative consequences of multiple-choice testing.

PubMed

Roediger, Henry L; Marsh, Elizabeth J

2005-09-01

Multiple-choice tests are commonly used in educational settings but with unknown effects on students' knowledge. The authors examined the consequences of taking a multiple-choice test on a later general knowledge test in which students were warned not to guess. A large positive testing effect was obtained: Prior testing of facts aided final cued-recall performance. However, prior testing also had negative consequences. Prior reading of a greater number of multiple-choice lures decreased the positive testing effect and increased production of multiple-choice lures as incorrect answers on the final test. Multiple-choice testing may inadvertently lead to the creation of false knowledge.

The Incidence of Clueing in Multiple Choice Testbank Questions in Accounting: Some Evidence from Australia

ERIC Educational Resources Information Center

Ibbett, Nicole L.; Wheldon, Brett J.

2016-01-01

In 2014 Central Queensland University (CQU) in Australia banned the use of multiple choice questions (MCQs) as an assessment tool. One of the reasons given for this decision was that MCQs provide an opportunity for students to "pass" by merely guessing their answers. The mathematical likelihood of a student passing by guessing alone can…

Incorporating Multiple-Choice Questions into an AACSB Assurance of Learning Process: A Course-Embedded Assessment Application to an Introductory Finance Course

ERIC Educational Resources Information Center

Santos, Michael R.; Hu, Aidong; Jordan, Douglas

2014-01-01

The authors offer a classification technique to make a quantitative skills rubric more operational, with the groupings of multiple-choice questions to match the student learning levels in knowledge, calculation, quantitative reasoning, and analysis. The authors applied this classification technique to the mid-term exams of an introductory finance…

Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing.

PubMed

Butler, Andrew C; Roediger, Henry L

2008-04-01

Multiple-choice tests are used frequently in higher education without much consideration of the impact this form of assessment has on learning. Multiple-choice testing enhances retention of the material tested (the testing effect); however, unlike other tests, multiple-choice can also be detrimental because it exposes students to misinformation in the form of lures. The selection of lures can lead students to acquire false knowledge (Roediger & Marsh, 2005). The present research investigated whether feedback could be used to boost the positive effects and reduce the negative effects of multiple-choice testing. Subjects studied passages and then received a multiple-choice test with immediate feedback, delayed feedback, or no feedback. In comparison with the no-feedback condition, both immediate and delayed feedback increased the proportion of correct responses and reduced the proportion of intrusions (i.e., lure responses from the initial multiple-choice test) on a delayed cued recall test. Educators should provide feedback when using multiple-choice tests.

Grade 12 Diploma Examination, English 30. Part B: Reading (Multiple Choice). Readings Booklet. June 1988 Edition.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Intended for students taking the English 30 Grade 12 Diploma Examinations in Alberta, Canada, this test (to be administered along with a questions booklet) contains the reading selections portion of Part B, the reading component of the June 1988 tests. Representing the genres of fiction, nonfiction, poetry, and drama, the 10 selections consist of:…

An Empirical Comparison of Five Linear Equating Methods for the NEAT Design

ERIC Educational Resources Information Center

Suh, Youngsuk; Mroch, Andrew A.; Kane, Michael T.; Ripkey, Douglas R.

2009-01-01

In this study, a data base containing the responses of 40,000 candidates to 90 multiple-choice questions was used to mimic data sets for 50-item tests under the "nonequivalent groups with anchor test" (NEAT) design. Using these smaller data sets, we evaluated the performance of five linear equating methods for the NEAT design with five levels of…

A Combination of Teacher-Led Assessment and Self-Assessment Drives the Learning Process in Online Master Degree in Transplantation

ERIC Educational Resources Information Center

Halawa, Ahmed; Sharma, Ajay; Bridson, Julie M.; Lyon, Sarah; Prescott, Denise; Guha, Arpan; Taylor, David

2017-01-01

Background: Good performance in a summative assessment does not always equate to educational gain following a course. An educational programme may focus on improving student's performance on a particular test instrument. For example, practicing multiple choice questions may lead to mastery of the instrument itself rather than testing the knowledge…

Grade 12 Diploma Examination, English 33. Part B: Reading (Multiple Choice). Readings Booklet. June 1988 Edition.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Intended for students taking the English 33 Grade 12 Diploma Examinations in Alberta, Canada, this test (to be administered along with a questions booklet) contains the reading selections portion of Part B, the reading component of the June 1988 tests. The following short selections taken from fiction, nonfiction, poetry, drama, and day-to-day…

On the Use of the Immediate Recall Task as a Measure of Second Language Reading Comprehension

ERIC Educational Resources Information Center

Chang, Yuh-Fang

2006-01-01

The immediate written recall task, a widely used measure of both first language (L1) and second language (L2) reading comprehension, has been advocated over traditional test methods such as multiple choice, cloze tests and open-ended questions because it is a direct and integrative assessment task. It has been, however, criticized as requiring…

Spatial abilities and anatomy knowledge assessment: A systematic review.

PubMed

Langlois, Jean; Bellemare, Christian; Toulouse, Josée; Wells, George A

2017-06-01

Anatomy knowledge has been found to include both spatial and non-spatial components. However, no systematic evaluation of studies relating spatial abilities and anatomy knowledge has been undertaken. The objective of this study was to conduct a systematic review of the relationship between spatial abilities test and anatomy knowledge assessment. A literature search was done up to March 20, 2014 in Scopus and in several databases on the OvidSP and EBSCOhost platforms. Of the 556 citations obtained, 38 articles were identified and fully reviewed yielding 21 eligible articles and their quality were formally assessed. Non-significant relationships were found between spatial abilities test and anatomy knowledge assessment using essays and non-spatial multiple-choice questions. Significant relationships were observed between spatial abilities test and anatomy knowledge assessment using practical examination, three-dimensional synthesis from two-dimensional views, drawing of views, and cross-sections. Relationships between spatial abilities test and anatomy knowledge assessment using spatial multiple-choice questions were unclear. The results of this systematic review provide evidence for spatial and non-spatial methods of anatomy knowledge assessment. Anat Sci Educ 10: 235-241. © 2016 American Association of Anatomists. © 2016 American Association of Anatomists.

Comparison between three option, four option and five option multiple choice question tests for quality parameters: A randomized study.

PubMed

Vegada, Bhavisha; Shukla, Apexa; Khilnani, Ajeetkumar; Charan, Jaykaran; Desai, Chetna

2016-01-01

Most of the academic teachers use four or five options per item of multiple choice question (MCQ) test as formative and summative assessment. Optimal number of options in MCQ item is a matter of considerable debate among academic teachers of various educational fields. There is a scarcity of the published literature regarding the optimum number of option in each item of MCQ in the field of medical education. To compare three options, four options, and five options MCQs test for the quality parameters - reliability, validity, item analysis, distracter analysis, and time analysis. Participants were 3 rd semester M.B.B.S. students. Students were divided randomly into three groups. Each group was given one set of MCQ test out of three options, four options, and five option randomly. Following the marking of the multiple choice tests, the participants' option selections were analyzed and comparisons were conducted of the mean marks, mean time, validity, reliability and facility value, discrimination index, point biserial value, distracter analysis of three different option formats. Students score more ( P = 0.000) and took less time ( P = 0.009) for the completion of three options as compared to four options and five options groups. Facility value was more ( P = 0.004) in three options group as compared to four and five options groups. There was no significant difference between three groups for the validity, reliability, and item discrimination. Nonfunctioning distracters were more in the four and five options group as compared to three option group. Assessment based on three option MCQs is can be preferred over four option and five option MCQs.

Condemned to Repeat It

ERIC Educational Resources Information Center

Gravois, John

2006-01-01

In this article, the author discusses the findings of the study entitled, "The Coming Crisis in Citizenship: Higher Education's Failure to Teach America's History and Institutions," by the Intercollegiate Studies Institute, a Delaware think tank. In the study, a 60-question, multiple-choice test was administered to 14,000 freshmen and seniors at…

Constructing Multiple-Choice Items to Measure Higher-Order Thinking

ERIC Educational Resources Information Center

Scully, Darina

2017-01-01

Across education, certification and licensure, there are repeated calls for the development of assessments that target "higher-order thinking," as opposed to mere recall of facts. A common assumption is that this necessitates the use of constructed response or essay-style test questions; however, empirical evidence suggests that this may…

Evaluating the Effectiveness of the Lecture versus Independent Study.

ERIC Educational Resources Information Center

DaRosa, Debra A.; And Others

1991-01-01

The impacts of independent study and the lecture approach on student test scores and study time were compared for 205 medical students studying surgery. Learning objective, multiple-choice, and essay questions were developed for selected topics related to surgery. Findings support increased individual active learning strategies and decreased…

An Examination of the Perceived Importance of Technical Competence in Acquisition Project Management

DTIC Science & Technology

1991-09-01

Develop (First Draft) Instructions Critique (Revision) Answerability Pilot Test (Second Draft) Analysis Response Mode Revision Useability Preparation...appropriate questionnaire items. Initially, the set of questions developed for the study reflected a few shortcomings. A pilot test of the first draft among...resulted. First, feedback from the pilot test indicated a need to reduce the completion time. Because the multiple choice format required several

Analysis of the Difficulty and Discrimination Indices of Multiple-Choice Questions According to Cognitive Levels in an Open and Distance Learning Context

ERIC Educational Resources Information Center

Koçdar, Serpil; Karadag, Nejdet; Sahin, Murat Dogan

2016-01-01

This is a descriptive study which intends to determine whether the difficulty and discrimination indices of the multiple-choice questions show differences according to cognitive levels of the Bloom's Taxonomy, which are used in the exams of the courses in a business administration bachelor's degree program offered through open and distance…

Effect of response format on cognitive reflection: Validating a two- and four-option multiple choice question version of the Cognitive Reflection Test.

PubMed

Sirota, Miroslav; Juanchich, Marie

2018-03-27

The Cognitive Reflection Test, measuring intuition inhibition and cognitive reflection, has become extremely popular because it reliably predicts reasoning performance, decision-making, and beliefs. Across studies, the response format of CRT items sometimes differs, based on the assumed construct equivalence of tests with open-ended versus multiple-choice items (the equivalence hypothesis). Evidence and theoretical reasons, however, suggest that the cognitive processes measured by these response formats and their associated performances might differ (the nonequivalence hypothesis). We tested the two hypotheses experimentally by assessing the performance in tests with different response formats and by comparing their predictive and construct validity. In a between-subjects experiment (n = 452), participants answered stem-equivalent CRT items in an open-ended, a two-option, or a four-option response format and then completed tasks on belief bias, denominator neglect, and paranormal beliefs (benchmark indicators of predictive validity), as well as on actively open-minded thinking and numeracy (benchmark indicators of construct validity). We found no significant differences between the three response formats in the numbers of correct responses, the numbers of intuitive responses (with the exception of the two-option version, which had a higher number than the other tests), and the correlational patterns of the indicators of predictive and construct validity. All three test versions were similarly reliable, but the multiple-choice formats were completed more quickly. We speculate that the specific nature of the CRT items helps build construct equivalence among the different response formats. We recommend using the validated multiple-choice version of the CRT presented here, particularly the four-option CRT, for practical and methodological reasons. Supplementary materials and data are available at https://osf.io/mzhyc/ .

Pushing Critical Thinking Skills With Multiple-Choice Questions: Does Bloom's Taxonomy Work?

PubMed

Zaidi, Nikki L Bibler; Grob, Karri L; Monrad, Seetha M; Kurtz, Joshua B; Tai, Andrew; Ahmed, Asra Z; Gruppen, Larry D; Santen, Sally A

2018-06-01

Medical school assessments should foster the development of higher-order thinking skills to support clinical reasoning and a solid foundation of knowledge. Multiple-choice questions (MCQs) are commonly used to assess student learning, and well-written MCQs can support learner engagement in higher levels of cognitive reasoning such as application or synthesis of knowledge. Bloom's taxonomy has been used to identify MCQs that assess students' critical thinking skills, with evidence suggesting that higher-order MCQs support a deeper conceptual understanding of scientific process skills. Similarly, clinical practice also requires learners to develop higher-order thinking skills that include all of Bloom's levels. Faculty question writers and examinees may approach the same material differently based on varying levels of knowledge and expertise, and these differences can influence the cognitive levels being measured by MCQs. Consequently, faculty question writers may perceive that certain MCQs require higher-order thinking skills to process the question, whereas examinees may only need to employ lower-order thinking skills to render a correct response. Likewise, seemingly lower-order questions may actually require higher-order thinking skills to respond correctly. In this Perspective, the authors describe some of the cognitive processes examinees use to respond to MCQs. The authors propose that various factors affect both the question writer and examinee's interaction with test material and subsequent cognitive processes necessary to answer a question.

Definite Integral Automatic Analysis Mechanism Research and Development Using the "Find the Area by Integration" Unit as an Example

ERIC Educational Resources Information Center

Ting, Mu Yu

2017-01-01

Using the capabilities of expert knowledge structures, the researcher prepared test questions on the university calculus topic of "finding the area by integration." The quiz is divided into two types of multiple choice items (one out of four and one out of many). After the calculus course was taught and tested, the results revealed that…

Emergency Medical Care. A Manual for the Paramedic in the Field--Workbook.

ERIC Educational Resources Information Center

National Highway Traffic Safety Administration (DOT), Washington, DC.

This workbook is designed to accompany the text of the same name and to serve as an aid to both learning and review during the course of study. The workbook consists of 15 module self-tests and vocabulary lists that follow the modules of the text. Tests consist of objective questions (multiple choice, fill-in-the-blank, short answers, and…

A Novel Multiple Choice Question Generation Strategy: Alternative Uses for Controlled Vocabulary Thesauri in Biomedical-Sciences Education.

PubMed

Lopetegui, Marcelo A; Lara, Barbara A; Yen, Po-Yin; Çatalyürek, Ümit V; Payne, Philip R O

2015-01-01

Multiple choice questions play an important role in training and evaluating biomedical science students. However, the resource intensive nature of question generation limits their open availability, reducing their contribution to evaluation purposes mainly. Although applied-knowledge questions require a complex formulation process, the creation of concrete-knowledge questions (i.e., definitions, associations) could be assisted by the use of informatics methods. We envisioned a novel and simple algorithm that exploits validated knowledge repositories and generates concrete-knowledge questions by leveraging concepts' relationships. In this manuscript we present the development and validation of a prototype which successfully produced meaningful concrete-knowledge questions, opening new applications for existing knowledge repositories, potentially benefiting students of all biomedical sciences disciplines.

Exploring undergraduates' understanding of photosynthesis using diagnostic question clusters.

PubMed

Parker, Joyce M; Anderson, Charles W; Heidemann, Merle; Merrill, John; Merritt, Brett; Richmond, Gail; Urban-Lurain, Mark

2012-01-01

We present a diagnostic question cluster (DQC) that assesses undergraduates' thinking about photosynthesis. This assessment tool is not designed to identify individual misconceptions. Rather, it is focused on students' abilities to apply basic concepts about photosynthesis by reasoning with a coordinated set of practices based on a few scientific principles: conservation of matter, conservation of energy, and the hierarchical nature of biological systems. Data on students' responses to the cluster items and uses of some of the questions in multiple-choice, multiple-true/false, and essay formats are compared. A cross-over study indicates that the multiple-true/false format shows promise as a machine-gradable format that identifies students who have a mixture of accurate and inaccurate ideas. In addition, interviews with students about their choices on three multiple-choice questions reveal the fragility of students' understanding. Collectively, the data show that many undergraduates lack both a basic understanding of the role of photosynthesis in plant metabolism and the ability to reason with scientific principles when learning new content. Implications for instruction are discussed.

Exploring Undergraduates' Understanding of Photosynthesis Using Diagnostic Question Clusters

PubMed Central

Parker, Joyce M.; Anderson, Charles W.; Heidemann, Merle; Merrill, John; Merritt, Brett; Richmond, Gail; Urban-Lurain, Mark

2012-01-01

We present a diagnostic question cluster (DQC) that assesses undergraduates' thinking about photosynthesis. This assessment tool is not designed to identify individual misconceptions. Rather, it is focused on students' abilities to apply basic concepts about photosynthesis by reasoning with a coordinated set of practices based on a few scientific principles: conservation of matter, conservation of energy, and the hierarchical nature of biological systems. Data on students' responses to the cluster items and uses of some of the questions in multiple-choice, multiple-true/false, and essay formats are compared. A cross-over study indicates that the multiple-true/false format shows promise as a machine-gradable format that identifies students who have a mixture of accurate and inaccurate ideas. In addition, interviews with students about their choices on three multiple-choice questions reveal the fragility of students' understanding. Collectively, the data show that many undergraduates lack both a basic understanding of the role of photosynthesis in plant metabolism and the ability to reason with scientific principles when learning new content. Implications for instruction are discussed. PMID:22383617

Grade 12 Diploma Examination, English 33. Part B: Reading (Multiple Choice). Readings Booklet.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Intended for students taking the Grade 12 Examination in English 33 in Alberta, Canada, this reading test (to be administered along with the questions booklet) contains short reading selections taken from fiction, nonfiction, poetry, and drama, including the following: an excerpt from "Catch-22" (Joseph Heller); "School Thief"…

The Influence of Distractor Strength and Response Order on MCQ Responding

ERIC Educational Resources Information Center

Kiat, John Emmanuel; Ong, Ai Rene; Ganesan, Asha

2018-01-01

Multiple-choice questions (MCQs) play a key role in standardised testing and in-class assessment. Research into the influence of within-item response order on MCQ characteristics has been mixed. While some researchers have shown preferential selection of response options presented earlier in the answer list, others have failed to replicate these…

Transferring Knowledge from the Classroom to the Real World: Redox Concepts.

ERIC Educational Resources Information Center

Soudani, Mohamed; Sivade, Andre; Cros, Daniele; Medimagh, Mohamed Salah

2000-01-01

Describes a study of the ability of fourth-year university student-teachers to use their theoretical knowledge of redox concepts to interpret natural phenomena of everyday life. Uses word-association tests, multiple-choice and open questions and reports that the students have difficulties in recognizing the scientific nature of well-known everyday…

Improving Student Performance through Computer-Based Assessment: Insights from Recent Research.

ERIC Educational Resources Information Center

Ricketts, C.; Wilks, S. J.

2002-01-01

Compared student performance on computer-based assessment to machine-graded multiple choice tests. Found that performance improved dramatically on the computer-based assessment when students were not required to scroll through the question paper. Concluded that students may be disadvantaged by the introduction of online assessment unless care is…

Grade 12 Diploma Examination, English 30. Part B: Reading (Multiple Choice). Readings Booklet. June 1989 Edition.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Intended for students taking the Grade 12 Examination in English 30 in Alberta, Canada, this reading test (to be administered along with a questions booklet) includes the following eight short selections taken from fiction, nonfiction, poetry, and drama: "Loyalties" (Roo Borson); "Clever Animals" (Lewis Thomas); "Death of…

Grade 12 Diploma Examination, English 30. Part B: Reading (Multiple Choice). Readings Booklet. 1988 Edition.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Intended for students taking the Grade 12 Examinations in English 30 in Alberta, Canada, this reading test (to be administered along with a questions booklet) includes the following nine short selections taken from fiction, nonfiction, poetry, and drama: "The Biggest Liar in the World" (Harry Mark Petrakis); "Victorian…

College Students' Prejudicial Biases against Instructors Who Smoke Cigarettes

ERIC Educational Resources Information Center

Oberle, Crystal D.; Engeling, Stephanie J.; Ontiberos, Senecae P.

2010-01-01

In an investigation of students' prejudicial biases against instructors who smoke, 61 female and 16 male undergraduates watched and listened to a 20-min lecture about parasomnias, completed a survey asking for instructor evaluation ratings and ratings of perceived learning, and completed a lecture-retention test with multiple-choice questions to…

Grade 12 Diploma Examination, English 30. Part B: Reading (Multiple Choice). Readings Booklet.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Intended for students taking the Grade 12 Examination in English 30 in Alberta, Canada, this reading test (to be administered along with the questions booklet) contains 10 short reading selections taken from fiction, nonfiction, poetry, and drama, including the following: an excerpt from "Where Did You Go?""Out." (Robert Paul…

Development and Validation of the Conceptual Assessment of Natural Selection (CANS)

ERIC Educational Resources Information Center

Kalinowski, Steven T.; Leonard, Mary J.; Taper, Mark L.

2016-01-01

We developed and validated the Conceptual Assessment of Natural Selection (CANS), a multiple-choice test designed to assess how well college students understand the central principles of natural selection. The expert panel that reviewed the CANS concluded its questions were relevant to natural selection and generally did a good job sampling the…

Child Abuse and Neglect: Training Needs of Student Teachers

ERIC Educational Resources Information Center

McKee, Bronagh E.; Dillenburger, Karola

2009-01-01

Increasing awareness of child abuse and neglect (CAN) raises questions about how well teachers are prepared for their role in child protection. This paper assesses and differentiates training needs of first-year students (n = 216) in Northern Ireland. Multiple-choice tests were used to assess knowledge of CAN statistics; recognising and reporting;…

Multiple choice answers: what to do when you have too many questions.

PubMed

Jupiter, Daniel C

2015-01-01

Carrying out too many statistical tests in a single study throws results into doubt, for reasons statistical and ethical. I discuss why this is the case and briefly mention ways to handle the problem. Copyright © 2015 American College of Foot and Ankle Surgeons. Published by Elsevier Inc. All rights reserved.

Grade 12 Diploma Examination, English 30. Part B: Reading (Multiple Choice). Readings Booklet. 1986 Edition.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Intended for students taking the Grade 12 Examination in English 30 in Alberta, Canada, this reading test (to be administered along with the questions booklet) contains 10 short reading selections taken from fiction, nonfiction, poetry, and drama, including the following: "My Magical Metronome" (Lewis Thomas); "Queen Street…

Measuring Gains in Critical Thinking in Food Science and Human Nutrition Courses: The Cornell Critical Thinking Test, Problem-Based Learning Activities, and Student Journal Entries

ERIC Educational Resources Information Center

Iwaoka, Wayne T.; Li, Yong; Rhee, Walter Y.

2010-01-01

The Cornell Critical Thinking Test (CCTT) is one of the many multiple-choice tests with validated questions that have been reported to measure general critical thinking (CT) ability. One of the IFT Education Standards for undergraduate degrees in Food Science is the emphasis on the development of critical thinking. While this skill is easy to list…

Manipulations of Choice Familiarity in Multiple-Choice Testing Support a Retrieval Practice Account of the Testing Effect

ERIC Educational Resources Information Center

Jang, Yoonhee; Pashler, Hal; Huber, David E.

2014-01-01

We performed 4 experiments assessing the learning that occurs when taking a test. Our experiments used multiple-choice tests because the processes deployed during testing can be manipulated by varying the nature of the choice alternatives. Previous research revealed that a multiple-choice test that includes "none of the above" (NOTA)…

Effects of Test Expectation on Multiple-Choice Performance and Subjective Ratings

ERIC Educational Resources Information Center

Balch, William R.

2007-01-01

Undergraduates studied the definitions of 16 psychology terms, expecting either a multiple-choice (n = 132) or short-answer (n = 122) test. All students then received the same multiple-choice test, requiring them to recognize the definitions as well as novel examples of the terms. Compared to students expecting a multiple-choice test, those…

The establisment of an achievement test for determination of primary teachers’ knowledge level of earthquake

DOE Office of Scientific and Technical Information (OSTI.GOV)

Aydin, Süleyman, E-mail: yupul@hotmail.com; Haşiloğlu, M. Akif, E-mail: mehmet.hasiloglu@hotmail.com; Kunduraci, Ayşe, E-mail: ayse-kndrc@hotmail.com

In this study it was aimed to improve an academic achievement test to establish the students’ knowledge about the earthquake and the ways of protection from earthquakes. In the method of this study, the steps that Webb (1994) was created to improve an academic achievement test for a unit were followed. In the developmental process of multiple choice test having 25 questions, was prepared to measure the pre-service teachers’ knowledge levels about the earthquake and the ways of protection from earthquakes. The multiple choice test was presented to view of six academics (one of them was from geographic field andmore » five of them were science educator) and two expert teachers in science Prepared test was applied to 93 pre-service teachers studying in elementary education department in 2014-2015 academic years. As a result of validity and reliability of the study, the test was composed of 20 items. As a result of these applications, Pearson Moments Multiplication half-reliability coefficient was found to be 0.94. When this value is adjusted according to Spearman Brown reliability coefficient the reliability coefficient was set at 0.97.« less

Development and Evaluation of Internet-Based Hypermedia Chemistry Tutorials

NASA Astrophysics Data System (ADS)

Tissue, Brian M.; Earp, Ronald L.; Yip, Ching-Wan; Anderson, Mark R.

1996-05-01

This progress report describes the development and student use of World-Wide-Web-based prelaboratory exercises in senior-level Instrumental Analysis during the 1995 Fall semester. The laboratory preparation exercises contained hypermedia tutorials and multiple-choice questions that were intended to familiarize the students with the experiments and instrumentation before their laboratory session. The overall goal of our work is to explore ways in which computer and network technology can be applied in education to improve the cost-effectiveness and efficacy of teaching. The course material can be accessed at http://www.chem.vt.edu/chem-ed/4114/Fall1995.html. The students were instructed to read their experimental procedure and to do the relevant laboratory preparation exercise. The individual tutorial documents were primarily text that provided basic theoretical and experimental descriptions of analytical and instrumental methods. The documents included hyperlinks to basic concepts, simple schematics, and color graphics of experimental set-ups or instrumentation. We chose the World-Wide Web (WWW) as the delivery platform for this project because of the ease of developing, distributing, and modifying hypermedia material in a client-server system. The disadvantage of the WWW is that network bandwidth limits the size and sophistication of the hypermedia material. To minimize internet transfer time, the individual documents were kept short and usually contained no more than 3 or 4 inline images. After reading the tutorial the students answered several multiple-choice questions. The figure shows one example of a multiple-choice question and the response page. Clicking on the "Submit answer" button calls a *.cgi file, which contains instructions in the PERL interpretive language, that generates the response page and saves the date, time, and student's answer to a file on the server. Usage and student perception of the on-line material was evaluated from server logs and student surveys. On-time completion of the assignments was 75%, but use of other on-line resources such as a question-and-answer page was minimal. Responses from student surveys indicated that the students had sufficient access to the internet. Approximately half of the students completed the prelaboratory exercises from one of several computers in the laboratory, and half worked from a workplace, university library, or home. Greater than 85% of all student usage from the laboratory computers occurred between 11 am and 4 pm. A mid-semester student survey indicated that the spectroscopy prelabs with three multiple-choice questions were better for increasing conceptual understanding rather than preparing the students for the actual lab work. An end-of-the-semester survey based on the electrochemistry assignments, which consisted of two multiple-choice questions and one clickable-map graphical exercise, produced a slightly higher rating for preparing students for the laboratory work. The differences between the spectroscopy and electrochemistry exercises prevent drawing any real conclusions from these two surveys, however, they do help guide the preparation of the content of future exercises. Next year's materials will contain three multiple-choice questions and one graphics-based exercise. The clickable-map graphics and at least one of the multiple-choice questions will be designed to test an understanding of the experimental procedure and instrument use to better prepare students for the actual laboratory work. Acknowledgment. We would like to thank Professor Gary Long for his assistance with the course, and the NSF for financial support through the Division of Undergraduate Education (DUE-9455382) and a CAREER award (CHE-9502460). Literature Cited. Laurillard, D. Rethinking Teaching, a Framework for the Effective Use of Educational Technology; Routledge: London, 1993. Tissue, B. M.; Earp, R. L.; Yip, C.-W. Chem. Educator 1996, 1(1), S1430-4171(96)01010-2. Only available at http://journals.springer-ny.com/chedr.

Assessment of High-school Students Engaged in the EarthLabs Climate Modules using the Climate Concept Inventory

NASA Astrophysics Data System (ADS)

McNeal, K.; Libarkin, J. C.; Ledley, T. S.; Gold, A. U.; Lynds, S. E.; Haddad, N.; Ellins, K.; Dunlap, C.; Bardar, E. W.; Youngman, E.

2015-12-01

Instructors must have on hand appropriate assessments that align with their teaching and learning goals in order to provide evidence of student learning. We have worked with curriculum developers and scientists to develop the Climate Concept Inventory (CCI), which meets goals of the EarthLabs Climate on-line curriculum. The developed concept inventory includes 19 content-driven multiple choice questions, six affective-based multiple choice questions, one confidence question, three open-ended questions, and eight demographic questions. Our analysis of the instrument applies item response theory and uses item characteristic curves. We have assessed over 500 students in nearly twenty high school classrooms in Mississippi and Texas that have engaged in the implementation of the EarthLabs curriculum and completed the CCI. Results indicate that students had pre-post gains on 9 out of 10 of the content-based multiple choice questions with positive gains in answer choice selection ranging from 1.72% to 42%. Students significantly reported increased confidence with 15% more students reporting that they were either very or fairly confident with their answers. Of the six affective questions posed, 5 out of 6 showed significant shifts towards gains in knowledge, awareness, and information about Earth's climate system. The research has resulted in a robust and validated climate concept inventory for use with advanced high school students, where we have been able to apply its use within the EarthLabs project.

Grade 12 Diploma Examination: Chemistry 30. June 1986. = Examen en vue du Diplome Douzieme Annee: Chimie 30. Juin 1986.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Chemistry 30 is a twelfth-grade chemistry course for students in Alberta, Canada. This document is a final test for the course. Both English and French versions of the test are provided. Intended for administration during June 1986, it contains 56 multiple-choice questions and three written-response problems. Two-and-one-half hours are allowed for…

Grade 12 Diploma Examination: Biology 30. June 1986. = Examen en vue du Diplome Douzieme Annee: Biologie 30. Juin 1986.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Biology 30 is a twelfth-grade biology course for students in Alberta, Canada. This document is a final test for the course. Both English and French versions of the test are provided. Intended for administration during June 1986, it contains 80 multiple-choice questions and six written-response problems. Two-and-one-half hours are allowed for…

Relationship of awards in multiple choice questions and structured answer questions in the undergraduate years and their effectiveness in evaluation.

PubMed

Khan, Junaid Sarfraz; Mukhtar, Osama; Tabasum, Saima; Shaheen, Naveed; Farooq, M; Irfan, M Abdul; Sattar, Ajmal; Nabeel, M; Imran, M; Rafique, Sadia; Iqbal, Maryam; Afzal, M Sheraz; Hameed, M Shahbaz; Habib, Maryam; Jabeen, Uzma; Mubbashar, Malik Hussain

2010-01-01

A number of evaluation tools for assessing the cognitive and affective domains in accordance with Bloom's taxonomy are available for summative assessment. At the University of Health Sciences, Lahore, Multiple Choice Questions (MCQs) and Structured Answer Questions (SAQs) are used for the evaluation of the cognitive domain at all six hierarch levels of taxonomy using the tables of specifications to ensure content validity. The rationale of having two evaluation tools seemingly similar in their evaluative competency yet differing in feasibility of construction, administration and marking is being challenged in this study. The MCQ and SAQ awards of the ten percent sample population amounting to 985 students in fifteen Medical and Dental Colleges across Punjab were entered into SPSS-15 and correlated according to the cognitive and affective level of assessment in relation to the Bloom's taxonomy and their grouping in the Tables of Specifications, using parametric tests. 3494 anonymously administered questionnaires were analyzed using ethnograph. No statistically significant difference was found in the mean marks obtained by the students when MCQs and SAQs were compared according to their groupings in the Tables of Specifications at all levels of cognitive hierarchical testing. End-of-yearcognitive level testing targets set were not met and more questions were set at the lower cognitive testing levels. Expenses incurred in setting MCQs and SAQs were comparable but conduct and assessment costs for MCQs and SAQs were 6% and 94% of the total respectively. In both MCQs and SAQs students performed better at higher cognitive testing levels whereas the SAQs and MCQs were able to marginally test the lower levels of affective domain only. Student's feedback showed that attempting MCQs required critical thinking, experience and practice. MCQs are more cost effective means at levels of cognitive domain assessment.

Simulation-Based Educational Module Improves Intern and Medical Student Performance of Closed Reduction and Percutaneous Pinning of Pediatric Supracondylar Humeral Fractures.

PubMed

Butler, Bennet A; Lawton, Cort D; Burgess, Jamie; Balderama, Earvin S; Barsness, Katherine A; Sarwark, John F

2017-12-06

Simulation-based education has been integrated into many orthopaedic residency programs to augment traditional teaching models. Here we describe the development and implementation of a combined didactic and simulation-based course for teaching medical students and interns how to properly perform a closed reduction and percutaneous pinning of a pediatric supracondylar humeral fracture. Subjects included in the study were either orthopaedic surgery interns or subinterns at our institution. Subjects all completed a combined didactic and simulation-based course on pediatric supracondylar humeral fractures. The first part of this course was an electronic (e)-learning module that the subjects could complete at home in approximately 40 minutes. The second part of the course was a 20-minute simulation-based skills learning session completed in the simulation center. Subject knowledge of closed reduction and percutaneous pinning of supracondylar humeral fractures was tested using a 30-question, multiple-choice, written test. Surgical skills were tested in the operating room or in a simulated operating room. Subject pre-intervention and post-intervention scores were compared to determine if and how much they had improved. A total of 21 subjects were tested. These subjects significantly improved their scores on both the written, multiple-choice test and skills test after completing the combined didactic and simulation module. Prior to the module, intern and subintern multiple-choice test scores were significantly worse than postgraduate year (PGY)-2 to PGY-5 resident scores (p < 0.01); after completion of the module, there was no significant difference in the multiple-choice test scores. After completing the module, there was no significant difference in skills test scores between interns and PGY-2 to PGY-5 residents. Both tests were validated using the scores obtained from PGY-2 to PGY-5 residents. Our combined didactic and simulation course significantly improved intern and subintern understanding of supracondylar humeral fractures and their ability to perform a closed reduction and percutaneous pinning of these fractures.

State Test Programs Mushroom as NCLB Mandate Kicks in: Nearly Half of States Are Expanding Their Testing Programs to Additional Grades This School Year to Comply with the Federal No Child Left Behind Act

ERIC Educational Resources Information Center

Olson, Lynn

2005-01-01

Twenty-three states are expanding their testing programs to additional grades this school year to comply with the federal No Child Left Behind Act. In devising the new tests, most states have defied predictions and chosen to go beyond multiple-choice items, by including questions that ask students to construct their own responses. But many state…

Virtual test: A student-centered software to measure student's critical thinking on human disease

NASA Astrophysics Data System (ADS)

Rusyati, Lilit; Firman, Harry

2016-02-01

The study "Virtual Test: A Student-Centered Software to Measure Student's Critical Thinking on Human Disease" is descriptive research. The background is importance of computer-based test that use element and sub element of critical thinking. Aim of this study is development of multiple choices to measure critical thinking that made by student-centered software. Instruments to collect data are (1) construct validity sheet by expert judge (lecturer and medical doctor) and professional judge (science teacher); and (2) test legibility sheet by science teacher and junior high school student. Participants consisted of science teacher, lecturer, and medical doctor as validator; and the students as respondent. Result of this study are describe about characteristic of virtual test that use to measure student's critical thinking on human disease, analyze result of legibility test by students and science teachers, analyze result of expert judgment by science teachers and medical doctor, and analyze result of trial test of virtual test at junior high school. Generally, result analysis shown characteristic of multiple choices to measure critical thinking was made by eight elements and 26 sub elements that developed by Inch et al.; complete by relevant information; and have validity and reliability more than "enough". Furthermore, specific characteristic of multiple choices to measure critical thinking are information in form science comic, table, figure, article, and video; correct structure of language; add source of citation; and question can guide student to critical thinking logically.

Grade 12 Diploma Examination, English 33. Part B: Reading. (Multiple Choice). Readings Booklet. 1988 Edition.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Intended for students taking Grade 12 Diploma Examinations in English 33 in Alberta, Canada, this reading test is designed to be administered with a questions booklet. The following short selections taken from fiction, nonfiction, poetry, drama, and day-to-day functional materials are included: (1) "M is for Mother" (Marjorie Riddle);…

Improving the Quality of Student Experience in Large Lectures Using Quick Polls

ERIC Educational Resources Information Center

Calma, Angelito; Webster, Beverley; Petry, Stefan; Pesina, Jenny

2014-01-01

A quick polling initiative was tested in finance classes using multiple choice questions to determine whether it can improve student interaction and engagement in a large class. Students (n = 446) responded using either a smartphone app (53%) or by using pen and paper (47%). Immediate feedback was provided to students using charts that were…

A Computer-Based Approach for Deriving and Measuring Individual and Team Knowledge Structure from Essay Questions

ERIC Educational Resources Information Center

Clariana, Roy B.; Wallace, Patricia

2007-01-01

This proof-of-concept investigation describes a computer-based approach for deriving the knowledge structure of individuals and of groups from their written essays, and considers the convergent criterion-related validity of the computer-based scores relative to human rater essay scores and multiple-choice test scores. After completing a…

Grade 12 Diploma Examination, English 30. Part B: Reading (Multiple Choice). Readings Booklet. 1987 Edition.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Intended for students taking the Grade 12 Examination in English 30 in Alberta, Canada, this reading test (to be administered along with a questions booklet) includes the following 10 short selections taken from fiction, nonfiction, poetry, and drama: "Parents as People (with Children)" (Ellen Goodman); "Everybody Knows about the…

Grade 12 Diploma Examination, English 30. Part B: Reading (Multiple Choice). Readings Booklet.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Intended for students taking the Grade 12 Examination in English 30 in Alberta, Canada, this reading test (to be administered along with the questions booklet) contains 10 short reading selections taken from fiction, nonfiction, poetry and drama, including the following: "At the Age at Which Mozart Was Dead Already" (Ellen Goodman);…

Differential Competencies Contributing to Children's Comprehension of Narrative and Expository Texts

ERIC Educational Resources Information Center

Best, Rachel M.; Floyd, Randy G.; Mcnamara, Danielle S.

2008-01-01

This study examined the influences of reading decoding skills and world knowledge on third graders' comprehension of narrative and expository texts. Children read a narrative text and an expository text. Comprehension of each text was assessed with a free recall prompt, three cued recall prompts, and 12 multiple-choice questions. Tests from the…

Evaluation of Modified Essay Questions (MEQ) and Multiple Choice Questions (MCQ) as a tool for Assessing the Cognitive Skills of Undergraduate Medical Students

PubMed Central

Khan, Moeen-uz-Zafar; Aljarallah, Badr Muhammad

2011-01-01

Objectives: Developing and testing the cognitive skills and abstract thinking of undergraduate medical students are the main objectives of problem based learning. Modified Essay Questions (MEQ) and Multiple Choice Questions (MCQ) may both be designed to test these skills. The objectives of this study were to assess the effectiveness of both forms of questions in testing the different levels of the cognitive skills of undergraduate medical students and to detect any item writing flaws in the questions. Methods: A total of 50 MEQs and 50 MCQs were evaluated. These questions were chosen randomly from various examinations given to different batches of undergraduate medical students taking course MED 411–412 at the Department of Medicine, Qassim University from the years 2005 to 2009. The effectiveness of the questions was determined by two assessors and was defined by the question’s ability to measure higher cognitive skills, as determined by modified Bloom’s taxonomy, and its quality as determined by the presence of item writing flaws. ‘SPSS15’ and ‘Medcalc’ programs were used to tabulate and analyze the data. Results: The percentage of questions testing the level III (problem solving) cognitive skills of the students was 40% for MEQs and 60% for the MCQs; the remaining questions merely assessed the recall and comprehension. No significant difference was found between MEQ and MCQ in relation to the type of questions (recall; comprehension or problem solving x2 = 5.3, p = 0.07).The agreement between the two assessors was quite high in case of MCQ (kappa=0.609; SE 0.093; 95%CI 0.426 – 0.792) but lower in case of MEQ (kappa=0.195; SE 0.073; 95%CI 0.052 – 0.338). 16% of the MEQs and 12% of the MCQs had item writing flaws. Conclusion: A well constructed MCQ is superior to MEQ in testing the higher cognitive skills of undergraduate medical students in a problem based learning setup. Constructing an MEQ for assessing the cognitive skills of a student is not a simple task and is more frequently associated with item writing flaws. PMID:22489228

Wolf Testing: Open Source Testing Software

NASA Astrophysics Data System (ADS)

Braasch, P.; Gay, P. L.

2004-12-01

Wolf Testing is software for easily creating and editing exams. Wolf Testing allows the user to create an exam from a database of questions, view it on screen, and easily print it along with the corresponding answer guide. The questions can be multiple choice, short answer, long answer, or true and false varieties. This software can be accessed securely from any location, allowing the user to easily create exams from home. New questions, which can include associated pictures, can be added through a web-interface. After adding in questions, they can be edited, deleted, or duplicated into multiple versions. Long-term test creation is simplified, as you are able to quickly see what questions you have asked in the past and insert them, with or without editing, into future tests. All tests are archived in the database. Written in PHP and MySQL, this software can be installed on any UNIX / Linux platform, including Macintosh OS X. The secure interface keeps students out, and allows you to decide who can create tests and who can edit information already in the database. Tests can be output as either html with pictures or rich text without pictures, and there are plans to add PDF and MS Word formats as well. We would like to thank Dr. Wolfgang Rueckner and the Harvard University Science Center for providing incentive to start this project, computers and resources to complete this project, and inspiration for the project's name. We would also like to thank Dr. Ronald Newburgh for his assistance in beta testing.

Climbing Bloom's taxonomy pyramid: Lessons from a graduate histology course.

PubMed

Zaidi, Nikki B; Hwang, Charles; Scott, Sara; Stallard, Stefanie; Purkiss, Joel; Hortsch, Michael

2017-09-01

Bloom's taxonomy was adopted to create a subject-specific scoring tool for histology multiple-choice questions (MCQs). This Bloom's Taxonomy Histology Tool (BTHT) was used to analyze teacher- and student-generated quiz and examination questions from a graduate level histology course. Multiple-choice questions using histological images were generally assigned a higher BTHT level than simple text questions. The type of microscopy technique (light or electron microscopy) used for these image-based questions did not result in any significant differences in their Bloom's taxonomy scores. The BTHT levels for teacher-generated MCQs correlated positively with higher discrimination indices and inversely with the percent of students answering these questions correctly (difficulty index), suggesting that higher-level Bloom's taxonomy questions differentiate well between higher- and lower-performing students. When examining BTHT scores for MCQs that were written by students in a Multiple-Choice Item Development Assignment (MCIDA) there was no significant correlation between these scores and the students' ability to answer teacher-generated MCQs. This suggests that the ability to answer histology MCQs relies on a different skill set than the aptitude to construct higher-level Bloom's taxonomy questions. However, students significantly improved their average BTHT scores from the midterm to the final MCIDA task, which indicates that practice, experience and feedback increased their MCQ writing proficiency. Anat Sci Educ 10: 456-464. © 2017 American Association of Anatomists. © 2017 American Association of Anatomists.

Strong Body, Strong Mind: The Effects of Implementing Physical Activity within a Mathematics Course for Deployed Sailors

ERIC Educational Resources Information Center

Edwards, Joshua Joel

2012-01-01

Test anxiety can act as a major inhibitor for students to perform to their ability. Students can find that what life decisions they are going to be afforded is going to be determined by a series of multiple choice mathematics questions. There must be a way to deal with their test anxiety so that the true nature of their knowledge is displayed, and…

An Exploratory Study of the Relationships between Reported Imagery and the Comprehension and Recall of a Story in Fifth Graders. Instructional Research Laboratory Technical Paper # R82007.

ERIC Educational Resources Information Center

Sadoski, Mark C.

A study investigated the role of visual imagery in the comprehension and retention of prose. Subjects were 48 fifth grade students who orally read a story and then completed three comprehension tasks directly related to the story: a retelling, an oral reading cloze test, and a multiple choice question test comprised of items demonstrated to be…

Fundamental Use of Surgical Energy (FUSE) certification: validation and predictors of success.

PubMed

Robinson, Thomas N; Olasky, Jaisa; Young, Patricia; Feldman, Liane S; Fuchshuber, Pascal R; Jones, Stephanie B; Madani, Amin; Brunt, Michael; Mikami, Dean; Jackson, Gretchen P; Mischna, Jessica; Schwaitzberg, Steven; Jones, Daniel B

2016-03-01

The Fundamental Use of Surgical Energy (FUSE) program includes a Web-based didactic curriculum and a high-stakes multiple-choice question examination with the goal to provide certification of knowledge on the safe use of surgical energy-based devices. The purpose of this study was (1) to set a passing score through a psychometrically sound process and (2) to determine what pretest factors predicted passing the FUSE examination. Beta-testing of multiple-choice questions on 62 topics of importance to the safe use of surgical energy-based devices was performed. Eligible test takers were physicians with a minimum of 1 year of surgical training who were recruited by FUSE task force members. A pretest survey collected baseline information. A total of 227 individuals completed the FUSE beta-test, and 208 completed the pretest survey. The passing/cut score for the first test form of the FUSE multiple-choice examination was determined using the modified Angoff methodology and for the second test form was determined using a linear equating methodology. The overall passing rate across the two examination forms was 81.5%. Self-reported time studying the FUSE Web-based curriculum for a minimum of >2 h was associated with a passing examination score (p < 0.001). Performance was not different based on increased years of surgical practice (p = 0.363), self-reported expertise on one or more types of energy-based devices (p = 0.683), participation in the FUSE postgraduate course (p = 0.426), or having reviewed the FUSE manual (p = 0.428). Logistic regression found that studying the FUSE didactics for >2 h predicted a passing score (OR 3.61; 95% CI 1.44-9.05; p = 0.006) independent of the other baseline characteristics recorded. The development of the FUSE examination, including the passing score, followed a psychometrically sound process. Self-reported time studying the FUSE curriculum predicted a passing score independent of other pretest characteristics such as years in practice and self-reported expertise.

Writing Multiple Choice Outcome Questions to Assess Knowledge and Competence.

PubMed

Brady, Erik D

2015-11-01

Few articles contemplate the need for good guidance in question item-writing in the continuing education (CE) space. Although many of the core principles of sound item design translate to the CE health education team, the need exists for specific examples for nurse educators that clearly describe how to measure changes in competence and knowledge using multiple choice items. In this article, some keys points and specific examples for nursing CE providers are shared. Copyright 2015, SLACK Incorporated.

Multiple Choice Testing and the Retrieval Hypothesis of the Testing Effect

ERIC Educational Resources Information Center

Sensenig, Amanda E.

2010-01-01

Taking a test often leads to enhanced later memory for the tested information, a phenomenon known as the "testing effect". This memory advantage has been reliably demonstrated with recall tests but not multiple choice tests. One potential explanation for this finding is that multiple choice tests do not rely on retrieval processes to the same…

Non-Hierarchical Clustering as a Method to Analyse an Open-Ended Questionnaire on Algebraic Thinking

ERIC Educational Resources Information Center

Di Paola, Benedetto; Battaglia, Onofrio Rosario; Fazio, Claudio

2016-01-01

The problem of taking a data set and separating it into subgroups, where the members of each subgroup are more similar to each other than they are to members outside the subgroup, has been extensively studied in science and mathematics education research. Student responses to written questions and multiple-choice tests have been characterised and…

Using the ePortfolio to Complement Standardized Testing in a Healthcare Professional Program: Better Education or More Busy Work?

ERIC Educational Resources Information Center

Chan, Clarence

2012-01-01

This article evaluates the full-scale integration of the ePortfolio into a healthcare professional program in an open admissions community college in the United States. The Physical Therapist Assistant program in question struggles to balance the dynamic tension between preparing students for a summative multiple-choice licensing examination and…

Assessment of Numeracy in Sports and Exercise Science Students at an Australian University

ERIC Educational Resources Information Center

Green, Simon; McGlynn, Susan; Stuart, Deidre; Fahey, Paul; Pettigrew, Jim; Clothier, Peter

2018-01-01

The effect of high school study of mathematics on numeracy performance of sports and exercise science (SES) students is not clear. To investigate this further, we tested the numeracy skills of 401 students enrolled in a Bachelor of Health Sciences degree in SES using a multiple-choice survey consisting of four background questions and 39 numeracy…

Processing and Memory of Central versus Peripheral Information as a Function of Reading Goals: Evidence from Eye-Movements

ERIC Educational Resources Information Center

Yeari, Menahem; van den Broek, Paul; Oudega, Marja

2015-01-01

The present study examined the effect of reading goals on the processing and memory of central and peripheral textual information. Using eye-tracking methodology, we compared the effect of four common reading goals--entertainment, presentation, studying for a close-ended (multiple-choice) questions test, and studying for an open-ended questions…

Multiple-Choice Answers: To Change or Not to Change? Perhaps Not Such a Simple Question

ERIC Educational Resources Information Center

Wainscott, Heidi

2016-01-01

When grading students' quizzes and exams, I find that students are seemingly always changing their answers from the right answer to the wrong answer. In fact, I have cautioned students against changing their answer. Colleagues have made similar observations and some books on test-taking strategies advise against answer-changing. In an effort to…

Can Item Analysis of MCQs Accomplish the Need of a Proper Assessment Strategy for Curriculum Improvement in Medical Education?

ERIC Educational Resources Information Center

Pawade, Yogesh R.; Diwase, Dipti S.

2016-01-01

Item analysis of Multiple Choice Questions (MCQs) is the process of collecting, summarizing and utilizing information from students' responses to evaluate the quality of test items. Difficulty Index (p-value), Discrimination Index (DI) and Distractor Efficiency (DE) are the parameters which help to evaluate the quality of MCQs used in an…

An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis.

PubMed

Tarrant, Marie; Ware, James; Mohammed, Ahmed M

2009-07-07

Four- or five-option multiple choice questions (MCQs) are the standard in health-science disciplines, both on certification-level examinations and on in-house developed tests. Previous research has shown, however, that few MCQs have three or four functioning distractors. The purpose of this study was to investigate non-functioning distractors in teacher-developed tests in one nursing program in an English-language university in Hong Kong. Using item-analysis data, we assessed the proportion of non-functioning distractors on a sample of seven test papers administered to undergraduate nursing students. A total of 514 items were reviewed, including 2056 options (1542 distractors and 514 correct responses). Non-functioning options were defined as ones that were chosen by fewer than 5% of examinees and those with a positive option discrimination statistic. The proportion of items containing 0, 1, 2, and 3 functioning distractors was 12.3%, 34.8%, 39.1%, and 13.8% respectively. Overall, items contained an average of 1.54 (SD = 0.88) functioning distractors. Only 52.2% (n = 805) of all distractors were functioning effectively and 10.2% (n = 158) had a choice frequency of 0. Items with more functioning distractors were more difficult and more discriminating. The low frequency of items with three functioning distractors in the four-option items in this study suggests that teachers have difficulty developing plausible distractors for most MCQs. Test items should consist of as many options as is feasible given the item content and the number of plausible distractors; in most cases this would be three. Item analysis results can be used to identify and remove non-functioning distractors from MCQs that have been used in previous tests.

The Positive and Negative Consequences of Multiple-Choice Testing

ERIC Educational Resources Information Center

Roediger, Henry L.; Marsh, Elizabeth J.

2005-01-01

Multiple-choice tests are commonly used in educational settings but with unknown effects on students' knowledge. The authors examined the consequences of taking a multiple-choice test on a later general knowledge test in which students were warned not to guess. A large positive testing effect was obtained: Prior testing of facts aided final…

Students' Geographic Knowledge and Skills in Different Kinds of Tests: Multiple-Choice versus Performance Assessment.

ERIC Educational Resources Information Center

Kon, Jane Heckley; Martin-Kniep, Giselle O.

1992-01-01

Describes a case study to determine whether performance tests are a feasible alternative to multiple-choice tests. Examines the difficulties of administering and scoring performance assessments. Explains that the study employed three performance tests and one multiple-choice test. Concludes that performance test administration and scoring was no…

Instructor perspectives of multiple-choice questions in summative assessment for novice programmers

NASA Astrophysics Data System (ADS)

Shuhidan, Shuhaida; Hamilton, Margaret; D'Souza, Daryl

2010-09-01

Learning to program is known to be difficult for novices. High attrition and high failure rates in foundation-level programming courses undertaken at tertiary level in Computer Science programs, are commonly reported. A common approach to evaluating novice programming ability is through a combination of formative and summative assessments, with the latter typically represented by a final examination. Preparation of such assessment is driven by instructor perceptions of student learning of programming concepts. This in turn may yield instructor perspectives of summative assessment that do not necessarily correlate with student expectations or abilities. In this article, we present results of our study around instructor perspectives of summative assessment for novice programmers. Both quantitative and qualitative data have been obtained via survey responses from programming instructors with varying teaching experience, and from novice student responses to targeted examination questions. Our findings highlight that most of the instructors believed that summative assessment is, and is meant to be, a valid measure of a student's ability to program. Most instructors further believed that Multiple-choice Questions (MCQs) provide a means of testing a low level of understanding, and a few added qualitative comments to suggest that MCQs are easy questions, and others refused to use them at all. There was no agreement around the proposition that if a question was designed to test a low level of skill, or a low level in a hierarchy of a body of knowledge, that such a question should or would be found to be easy by the student. To aid our analysis of assessment questions, we introduced four measures: Syntax Knowledge; Semantic Knowledge; Problem Solving Skill and the Level of Difficulty of the Problem. We applied these measures to selected examination questions, and have identified gaps between the instructor perspectives of what is considered to be an easy question and also in what is required to be assessed to determine whether students have achieved the goals of their course.

Students’ Conception on Heat and Temperature toward Science Process Skill

NASA Astrophysics Data System (ADS)

Ratnasari, D.; Sukarmin, S.; Suparmi, S.; Aminah, N. S.

2017-09-01

This research is aimed to analyze the effect of students’ conception toward science process skill. This is a descriptive research with subjects of the research were 10th-grade students in Surakarta from high, medium and low categorized school. The sample selection uses purposive sampling technique based on physics score in national examination four latest years. Data in this research collecting from essay test, two-tier multiple choice test, and interview. Two-tier multiple choice test consists of 30 question that contains an indicator of science process skill. Based on the result of the research and analysis, it shows that students’ conception of heat and temperature affect science process skill of students. The students’ conception that still contains the wrong concept can emerge misconception. For the future research, it is suggested to improve students’ conceptual understanding and students’ science process skill with appropriate learning method and assessment instrument because heat and temperature is one of physics material that closely related with students’ daily life.

Glucose as the Sole Metabolic Fuel: A Study on the Possible Influence of Teachers' Knowledge on the Establishment of a Misconception among Brazilian High School Students

ERIC Educational Resources Information Center

da Luz, Mauricio Roberto Motta Pinto

2008-01-01

In the present work, I investigated the origin of the misconception that glucose is the sole metabolic fuel previously described among Brazilian high school students. The results of a multiple-choice test composed of 24 questions about a broad range of biology subjects were analyzed. The test was part of a contest and was answered by a sample…

A Comparison of Alternate-Choice and True-False Item Forms Used in Classroom Examinations.

ERIC Educational Resources Information Center

Maihoff, N. A.; Mehrens, Wm. A.

A comparison is presented of alternate-choice and true-false item forms used in an undergraduate natural science course. The alternate-choice item is a modified two-choice multiple-choice item in which the two responses are included within the question stem. This study (1) compared the difficulty level, discrimination level, reliability, and…

Navigating the feminine in massively multiplayer online games: gender in World of Warcraft.

PubMed

Brehm, Audrey L

2013-01-01

The objective of the study is to present and discuss attitudes, perceptions and opinions about sexism and gendered play in the massively multiplayer online roleplaying game (MMO), World of Warcraft. Through the use of an online survey which includes both multiple choice questions and open-ended questions, the research discusses the major themes and findings expressed by the World of Warcraft forum users (N = 294). The descriptive statistical findings presented are derived from the multiple choice questions. Within the sample, the results reveal that sexism is a contentious topic in the World of Warcraft community. 63.6% (n = 75) of female respondents reported experiencing sexism within the game. 27.5% (n = 44) of male respondents and 45.3% (n = 53) of female respondents believe that sexism is a problem in the game. Overall, 64.4% (n = 183) of the respondents reported sexism as a non-issue in the game. Themes surrounding the topic of sexism experienced within the game are presented based on frequency of homogenous responses. Based on the multiple choice questions and the open-ended questions, the research argues that sexism and gendered play in gaming should be studied more closely, as the results reveal that many MMO players are affected negatively by it.

M-OSCE as a method to measure dental hygiene students' critical thinking: a pilot study.

PubMed

McComas, Martha J; Wright, Rebecca A; Mann, Nancy K; Cooper, Mary D; Jacks, Mary E

2013-04-01

Educators in all academic disciplines have been encouraged to utilize assessment strategies to evaluate students' critical thinking. The purpose of this study was to assess the viability of the modified objective structured clinical examination (m-OSCE) to evaluate critical thinking in dental hygiene education. This evaluation utilized a convenience sample of senior dental hygiene students. Students participated in the m-OSCE in which portions of a patient case were revealed at four stations. The exam consisted of multiple-choice questions intended to measure students' ability to utilize critical thinking skills. Additionally, there was one fill-in-the-blank question and a treatment plan that was completed at the fifth station. The results of this study revealed that the m-OSCE did not reliably measure dental hygiene students' critical thinking. Statistical analysis found no satisfactory reliability within the multiple-choice questions and moderately reliable results within the treatment planning portion of the examination. In addition, the item analysis found gaps in students' abilities to transfer clinical evidence/data to basic biomedical knowledge as demonstrated through the multiple-choice questioning results. This outcome warrants further investigation of the utility of the m-OSCE, with a focus on modifications to the evaluation questions, grading rubric, and patient case.

Navigating the feminine in massively multiplayer online games: gender in World of Warcraft

PubMed Central

Brehm, Audrey L.

2013-01-01

The objective of the study is to present and discuss attitudes, perceptions and opinions about sexism and gendered play in the massively multiplayer online roleplaying game (MMO), World of Warcraft. Through the use of an online survey which includes both multiple choice questions and open-ended questions, the research discusses the major themes and findings expressed by the World of Warcraft forum users (N = 294). The descriptive statistical findings presented are derived from the multiple choice questions. Within the sample, the results reveal that sexism is a contentious topic in the World of Warcraft community. 63.6% (n = 75) of female respondents reported experiencing sexism within the game. 27.5% (n = 44) of male respondents and 45.3% (n = 53) of female respondents believe that sexism is a problem in the game. Overall, 64.4% (n = 183) of the respondents reported sexism as a non-issue in the game. Themes surrounding the topic of sexism experienced within the game are presented based on frequency of homogenous responses. Based on the multiple choice questions and the open-ended questions, the research argues that sexism and gendered play in gaming should be studied more closely, as the results reveal that many MMO players are affected negatively by it. PMID:24363650

Measures of Partial Knowledge and Unexpected Responses in Multiple-Choice Tests

ERIC Educational Resources Information Center

Chang, Shao-Hua; Lin, Pei-Chun; Lin, Zih-Chuan

2007-01-01

This study investigates differences in the partial scoring performance of examinees in elimination testing and conventional dichotomous scoring of multiple-choice tests implemented on a computer-based system. Elimination testing that uses the same set of multiple-choice items rewards examinees with partial knowledge over those who are simply…

A Method for Writing Open-Ended Curved Arrow Notation Questions for Multiple-Choice Exams and Electronic-Response Systems

ERIC Educational Resources Information Center

Ruder, Suzanne M.; Straumanis, Andrei R.

2009-01-01

A critical stage in the process of developing a conceptual understanding of organic chemistry is learning to use curved arrow notation. From this stems the ability to predict reaction products and mechanisms beyond the realm of memorization. Since evaluation (i.e., testing) is known to be a key driver of student learning, it follows that a new…

Delay and probability discounting of multiple commodities in smokers and never-smokers using multiple-choice tasks.

PubMed

Poltavski, Dmitri V; Weatherly, Jeffrey N

2013-12-01

The purpose of the present study was to investigate temporal and probabilistic discounting in smokers and never-smokers, across a number of commodities, using a multiple-choice method. One hundred and eighty-two undergraduate university students, of whom 90 had never smoked, 73 were self-reported light smokers (<10 cigarettes/day), and 17 were heavy smokers (10+cigarettes/day), completed computerized batteries of delay and probability discounting questions pertaining to a total of eight commodities and administered in a multiple-choice format. In addition to cigarettes, monetary rewards, and health outcomes, the tasks included novel commodities such as ideal dating partner and retirement income. The results showed that heavy smokers probability discounted commodities at a significantly shallower rate than never-smokers, suggesting greater risk-taking. No effect of smoking status was observed for delay discounting questions. The only commodity that was probability discounted significantly less than others was 'finding an ideal dating partner'. The results suggest that probability discounting tasks using the multiple-choice format can discriminate between non-abstaining smokers and never-smokers and could be further explored in the context of behavioral and drug addictions.

Academic performance of day scholars versus boarders in pharmacology examinations of a medical school in Pakistan.

PubMed

Faisal, Rizwan; Shinwari, Laiyla; Izzat, Saadia

2016-09-01

To compare the academic performance of day scholar and boarder students in Pharmacology examinations. This comparative study was conducted at Rehman Medical College, Peshawar, Pakistan, from June to September, 2015. It comprised third-year medical students of the sessions 2013-14 and 2014-15.The record of the results of examinations, which had already been conducted, were assessed. All the exams had two components, i.e. multiple-choice questions and short-essay questions. Students were categorised into 4 groups according to their academic performance: those who got <50% marks (Group 1); 51-69% marks (Group 2); 70-80% marks (Group 3); and >80% marks (Group 4). SPSS 20 was used for data analysis. Of the 200 students, 159(79.5%) were day scholars and 41(20.5%) were boarders. In multiple-choice questions, 29(70.7%) boarder students were in Group 2, while none of them was in Group 4. In short-essay questions, 11(26.8%) of them were in Group 1 and 17(41.5%) in Group 2. Results of day scholars' multiple-choice questions exams showed 93(58.5%) were in Group 2 and 2(1.3%) in Group 4. In short-essay questions, 63(39.6%) were in Group 2 (p>o.o5 each). No significant difference was found between the academic performance of boarders and day scholars.

Exam Question Exchange.

ERIC Educational Resources Information Center

Alexander, John J., Ed.

1987-01-01

Contains two articles relating to chemistry examination questions. One provides examples of how to sequence multiple choice questions so that partial credit may be given for some responses. The second includes a question and solution dealing with stereoisomerism as a result of free radical chlorination of a nonstereoisometic substance. (TW)

Use of the Moodle Platform to Promote an Ongoing Learning When Lecturing General Physics in the Physics, Mathematics and Electronic Engineering Programmes at the University of the Basque Country UPV/EHU

NASA Astrophysics Data System (ADS)

López, Gabriel A.; Sáenz, Jon; Leonardo, Aritz; Gurtubay, Idoia G.

2016-08-01

The Moodle platform has been used to put into practice an ongoing evaluation of the students' Physics learning process. The evaluation has been done on the frame of the course General Physics, which is lectured during the first year of the Physics, Mathematics and Electronic Engineering Programmes at the Faculty of Science and Technology of the University of the Basque Country (UPV/EHU). A test bank with more than 1000 multiple-choice questions, including conceptual and numerical problems, has been prepared. Throughout the course, the students have to answer a 10-question multiple-choice test for every one of the blocks the course is divided in and which were previously treated and worked in the theoretical lectures and problem-solving sessions. The tests are automatically corrected by Moodle, and under certain criteria, the corresponding mark is taken into account for the final mark of the course. According to the results obtained from a statistical study of the data on the student performances during the last four academic years, it has been observed that there exists an actual correlation between the marks obtained in the Moodle tests and the final mark of the course. In addition, it could be deduced that students who have passed the Moodle tests increase their possibilities of passing the course by an odds ratio close to 3.

Identification of technical item flaws leads to improvement of the quality of single best Multiple Choice Questions.

PubMed

Fayyaz Khan, Humaira; Farooq Danish, Khalid; Saeed Awan, Azra; Anwar, Masood

2013-05-01

The purpose of the study was to identify technical item flaws in the multiple choice questions submitted for the final exams for the years 2009, 2010 and 2011. This descriptive analytical study was carried out in Islamic International Medical College (IIMC). The Data was collected from the MCQ's submitted by the faculty for the final exams for the year 2009, 2010 and 2011. The data was compiled and evaluated by a three member assessment committee. The data was analyzed for frequency and percentages the categorical data was analyzed by chi-square test. Overall percentage of flawed item was 67% for the year 2009 of which 21% were for testwiseness and 40% were for irrelevant difficulty. In year 2010 the total item flaws were 36% and 11% testwiseness and 22% were for irrelevant difficulty. The year 2011 data showed decreased overall flaws of 21%. The flaws of testwisness were 7%, irrelevant difficulty were 11%. Technical item flaws are frequently encountered during MCQ construction, and the identification of flaws leads to improved quality of the single best MCQ's.

Reducing the Need for Guesswork in Multiple-Choice Tests

ERIC Educational Resources Information Center

Bush, Martin

2015-01-01

The humble multiple-choice test is very widely used within education at all levels, but its susceptibility to guesswork makes it a suboptimal assessment tool. The reliability of a multiple-choice test is partly governed by the number of items it contains; however, longer tests are more time consuming to take, and for some subject areas, it can be…

The Impact of Escape Alternative Position Change in Multiple-Choice Test on the Psychometric Properties of a Test and Its Items Parameters

ERIC Educational Resources Information Center

Hamadneh, Iyad Mohammed

2015-01-01

This study aimed at investigating the impact changing of escape alternative position in multiple-choice test on the psychometric properties of a test and it's items parameters (difficulty, discrimination & guessing), and estimation of examinee ability. To achieve the study objectives, a 4-alternative multiple choice type achievement test…

Does retrieval practice enhance learning and transfer relative to restudy for term-definition facts?

PubMed

Pan, Steven C; Rickard, Timothy C

2017-09-01

In many pedagogical contexts, term-definition facts that link a concept term (e.g., "vision") with its corresponding definition (e.g., "the ability to see") are learned. Does retrieval practice involving retrieval of the term (given the definition) or the definition (given the term) enhance subsequent recall, relative to restudy of the entire fact? Moreover, does any benefit of retrieval practice for the term transfer to later recall of the definition, or vice versa? We addressed those questions in 4 experiments. In each, subjects first studied term-definition facts and then trained on two thirds of the facts using multiple-choice tests with feedback. Half of the test questions involved recalling terms; the other half involved recalling definitions. The remaining facts were either not trained (Experiment 1) or restudied (Experiments 2-4). A 48-hr delayed multiple-choice (Experiments 1-2) or short answer (Experiments 3a-4) final test assessed recall of all terms or all definitions. Replicating and extending prior research, retrieval practice yielded improved recall and positive transfer relative to no training. Relative to restudy, however, retrieval practice consistently enhanced subsequent term retrieval, enhanced subsequent definition retrieval only after repeated practice, and consistently yielded at best minimal positive transfer in either direction. Theoretical and practical implications are discussed. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

Comparison of Integrated Testlet and Constructed-Response Question Formats

ERIC Educational Resources Information Center

Slepkov, Aaron D.; Shiell, Ralph C.

2014-01-01

Constructed-response (CR) questions are a mainstay of introductory physics textbooks and exams. However, because of the time, cost, and scoring reliability constraints associated with this format, CR questions are being increasingly replaced by multiple-choice (MC) questions in formal exams. The integrated testlet (IT) is a recently developed…

Environmental Problems, Causes, and Solutions: An Open Question

ERIC Educational Resources Information Center

Negev, Maya; Garb, Yaakov; Biller, Roni; Sagy, Gonen; Tal, Alon

2010-01-01

In a national evaluation of environmental literacy in Israel, (Negev, Sagy, Garb, Salzberg, & Tal, 2008), the authors included both multiple choice questions and open questions. In this article the authors describe the qualitative analysis of the answers to an open question regarding a local environmental problem. Most participants specified…

Primary FRCS Eng

PubMed Central

Hobsley, Michael

1974-01-01

In five consecutive Primary Examinations for the Fellowship of the Royal College of Surgeons of England, the scores of candidates in the multiple choice question paper, written paper, and oral interview have been analysed for mutual correlations and for the reproducibility of the written paper score. The conclusions reached were that all these scores correlate with each other, that no score can be left out without reducing the reliability of the examination, that the marking of written papers in a close-marking system is remarkably reproducible, and that the oral score contributes most, the multiple choice question paper the least, to the overall assessment. PMID:4417893

Are Multiple Choice Tests Fair to Medical Students with Specific Learning Disabilities?

ERIC Educational Resources Information Center

Ricketts, Chris; Brice, Julie; Coombes, Lee

2010-01-01

The purpose of multiple choice tests of medical knowledge is to estimate as accurately as possible a candidate's level of knowledge. However, concern is sometimes expressed that multiple choice tests may also discriminate in undesirable and irrelevant ways, such as between minority ethnic groups or by sex of candidates. There is little literature…

Do Streaks Matter in Multiple-Choice Tests?

ERIC Educational Resources Information Center

Kiss, Hubert János; Selei, Adrienn

2018-01-01

Success in life is determined to a large extent by school performance, which in turn depends heavily on grades obtained in exams. In this study, we investigate a particular type of exam: multiple-choice tests. More concretely, we study if patterns of correct answers in multiple-choice tests affect performance. We design an experiment to study if…

The Effects of Clinically Relevant Multiple-Choice Items on the Statistical Discrimination of Physician Clinical Competence.

ERIC Educational Resources Information Center

Downing, Steven M.; Maatsch, Jack L.

To test the effect of clinically relevant multiple-choice item content on the validity of statistical discriminations of physicians' clinical competence, data were collected from a field test of the Emergency Medicine Examination, test items for the certification of specialists in emergency medicine. Two 91-item multiple-choice subscales were…

Developing, Analyzing, and Using Distractors for Multiple-Choice Tests in Education: A Comprehensive Review

ERIC Educational Resources Information Center

Gierl, Mark J.; Bulut, Okan; Guo, Qi; Zhang, Xinxin

2017-01-01

Multiple-choice testing is considered one of the most effective and enduring forms of educational assessment that remains in practice today. This study presents a comprehensive review of the literature on multiple-choice testing in education focused, specifically, on the development, analysis, and use of the incorrect options, which are also…

Time-Series Analysis: Assessing the Effects of Multiple Educational Interventions in a Small-Enrollment Course

NASA Astrophysics Data System (ADS)

Warren, Aaron R.

2009-11-01

Time-series designs are an alternative to pretest-posttest methods that are able to identify and measure the impacts of multiple educational interventions, even for small student populations. Here, we use an instrument employing standard multiple-choice conceptual questions to collect data from students at regular intervals. The questions are modified by asking students to distribute 100 Confidence Points among the options in order to indicate the perceived likelihood of each answer option being the correct one. Tracking the class-averaged ratings for each option produces a set of time-series. ARIMA (autoregressive integrated moving average) analysis is then used to test for, and measure, changes in each series. In particular, it is possible to discern which educational interventions produce significant changes in class performance. Cluster analysis can also identify groups of students whose ratings evolve in similar ways. A brief overview of our methods and an example are presented.

Review of multiple-choice-questions and group performance - A comparison of face-to-face and virtual groups with and without facilitation

PubMed Central

Kazubke, Edda; Schüttpelz-Brauns, Katrin

2010-01-01

Background: Multiple choice questions (MCQs) are often used in exams of medical education and need careful quality management for example by the application of review committees. This study investigates whether groups communicating virtually by email are similar to face-to-face groups concerning their review process performance and whether a facilitator has positive effects. Methods: 16 small groups of students were examined, which had to evaluate and correct MCQs under four different conditions. In the second part of the investigation the changed questions were given to a new random sample for the judgement of the item quality. Results: There was no significant influence of the variables “form of review committee” and “facilitation”. However, face-to-face and virtual groups clearly differed in the required treatment times. The test condition “face to face without facilitation” was generally valued most positively concerning taking over responsibility, approach to work, sense of well-being, motivation and concentration on the task. Discussion: Face-to-face and virtual groups are equally effective in the review of MCQs but differ concerning their efficiency. The application of electronic review seems to be possible but is hardly recommendable because of the long process time and technical problems. PMID:21818213

Are Faculty Predictions or Item Taxonomies Useful for Estimating the Outcome of Multiple-Choice Examinations?

ERIC Educational Resources Information Center

Kibble, Jonathan D.; Johnson, Teresa

2011-01-01

The purpose of this study was to evaluate whether multiple-choice item difficulty could be predicted either by a subjective judgment by the question author or by applying a learning taxonomy to the items. Eight physiology faculty members teaching an upper-level undergraduate human physiology course consented to participate in the study. The…

Australian Item Bank Program: Science Item Bank. Book 3: Biology.

ERIC Educational Resources Information Center

Australian Council for Educational Research, Hawthorn.

The Australian Science Item Bank consists of three volumes of multiple-choice questions. Book 3 contains questions on the biological sciences. The questions are designed to be suitable for high school students (year 8 to year 12 in Australian schools). The questions are classified by the subject content of the question, the cognitive skills…

Making School Choice Work

ERIC Educational Resources Information Center

DeArmond, Michael; Jochim, Ashley; Lake, Robin

2014-01-01

School choice is increasingly the new normal in urban education. But in cities with multiple public school options, how can civic leaders create a choice system that works for all families, whether they choose a charter or district public school? To answer this question, the Center on Reinventing Public Education (CRPE) researchers surveyed 4,000…

Algorithms for Developing Test Questions from Sentences in Instructional Materials: an Extension of an Earlier Study

DTIC Science & Technology

1980-01-01

Silverfish, Canine, and Cicadas . b. ’Mgorithinically--Sdverfish, Females, Individuals, and Wasps. This process resulted in 16.0 multiple-choice items: 20...and Their Text Frequency Nouns Adjectives Rare Singleton Keyword Rare Singleton Keyword Instars Insect (8) Plant-feeding Immature (3) Cicadas ...Sllverflsh (c) Caniru’S (d) Cicadas c. Foils Produced Algorl ..nii-allv: Sllverflsh Fenvilea ItuliviJu.i Is Wanps ?. Kevword Adjective—Immature

Advanced Placement Economics. Macroeconomics: Student Activities.

ERIC Educational Resources Information Center

Morton, John S.

This book is designed to help advanced placement students better understand macroeconomic concepts through various activities. The book contains 6 units with 64 activities, sample multiple-choice questions, sample short essay questions, and sample long essay questions. The units are entitled: (1) "Basic Economic Concepts"; (2) "Measuring Economic…

A comparative study of students' performance in preclinical physiology assessed by multiple choice and short essay questions.

PubMed

Oyebola, D D; Adewoye, O E; Iyaniwura, J O; Alada, A R; Fasanmade, A A; Raji, Y

2000-01-01

This study was designed to compare the performance of medical students in physiology when assessed by multiple choice questions (MCQs) and short essay questions (SEQs). The study also examined the influence of factors such as age, sex, O/level grades and JAMB scores on performance in the MCQs and SEQs. A structured questionnaire was administered to 264 medical students' four months before the Part I MBBS examination. Apart from personal data of each student, the questionnaire sought information on the JAMB scores and GCE O' Level grades of each student in English Language, Biology, Chemistry, Physics and Mathematics. The physiology syllabus was divided into five parts and the students were administered separate examinations (tests) on each part. Each test consisted of MCQs and SEQs. The performance in MCQs and SEQs were compared. Also, the effects of JAMB scores and GCE O/level grades on the performance in both the MCQs and SEQs were assessed. The results showed that the students performed better in all MCQ tests than in the SEQs. JAMB scores and O' level English Language grade had no significant effect on students' performance in MCQs and SEQs. However O' level grades in Biology, Chemistry, Physics and Mathematics had significant effects on performance in MCQs and SEQs. Inadequate knowledge of physiology and inability to present information in a logical sequence are believed to be major factors contributing to the poorer performance in the SEQs compared with MCQs. In view of the finding of significant association between performance in MCQs and SEQs and GCE O/level grades in science subjects and mathematics, it was recommended that both JAMB results and the GCE results in the four O/level subjects above may be considered when selecting candidates for admission into the medical schools.

On the Equivalence of Constructed-Response and Multiple-Choice Tests.

ERIC Educational Resources Information Center

Traub, Ross E.; Fisher, Charles W.

Two sets of mathematical reasoning and two sets of verbal comprehension items were cast into each of three formats--constructed response, standard multiple-choice, and Coombs multiple-choice--in order to assess whether tests with indentical content but different formats measure the same attribute, except for possible differences in error variance…

Student perception and post-exam analysis of one best MCQs and one correct MCQs: A comparative study.

PubMed

Adhi, Mohammad Idrees; Aly, Syed Moyn

2018-04-01

To find differences between One-Correct and One-Best multiple-choice questions with relation to student scores, post-exam item analyses results and student perception. This comparative cross-sectional study was conducted at the Dow University of Health Sciences, Karachi, from November 2010 to April 2011, and comprised medical students. Data was analysed using SPSS 18. Of the 207 participants, 16(7.7%) were boys and 191(92.3%) were girls. The mean score in Paper I was 18.62±4.7, while in Paper II it was 19.58±6.1. One-Best multiple-choice questions performed better than One-Correct. There was no statistically significant difference in the mean scores of the two papers or in the difficulty indices. Difficulty and discrimination indices correlated well in both papers. Cronbach's alpha of paper I was 0.584 and that of paper II was 0.696. Point-biserial values were better for paper II than for paper I. Most students expressed dissatisfaction with paper II. One-Best multiple-choice questions showed better scores, higher reliability, better item performance and correlation values.

Learning from peer feedback on student-generated multiple choice questions: Views of introductory physics students

NASA Astrophysics Data System (ADS)

Kay, Alison E.; Hardy, Judy; Galloway, Ross K.

2018-06-01

PeerWise is an online application where students are encouraged to generate a bank of multiple choice questions for their classmates to answer. After answering a question, students can provide feedback to the question author about the quality of the question and the question author can respond to this. Student use of, and attitudes to, this online community within PeerWise was investigated in two large first year undergraduate physics courses, across three academic years, to explore how students interact with the system and the extent to which they believe PeerWise to be useful to their learning. Most students recognized that there is value in engaging with PeerWise, and many students engaged deeply with the system, thinking critically about the quality of their submissions and reflecting on feedback provided to them. Students also valued the breadth of topics and level of difficulty offered by the questions, recognized the revision benefits afforded by the resource, and were often willing to contribute to the community by providing additional explanations and engaging in discussion.

Approaches to learning for the ANZCA Final Examination and validation of the revised Study Process Questionnaire in specialist medical training.

PubMed

Weller, J M; Henning, M; Civil, N; Lavery, L; Boyd, M J; Jolly, B

2013-09-01

When evaluating assessments, the impact on learning is often overlooked. Approaches to learning can be deep, surface and strategic. To provide insights into exam quality, we investigated the learning approaches taken by trainees preparing for the Australian and New Zealand College of Anaesthetists (ANZCA) Final Exam. The revised two-factor Study Process Questionnaire (R-SPQ-2F) was modified and validated for this context and was administered to ANZCA advanced trainees. Additional questions were asked about perceived value for anaesthetic practice, study time and approaches to learning for each exam component. Overall, 236 of 690 trainees responded (34%). Responses indicated both deep and surface approaches to learning with a clear preponderance of deep approaches. The anaesthetic viva was valued most highly and the multiple choice question component the least. Despite this, respondents spent the most time studying for the multiple choice questions. The traditionally low short answer questions pass rate could not be explained by limited study time, perceived lack of value or study approaches. Written responses suggested that preparation for multiple choice questions was characterised by a surface approach, with rote memorisation of past questions. Minimal reference was made to the ANZCA syllabus as a guide for learning. These findings indicate that, although trainees found the exam generally relevant to practice and adopted predominantly deep learning approaches, there was considerable variation between the four components. These results provide data with which to review the existing ANZCA Final Exam and comparative data for future studies of the revisions to the ANZCA curriculum and exam process.

Emotion modulates eye movement patterns and subsequent memory for the gist and details of movie scenes.

PubMed

Subramanian, Ramanathan; Shankar, Divya; Sebe, Nicu; Melcher, David

2014-03-26

A basic question in vision research regards where people look in complex scenes and how this influences their performance in various tasks. Previous studies with static images have demonstrated a close link between where people look and what they remember. Here, we examined the pattern of eye movements when participants watched neutral and emotional clips from Hollywood-style movies. Participants answered multiple-choice memory questions concerning visual and auditory scene details immediately upon viewing 1-min-long neutral or emotional movie clips. Fixations were more narrowly focused for emotional clips, and immediate memory for object details was worse compared to matched neutral scenes, implying preferential attention to emotional events. Although we found the expected correlation between where people looked and what they remembered for neutral clips, this relationship broke down for emotional clips. When participants were subsequently presented with key frames (static images) extracted from the movie clips such that presentation duration of the target objects (TOs) corresponding to the multiple-choice questions was matched and the earlier questions were repeated, more fixations were observed on the TOs, and memory performance also improved significantly, confirming that emotion modulates the relationship between gaze position and memory performance. Finally, in a long-term memory test, old/new recognition performance was significantly better for emotional scenes as compared to neutral scenes. Overall, these results are consistent with the hypothesis that emotional content draws eye fixations and strengthens memory for the scene gist while weakening encoding of peripheral scene details.

Stereotype threat? Effects of inquiring about test takers' gender on conceptual test performance in physics

NASA Astrophysics Data System (ADS)

Maries, Alexandru; Singh, Chandralekha

2015-12-01

It has been found that activation of a stereotype, for example by indicating one's gender before a test, typically alters performance in a way consistent with the stereotype, an effect called "stereotype threat." On a standardized conceptual physics assessment, we found that asking test takers to indicate their gender right before taking the test did not deteriorate performance compared to an equivalent group who did not provide gender information. Although a statistically significant gender gap was present on the standardized test whether or not students indicated their gender, no gender gap was observed on the multiple-choice final exam students took, which included both quantitative and conceptual questions on similar topics.

The development of a computer assisted instruction and assessment system in pharmacology.

PubMed

Madsen, B W; Bell, R C

1977-01-01

We describe the construction of a computer based system for instruction and assessment in pharmacology, utilizing a large bank of multiple choice questions. Items were collected from many sources, edited and coded for student suitability, topic, taxonomy and difficulty and text references. Students reserve a time during the day, specify the type of test desired and questions are presented randomly from the subset satisfying their criteria. Answers are scored after each question and a summary given at the end of every test; details on item performance are recorded automatically. The biggest hurdle in implementation was the assembly, review, classification and editing of items, while the programming was relatively straight-forward. A number of modifications had to be made to the initial plans and changes will undoubtedly continue with further experience. When fully operational the system will possess a number of advantages including: elimination of test preparation, editing and marking; facilitated item review opportunities; increased objectivity, feedback, flexibility and descreased anxiety in students.

Teaching habitat and animal classification to fourth graders using an engineering-design model

NASA Astrophysics Data System (ADS)

Marulcu, Ismail

2014-05-01

Background: The motivation for this work is built upon the premise that there is a need for research-based materials for design-based science instruction. In this paper, a small portion of our work investigating the impact of a LEGOTM engineering unit on fourth grade students' preconceptions and understanding of animals is presented. Purpose: The driving questions for our work are: (1) What is the impact of an engineering-design-based curricular module on students' understanding of habitat and animal classification? (2) What are students' misconceptions regarding animal classification and habitat? Sample: The study was conducted in an inner-city K-8 school in the northeastern region of the United States. There were two fourth grade classrooms in the school. The first classroom included seven girls and nine boys, whereas the other classroom included eight girls and eight boys. All fourth grade students participated in the study. Design and methods: In answering the research questions mixed-method approaches are used. Data collection methods included pre- and post-tests, pre- and post-interviews, student journals, and classroom observations. Identical pre- and post-tests were administered to measure students' understanding of animals. They included four multiple-choice and six open-ended questions. Identical pre- and post-interviews were administered to explore students' in-depth understanding of animals. Results: Our results show that students significantly increased their performance after instruction on both the multiple-choice questions (t = -3.586, p = .001) and the open-ended questions (t = -5.04, p = .000). They performed better on the post interviews as well. Also, it is found that design-based instruction helped students comprehend core concepts of a life science subject, animals. Conclusions: Based on these results, the main argument of the study is that engineering design is a useful framework for teaching not only physical science-related subjects, but also life science subjects in elementary science classrooms.

Multiple-choice pretesting potentiates learning of related information.

PubMed

Little, Jeri L; Bjork, Elizabeth Ligon

2016-10-01

Although the testing effect has received a substantial amount of empirical attention, such research has largely focused on the effects of tests given after study. The present research examines the effect of using tests prior to study (i.e., as pretests), focusing particularly on how pretesting influences the subsequent learning of information that is not itself pretested but that is related to the pretested information. In Experiment 1, we found that multiple-choice pretesting was better for the learning of such related information than was cued-recall pretesting or a pre-fact-study control condition. In Experiment 2, we found that the increased learning of non-pretested related information following multiple-choice testing could not be attributed to increased time allocated to that information during subsequent study. Last, in Experiment 3, we showed that the benefits of multiple-choice pretesting over cued-recall pretesting for the learning of related information persist over 48 hours, thus demonstrating the promise of multiple-choice pretesting to potentiate learning in educational contexts. A possible explanation for the observed benefits of multiple-choice pretesting for enhancing the effectiveness with which related nontested information is learned during subsequent study is discussed.

Development and Validation of the Conceptual Assessment of Natural Selection (CANS)

PubMed Central

Kalinowski, Steven T.; Leonard, Mary J.; Taper, Mark L.

2016-01-01

We developed and validated the Conceptual Assessment of Natural Selection (CANS), a multiple-choice test designed to assess how well college students understand the central principles of natural selection. The expert panel that reviewed the CANS concluded its questions were relevant to natural selection and generally did a good job sampling the specific concepts they were intended to assess. Student interviews confirmed questions on the CANS provided accurate reflections of how students think about natural selection. And, finally, statistical analysis of student responses using item response theory showed that the CANS did a very good job of estimating how well students understood natural selection. The empirical reliability of the CANS was substantially higher than the Force Concept Inventory, a highly regarded test in physics that has a similar purpose. PMID:27856552

The effect of content delivery style on student performance in anatomy.

PubMed

White, Lloyd J; McGowan, Heath W; McDonald, Aaron C

2018-04-12

The development of new technologies and ensuing pedagogical research has led many tertiary institutions to integrate and adopt online learning strategies. The authors of this study have incorporated online learning strategies into existing educational practices of a second year anatomy course, resulting in half of the course content delivered via face-to-face lectures, and half delivered online via tailored video vignettes, with accompanying worksheets and activities. The effect of the content delivery mode on student learning was analyzed by tailoring questions to content presented either face-to-face or online. Four practical tests were conducted across the semester with each consisting of four questions. Within each test, two questions were based on content delivered face-to-face, and two questions were based on content delivered online. Examination multiple choice questions were similarly divided and assessed. Findings indicate that student learning is consistent regardless of the mode of content delivery. However, student viewing habits had a significant impact on learning, with students who viewed videos multiple times achieving higher marks than those less engaged with the online content. Student comments also indicated that content delivery mode was not an influence on learning. Therefore student engagement, rather than the mode of content delivery, is a determinant of student learning and performance in human anatomy. Anat Sci Educ. © 2018 American Association of Anatomists. © 2018 American Association of Anatomists.

Beyond hypercorrection: remembering corrective feedback for low-confidence errors.

PubMed

Griffiths, Lauren; Higham, Philip A

2018-02-01

Correcting errors based on corrective feedback is essential to successful learning. Previous studies have found that corrections to high-confidence errors are better remembered than low-confidence errors (the hypercorrection effect). The aim of this study was to investigate whether corrections to low-confidence errors can also be successfully retained in some cases. Participants completed an initial multiple-choice test consisting of control, trick and easy general-knowledge questions, rated their confidence after answering each question, and then received immediate corrective feedback. After a short delay, they were given a cued-recall test consisting of the same questions. In two experiments, we found high-confidence errors to control questions were better corrected on the second test compared to low-confidence errors - the typical hypercorrection effect. However, low-confidence errors to trick questions were just as likely to be corrected as high-confidence errors. Most surprisingly, we found that memory for the feedback and original responses, not confidence or surprise, were significant predictors of error correction. We conclude that for some types of material, there is an effortful process of elaboration and problem solving prior to making low-confidence errors that facilitates memory of corrective feedback.

Predicting social and communicative ability in school-age children with autism spectrum disorder: A pilot study of the Social Attribution Task, Multiple Choice.

PubMed

Burger-Caplan, Rebecca; Saulnier, Celine; Jones, Warren; Klin, Ami

2016-11-01

The Social Attribution Task, Multiple Choice is introduced as a measure of implicit social cognitive ability in children, addressing a key challenge in quantification of social cognitive function in autism spectrum disorder, whereby individuals can often be successful in explicit social scenarios, despite marked social adaptive deficits. The 19-question Social Attribution Task, Multiple Choice, which presents ambiguous stimuli meant to elicit social attribution, was administered to children with autism spectrum disorder (N = 23) and to age-matched and verbal IQ-matched typically developing children (N = 57). The Social Attribution Task, Multiple Choice performance differed between autism spectrum disorder and typically developing groups, with typically developing children performing significantly better than children with autism spectrum disorder. The Social Attribution Task, Multiple Choice scores were positively correlated with age (r = 0.474) while being independent from verbal IQ (r = 0.236). The Social Attribution Task, Multiple Choice was strongly correlated with Vineland Adaptive Behavior Scales Communication (r = 0.464) and Socialization (r = 0.482) scores, but not with Daily Living Skills scores (r = 0.116), suggesting that the implicit social cognitive ability underlying performance on the Social Attribution Task, Multiple Choice is associated with real-life social adaptive function. © The Author(s) 2016.

To Click or Not to Click

NASA Astrophysics Data System (ADS)

Abramzon, Nina; Sadaghiani, Homeyra

2009-03-01

A comparison of clickers v. flashcards in a controlled setting was done to test a) whether clickers show an improvement over flashcards in students learning the following concepts: i) Coulomb's force law and ii) magnetic fields caused by currents, and b) if students using clickers are more open towards conceptual questions and the peer instruction method compared to students using flashcards. Two classes taught concurrently by the same instructor were taught identically, except that in one class the collection of answers to concept questions was done using clickers, and in the other using flashcards. To test which students learned the concepts better, a few multiple choice questions from a standard exam used in physics education were included in the final exam of both classes, and the performance of the two classes was compared. In addition, a questionnaire was given to each class to evaluate students' opinions about the benefits of lectures including conceptual TPS questions and the use of related conceptual questions on exams. The results of the survey were compared between the two classes. The experimental design and results of the study will be presented.

Gender differences in response to questions on the australian national chemistry quiz

NASA Astrophysics Data System (ADS)

Walding, Richard; Fogliani, Charles; Over, Ray; Bain, John D.

In contrast to the attention given to the relative levels of achievement of boys and girls in mathematics, the question of whether there are sex differences in the solution of chemistry questions has not attracted much attention. This study compares the performance of boys and girls in the Australian National Chemistry Quiz (Chem Quiz), a multiple-choice test conducted by the Royal Australian Chemical Institute. The analyses were based on results from 27,000 students in Years 11 and 12 and 16,000 students in Year 10 who completed the Chem Quiz in 1991. Although some questions in the Chem Quiz were solved equally well by boys and girls, on many questions boys outperformed girls. The extreme case was a question answered correctly by 67% of Year 10 boys in contrast to 48% of Year 10 girls. Several reasons why boys and girls might differ in the rates they solve at least some chemistry questions are discussed, and directions for identifying the nature, extent, and basis for sex differences are outlined.

The influence of question design on the response to self-assessment in www.elearnSCI.org: a submodule pilot study.

PubMed

Liu, N; Li, X-W; Zhou, M-W; Biering-Sørensen, F

2015-08-01

This is an interventional training session. The objective of this study was to investigate the difference in response to self-assessment questions in the original and an adjusted version for a submodule of www.elearnSCI.org for student nurses. The study was conducted in a teaching hospital affiliated to Peking University, China. In all, 28 student nurses divided into two groups (groups A and B; 14 in each) received a print-out of a Chinese translation of the slides from the 'Maintaining skin integrity following spinal cord injury' submodule in www.elearnSCI.org for self-study. Both groups were then tested using the 10 self-assessment multiple-choice questions (MCQs) related to the same submodule. Group A used the original questions, whereas group B received an adjusted questionnaire. The responses to four conventional single-answer MCQs were nearly all correct in both groups. However, in three questions, group A, with the option 'All of the above', had a higher number of correct answers than group B, with multiple-answer MCQs. In addition, in another three questions, group A, using the original multiple-answer MCQs, had fewer correct answers than group B, where it was only necessary to tick a single incorrect answer. Variations in design influence the response to questions. The use of conventional single-answer MCQs should be reconsidered, as they only examine the recall of isolated knowledge facts. The 'All of the above' option should be avoided because it would increase the number of correct answers arrived at by guessing. When using multiple-answer MCQs, it is recommended that the questions asked should be in accordance with the content within the www.elearnSCI.org.

Open problem-based instruction impacts understanding of physiological concepts differently in undergraduate students

PubMed Central

Franklin, Brandon M.; Xiang, Lin; Collett, Jason A.; Rhoads, Megan K.

2015-01-01

Student populations are diverse such that different types of learners struggle with traditional didactic instruction. Problem-based learning has existed for several decades, but there is still controversy regarding the optimal mode of instruction to ensure success at all levels of students' past achievement. The present study addressed this problem by dividing students into the following three instructional groups for an upper-level course in animal physiology: traditional lecture-style instruction (LI), guided problem-based instruction (GPBI), and open problem-based instruction (OPBI). Student performance was measured by three summative assessments consisting of 50% multiple-choice questions and 50% short-answer questions as well as a final overall course assessment. The present study also examined how students of different academic achievement histories performed under each instructional method. When student achievement levels were not considered, the effects of instructional methods on student outcomes were modest; OPBI students performed moderately better on short-answer exam questions than both LI and GPBI groups. High-achieving students showed no difference in performance for any of the instructional methods on any metric examined. In students with low-achieving academic histories, OPBI students largely outperformed LI students on all metrics (short-answer exam: P < 0.05, d = 1.865; multiple-choice question exam: P < 0.05, d = 1.166; and final score: P < 0.05, d = 1.265). They also outperformed GPBI students on short-answer exam questions (P < 0.05, d = 1.109) but not multiple-choice exam questions (P = 0.071, d = 0.716) or final course outcome (P = 0.328, d = 0.513). These findings strongly suggest that typically low-achieving students perform at a higher level under OPBI as long as the proper support systems (formative assessment and scaffolding) are provided to encourage student success. PMID:26628656

The Colorado Humanitarian Surgical Skills Workshop: A Cadaver-Based Workshop to Prepare Residents for Surgery in Austere Settings.

PubMed

Lin, Yihan; Mukhopadhyay, Swagoto; Meguid, Robert A; Kuwayama, David P

Interest in humanitarian surgery is high among surgical and obstetric residents. The Colorado Humanitarian Surgical Skills Workshop is an annual 2-day course exposing senior residents to surgical techniques essential in low- and middle-income countries but not traditionally taught in US residencies. We evaluated the course's ability to foster resident comfort, knowledge, and competence in these skills. The cohort of course participants was studied prospectively. Participants attended didactic sessions followed by skills sessions using cadavers. Sample areas of focus included general surgery (mesh-free hernia repair), orthopedics (powerless external fixation), and neurosurgery (powerless craniotomy). Before and after the course, participants answered a questionnaire assessing confidence with taught skills; took a knowledge-based test composed of multiple choice and open-ended questions; and participated in a manual skills test of tibial external fixation. The Center for Surgical Innovation, University of Colorado School of Medicine. A total of 12 residents (11 general surgical and 1 obstetric) from ten US institutions. After the course, participants perceived increased confidence in performing all 27 taught procedures and ability to practice in low- and middle-income countries. In knowledge-based testing, 10 of 12 residents demonstrated improvement on multiple choice questioning and 9 of 12 residents demonstrated improvement on open-ended questioning with structured scoring. In manual skills testing, all external fixator constructs demonstrated objective improvement on structured scoring and subjective improvement on stability assessment. For senior residents interested in humanitarian surgery, a combination of skills-focused teaching and manual practice led to self-perceived and objective improvement in relevant surgical knowledge and skills. The Colorado Humanitarian Surgical Skills Workshop represents an effective model for transmitting essential surgical principles and techniques of value in low-resource settings. Copyright © 2017 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

A survey of Canadian medical physicists: software quality assurance of in-house software.

PubMed

Salomons, Greg J; Kelly, Diane

2015-01-05

This paper reports on a survey of medical physicists who write and use in-house written software as part of their professional work. The goal of the survey was to assess the extent of in-house software usage and the desire or need for related software quality guidelines. The survey contained eight multiple-choice questions, a ranking question, and seven free text questions. The survey was sent to medical physicists associated with cancer centers across Canada. The respondents to the survey expressed interest in having guidelines to help them in their software-related work, but also demonstrated extensive skills in the area of testing, safety, and communication. These existing skills form a basis for medical physicists to establish a set of software quality guidelines.

Feedback in Technology-Based Instruction: Learner Preferences

ERIC Educational Resources Information Center

Lefevre, David; Cox, Benita

2016-01-01

This research investigates learner preferences for the format of feedback?when using technology-based instruction (TBI). The primary method of data collection was to provide subjects with a range of options for TBI feedback following responses to multiple-choice questions and then observe their choices. A software tool both presented the feedback…

Wrong Answers on Multiple-Choice Achievement Tests: Blind Guesses or Systematic Choices?.

ERIC Educational Resources Information Center

Powell, J. C.

A multi-faceted model for the selection of answers for multiple-choice tests was developed from the findings of a series of exploratory studies. This model implies that answer selection should be curvilinear. A series of models were tested for fit using the chi square procedure. Data were collected from 359 elementary school students ages 9-12.…

How do clinical genetics consent forms address the familial approach to confidentiality and incidental findings? A mixed-methods study.

PubMed

Dheensa, Sandi; Crawford, Gillian; Salter, Claire; Parker, Michael; Fenwick, Angela; Lucassen, Anneke

2018-01-01

Genetic test results can be relevant to patients and their relatives. Questions thus arise around whether clinicians regard genetic information as confidential to individuals or to families, and about how they broach this and other issues, including the potential for incidental findings, in consent (forms) for genetic testing. We conducted a content analysis of UK-wide genetic testing consent forms and interviewed 128 clinicians/laboratory scientists. We found that almost all genetic services offered patients multiple, sometimes unworkable, choices on forms, including an option to veto the use of familial genetic information to benefit relatives. Participants worried that documented choices were overriding professional judgement and cautioned against any future forms dictating practice around incidental findings. We conclude that 'tick-box' forms, which do little to enhance autonomy, are masking valid consent processes in clinical practice. As genome-wide testing becomes commonplace, we must re-consider consent processes, so that they protects patients'-and relatives'-interests.

Concept Learning versus Problem Solving: Evaluating a Threat to the Validity of a Particulate Gas Law Question

ERIC Educational Resources Information Center

Sanger, Michael J.; Vaughn, C. Kevin; Binkley, David A.

2013-01-01

Three different samples of students were asked to answer five multiple-choice questions concerning the properties of a sample of helium gas (particle speed, state of matter, sample volume, sample pressure, and particle distribution), including a particulate question first used by Nurrenbern and Pickering (particle distribution). In the first…

The Effects of Different Feedback Strategies Using Computer-Administered Multiple-Choice Questions as Instruction.

ERIC Educational Resources Information Center

Clariana, Roy B.; And Others

The present study investigated the effects of using different forms of material with 100 eleventh grade students enrolled in a 5-week CBI (computer based instruction) summer enrichment program in Memphis, Tennessee. The basic design consisted of two conditions of instructional support (text and questions vs. questions only), two testings…

Cognitive task analysis for teaching technical skills in an inanimate surgical skills laboratory.

PubMed

Velmahos, George C; Toutouzas, Konstantinos G; Sillin, Lelan F; Chan, Linda; Clark, Richard E; Theodorou, Demetrios; Maupin, Fredric

2004-01-01

The teaching of surgical skills is based mostly on the traditional "see one, do one, teach one" resident-to-resident method. Surgical skills laboratories provide a new environment for teaching skills but their effectiveness has not been adequately tested. Cognitive task analysis is an innovative method to teach skills, used successfully in nonmedical fields. The objective of this study is to evaluate the effectiveness of a 3-hour surgical skills laboratory course on central venous catheterization (CVC), taught by the principles of cognitive task analysis to surgical interns. Upon arrival to the Department of Surgery, 26 new interns were randomized to either receive a surgical skills laboratory course on CVC ("course" group, n = 12) or not ("traditional" group, n = 14). The course consisted mostly of hands-on training on inanimate CVC models. All interns took a 15-item multiple-choice question test on CVC at the beginning of the study. Within two and a half months all interns performed CVC on critically ill patients. The outcome measures were cognitive knowledge and technical-skill competence on CVC. These outcomes were assessed by a 14-item checklist evaluating the interns while performing CVC on a patient and by the 15-item multiple-choice-question test, which was repeated at that time. There were no differences between the two groups in the background characteristics of the interns or the patients having CVC. The scores at the initial multiple-choice test were similar (course: 7.33 +/- 1.07, traditional: 8 +/- 2.15, P = 0.944). However, the course interns scored significantly higher in the repeat test compared with the traditional interns (11 +/- 1.86 versus 8.64 +/- 1.82, P = 0.03). Also, the course interns achieved a higher score on the 14-item checklist (12.6 +/- 1.1 versus 7.5 +/- 2.2, P <0.001). They required fewer attempts to find the vein (3.3 +/- 2.2 versus 6.4 +/- 4.2, P = 0.046) and showed a trend toward less time to complete the procedure (15.4 +/- 9.5 versus 20.6 +/- 9.1 minutes, P = 0.149). A surgical skills laboratory course on CVC, taught by the principles of cognitive task analysis and using inanimate models, improves the knowledge and technical skills of new surgical interns on this task.

A Pretest for Introductory Crops Students.

ERIC Educational Resources Information Center

Elkins, Donald M.

1987-01-01

Discusses the advantages of using a pretest in introductory agronomy courses. Provides a pretest that has been developed for use in an introductory crops course taught at Southern Illinois University. Includes 25 definitions, 17 true-false and multiple choice questions, and 6 short answer questions. (TW)

Developing multiple-choices test items as tools for measuring the scientific-generic skills on solar system

NASA Astrophysics Data System (ADS)

Bhakti, Satria Seto; Samsudin, Achmad; Chandra, Didi Teguh; Siahaan, Parsaoran

2017-05-01

The aim of research is developing multiple-choices test items as tools for measuring the scientific of generic skills on solar system. To achieve the aim that the researchers used the ADDIE model consisting Of: Analyzing, Design, Development, Implementation, dan Evaluation, all of this as a method research. While The scientific of generic skills limited research to five indicator including: (1) indirect observation, (2) awareness of the scale, (3) inference logic, (4) a causal relation, and (5) mathematical modeling. The participants are 32 students at one of junior high schools in Bandung. The result shown that multiple-choices that are constructed test items have been declared valid by the expert validator, and after the tests show that the matter of developing multiple-choices test items be able to measuring the scientific of generic skills on solar system.

Optimizing Multiple-Choice Tests as Learning Events

ERIC Educational Resources Information Center

Little, Jeri Lynn

2011-01-01

Although generally used for assessment, tests can also serve as tools for learning--but different test formats may not be equally beneficial. Specifically, research has shown multiple-choice tests to be less effective than cued-recall tests in improving the later retention of the tested information (e.g., see meta-analysis by Hamaker, 1986),…

Validation of science virtual test to assess 8th grade students' critical thinking on living things and environmental sustainability theme

NASA Astrophysics Data System (ADS)

Rusyati, Lilit; Firman, Harry

2017-05-01

This research was motivated by the importance of multiple-choice questions that indicate the elements and sub-elements of critical thinking and implementation of computer-based test. The method used in this research was descriptive research for profiling the validation of science virtual test to measure students' critical thinking in junior high school. The participant is junior high school students of 8th grade (14 years old) while science teacher and expert as the validators. The instrument that used as a tool to capture the necessary data are sheet of an expert judgment, sheet of legibility test, and science virtual test package in multiple choice form with four possible answers. There are four steps to validate science virtual test to measure students' critical thinking on the theme of "Living Things and Environmental Sustainability" in 7th grade Junior High School. These steps are analysis of core competence and basic competence based on curriculum 2013, expert judgment, legibility test and trial test (limited and large trial test). The test item criterion based on trial test are accepted, accepted but need revision, and rejected. The reliability of the test is α = 0.747 that categorized as `high'. It means the test instruments used is reliable and high consistency. The validity of Rxy = 0.63 means that the validity of the instrument was categorized as `high' according to interpretation value of Rxy (correlation).

Set of Criteria for Efficiency of the Process Forming the Answers to Multiple-Choice Test Items

ERIC Educational Resources Information Center

Rybanov, Alexander Aleksandrovich

2013-01-01

Is offered the set of criteria for assessing efficiency of the process forming the answers to multiple-choice test items. To increase accuracy of computer-assisted testing results, it is suggested to assess dynamics of the process of forming the final answer using the following factors: loss of time factor and correct choice factor. The model…

Using the Flipped Classroom to Bridge the Gap to Generation Y

PubMed Central

Gillispie, Veronica

2016-01-01

Background: The flipped classroom is a student-centered approach to learning that increases active learning for the student compared to traditional classroom-based instruction. In the flipped classroom model, students are first exposed to the learning material through didactics outside of the classroom, usually in the form of written material, voice-over lectures, or videos. During the formal teaching time, an instructor facilitates student-driven discussion of the material via case scenarios, allowing for complex problem solving, peer interaction, and a deep understanding of the concepts. A successful flipped classroom should have three goals: (1) allow the students to become critical thinkers, (2) fully engage students and instructors, and (3) stimulate the development of a deep understanding of the material. The flipped classroom model includes teaching and learning methods that can appeal to all four generations in the academic environment. Methods: During the 2015 academic year, we implemented the flipped classroom in the obstetrics and gynecology clerkship for the Ochsner Clinical School in New Orleans, LA. Voice-over presentations of the lectures that had been given to students in prior years were recorded and made available to the students through an online classroom. Weekly problem-based learning sessions matched to the subjects of the traditional lectures were held, and the faculty who had previously presented the information in the traditional lecture format facilitated the problem-based learning sessions. The knowledge base of students was evaluated at the end of the rotation via a multiple-choice question examination and the Objective Structured Clinical Examination (OSCE) as had been done in previous years. We compared demographic information and examination scores for traditional teaching and flipped classroom groups of students. The traditional teaching group consisted of students from Rotation 2 and Rotation 3 of the 2014 academic year who received traditional classroom-based instruction. The flipped classroom group consisted of students from Rotation 2 and Rotation 3 of the 2015 academic year who received formal didactics via voice-over presentation and had the weekly problem-based learning sessions. Results: When comparing the students taught by traditional methods to those taught in the flipped classroom model, we saw a statistically significant increase in test scores on the multiple-choice question examination in both the obstetrics and gynecology sections in Rotation 2. While the average score for the flipped classroom group increased in Rotation 3 on the obstetrics section of the multiple-choice question examination, the difference was not statistically significant. Unexpectedly, the average score on the gynecology portion of the multiple-choice question examination decreased among the flipped classroom group compared to the traditional teaching group, and this decrease was statistically significant. For both the obstetrics and the gynecology portions of the OSCE, we saw statistically significant increases in the scores for the flipped classroom group in both Rotation 2 and Rotation 3 compared to the traditional teaching group. With the exception of the gynecology portion of the multiple-choice question examination in Rotation 3, we saw improvement in scores after the implementation of the flipped classroom. Conclusion: The flipped classroom is a feasible and useful alternative to the traditional classroom. It is a method that embraces Generation Y's need for active learning in a group setting while maintaining a traditional classroom method for introducing the information. Active learning increases student engagement and can lead to improved retention of material as demonstrated on standard examinations. PMID:27046401

Using the Flipped Classroom to Bridge the Gap to Generation Y.

PubMed

Gillispie, Veronica

2016-01-01

The flipped classroom is a student-centered approach to learning that increases active learning for the student compared to traditional classroom-based instruction. In the flipped classroom model, students are first exposed to the learning material through didactics outside of the classroom, usually in the form of written material, voice-over lectures, or videos. During the formal teaching time, an instructor facilitates student-driven discussion of the material via case scenarios, allowing for complex problem solving, peer interaction, and a deep understanding of the concepts. A successful flipped classroom should have three goals: (1) allow the students to become critical thinkers, (2) fully engage students and instructors, and (3) stimulate the development of a deep understanding of the material. The flipped classroom model includes teaching and learning methods that can appeal to all four generations in the academic environment. During the 2015 academic year, we implemented the flipped classroom in the obstetrics and gynecology clerkship for the Ochsner Clinical School in New Orleans, LA. Voice-over presentations of the lectures that had been given to students in prior years were recorded and made available to the students through an online classroom. Weekly problem-based learning sessions matched to the subjects of the traditional lectures were held, and the faculty who had previously presented the information in the traditional lecture format facilitated the problem-based learning sessions. The knowledge base of students was evaluated at the end of the rotation via a multiple-choice question examination and the Objective Structured Clinical Examination (OSCE) as had been done in previous years. We compared demographic information and examination scores for traditional teaching and flipped classroom groups of students. The traditional teaching group consisted of students from Rotation 2 and Rotation 3 of the 2014 academic year who received traditional classroom-based instruction. The flipped classroom group consisted of students from Rotation 2 and Rotation 3 of the 2015 academic year who received formal didactics via voice-over presentation and had the weekly problem-based learning sessions. When comparing the students taught by traditional methods to those taught in the flipped classroom model, we saw a statistically significant increase in test scores on the multiple-choice question examination in both the obstetrics and gynecology sections in Rotation 2. While the average score for the flipped classroom group increased in Rotation 3 on the obstetrics section of the multiple-choice question examination, the difference was not statistically significant. Unexpectedly, the average score on the gynecology portion of the multiple-choice question examination decreased among the flipped classroom group compared to the traditional teaching group, and this decrease was statistically significant. For both the obstetrics and the gynecology portions of the OSCE, we saw statistically significant increases in the scores for the flipped classroom group in both Rotation 2 and Rotation 3 compared to the traditional teaching group. With the exception of the gynecology portion of the multiple-choice question examination in Rotation 3, we saw improvement in scores after the implementation of the flipped classroom. The flipped classroom is a feasible and useful alternative to the traditional classroom. It is a method that embraces Generation Y's need for active learning in a group setting while maintaining a traditional classroom method for introducing the information. Active learning increases student engagement and can lead to improved retention of material as demonstrated on standard examinations.

Dental Student Academic Integrity in U.S. Dental Schools: Current Status and Recommendations for Enhancement.

PubMed

Graham, Bruce S; Knight, G William; Graham, Linda

2016-01-01

Cheating incidents in 2006-07 led U.S. dental schools to heighten their efforts to enhance the environment of academic integrity in their institutions. The aims of this study were to document the measures being used by U.S. dental schools to discourage student cheating, determine the current incidence of reported cheating, and make recommendations for enhancing a culture of integrity in dental education. In late 2014-early 2015, an online survey was distributed to academic deans of all 61 accredited U.S. dental schools that had four classes of dental students enrolled; 50 (82%) responded. Among measures used, 98% of respondents reported having policy statements regarding student academic integrity, 92% had an Honor Code, 96% provided student orientation to integrity policies, and most used proctoring of final exams (91%) and tests (93%). Regarding disciplinary processes, 27% reported their faculty members only rarely reported suspected cheating (though required in 76% of the schools), and 40% disseminated anonymous results of disciplinary hearings. A smaller number of schools (n=36) responded to the question about student cheating than to other questions; those results suggested that reported cheating had increased almost threefold since 1998. The authors recommend that schools add cheating case scenarios to professional ethics curricula; disseminate outcomes of cheating enforcement actions; have students sign a statement attesting to compliance with academic integrity policies at every testing activity; add curricular content on correct writing techniques to avoid plagiarism; require faculty to distribute retired test items; acquire examination-authoring software programs to enable faculty to generate new multiple-choice items and different versions of the same multiple-choice tests; avoid take-home exams when assessing independent student knowledge; and utilize student assessment methods directly relevant to clinical practice.

Measuring more than we know? An examination of the motivational and situational influences in science achievement

NASA Astrophysics Data System (ADS)

Haydel, Angela Michelle

The purpose of this dissertation was to advance theoretical understanding about fit between the personal resources of individuals and the characteristics of science achievement tasks. Testing continues to be pervasive in schools, yet we know little about how students perceive tests and what they think and feel while they are actually working on test items. This study focused on both the personal (cognitive and motivational) and situational factors that may contribute to individual differences in achievement-related outcomes. 387 eighth grade students first completed a survey including measures of science achievement goals, capability beliefs, efficacy related to multiple-choice items and performance assessments, validity beliefs about multiple-choice items and performance assessments, and other perceptions of these item formats. Students then completed science achievement tests including multiple-choice items and two performance assessments. A sample of students was asked to verbalize both thoughts and feelings as they worked through the test items. These think-alouds were transcribed and coded for evidence of cognitive, metacognitive and motivational engagement. Following each test, all students completed measures of effort, mood, energy level and strategy use during testing. Students reported that performance assessments were more challenging, authentic, interesting and valid than multiple-choice tests. They also believed that comparisons between students were easier using multiple-choice items. Overall, students tried harder, felt better, had higher levels of energy and used more strategies while working on performance assessments. Findings suggested that performance assessments might be more congruent with a mastery achievement goal orientation, while multiple-choice tests might be more congruent with a performance achievement goal orientation. A variable-centered analytic approach including regression analyses provided information about how students, on average, who differed in terms of their teachers' ratings of their science ability, achievement goals, capability beliefs and experiences with science achievement tasks perceived, engaged in, and performed on multiple-choice items and performance assessments. Person-centered analyses provided information about the perceptions, engagement and performance of subgroups of individuals who had different motivational characteristics. Generally, students' personal goals and capability beliefs related more strongly to test perceptions, but not performance, while teacher ratings of ability and test-specific beliefs related to performance.

Studies of the Seriousness of Three Threats to Passage Dependence.

ERIC Educational Resources Information Center

Hanna, Gerald S.; Oaster, Thomas R.

1980-01-01

Certain kinds of multiple-choice reading comprehension questions may be answered correctly at the higher-than-chance level when they are administered without the accompanying passage. These high risk questions do not necessarily lead to passage dependence invalidity. They threaten but do not prove invalidity. (Author/CP)

Exam Question Sequencing Effects and Context Cues

ERIC Educational Resources Information Center

Davis, Doris Bitler

2017-01-01

Providing two or more versions of multiple-choice exams has long been a popular strategy for reducing the opportunity for students to engage in academic dishonesty. While the results of studies comparing exam scores under different question-order conditions have been inconclusive, the potential importance of contextual cues to aid student recall…

The Use of Technology by Nonformal Environmental Educators

ERIC Educational Resources Information Center

Peffer, Tamara Elizabeth; Bodzin, Alec M.; Smith, Judith Duffield

2013-01-01

This study examined the use of instructional and learning technologies by nonformal environmental educators. A 40-question survey was developed to inquire about practitioner demographics, technology use in practice, and beliefs about technology. The survey consisted of multiple choice, open-ended questions, and a Likert-type scale component--the…

Validation and Structural Analysis of the Kinematics Concept Test

ERIC Educational Resources Information Center

Lichtenberger, A.; Wagner, C.; Hofer, S. I.; Stem, E.; Vaterlaus, A.

2017-01-01

The kinematics concept test (KCT) is a multiple-choice test designed to evaluate students' conceptual understanding of kinematics at the high school level. The test comprises 49 multiple-choice items about velocity and acceleration, which are based on seven kinematic concepts and which make use of three different representations. In the first part…

Multiple-Choice Test Bias Due to Answering Strategy Variation.

ERIC Educational Resources Information Center

Frary, Robert B.; Giles, Mary B.

This paper describes the development and investigation of a new approach to determining the existence of bias in multiple-choice test scores. Previous work in this area has concentrated almost exclusively on bias attributable to specific test items or to differences in test score distributions across racial or ethnic groups. In contrast, the…

Redesigning the Preexam Review Session

ERIC Educational Resources Information Center

King, Daniel

2010-01-01

In a large enrollment, multiple-section course, review sessions enable efficient and consistent delivery of information to all students. A redesigned review session has been implemented to increase attendance and improve effectiveness. The new design involves a practice quiz consisting of about 20 multiple-choice questions that students answer…

Cognitive Diagnostic Models for Tests with Multiple-Choice and Constructed-Response Items

ERIC Educational Resources Information Center

Kuo, Bor-Chen; Chen, Chun-Hua; Yang, Chih-Wei; Mok, Magdalena Mo Ching

2016-01-01

Traditionally, teachers evaluate students' abilities via their total test scores. Recently, cognitive diagnostic models (CDMs) have begun to provide information about the presence or absence of students' skills or misconceptions. Nevertheless, CDMs are typically applied to tests with multiple-choice (MC) items, which provide less diagnostic…

Samejima Items in Multiple-Choice Tests: Identification and Implications

ERIC Educational Resources Information Center

Rahman, Nazia

2013-01-01

Samejima hypothesized that non-monotonically increasing item response functions (IRFs) of ability might occur for multiple-choice items (referred to here as "Samejima items") if low ability test takers with some, though incomplete, knowledge or skill are drawn to a particularly attractive distractor, while very low ability test takers…

Optimizing the Performance of Radionuclide Identification Software in the Hunt for Nuclear Security Threats

DOE Office of Scientific and Technical Information (OSTI.GOV)

Fotion, Katherine A.

2016-08-18

The Radionuclide Analysis Kit (RNAK), my team’s most recent nuclide identification software, is entering the testing phase. A question arises: will removing rare nuclides from the software’s library improve its overall performance? An affirmative response indicates fundamental errors in the software’s framework, while a negative response confirms the effectiveness of the software’s key machine learning algorithms. After thorough testing, I found that the performance of RNAK cannot be improved with the library choice effect, thus verifying the effectiveness of RNAK’s algorithms—multiple linear regression, Bayesian network using the Viterbi algorithm, and branch and bound search.

Computer-Aided Assessment in Mechanics: Question Design and Test Evaluation

ERIC Educational Resources Information Center

Gill, M.; Greenhow, M.

2007-01-01

This article describes pedagogic issues in setting objective tests in mechanics using Question Mark Perception, coupled with MathML mathematics mark-up and the Scalable Vector Graphics (SVG) syntax for producing diagrams. The content of the questions (for a range of question types such as multi-choice, numerical input and variants such as…

Objective questions in orthopaedic nursing objective questions in orthopaedic nursing Cole P Churchill Lovingstone 212pp £8.50 0-443-02473-1.

PubMed

1988-01-09

The presentation of Elizabeth Evans's book is good; typography and her style make it very readable; and it is well illustrated throughout. The first of two sections deals with multiple choice (MCQs) and the true/ false type questions. The second contains nursing care studies with relevant MCQs for the reader to answer.

Technical flaws in multiple-choice questions in the access exam to medical specialties ("examen MIR") in Spain (2009-2013).

PubMed

Rodríguez-Díez, María Cristina; Alegre, Manuel; Díez, Nieves; Arbea, Leire; Ferrer, Marta

2016-02-03

The main factor that determines the selection of a medical specialty in Spain after obtaining a medical degree is the MIR ("médico interno residente", internal medical resident) exam. This exam consists of 235 multiple-choice questions with five options, some of which include images provided in a separate booklet. The aim of this study was to analyze the technical quality of the multiple-choice questions included in the MIR exam over the last five years. All the questions included in the exams from 2009 to 2013 were analyzed. We studied the proportion of questions including clinical vignettes, the number of items related to an image and the presence of technical flaws in the questions. For the analysis of technical flaws, we adapted the National Board of Medical Examiners (NBME) guidelines. We looked for 18 different issues included in the manual, grouped into two categories: issues related to testwiseness and issues related to irrelevant difficulties. The final number of questions analyzed was 1,143. The percentage of items based on clinical vignettes increased from 50% in 2009 to 56-58% in the following years (2010-2013). The percentage of items based on an image increased progressively from 10% in 2009 to 15% in 2012 and 2013. The percentage of items with at least one technical flaw varied between 68 and 72%. We observed a decrease in the percentage of items with flaws related to testwiseness, from 30% in 2009 to 20% in 2012 and 2013. While most of these issues decreased dramatically or even disappeared (such as the imbalance in the correct option numbers), the presence of non-plausible options remained frequent. With regard to technical flaws related to irrelevant difficulties, no improvement was observed; this is especially true with respect to negative stem questions and "hinged" questions. The formal quality of the MIR exam items has improved over the last five years with regard to testwiseness. A more detailed revision of the items submitted, checking systematically for the presence of technical flaws, could improve the validity and discriminatory power of the exam, without increasing its difficulty.

A 2-year study of Gram stain competency assessment in 40 clinical laboratories.

PubMed

Goodyear, Nancy; Kim, Sara; Reeves, Mary; Astion, Michael L

2006-01-01

We used a computer-based competency assessment tool for Gram stain interpretation to assess the performance of 278 laboratory staff from 40 laboratories on 40 multiple-choice questions. We report test reliability, mean scores, median, item difficulty, discrimination, and analysis of the highest- and lowest-scoring questions. The questions were reliable (KR-20 coefficient, 0.80). Overall mean score was 88% (range, 63%-98%). When categorized by cell type, the means were host cells, 93%; other cells (eg, yeast), 92%; gram-positive, 90%; and gram-negative, 88%. When categorized by type of interpretation, the means were other (eg, underdecolorization), 92%; identify by structure (eg, bacterial morphologic features), 91%; and identify by name (eg, genus and species), 87%. Of the 6 highest-scoring questions (mean scores, > or = 99%) 5 were identify by structure and 1 was identify by name. Of the 6 lowest-scoring questions (mean scores, < 75%) 5 were gram-negative and 1 was host cells. By type of interpretation, 2 were identify by structure and 4 were identify by name. Computer-based Gram stain competency assessment examinations are reliable. Our analysis helps laboratories identify areas for continuing education in Gram stain interpretation and will direct future revisions of the tests.

Format of Options in Multiple Choice Test vis-a-vis Test Performance

ERIC Educational Resources Information Center

Bendulo, Hermabeth O.; Tibus, Erlinda D.; Bande, Rhodora A.; Oyzon, Voltaire Q.; Milla, Norberto E.; Macalinao, Myrna L.

2017-01-01

Testing or evaluation in an educational context is primarily used to measure or evaluate and authenticate the academic readiness, learning advancement, acquisition of skills, or instructional needs of learners. This study tried to determine whether the varied combinations of arrangements of options and letter cases in a Multiple-Choice Test (MCT)…

Sex Differences in the Tendency to Omit Items on Multiple-Choice Tests: 1980-2000

ERIC Educational Resources Information Center

von Schrader, Sarah; Ansley, Timothy

2006-01-01

Much has been written concerning the potential group differences in responding to multiple-choice achievement test items. This discussion has included references to possible disparities in tendency to omit such test items. When test scores are used for high-stakes decision making, even small differences in scores and rankings that arise from male…

Equal Opportunity in the Classroom: Test Construction in a Diversity-Sensitive Environment.

ERIC Educational Resources Information Center

Ghorpade, Jai; Lackritz, James R.

1998-01-01

Two multiple-choice tests and one essay test were taken by 231 students (50/50 male/female, 192 White, 39 East Asian, Black, Mexican American, or Middle Eastern). Multiple-choice tests showed no significant differences in equal employment opportunity terms; women and men scored about the same on essays, but minority students had significantly…

Chemistry 300. Administration Manual for Supervising Teachers, Provincial Examination, Answer Key--Multiple-Choice and Written-Answer Questions, and Provincial Summary Report = Chimie 300. Guide d'administration a l'intention des surveillants d'examen, Examen provincial, Cle de correction--Questions choix multiple et Questions responses ouvertes, et Rapport sommaire provincial.

ERIC Educational Resources Information Center

Manitoba Dept. of Education and Training, Winnipeg.

This collection of manuals contains the Manitoba Provincial Chemistry Examination for students seeking credit in Senior 4 Chemistry (Chemistry 300) and instructions for its use and grading. The examination is based on the Core Topics of the Senior 4 Chemistry course and accounts for 30% of the student's final grade in the course. The examination…

Measuring the Consistency in Change in Hepatitis B Knowledge among Three Different Types of Tests: True/False, Multiple Choice, and Fill in the Blanks Tests.

ERIC Educational Resources Information Center

Sahai, Vic; Demeyere, Petra; Poirier, Sheila; Piro, Felice

1998-01-01

The recall of information about Hepatitis B demonstrated by 180 seventh graders was tested with three test types: (1) short-answer; (2) true/false; and (3) multiple-choice. Short answer testing was the most reliable. Suggestions are made for the use of short-answer tests in evaluating student knowledge. (SLD)

Interactive anatomical and surgical live stream lectures improve students' academic performance in applied clinical anatomy.

PubMed

Shiozawa, Thomas; Butz, Benjamin; Herlan, Stephan; Kramer, Andreas; Hirt, Bernhard

2017-01-01

Tuebingen's Sectio Chirurgica (TSC) is an innovative, interactive, multimedia, and transdisciplinary teaching method designed to complement dissection courses. The Tuebingen's Sectio Chirurgica (TSC) allows clinical anatomy to be taught via interactive live stream surgeries moderated by an anatomist. This method aims to provide an application-oriented approach to teaching anatomy that offers students a deeper learning experience. A cohort study was devised to determine whether students who participated in the TSC were better able to solve clinical application questions than students who did not participate. A total of 365 students participated in the dissection course during the winter term of the 2012/2013 academic year. The final examination contained 40 standard multiple-choice (S-MC) and 20 clinically-applied multiple-choice (CA-MC) items. The CA-MC items referred to clinical cases but could be answered solely using anatomical knowledge. Students who regularly participated in the TSC answered the CA-MC questions significantly better than the control group (75% and 65%, respectively; P < 0.05, Mann-Whitney U test). The groups exhibited no differences on the S-MC questions (85% and 82.5%, respectively; P > 0.05). The CA-MC questions had a slightly higher level of difficulty than the S-MC questions (0.725 and 0.801, respectively; P = 0.083). The discriminatory power of the items was comparable (S-MC median Pearson correlations: 0.321; CA-MC: 0.283). The TSC successfully teaches the clinical application of anatomical knowledge. Students who attended the TSC in addition to the dissection course were able to answer CA-MC questions significantly better than students who did not attend the TSC. Thus, attending the TSC in addition to the dissection course supported students' clinical learning goals. Anat Sci Educ 10: 46-52. © 2016 American Association of Anatomists. © 2016 American Association of Anatomists.

Crime in the Classroom Part IV: Conclusions

ERIC Educational Resources Information Center

Harpp, David N.

2008-01-01

In 1990, the McGill University Senate established regulations governing how multiple-choice exams are to be conducted. The primary rules require multiple-version exams (scrambled question and if possible, scrambled answer positions) as well as assigned seating or seating with alternating rows of students from different courses. In recent years, we…

WordBytes: Exploring an Intermediate Constraint Format for Rapid Classification of Student Answers on Constructed Response Assessments

ERIC Educational Resources Information Center

Kim, Kerry J.; Meir, Eli; Pope, Denise S.; Wendel, Daniel

2017-01-01

Computerized classification of student answers offers the possibility of instant feedback and improved learning. Open response (OR) questions provide greater insight into student thinking and understanding than more constrained multiple choice (MC) questions, but development of automated classifiers is more difficult, often requiring training a…

Interactive Learning in the Classroom: Is Student Response Method Related to Performance?

ERIC Educational Resources Information Center

Elicker, Joelle D.; McConnell, Nicole L.

2011-01-01

This study examined three methods of responding to in-class multiple-choice concept questions in an Introduction to Psychology course. Specifically, this study compared exam performance and student reactions using three methods of responding to concept questions: (a) a technology-based network system, (b) hand-held flashcards, and (c) hand…

Classroom Voting Questions to Stimulate Discussions in Precalculus

ERIC Educational Resources Information Center

Cline, Kelly; Zullo, Holly; Huckaby, David A.; Storm, Christopher; Stewart, Ann

2018-01-01

Classroom voting can be an effective way to stimulate student discussions. In this pedagogy, the instructor poses a multiple-choice question to the class, and then allows a few minutes for consideration and small-group discussion before students vote, either with clickers, cell phones, or a non-electronic method. After the vote the instructor…

Gender Bias in Alberta Social Studies 30 Examinations: Cause and Effect.

ERIC Educational Resources Information Center

Walter, Connie; Young, Beth

1997-01-01

Reports on an exploration of gender bias in the multiple-choice portions of six Canadian social studies examinations. Considers the lack of women's experiences reflected in the questions, formal content, and epistemological stance. Concludes that gender biases do exist in the questions and may have contributed to differences in achievement. (MJP)

76 FR 77203 - Notice of Intent To Seek Approval To Collect Information

Federal Register 2010, 2011, 2012, 2013, 2014

2011-12-12

... Web pages created and maintained by component organizations of the NAL. On average, 2 million people... interest collections have established a Web presence with a home page and links to sub-pages that provide... Semantic Differential Scale or multiple choice questions, and no more than 4 open-ended response questions...

Exit Exam as Academic Performance Indicator

ERIC Educational Resources Information Center

Al Ahmad, Mahmoud; Al Marzouqi, Ali H.; Hussien, Mousa

2014-01-01

This paper focuses on the impact of exit exams on different elements of the educational process, namely: curriculum development, students and instructors. A 50-question multiple-choice Exit Exam was prepared by Electrical Engineering (EE) faculty members covering a poll of questions from EE core courses. A copy of the Exit Exam applied during each…

Piloting a Polychotomous Partial-Credit Scoring Procedure in a Multiple-Choice Test

ERIC Educational Resources Information Center

Tsopanoglou, Antonios; Ypsilandis, George S.; Mouti, Anna

2014-01-01

Multiple-choice (MC) tests are frequently used to measure language competence because they are quick, economical and straightforward to score. While degrees of correctness have been investigated for partially correct responses in combined-response MC tests, degrees of incorrectness in distractors and the role they play in determining the…

Preliminary Findings on the Computer-Administered Multiple-Choice Online Causal Comprehension Assessment, a Diagnostic Reading Comprehension Test

ERIC Educational Resources Information Center

Davison, Mark L.; Biancarosa, Gina; Carlson, Sarah E.; Seipel, Ben; Liu, Bowen

2018-01-01

The computer-administered Multiple-Choice Online Causal Comprehension Assessment (MOCCA) for Grades 3 to 5 has an innovative, 40-item multiple-choice structure in which each distractor corresponds to a comprehension process upon which poor comprehenders have been shown to rely. This structure requires revised thinking about measurement issues…

Post-Graduate Student Performance in "Supervised In-Class" vs. "Unsupervised Online" Multiple Choice Tests: Implications for Cheating and Test Security

ERIC Educational Resources Information Center

Ladyshewsky, Richard K.

2015-01-01

This research explores differences in multiple choice test (MCT) scores in a cohort of post-graduate students enrolled in a management and leadership course. A total of 250 students completed the MCT in either a supervised in-class paper and pencil test or an unsupervised online test. The only statistically significant difference between the nine…

Do Sequentially-Presented Answer Options Prevent the Use of Testwiseness Cues on Continuing Medical Education Tests?

ERIC Educational Resources Information Center

Willing, Sonja; Ostapczuk, Martin; Musch, Jochen

2015-01-01

Testwiseness--that is, the ability to find subtle cues towards the solution by the simultaneous comparison of the available answer options--threatens the validity of multiple-choice (MC) tests. Discrete-option multiple-choice (DOMC) has recently been proposed as a computerized alternative testing format for MC tests, and presumably allows for a…

Developing Achievement Test: A Research for Assessment of 5th Grade Biology Subject

ERIC Educational Resources Information Center

Sener, Nilay; Tas, Erol

2017-01-01

The purpose of this study is to prepare a multiple-choice achievement test with high reliability and validity for the "Let's Solve the Puzzle of Our Body" unit. For this purpose, a multiple choice achievement test consisting of 46 items was applied to 178 fifth grade students in total. As a result of the test and material analysis…

Backwash Effects of Language-Testing in Primary and Secondary Education.

ERIC Educational Resources Information Center

Wesdorp, H.

A debate has been carried on in Dutch educational circles about the widespread use of multiple-choice tests, and a number of objections have been raised against the use of such tests. This paper reports on research into the validity of the objections, in particular with respect to the possible effect of multiple-choice tests on the teaching of…

Theodore E. Woodward Award: Spare Me the Powerpoint and Bring Back the Medical Textbook

PubMed Central

Southwick, Frederick S.

2007-01-01

A tutorial for 4th year medical students revealed absent long-term retention of microbiology and infectious disease facts taught during the 2nd year. Students were suffering from the Ziegarnik effect, the loss of memory after completion of a task. PowerPoint lectures and PowerPoint notes combined with multiple-choice questions may have encouraged this outcome; this teaching format was also associated with minimal use of the course textbook. During the subsequent year, active learning techniques, Just-in-Time Teaching (JiTT) and Peer Instruction (PI) were used, and instructors specifically taught from the textbook. Essays and short answer questions were combined with multiple-choice questions to encourage understanding and recall. Performance on the National Board Shelf exam improved from the 59th percentile (2002–2004) to the 83rd percentile (2005), and textbook use increased from 1.6% to 79%. This experience demonstrates that strategies incorporating active learning and textbook use correlate with striking improvement in medical student performance. PMID:18528495

Measuring student learning using initial and final concept test in an STEM course

NASA Astrophysics Data System (ADS)

Kaw, Autar; Yalcin, Ali

2012-06-01

Effective assessment is a cornerstone in measuring student learning in higher education. For a course in Numerical Methods, a concept test was used as an assessment tool to measure student learning and its improvement during the course. The concept test comprised 16 multiple choice questions and was given in the beginning and end of the class for three semesters. Hake's gain index, a measure of learning gains from pre- to post-tests, of 0.36 to 0.41 were recorded. The validity and reliability of the concept test was checked via standard measures such as Cronbach's alpha, content and criterion-related validity, item characteristic curves and difficulty and discrimination indices. The performance of various subgroups such as pre-requisite grades, transfer students, gender and age were also studied.

Effect of differing PowerPoint slide design on multiple-choice test scores for assessment of knowledge and retention in a theriogenology course.

PubMed

Root Kustritz, Margaret V

2014-01-01

Third-year veterinary students in a required theriogenology diagnostics course were allowed to self-select attendance at a lecture in either the evening or the next morning. One group was presented with PowerPoint slides in a traditional format (T group), and the other group was presented with PowerPoint slides in the assertion-evidence format (A-E group), which uses a single sentence and a highly relevant graphic on each slide to ensure attention is drawn to the most important points in the presentation. Students took a multiple-choice pre-test, attended lecture, and then completed a take-home assignment. All students then completed an online multiple-choice post-test and, one month later, a different online multiple-choice test to evaluate retention. Groups did not differ on pre-test, assignment, or post-test scores, and both groups showed significant gains from pre-test to post-test and from pre-test to retention test. However, the T group showed significant decline from post-test to retention test, while the A-E group did not. Short-term differences between slide designs were most likely unaffected due to required coursework immediately after lecture, but retention of material was superior with the assertion-evidence slide design.

Assessing secondary science students' knowledge of molecule movement, concentration gradients, and equilibrium through multiple contexts

NASA Astrophysics Data System (ADS)

Raven, Sara

2015-09-01

Background: Studies have shown that students' knowledge of osmosis and diffusion and the concepts associated with these processes is often inaccurate. This is important to address, as these concepts not only provide the foundation for more advanced topics in biology and chemistry, but are also threaded throughout both state and national science standards. Purpose: In this study, designed to determine the completeness and accuracy of three specific students' knowledge of molecule movement, concentration gradients, and equilibrium, I sought to address the following question: Using multiple evaluative methods, how can students' knowledge of molecule movement, concentration gradients, and equilibrium be characterized? Sample: This study focuses on data gathered from three students - Emma, Henry, and Riley - all of whom were gifted/honors ninth-grade biology students at a suburban high school in the southeast United States. Design and Methods: Using various qualitative data analysis techniques, I analyzed multiple sources of data from the three students, including multiple-choice test results, written free-response answers, think-aloud interview responses, and student drawings. Results: Results of the analysis showed that students maintained misconceptions about molecule movement, concentration gradients, and equilibrium. The conceptual knowledge students demonstrated differed depending on the assessment method, with the most distinct differences appearing on the multiple-choice versus the free-response questions, and in verbal versus written formats. Conclusions: Multiple levels of assessment may be required to obtain an accurate picture of content knowledge, as free-response and illustrative tasks made it difficult for students to conceal any misconceptions. Using a variety of assessment methods within a section of the curriculum can arguably help to provide a deeper understanding of student knowledge and learning, as well as illuminate misconceptions that may have remained unknown if only one assessment method was used. Furthermore, beyond simply evaluating past learning, multiple assessment methods may aid in student comprehension of key concepts.

The Meaning of "Choice and Control" for People with Intellectual Disabilities Who Are Planning Their Social Care and Support

ERIC Educational Resources Information Center

Williams, Val; Porter, Sue

2017-01-01

Background: This paper questions consumerist assumptions in current English social care policy and aims to look behind the processes of personalization to interrogate what "choice and control" means in the lives of a diverse group of people with intellectual disabilities. Methods: Data were from multiple interviews and direct practice…

Senior high school students’ need analysis of Three-Tier Multiple Choice (3TMC) diagnostic test about acid-base and solubility equilibrium

NASA Astrophysics Data System (ADS)

Ardiansah; Masykuri, M.; Rahardjo, S. B.

2018-05-01

Students’ conceptual understanding is the most important comprehension to obtain related comprehension. However, they held their own conception. With this need analysis, we will elicit student need of 3TMC diagnostic test to measure students’ conception about acid-base and solubility equilibrium. The research done by a mixed method using questionnaire analysis based on descriptive of quantitative and qualitative. The research subject was 96 students from 4 senior high schools and 4 chemistry teachers chosen by random sampling technique. Data gathering used a questionnaire with 10 questions for student and 28 questions for teachers. The results showed that 97% of students stated that the development this instrument is needed. In addition, there were several problems obtained in this questionnaire include learning activity, teacher’s test and guessing. In conclusion, this is necessary to develop the 3TMC instrument that can diagnose and measure the student’s conception in acid-base and solubility equilibrium.

Multiple-Choice versus Constructed-Response Tests in the Assessment of Mathematics Computation Skills.

ERIC Educational Resources Information Center

Gadalla, Tahany M.

The equivalence of multiple-choice (MC) and constructed response (discrete) (CR-D) response formats as applied to mathematics computation at grade levels two to six was tested. The difference between total scores from the two response formats was tested for statistical significance, and the factor structure of items in both response formats was…

Developing Multiple Choice Tests: Tips & Techniques

ERIC Educational Resources Information Center

McCowan, Richard J.

1999-01-01

Item writing is a major responsibility of trainers. Too often, qualified staff who prepare lessons carefully and teach conscientiously use inadequate tests that do not validly reflect the true level of trainee achievement. This monograph describes techniques for constructing multiple-choice items that measure student performance accurately. It…

No Computer Left Behind

ERIC Educational Resources Information Center

Cohen, Daniel J.; Rosenzweig, Roy

2006-01-01

The combination of the Web and the cell phone forecasts the end of the inexpensive technologies of multiple-choice tests and grading machines. These technological developments are likely to bring the multiple-choice test to the verge of obsolescence, mounting a substantial challenge to the presentation of history and other disciplines.

An Investigation of Current Endodontic Practice in Turkey

PubMed Central

Kaptan, R. F.; Haznedaroglu, F.; Kayahan, M. B.; Basturk, F. B.

2012-01-01

Objectives. The aim of this study was to gather information about the quality and quantity of root canal treatments carried out by general dental practitioners in Turkey. Methods. Questionnaires were given to 1400 dentists who attended the 16th National Congress organized by the Turkish Dental Association. The participants were asked to answer 34 multiple-choice questions. The questions were subdivided into 3 main topics; general information; general approach to endodontic treatment; and cleaning, shaping, and obturation of root canals. The statistical analysis was carried out by an χ 2-test to compare the means at a significance level of P < 0.05. Results. The response rate for this study was 43%. There was a wide variation in the number of root canal treatments completed per month. Nearly 92% of practitioners stated that they never used rubber dam. The most commonly used working length determination technique was radiographic evaluation (P < 0.05). Sodium hypochlorite was the irrigant of choice with varying concentrations and AH Plus was the sealer of choice (P < 0.05). Resin composite was the most frequently used material for final restorations. Conclusions. Endodontic procedures in general practice in Turkey have differences from widely acknowledged quality guidelines. Despite the introduction of new instruments and techniques, most of the general practitioners chose conventional methods. PMID:23251103

The Effects of Item Preview on Video-Based Multiple-Choice Listening Assessments

ERIC Educational Resources Information Center

Koyama, Dennis; Sun, Angela; Ockey, Gary J.

2016-01-01

Multiple-choice formats remain a popular design for assessing listening comprehension, yet no consensus has been reached on how multiple-choice formats should be employed. Some researchers argue that test takers must be provided with a preview of the items prior to the input (Buck, 1995; Sherman, 1997); others argue that a preview may decrease the…

Utilization of Fact Retrieval and Inferential Reasoning in Young, Middle-Aged, and Elderly Adults.

ERIC Educational Resources Information Center

Camp, Cameron J.; Pignatiello, Michael F.

World knowledge is defined as information that is acquired by adults from life experiences. To investigate question answering processes involving world knowledge systems, 120 young, middle-aged and older adults were given questions intended to induce either fact retrieval or inferential reasoning. Multiple-choice and true/false formats were used.…

A Dialogue about MCQs, Reliability, and Item Response Modelling

ERIC Educational Resources Information Center

Wright, Daniel B.; Skagerberg, Elin M.

2006-01-01

Multiple choice questions (MCQs) are becoming more common in UK psychology departments and the need to assess their reliability is apparent. Having examined the reliability of MCQs in our department we faced many questions from colleagues about why we were examining reliability, what it was that we were doing, and what should be reported when…

Physics 30: Grade 12 Diploma Examination, June 1997.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton. Student Evaluation Branch.

This document is a Physics 30 Grade 12 Diploma Examination from Alberta Education. It is a 2.5 hour closed-book examination consisting of 37 multiple-choice and 12 numerical-response questions of equal value worth 70% of the examination and two written-response questions of equal value worth 30% of the examination. The exam contains sets of…

Assessment of musculoskeletal physical examination skills and attitudes of orthopaedic residents.

PubMed

Beran, Matthew C; Awan, Hisham; Rowley, David; Samora, Julie Balch; Griesser, Michael J; Bishop, Julie Y

2012-03-21

Although the musculoskeletal physical examination is an essential part of patient encounters, we believe that it is underemphasized in residency education and that residents' physical examination skills may be lacking. We sought to assess attitudes regarding teaching of the physical examination in orthopaedic residencies, to assess physical examination knowledge and skills among residents, and to develop a method to track the skill level of residents in order to improve our physical examination curriculum. We created a thirty-question multiple-choice musculoskeletal physical examination test and administered it to our residents. We created a five-question survey assessing attitudes toward physical examination teaching in orthopaedic residencies and distributed it to U.S. orthopaedic department chairs We developed an Objective Structured Clinical Examination (OSCE), in which standardized patients enact four clinical scenarios, to observe and assess physical examination skills. The mean score on the multiple-choice physical examination test was 76% despite the fact that our residents consistently scored above 90% on the Orthopaedic In-Training Examination. Department chairs and residents agreed that, although learning to perform the physical examination is important, there is not enough time in the clinical setting to observe and critique a resident's patient examination. The overall score of our residents on the OSCE was 66%. We have exposed a deficiency in the physical examination knowledge and skills of our residents. Although the musculoskeletal physical examination is a vital practice component, our data indicate that it is likely underemphasized in training. Clinic time alone is likely insufficient for the teaching and learning of the musculoskeletal physical examination.

Pointing with Power or Creating with Chalk

ERIC Educational Resources Information Center

Rudow, Sasha R.; Finck, Joseph E.

2015-01-01

This study examines the attitudes of students on the use of PowerPoint and chalk/white boards in college science lecture classes. Students were asked to complete a survey regarding their experiences with PowerPoint and chalk/white boards in their science classes. Both multiple-choice and short answer questions were used. The multiple-choice…

Performance of Men and Women on Multiple-Choice and Constructed-Response Tests for Beginning Teachers. Research Report. ETS RR-04-48

ERIC Educational Resources Information Center

Livingston, Samuel A.; Rupp, Stacie L.

2004-01-01

Some previous research results imply that women tend to perform better, relative to men, on constructed-response (CR) tests than on multiple-choice (MC) tests in the same subjects. An analysis of data from several tests used in the licensing of beginning teachers supported this hypothesis, to varying degrees, in most of the tests investigated. The…

Of Small Beauties and Large Beasts: The Quality of Distractors on Multiple-Choice Tests Is More Important than Their Quantity

ERIC Educational Resources Information Center

Papenberg, Martin; Musch, Jochen

2017-01-01

In multiple-choice tests, the quality of distractors may be more important than their number. We therefore examined the joint influence of distractor quality and quantity on test functioning by providing a sample of 5,793 participants with five parallel test sets consisting of items that differed in the number and quality of distractors.…

Does the Position of Response Options in Multiple-Choice Tests Matter?

ERIC Educational Resources Information Center

Hohensinn, Christine; Baghaei, Purya

2017-01-01

In large scale multiple-choice (MC) tests alternate forms of a test may be developed to prevent cheating by changing the order of items or by changing the position of the response options. The assumption is that since the content of the test forms are the same the order of items or the positions of the response options do not have any effect on…

The Geoscience Concept Test: A New Assessment Tool Based on Student Misconceptions

NASA Astrophysics Data System (ADS)

Libarkin, J.; Anderson, S. W.; Boone, W. J.; Beilfuss, M.; Dahl, J.

2002-12-01

We developed and began pilot testing of an earth science assessment tool called the geoscience concept test (GCT). The GCT uses student misconceptions as distractors in a 30 item multiple-choice instrument. Student misconceptions were first assessed through the analysis of nearly 300 questionnaires administered in introductory geology courses at three institutions. Results from the questionnaires guided the development of an interview protocol that was used by four interviewers at four different institutions. Over 100 in-depth student interviews lasting from 0.5 to 1 hour probed topics related to the Earth's interior, geologic time, and the formation of Earth surface features such as mountains and volcanoes to better define misconceptions. Thematic content analysis of the interviews identified a number of widely held misconceptions, which were then incorporated into the GCT as multiple-choice distractors (wrong answers). For content validity, the initial GCT was reviewed by seven experts (3 geoscientists and 4 science educators) and revised before pilot testing. Approximately 100 introductory and non-science major college students from four institutions were assessed with the GCT pilot in the spring of 2002. Rasch model analysis of this data showed that students found the pilot test difficult, and the level of difficulty was consistent between the four institutions. Analysis of individual items showed that students had fewer misconceptions regarding the locations of earthquakes, and many misconceptions regarding the locations of volcanoes on the Earth's surface, suggesting a disconnect in their understanding of the role of plate tectonics in these phenomena. Analysis of the misfit statistic for each item showed that none of the questions misfit, although we dropped one question and modified the wording of another for clarity in the next round of piloting. A second round of piloting scheduled for the fall of 2002 includes nearly 3000 students from 34 institutions in 19 states.

The Effectiveness of learning materials based on multiple intelligence on the understanding of global warming

NASA Astrophysics Data System (ADS)

Liliawati, W.; Purwanto; Zulfikar, A.; Kamal, R. N.

2018-05-01

This study aims to examine the effectiveness of the use of teaching materials based on multiple intelligences on the understanding of high school students’ material on the theme of global warming. The research method used is static-group pretest-posttest design. Participants of the study were 60 high school students of XI class in one of the high schools in Bandung. Participants were divided into two classes of 30 students each for the experimental class and control class. The experimental class uses compound-based teaching materials while the experimental class does not use a compound intelligence-based teaching material. The instrument used is a test of understanding of the concept of global warming with multiple choices form amounted to 15 questions and 5 essay items. The test is given before and after it is applied to both classes. Data analysis using N-gain and effect size. The results obtained that the N-gain for both classes is in the medium category and the effectiveness of the use of teaching materials based on the results of effect-size test results obtained in the high category.

Valuing Assessment in Teacher Education - Multiple-Choice Competency Testing

ERIC Educational Resources Information Center

Martin, Dona L.; Itter, Diane

2014-01-01

When our focus is on assessment educators should work to value the nature of assessment. This paper presents a new approach to multiple-choice competency testing in mathematics education. The instrument discussed here reflects student competence, encourages self-regulatory learning behaviours and links content with current curriculum documents and…

How to Assess Student Performance in Science: Going beyond Multiple-Choice Tests. Third Edition

ERIC Educational Resources Information Center

Butler, Susan M.; McColskey, Wendy; O'Sullivan, Rita

2005-01-01

Educational systems promote student growth in a variety of dimensions. Basic content knowledge can be effectively assessed with multiple-choice and completion tests. However educational reforms have become more concerned with higher-order cognitive dimensions (problem-solving, creativity), social dimensions (communication skills, ability to work…

Choose and choose again: appearance-reality errors, pragmatics and logical ability.

PubMed

Deák, Gedeon O; Enright, Brian

2006-05-01

In the Appearance/Reality (AR) task some 3- and 4-year-old children make perseverative errors: they choose the same word for the appearance and the function of a deceptive object. Are these errors specific to the AR task, or signs of a general question-answering problem? Preschoolers completed five tasks: AR; simple successive forced-choice question pairs (QP); flexible naming of objects (FN); working memory (WM) span; and indeterminacy detection (ID). AR errors correlated with QP errors. Insensitivity to indeterminacy predicted perseveration in both tasks. Neither WM span nor flexible naming predicted other measures. Age predicted sensitivity to indeterminacy. These findings suggest that AR tests measure a pragmatic understanding; specifically, different questions about a topic usually call for different answers. This understanding is related to the ability to detect indeterminacy of each question in a series. AR errors are unrelated to the ability to represent an object as belonging to multiple categories, to working memory span, or to inhibiting previously activated words.

Evaluating learning among undergraduate medical students in schools with traditional and problem-based curricula.

PubMed

Meo, Sultan Ayoub

2013-09-01

This study aimed to assess knowledge and skills in a respiratory physiology course in traditional versus problem-based learning (PBL) groups in two different medical schools. Two different undergraduate medical schools were selected for this study. The first medical school followed the traditional [lecture-based learning (LBL)] curriculum, and the second medical school followed the PBL curriculum. Sixty first-year male medical students (30 students from each medical school) volunteered; they were apparently healthy and of the same age, sex, nationality, and regional and cultural background. Students were taught respiratory physiology according to their curriculum for a period of 2 wk. At the completion of the study period, knowledge was measured based on a single best multiple-choice question examination, and skill was measured based on the objective structured practical examination in the lung function laboratory (respiratory physiology). A Student's t-test was applied for the analysis of the data, and the level of significance was set at P < 0.05. Students belonging to the PBL curriculum obtained a higher score in the multiple-choice question examination (P = 0.001) and objective structured practical examination (P = 0.0001) compared with traditional (LBL) students. Students in the PBL group obtained significantly higher knowledge and skill scores in the respiratory physiology course compared with students in the traditional (LBL) style of medical schools.

Temporally contiguous pencast instruction promotes meaningful learning for dental and dental hygiene students in physiology.

PubMed

Roesch, Darren M

2014-01-01

Smartpens allow for the creation of computerized "pencasts" that combine voice narration with handwritten notes and illustrations. The purpose of this study was to test the effects of voluntary participation in extracurricular instruction with a pencast on student learning. Dental and dental hygiene students were given instruction in a complex physiological topic using lecture and static slides. An Internet link to a pencast that covered the complex topic in a more temporally contiguous fashion was also provided for voluntary review. The students were given a multiple-choice exam that consisted of retention and transfer test questions. Sixty-nine percent of the students who did not watch the pencast and 89 percent of the students who watched the pencast answered the retention test question correctly (p=0.08). Fifty-four percent of the students who did not watch the pencast and 90 percent of the students who watched the pencast answered the transfer test question correctly (p=0.005). This finding indicates that students who watched the pencast performed better on a transfer test, a measurement of meaningful learning, than students who received only the narrated instruction with static images. This supports the hypothesis that temporally contiguous instruction promotes more meaningful learning than lecture accompanied only by static slide images.

Predictive Validity of a Multiple-Choice Test for Placement in a Community College

ERIC Educational Resources Information Center

Verbout, Mary F.

2013-01-01

Multiple-choice tests of punctuation and usage are used throughout the United States to assess the writing skills of new community college students in order to place them in either a basic writing course or first-year composition. To determine whether using the COMPASS Writing Test (CWT) is a valid placement at a community college, student test…

Teaching Composition Skills with Weekly Multiple Choice Tests in Lieu of Theme Writing. Final Report.

ERIC Educational Resources Information Center

Scannell, Dale P.; Haugh, Oscar M.

The purpose of the study was to compare the effectiveness with which composition skills could be taught by the traditional theme-assignment approach and by an experimental method using weekly multiple-choice composition tests in lieu of theme writing. The weekly tests were based on original but typical first-draft compositions and covered problems…

What is adaptive about adaptive decision making? A parallel constraint satisfaction account.

PubMed

Glöckner, Andreas; Hilbig, Benjamin E; Jekel, Marc

2014-12-01

There is broad consensus that human cognition is adaptive. However, the vital question of how exactly this adaptivity is achieved has remained largely open. Herein, we contrast two frameworks which account for adaptive decision making, namely broad and general single-mechanism accounts vs. multi-strategy accounts. We propose and fully specify a single-mechanism model for decision making based on parallel constraint satisfaction processes (PCS-DM) and contrast it theoretically and empirically against a multi-strategy account. To achieve sufficiently sensitive tests, we rely on a multiple-measure methodology including choice, reaction time, and confidence data as well as eye-tracking. Results show that manipulating the environmental structure produces clear adaptive shifts in choice patterns - as both frameworks would predict. However, results on the process level (reaction time, confidence), in information acquisition (eye-tracking), and from cross-predicting choice consistently corroborate single-mechanisms accounts in general, and the proposed parallel constraint satisfaction model for decision making in particular. Copyright © 2014 Elsevier B.V. All rights reserved.

Multiple-Choice Exams and Guessing: Results from a One-Year Study of General Chemistry Tests Designed to Discourage Guessing

ERIC Educational Resources Information Center

Campbell, Mark L.

2015-01-01

Multiple-choice exams, while widely used, are necessarily imprecise due to the contribution of the final student score due to guessing. This past year at the United States Naval Academy the construction and grading scheme for the department-wide general chemistry multiple-choice exams were revised with the goal of decreasing the contribution of…

Assessment of representational competence in kinematics

NASA Astrophysics Data System (ADS)

Klein, P.; Müller, A.; Kuhn, J.

2017-06-01

A two-tier instrument for representational competence in the field of kinematics (KiRC) is presented, designed for a standard (1st year) calculus-based introductory mechanics course. It comprises 11 multiple choice (MC) and 7 multiple true-false (MTF) questions involving multiple representational formats, such as graphs, pictures, and formal (mathematical) expressions (1st tier). Furthermore, students express their answer confidence for selected items, providing additional information (2nd tier). Measurement characteristics of KiRC were assessed in a validation sample (pre- and post-test, N =83 and N =46 , respectively), including usefulness for measuring learning gain. Validity is checked by interviews and by benchmarking KiRC against related measures. Values for item difficulty, discrimination, and consistency are in the desired ranges; in particular, a good reliability was obtained (KR 20 =0.86 ). Confidence intervals were computed and a replication study yielded values within the latter. For practical and research purposes, KiRC as a diagnostic tool goes beyond related extant instruments both for the representational formats (e.g., mathematical expressions) and for the scope of content covered (e.g., choice of coordinate systems). Together with the satisfactory psychometric properties it appears a versatile and reliable tool for assessing students' representational competency in kinematics (and of its potential change). Confidence judgments add further information to the diagnostic potential of the test, in particular for representational misconceptions. Moreover, we present an analytic result for the question—arising from guessing correction or educational considerations—of how the total effect size (Cohen's d ) varies upon combination of two test components with known individual effect sizes, and then discuss the results in the case of KiRC (MC and MTF combination). The introduced method of test combination analysis can be applied to any test comprising two components for the purpose of finding effect size ranges.

Examining Two Strategies to Link Mixed-Format Tests Using Multiple-Choice Anchors. Research Report. ETS RR-10-18

ERIC Educational Resources Information Center

Walker, Michael E.; Kim, Sooyeon

2010-01-01

This study examined the use of an all multiple-choice (MC) anchor for linking mixed format tests containing both MC and constructed-response (CR) items, in a nonequivalent groups design. An MC-only anchor could effectively link two such test forms if either (a) the MC and CR portions of the test measured the same construct, so that the MC anchor…

Models for Scoring Missing Responses to Multiple-Choice Items. Program Statistics Research Technical Report No. 94-1.

ERIC Educational Resources Information Center

Longford, Nicholas T.

This study is a critical evaluation of the roles for coding and scoring of missing responses to multiple-choice items in educational tests. The focus is on tests in which the test-takers have little or no motivation; in such tests omitting and not reaching (as classified by the currently adopted operational rules) is quite frequent. Data from the…

A Multiple Choice Version of the Sentence Completion Method

ERIC Educational Resources Information Center

Shouval, Ron; And Others

1975-01-01

It was concluded that a multiple choice form corresponding to a sentence completion measure, test clearly defined personality areas (such as autonomy) could be a reasonable alternative for many purposes. (Author/DEP)

Violating Conventional Wisdom in Multiple Choice Test Construction

ERIC Educational Resources Information Center

Taylor, Annette Kujawski

2005-01-01

This research examined 2 elements of multiple-choice test construction, balancing the key and optimal number of options. In Experiment 1 the 3 conditions included a balanced key, overrepresentation of a and b responses, and overrepresentation of c and d responses. The results showed that error-patterns were independent of the key, reflecting…

Multiple-Choice Tests with Correction Allowed in Autism: An Excel Applet

ERIC Educational Resources Information Center

Martinez, Elisabetta Monari

2010-01-01

The valuation of academic achievements in students with severe language impairment is problematic if they also have difficulties in sustaining attention and in praxic skills. In severe autism all of these difficulties may occur together. Multiple-choice tests offer the advantage that simple praxic skills are required, allowing the tasks to be…

Application of a Multidimensional Nested Logit Model to Multiple-Choice Test Items

ERIC Educational Resources Information Center

Bolt, Daniel M.; Wollack, James A.; Suh, Youngsuk

2012-01-01

Nested logit models have been presented as an alternative to multinomial logistic models for multiple-choice test items (Suh and Bolt in "Psychometrika" 75:454-473, 2010) and possess a mathematical structure that naturally lends itself to evaluating the incremental information provided by attending to distractor selection in scoring. One potential…

A Practical Methodology for the Systematic Development of Multiple Choice Tests.

ERIC Educational Resources Information Center

Blumberg, Phyllis; Felner, Joel

Using Guttman's facet design analysis, four parallel forms of a multiple-choice test were developed. A mapping sentence, logically representing the universe of content of a basic cardiology course, specified the facets of the course and the semantic structural units linking them. The facets were: cognitive processes, disease priority, specific…

The Relationship of Expert-System Scored Constrained Free-Response Items to Multiple-Choice and Open-Ended Items.

ERIC Educational Resources Information Center

Bennett, Randy Elliot; And Others

1990-01-01

The relationship of an expert-system-scored constrained free-response item type to multiple-choice and free-response items was studied using data for 614 students on the College Board's Advanced Placement Computer Science (APCS) Examination. Implications for testing and the APCS test are discussed. (SLD)

Investigation of the Effects of Scorm Adapted Whiteboard Movie Techniques on Students' Success in Physics Classes

ERIC Educational Resources Information Center

Basaran, Bulent; Gonen, Selahattin

2012-01-01

In this study, a web site including instructional materials such as white sheet videos, simulations, and animations and problem solving materials such as true-false questions, fill-in-the-blanks, puzzles, open-ended and multiple-choice questions regarding such important physics units as "Force" and "Movement" was designed. The purpose was to…

Using Diagnostic Assessment to Help Teachers Understand the Chemistry of the Lead-Acid Battery

ERIC Educational Resources Information Center

Cheung, Derek

2011-01-01

Nineteen pre-service and in-service teachers taking a chemistry teaching methods course at a university in Hong Kong were asked to take a diagnostic assessment. It consisted of seven multiple-choice questions about the chemistry of the lead-acid battery. Analysis of the teachers' responses to the questions indicated that they had difficulty in…

Automatic Generation of Analogy Questions for Student Assessment: An Ontology-Based Approach

ERIC Educational Resources Information Center

Alsubait, Tahani; Parsia, Bijan; Sattler, Uli

2012-01-01

Different computational models for generating analogies of the form "A is to B as C is to D" have been proposed over the past 35 years. However, analogy generation is a challenging problem that requires further research. In this article, we present a new approach for generating analogies in Multiple Choice Question (MCQ) format that can be used…

The Development and Evaluation of Software to Foster Professional Development in Educational Assessment

ERIC Educational Resources Information Center

Benton, Morgan C.

2008-01-01

This dissertation sought to answer the question: Is it possible to build a software tool that will allow teachers to write better multiple-choice questions? The thesis proceeded from the finding that the quality of teaching is very influential in the amount that students learn. A basic premise of this research, then, is that improving teachers…

The Inclusion of Science Process Skills in Multiple Choice Questions: Are We Getting Any Better?

ERIC Educational Resources Information Center

Elmas, Ridvan; Bodner, George M.; Aydogdu, Bulent; Saban, Yakup

2018-01-01

The goal of this study was to analyze the science and technology questions with respect to science process skills (SPS) included in the "Transition from Primary to Secondary Education" (TEOG) examination developed for use with 8th-grade students in Turkey. The 12 TEOG exams administered in the course of three academic years from 2014…

Helping students learn effective problem solving strategies by reflecting with peers

NASA Astrophysics Data System (ADS)

Mason, Andrew; Singh, Chandralekha

2010-07-01

We study how introductory physics students engage in reflection with peers about problem solving. The recitations for an introductory physics course with 200 students were broken into a "peer reflection" (PR) group and a traditional group. Each week in recitation, small teams of students in the PR group reflected on selected problems from the homework and discussed why the solutions of some students employed better problem solving strategies than others. The graduate and undergraduate teaching assistants in the PR recitations provided guidance and coaching to help students learn effective problem solving heuristics. In the traditional group recitations students could ask the graduate TA questions about the homework before they took a weekly quiz. The traditional group recitation quiz questions were similar to the homework questions selected for peer reflection in the PR group recitations. As one measure of the impact of this intervention, we investigated how likely students were to draw diagrams to help with problem solving on the final exam with only multiple-choice questions. We found that the PR group drew diagrams on more problems than the traditional group even when there was no explicit reward for doing so. Also, students who drew more diagrams for the multiple-choice questions outperformed those who did not, regardless of which group they were a member.

Recall in older cancer patients: measuring memory for medical information.

PubMed

Jansen, Jesse; van Weert, Julia; van der Meulen, Nienke; van Dulmen, Sandra; Heeren, Thea; Bensing, Jozien

2008-04-01

Remembering medical treatment information may be particularly taxing for older cancer patients, but to our knowledge this ability has never been assessed in this specific age group only. Our purpose in this study was to investigate older cancer patients' recall of information after patient education preceding chemotherapy. We constructed a recall questionnaire consisting of multiple-choice questions, completion items, and open-ended questions related to information about treatment and recommendations on how to handle side effects. Immediately after a nursing consultation preceding chemotherapy treatment, 69 older patients (M = 71.8 years, SD = 4.1) completed the questionnaire. We checked recall against the actual communication in video recordings of the consultations. On average, 82.2 items were discussed during the consultations. The mean percentage of information recalled correctly was 23.2% for open-ended questions, 68.0% for completion items, and 80.2% for multiple-choice questions. Older cancer patients are confronted with a lot of information. Recall of information strongly depended on question format; especially active reproduction appeared to be poor. To improve treatment outcomes, it is important that cancer patients are able to actively retrieve knowledge about how to prevent and recognize adverse side effects and that this is checked by the health professional. We make suggestions on how to make information more memorable for older cancer patients.

A Analysis of Saudi Arabian High School Students' Misconceptions about Physics Concepts.

NASA Astrophysics Data System (ADS)

Al-Rubayea, Abdullah A. M.

This study was conducted to explore Saudi high students' misconceptions in selected physics concepts. It also detected the effects of gender, grade level and location of school on Saudi high school students' misconceptions. In addition, a further analysis of students' misconceptions in each question was investigated and a correlation between students' responses, confidence in answers and sensibleness was conducted. There was an investigation of sources of students' answers in this study. Finally, this study included an analysis of students' selection of reasons only in the instrument. The instrument used to detect the students' misconceptions was a modified form of the Misconception Identification in Science Questionnaire (MISQ). This instrument was developed by Franklin (1992) to detected students' misconceptions in selected physics concepts. This test is a two-tier multiple choice test that examines four areas of physics: Force and motion, heat and temperature, light and color and electricity and magnetism. This study included a sample of 1080 Saudi high school students who were randomly selected from six Saudi educational districts. This study also included both genders, the three grade levels of Saudi high schools, six different educational districts, and a city and a town in each educational district. The sample was equally divided between genders, grade levels, and educational districts. The result of this study revealed that Saudi Arabian high school students hold numerous misconceptions about selected physics concepts. It also showed that tenth grade students were significantly different than the other grades. The result also showed that different misconceptions are held by the students for each concept in the MISQ. A positive correlation between students' responses, confidence in answers and sensibleness in many questions was shown. In addition, it showed that guessing was the most dominant source of misconceptions. The result revealed that gender and grade level had an affect on students' choice of decision on the MISQ items. A positive change in the means of gender and grade levels in the multiple choice test and gender differences in selection of reason may be associated with specific concepts. No significant difference in frequencies of the reasons chosen by the student to justify their answers were found in most of the items (10 items).

Multimodal Likelihoods in Educational Assessment: Will the Real Maximum Likelihood Score Please Stand up?

ERIC Educational Resources Information Center

Wothke, Werner; Burket, George; Chen, Li-Sue; Gao, Furong; Shu, Lianghua; Chia, Mike

2011-01-01

It has been known for some time that item response theory (IRT) models may exhibit a likelihood function of a respondent's ability which may have multiple modes, flat modes, or both. These conditions, often associated with guessing of multiple-choice (MC) questions, can introduce uncertainty and bias to ability estimation by maximum likelihood…

Score Increase and Partial-Credit Validity When Administering Multiple-Choice Tests Using an Answer-Until-Correct Format

ERIC Educational Resources Information Center

Slepkov, Aaron D.; Vreugdenhil, Andrew J.; Shiell, Ralph C.

2016-01-01

There are numerous benefits to answer-until-correct (AUC) approaches to multiple-choice testing, not the least of which is the straightforward allotment of partial credit. However, the benefits of granting partial credit can be tempered by the inevitable increase in test scores and by fears that such increases are further contaminated by a large…

Dividing the Force Concept Inventory into two equivalent half-length tests

NASA Astrophysics Data System (ADS)

Han, Jing; Bao, Lei; Chen, Li; Cai, Tianfang; Pi, Yuan; Zhou, Shaona; Tu, Yan; Koenig, Kathleen

2015-06-01

The Force Concept Inventory (FCI) is a 30-question multiple-choice assessment that has been a building block for much of the physics education research done today. In practice, there are often concerns regarding the length of the test and possible test-retest effects. Since many studies in the literature use the mean score of the FCI as the primary variable, it would be useful then to have different shorter tests that can produce FCI-equivalent scores while providing the benefits of being quicker to administer and overcoming the test-retest effects. In this study, we divide the 1995 version of the FCI into two half-length tests; each contains a different subset of the original FCI questions. The two new tests are shorter, still cover the same set of concepts, and produce mean scores equivalent to those of the FCI. Using a large quantitative data set collected at a large midwestern university, we statistically compare the assessment features of the two half-length tests and the full-length FCI. The results show that the mean error of equivalent scores between any two of the three tests is within 3%. Scores from all tests are well correlated. Based on the analysis, it appears that the two half-length tests can be a viable option for score based assessment that need to administer tests quickly or need to measure short-term gains where using identical pre- and post-test questions is a concern.

Comparison of individual answer and group answer with and without structured peer assessment

NASA Astrophysics Data System (ADS)

Kablan, Zeynel

2014-09-01

Background:Cooperative learning activities provide active participation of students leading to better learning. The literature suggests that cooperative learning activities need to be structured for a more effective and productive interaction. Purpose: This study aimed to test the differences among three instructional conditions in terms of science achievement. Sample:A total of 79 fifth-grade students, 42 males (53%) and 37 females (47%), participated in the study. Design and Methods:In the first condition, students answered the teacher's questions individually by raising hands. In the second condition, students discussed the answer in groups and came up with a single group answer. In this condition, the teacher provided only verbal directions to the groups without using any strategy or material. In the third condition, students used a 'peer assessment form' before giving the group answer. A pre-/post-test experimental design was used. Multiple-choice and open-ended tests were used for data collection. One-way analysis of variance (ANOVA) was conducted to test the differences in the test scores between the three groups (individual answer, unstructured group answer and structured group answer). Results:Results showed that there were no significant differences among the three learning conditions in terms of their multiple-choice test scores. In terms of the open-ended test scores, students in the structured group answer condition scored significantly higher than the students in the individual answer condition. Conclusions:Structuring the group work through peer assessment helped to monitor the group discussion, provided a better learning compared to the individual answer condition, and helped students to participate in the activity equally.

Comparison of traditional and interactive teaching methods in a UK emergency department.

PubMed

Armstrong, Peter; Elliott, Tim; Ronald, Julie; Paterson, Brodie

2009-12-01

Didactic teaching remains a core component of undergraduate education, but developing computer assisted learning (CAL) packages may provide useful alternatives. We compared the effectiveness of interactive multimedia-based tutorials with traditional, lecture-based models for teaching arterial blood gas interpretation to fourth year medical students. Participants were randomized to complete a tutorial in either lecture or multimedia format containing identical content. Upon completion, students answered five multiple choice questions assessing post-tutorial knowledge, and provided feedback on their allocated learning method. Marks revealed no significant difference between either group. All lecture candidates rated their teaching as good, compared with 89% of the CAL group. All CAL users found multiple choice questions assessment useful, compared with 83% of lecture participants. Both groups highlighted the importance of interaction. CAL complements other teaching methods, but should be seen as an adjunct to, rather than a replacement for, traditional methods, thus offering students a blended learning environment.

[To what extent do reviewers of multiple-choice questions need to be trained? A comparison between handing out information sheets and brief workshop sessions].

PubMed

Öchsner, Wolfgang; Böckers, Anja

2016-01-01

A competent review process is crucial to ensure the quality of multiple-choice (MC) questions. However, the acquisition of reviewing skills should not cause any unnecessary additional burden for a medical staff that is already facing heavy workloads. 100 MC questions, for which an expert review existed, were presented to 12 novices. In advance, six participants received a specific information sheet covering critical information for high-calibre review; the other six participants attended a 2.5-hour workshop covering the same information. The review results of both groups were analysed with a licensed version of the IBM software SPSS 19.0 (SPSS Inc., Chicago, IL). The results of the workshop group were distinctly closer to the experts' results (gold standard) than those of the information sheet group. For the quantitatively important category of medium quality MC questions, the results of the workshop group did not significantly differ from the experts' results. In the information sheet group the results were significantly poorer than the experts', regardless of the quality of the questions. Distributing specific information sheets to MC question reviewers is not sufficient for ensuring the quality of the review so that - regardless of the increased effort involved - a recommendation to conduct specific workshops must be made. Copyright © 2014. Published by Elsevier GmbH.

Teaching About Genetics and Sickle Cell Disease In Fifth Grade.

PubMed

Day, Lucille Lang; Murray, Eileen; Treadwell, Marsha J; Lubin, Bertram H

2015-02-01

We are grateful to Laura McVittie Gray for her work on the development of the student activities described in this article. This work was made possible by a Science Education Partnership Award (SEPA), Grant Number R25RR020449, from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH). Additional support for this SEPA-funded project was provided by Grant Number UL1RR024131-01 from NCRR. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of NCRR or NIH. A 5-lesson, 5th-grade instructional unit, "Genetics and Sickle Cell Disease," was developed and tested as part of a 40-lesson curriculum entitled SEEK (Science Exploration, Excitement, and Knowledge): A Curriculum in Health and Biomedical Science for Diverse 4th and 5th Grade Students. The genetics lessons include hands-on activities (e.g., DNA extraction from cheek cells), a simulated plant genetics experiment, and a classroom visit by a person with sickle cell disease, as well as by a health care practitioner who works with sickle cell patients or a scientist specializing in genetics. The unit was tested with 82 5th-grade students at public elementary schools in Oakland, CA; 96% were racial and ethnic minorities. The comparison group consisted of 84 5th-grade Oakland students racially/ ethnically, academically, and socio-economically matched to those in the experimental group. Both groups completed a 20-question, multiple-choice pre/posttest covering science concepts, scientific process, lifestyle choices, and careers. The experimental group showed significant improvement on 13 of 20 questions (P<.05, t-tests) and on the test as a whole, whereas the comparison group did not show significant improvement either on any of the questions or on the test as a whole. The experimental group improved on 10 concept questions, 2 scientific process questions, and 1 lifestyle question. Teachers rated the educational value of the unit as 9.5 on a scale from 1 (low) to 10 (high). These results show that genetics and sickle cell disease can be taught successfully in 5th grade, although they are not typically covered at this level. © 2015 National Medical Association. Published by Elsevier Inc. All rights reserved.

Question Choice in G.C.E. Ordinary Level Chemistry: Candidates' Perceptions

ERIC Educational Resources Information Center

Francis, J. C.; Owen, S. J.

1978-01-01

Investigates how student ability, anxiety level, question wiseness, and relative difficulty of questions influence students' opinions of and performance on a chemistry examination. Examines strategies employed by students in selecting questions. Concludes that the major problem for test candidates is anxiety. (Author/DB)

The Effect of the Multiple-Choice Item Format on the Measurement of Knowledge of Language Structure

ERIC Educational Resources Information Center

Currie, Michael; Chiramanee, Thanyapa

2010-01-01

Noting the widespread use of multiple-choice items in tests in English language education in Thailand, this study compared their effect against that of constructed-response items. One hundred and fifty-two university undergraduates took a test of English structure first in constructed-response format, and later in three, stem-equivalent…

A participatory learning approach to biochemistry using student authored and evaluated multiple-choice questions.

PubMed

Bottomley, Steven; Denny, Paul

2011-01-01

A participatory learning approach, combined with both a traditional and a competitive assessment, was used to motivate students and promote a deep approach to learning biochemistry. Students were challenged to research, author, and explain their own multiple-choice questions (MCQs). They were also required to answer, evaluate, and discuss MCQs written by their peers. The technology used to support this activity was PeerWise--a freely available, innovative web-based system that supports students in the creation of an annotated question repository. In this case study, we describe students' contributions to, and perceptions of, the PeerWise system for a cohort of 107 second-year biomedical science students from three degree streams studying a core biochemistry subject. Our study suggests that the students are eager participants and produce a large repository of relevant, good quality MCQs. In addition, they rate the PeerWise system highly and use higher order thinking skills while taking an active role in their learning. We also discuss potential issues and future work using PeerWise for biomedical students. Copyright © 2011 Wiley Periodicals, Inc.

Simultaneous modeling of visual saliency and value computation improves predictions of economic choice.

PubMed

Towal, R Blythe; Mormann, Milica; Koch, Christof

2013-10-01

Many decisions we make require visually identifying and evaluating numerous alternatives quickly. These usually vary in reward, or value, and in low-level visual properties, such as saliency. Both saliency and value influence the final decision. In particular, saliency affects fixation locations and durations, which are predictive of choices. However, it is unknown how saliency propagates to the final decision. Moreover, the relative influence of saliency and value is unclear. Here we address these questions with an integrated model that combines a perceptual decision process about where and when to look with an economic decision process about what to choose. The perceptual decision process is modeled as a drift-diffusion model (DDM) process for each alternative. Using psychophysical data from a multiple-alternative, forced-choice task, in which subjects have to pick one food item from a crowded display via eye movements, we test four models where each DDM process is driven by (i) saliency or (ii) value alone or (iii) an additive or (iv) a multiplicative combination of both. We find that models including both saliency and value weighted in a one-third to two-thirds ratio (saliency-to-value) significantly outperform models based on either quantity alone. These eye fixation patterns modulate an economic decision process, also described as a DDM process driven by value. Our combined model quantitatively explains fixation patterns and choices with similar or better accuracy than previous models, suggesting that visual saliency has a smaller, but significant, influence than value and that saliency affects choices indirectly through perceptual decisions that modulate economic decisions.

Simultaneous modeling of visual saliency and value computation improves predictions of economic choice

PubMed Central

Towal, R. Blythe; Mormann, Milica; Koch, Christof

2013-01-01

Many decisions we make require visually identifying and evaluating numerous alternatives quickly. These usually vary in reward, or value, and in low-level visual properties, such as saliency. Both saliency and value influence the final decision. In particular, saliency affects fixation locations and durations, which are predictive of choices. However, it is unknown how saliency propagates to the final decision. Moreover, the relative influence of saliency and value is unclear. Here we address these questions with an integrated model that combines a perceptual decision process about where and when to look with an economic decision process about what to choose. The perceptual decision process is modeled as a drift–diffusion model (DDM) process for each alternative. Using psychophysical data from a multiple-alternative, forced-choice task, in which subjects have to pick one food item from a crowded display via eye movements, we test four models where each DDM process is driven by (i) saliency or (ii) value alone or (iii) an additive or (iv) a multiplicative combination of both. We find that models including both saliency and value weighted in a one-third to two-thirds ratio (saliency-to-value) significantly outperform models based on either quantity alone. These eye fixation patterns modulate an economic decision process, also described as a DDM process driven by value. Our combined model quantitatively explains fixation patterns and choices with similar or better accuracy than previous models, suggesting that visual saliency has a smaller, but significant, influence than value and that saliency affects choices indirectly through perceptual decisions that modulate economic decisions. PMID:24019496

Hiring for smarts.

PubMed

Menkes, Justin

2005-11-01

Yes, it's nice when a leader is charismatic and confident. And a great resume can tell you a lot about a person's knowledge and experience. But such assets are no substitute for sheer business intelligence, and they reveal very little about a leader's ability to consistently reach the "right" answer. How can hiring managers flag individuals with such smarts? Historically, the only reliable measure of brainpower has been the standard IQ test, which is rarely used in business settings because of the specific subjects it tests for-math, reading, and spatial reasoning-and because of its multiple-choice format. Despite its shortcomings, the standard IQ test is still a better predictor of managerial success than any other assessment tool companies currently use, Justin Menkes argues. It's true that there isn't a version of IQ testing that applies to the corporate world, but in rejecting IQ tests altogether, hiring managers have thwarted their own attempts to identify true business stars. The author defines the specific subjects that make up "executive intelligence"-namely, accomplishing tasks, working with people, and judging oneself. He describes how to formulate questions to test job candidates for their mastery of these subjects, offering several examples based on real situations. Knowledge questions, such as those used in standard behavioral interviews, require people to recite what they have learned or experienced; intelligence questions call for individuals to demonstrate their abilities. Therefore, the questions in an executive intelligence test shouldn't require specific industry expertise or experience; any knowledge they call for must be rudimentary and common to all executives. And the questions should not be designed to ask whether the candidate has a particular skill; they should be configured so that the candidate will have to demonstrate that skill in the course of answering them.

The effects of computer simulation versus hands-on dissection and the placement of computer simulation within the learning cycle on student achievement and attitude

NASA Astrophysics Data System (ADS)

Hopkins, Kathryn Susan

The value of dissection as an instructional strategy has been debated, but not evidenced in research literature. The purpose of this study was to examine the efficacy of using computer simulated frog dissection as a substitute for traditional hands-on frog dissection and to examine the possible enhancement of achievement by combining the two strategies in a specific sequence. In this study, 134 biology students at two Central Texas schools were divided into the five following treatment groups: computer simulation of frog dissection, computer simulation before dissection, traditional hands-on frog dissection, dissection before computer simulation, and textual worksheet materials. The effects on achievement were evaluated by labeling 10 structures on three diagrams, identifying 11 pinned structures on a prosected frog, and answering 9 multiple-choice questions over the dissection process. Attitude was evaluated using a thirty item survey with a five-point Likert scale. The quasi-experimental design was pretest/post-test/post-test nonequivalent group for both control and experimental groups, a 2 x 2 x 5 completely randomized factorial design (gender, school, five treatments). The pretest/post-test design was incorporated to control for prior knowledge using analysis of covariance. The dissection only group evidenced a significantly higher performance than all other treatments except dissection-then-computer on the post-test segment requiring students to label pinned anatomical parts on a prosected frog. Interactions between treatment and school in addition to interaction between treatment and gender were found to be significant. The diagram and attitude post-tests evidenced no significant difference. Results on the nine multiple-choice questions about dissection procedures indicated a significant difference between schools. The interaction between treatment and school was also found to be significant. On a delayed post-test, a significant difference in gender was found on the diagram labeling segment of the post-test. Males were reported to have the higher score. Since existing research conflicts with this study's results, additional research using authentic assessment is recommended. Instruction should be aligned with dissection content and process objectives for each treatment group, and the teacher variable should be controlled.

Computer grading of examinations

NASA Technical Reports Server (NTRS)

Frigerio, N. A.

1969-01-01

A method, using IBM cards and computer processing, automates examination grading and recording and permits use of computational problems. The student generates his own answers, and the instructor has much greater freedom in writing questions than is possible with multiple choice examinations.

Incorporation of core competency questions into an annual national self-assessment examination for residents in physical medicine and rehabilitation: results and implications.

PubMed

Webster, Joseph B

2009-03-01

To determine the performance and change over time when incorporating questions in the core competency domains of practice-based learning and improvement (PBLI), systems-based practice (SBP), and professionalism (PROF) into the national PM&R Self-Assessment Examination for Residents (SAER). Prospective, longitudinal analysis. The national Self-Assessment Examination for Residents (SAER) in Physical Medicine and Rehabilitation, which is administered annually. Approximately 1100 PM&R residents who take the examination annually. Inclusion of progressively more challenging questions in the core competency domains of PBLI, SBP, and PROF. Individual test item level of difficulty (P value) and discrimination (point biserial index). Compared with the overall test, questions in the subtopic areas of PBLI, SBP, and PROF were relatively easier and less discriminating (correlation of resident performance on these domains compared with that on the total test). These differences became smaller during the 3-year time period. The difficulty level of the questions in each of the subtopic domains was raised during the 3 year period to a level close to the overall exam. Discrimination of the test items improved or remained stable. This study demonstrates that, with careful item writing and review, multiple-choice items in the PBLI, SBP, and PROF domains can be successfully incorporated into an annual, national self-assessment examination for residents. The addition of these questions had value in assessing competency while not compromising the overall validity and reliability of the exam. It is yet to be determined if resident performance on these questions corresponds to performance on other measures of competency in the areas of PBLI, SBP, and PROF.

Chemistry 30: Grade 12 Diploma Examination = Chimie 30: Examen en vue du diplome 12 annee.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton. Student Evaluation Branch.

This document, in both French and English versions, is the Chemistry 30 Grade 12 Diploma Examination from Alberta Education. It is a 2.5 hour closed-book examination consisting of 44 multiple-choice and 12 numerical-response questions of equal value that are worth 70% of the examination, and 2 written-response questions of equal value worth 30% of…

Biology 30: Grade 12 Diploma Examination = Biologie 30: Examen en vue du diplome 12 annee.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton. Student Evaluation Branch.

This document, in both English and French versions, is the Biology 30 Grade 12 Diploma Examination from Alberta Education. It is a 2.5 hour closed-book examination consisting of 48 multiple-choice and 8 numerical-response questions of equal value that are worth 70% of the examination, and 2 written-response questions of equal value worth 30% of…

Using Student Writing and Lexical Analysis to Reveal Student Thinking about the Role of Stop Codons in the Central Dogma

ERIC Educational Resources Information Center

Prevost, Luanna B.; Smith, Michelle K.; Knight, Jennifer K.

2016-01-01

Previous work has shown that students have persistent difficulties in understanding how central dogma processes can be affected by a stop codon mutation. To explore these difficulties, we modified two multiple-choice questions from the Genetics Concept Assessment into three open-ended questions that asked students to write about how a stop codon…

Physics 30: Grade 12 Diploma Examination = Physique 30: Examen en vue du diplome 12 annee.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton. Student Evaluation Branch.

This document, in both English and French versions, is the Physics 30 Grade 12 Diploma Examination from Alberta Education. It is a 2.5 hour closed-book examination consisting of 37 multiple-choice and 12 numerical-response questions of equal value that are worth 70% of the examination, and 2 written-response questions of equal value worth 30% of…

Impact of interdisciplinary learning on critical thinking using case study method in allied health care graduate students.

PubMed

Zimmerman, Scott D; Lester Short, Glenda F; Hendrix, E Michael

2011-01-01

It remains unclear which classroom experiences, if any, foster critical think ability. We measured the effectiveness of interdisciplinary, case-based learning on the critical-thinking ability of graduate students enrolled in allied health care programs. We designed a voluntary classroom experience to examine the effectiveness of case studies used in an interdisciplinary setting to increase critical-thinking ability. Two groups of students were measured for their critical thinking ability using an online assessment both before and after their respective classroom experiences. One group of 14 graduate students from 4 different allied health care programs (interdisciplinary, ID) discussed complex interdisciplinary case studies and answered multiple-choice type questions formed around the cases. The second group was composed of graduate students (n = 28) from a single disciple enrolled in a clinical anatomy course (discipline specific, DS). They discussed complex case studies specific to their discipline and answered multiple-choice questions formed around the cases. There was no overall change in critical-thinking scores from the pre- to post-test in either group (delta scores: ID 1.5 ± 5.3, DS -1.7 ± 5.7). However, ID students scoring below the median on the pretest improved significantly (paired t-test, pre 50.7 ± 3.8, post 54.2 ± 1.7, p = 0.02). The interdisciplinary learning experience improved critical-thinking ability in students with the least proficiency. As case studies have long been used to advance deeper learning, these data provide evidence for a broader impact of cases when used in an interdisciplinary setting, especially for those students coming in with the least ability.

Comparison in the quality of distractors in three and four options type of multiple choice questions

PubMed Central

Rahma, Nourelhouda A A; Shamad, Mahdi M A; Idris, Muawia E A; Elfaki, Omer Abdelgadir; Elfakey, Walyedldin E M; Salih, Karimeldin M A

2017-01-01

Introduction The number of distractors needed for high quality multiple choice questions (MCQs) will be determined by many factors. These include firstly whether English language is their mother tongue or a foreign language; secondly whether the instructors who construct the questions are experts or not; thirdly the time spent on constructing the options is also an important factor. It has been observed by Tarrant et al that more time is often spent on constructing questions than on tailoring sound, reliable, and valid distractors. Objectives Firstly, to investigate the effects of reducing the number of options on psychometric properties of the item. Secondly, to determine the frequency of functioning distractors among three or four options in the MCQs examination of the dermatology course in University of Bahri, College of Medicine. Materials and methods This is an experimental study which was performed by means of a dermatology exam, MCQs type. Forty MCQs, with one correct answer for each question were constructed. Two sets of this exam paper were prepared: in the first one, four options were given, including one key answer and three distractors. In the second set, one of the three distractors was deleted randomly, and the sequence of the questions was kept in the same order. Any distracter chosen by less than 5% of the students was regarded as non-functioning. Kuder-Richardson Formula 20 (Kr-20) measures the internal consistency and reliability of an examination with an acceptable range 0.8–1.0. Chi square test was used to compare the distractors in the two exams. Results A significant difference was observed in discrimination and difficulty indexes for both sets of MCQs. More distractors were non-functional for set one (of four options), but slightly more reliable. The reliability (Kr-20) was slightly higher for set one (of four options). The average marks in option three and four were 34.163 and 33.140, respectively. Conclusion Compared to set 1 (four options), set 2 (of three options) was more discriminating and associated with low difficulty index but its reliability was low. PMID:28442942

Comparison in the quality of distractors in three and four options type of multiple choice questions.

PubMed

Rahma, Nourelhouda A A; Shamad, Mahdi M A; Idris, Muawia E A; Elfaki, Omer Abdelgadir; Elfakey, Walyedldin E M; Salih, Karimeldin M A

2017-01-01

The number of distractors needed for high quality multiple choice questions (MCQs) will be determined by many factors. These include firstly whether English language is their mother tongue or a foreign language; secondly whether the instructors who construct the questions are experts or not; thirdly the time spent on constructing the options is also an important factor. It has been observed by Tarrant et al that more time is often spent on constructing questions than on tailoring sound, reliable, and valid distractors. Firstly, to investigate the effects of reducing the number of options on psychometric properties of the item. Secondly, to determine the frequency of functioning distractors among three or four options in the MCQs examination of the dermatology course in University of Bahri, College of Medicine. This is an experimental study which was performed by means of a dermatology exam, MCQs type. Forty MCQs, with one correct answer for each question were constructed. Two sets of this exam paper were prepared: in the first one, four options were given, including one key answer and three distractors. In the second set, one of the three distractors was deleted randomly, and the sequence of the questions was kept in the same order. Any distracter chosen by less than 5% of the students was regarded as non-functioning. Kuder-Richardson Formula 20 (Kr-20) measures the internal consistency and reliability of an examination with an acceptable range 0.8-1.0. Chi square test was used to compare the distractors in the two exams. A significant difference was observed in discrimination and difficulty indexes for both sets of MCQs. More distractors were non-functional for set one (of four options), but slightly more reliable. The reliability (Kr-20) was slightly higher for set one (of four options). The average marks in option three and four were 34.163 and 33.140, respectively. Compared to set 1 (four options), set 2 (of three options) was more discriminating and associated with low difficulty index but its reliability was low.

Assessing Understanding of the Learning Cycle: The ULC

NASA Astrophysics Data System (ADS)

Marek, Edmund A.; Maier, Steven J.; McCann, Florence

2008-08-01

An 18-item, multiple choice, 2-tiered instrument designed to measure understanding of the learning cycle (ULC) was developed and field-tested from the learning cycle test (LCT) of Odom and Settlage ( Journal of Science Teacher Education, 7, 123 142, 1996). All question sets of the LCT were modified to some degree and 5 new sets were added, resulting in the ULC. The ULC measures (a) understandings and misunderstandings of the learning cycle, (b) the learning cycle’s association with Piaget’s ( Biology and knowledge theory: An essay on the relations between organic regulations and cognitive processes, 1975) theory of mental functioning, and (c) applications of the learning cycle. The resulting ULC instrument was evaluated for internal consistency with Cronbach’s alpha, yielding a coefficient of .791.

Development and Evaluation of a Pharmacogenomics Educational Program for Pharmacists

PubMed Central

Formea, Christine M.; Nicholson, Wayne T.; McCullough, Kristen B.; Berg, Kevin D.; Berg, Melody L.; Cunningham, Julie L.; Merten, Julianna A.; Ou, Narith N.; Stollings, Joanna L.

2013-01-01

Objectives. To evaluate hospital and outpatient pharmacists’ pharmacogenomics knowledge before and 2 months after participating in a targeted, case-based pharmacogenomics continuing education program. Design. As part of a continuing education program accredited by the Accreditation Council for Pharmacy Education (ACPE), pharmacists were provided with a fundamental pharmacogenomics education program. Evaluation. An 11-question, multiple-choice, electronic survey instrument was distributed to 272 eligible pharmacists at a single campus of a large, academic healthcare system. Pharmacists improved their pharmacogenomics test scores by 0.7 questions (pretest average 46%; posttest average 53%, p=0.0003). Conclusions. Although pharmacists demonstrated improvement, overall retention of educational goals and objectives was marginal. These results suggest that the complex topic of pharmacogenomics requires a large educational effort in order to increase pharmacists’ knowledge and comfort level with this emerging therapeutic opportunity. PMID:23459098

Development and evaluation of a pharmacogenomics educational program for pharmacists.

PubMed

Formea, Christine M; Nicholson, Wayne T; McCullough, Kristen B; Berg, Kevin D; Berg, Melody L; Cunningham, Julie L; Merten, Julianna A; Ou, Narith N; Stollings, Joanna L

2013-02-12

Objectives. To evaluate hospital and outpatient pharmacists' pharmacogenomics knowledge before and 2 months after participating in a targeted, case-based pharmacogenomics continuing education program.Design. As part of a continuing education program accredited by the Accreditation Council for Pharmacy Education (ACPE), pharmacists were provided with a fundamental pharmacogenomics education program.Evaluation. An 11-question, multiple-choice, electronic survey instrument was distributed to 272 eligible pharmacists at a single campus of a large, academic healthcare system. Pharmacists improved their pharmacogenomics test scores by 0.7 questions (pretest average 46%; posttest average 53%, p=0.0003).Conclusions. Although pharmacists demonstrated improvement, overall retention of educational goals and objectives was marginal. These results suggest that the complex topic of pharmacogenomics requires a large educational effort in order to increase pharmacists' knowledge and comfort level with this emerging therapeutic opportunity.

Multiple-Choice Testing Using Immediate Feedback--Assessment Technique (IF AT®) Forms: Second-Chance Guessing vs. Second-Chance Learning?

ERIC Educational Resources Information Center

Merrel, Jeremy D.; Cirillo, Pier F.; Schwartz, Pauline M.; Webb, Jeffrey A.

2015-01-01

Multiple choice testing is a common but often ineffective method for evaluating learning. A newer approach, however, using Immediate Feedback Assessment Technique (IF AT®, Epstein Educational Enterprise, Inc.) forms, offers several advantages. In particular, a student learns immediately if his or her answer is correct and, in the case of an…

Does Linking Mixed-Format Tests Using a Multiple-Choice Anchor Produce Comparable Results for Male and Female Subgroups? Research Report. ETS RR-11-44

ERIC Educational Resources Information Center

Kim, Sooyeon; Walker, Michael E.

2011-01-01

This study examines the use of subpopulation invariance indices to evaluate the appropriateness of using a multiple-choice (MC) item anchor in mixed-format tests, which include both MC and constructed-response (CR) items. Linking functions were derived in the nonequivalent groups with anchor test (NEAT) design using an MC-only anchor set for 4…

Does Self-Assessment Improve the Effectiveness of Grand Rounds Lectures in a Community-Based Teaching Hospital?

PubMed

Winton, Lisa M; Ferguson, Elizabeth M N; Hsu, Chiu-Hsieh; Agee, Neal; Eubanks, Ryan D; O'Neill, Patrick J; Goldberg, Ross F; Kopelman, Tammy R; Nodora, Jesse N; Caruso, Daniel M; Komenaka, Ian K

To determine whether use of self-assessment (SA) questions affects the effectiveness of weekly didactic grand rounds presentations. From 26 consecutive grand rounds presentations from August 2013 to April 2014, a 52-question multiple-choice test was administered based on 2 questions from each presentation. Community teaching institution. General surgery residents, students, and attending physicians. The test was administered to 66 participants. The mean score was 41.8%. There was no difference in test score based on experience with similar scores for junior residents, senior residents, and attending surgeons (43%, 46%, and 44%; p = 0.13). Most participants felt they would be most interested in presentations directly related to their surgical specialty. Participants, however, did not score differently on topics which were the focus of the program (40% vs. 42%; p = 0.85). Journal club presentations (39% vs. others 42%; p = 0.33) also did not affect the score. The Pearson correlation coefficient for attendance was 0.49 (p < 0.0001) demonstrated that attendance was very important. Participation in the weekly SA was significantly associated with improved score as those who participated in SA scored over 20% higher than those who did not (59% vs. 38%; p < 0.0001). Based on multiple linear regression for mean score, SA explained the variation in score more than attendance. The current study found that without preparation approximately 40% of material presented is retained after 10 months. Participation in weekly SA significantly improved retention of information from grand rounds presentations. Copyright © 2016 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Accommodations for Multiple Choice Tests

ERIC Educational Resources Information Center

Trammell, Jack

2011-01-01

Students with learning or learning-related disabilities frequently struggle with multiple choice assessments due to difficulty discriminating between items, filtering out distracters, and framing a mental best answer. This Practice Brief suggests accommodations and strategies that disability service providers can utilize in conjunction with…

Assessment of numeracy in sports and exercise science students at an Australian university

NASA Astrophysics Data System (ADS)

Green, Simon; McGlynn, Susan; Stuart, Deidre; Fahey, Paul; Pettigrew, Jim; Clothier, Peter

2018-05-01

The effect of high school study of mathematics on numeracy performance of sports and exercise science (SES) students is not clear. To investigate this further, we tested the numeracy skills of 401 students enrolled in a Bachelor of Health Sciences degree in SES using a multiple-choice survey consisting of four background questions and 39 numeracy test questions. Background questions (5-point scale) focused on highest level of mathematics studied at high school, self-perception of mathematics proficiency, perceived importance of mathematics to SES and likelihood of seeking help with mathematics. Numeracy questions focused on rational number, ratios and rates, basic algebra and graph interpretation. Numeracy performance was based on answers to these questions (1 mark each) and represented by the total score (maximum = 39). Students from first (n = 212), second (n = 78) and third (n = 111) years of the SES degree completed the test. The distribution of numeracy test scores for the entire cohort was negatively skewed with a median (IQR) score of 27(11). We observed statistically significant associations between test scores and the highest level of mathematics studied (P < 0.05), being lowest in students who studied Year 10 Mathematics (20 (9)), intermediate in students who studied Year 12 General Mathematics (26 (8)) and highest in two groups of students who studied higher-level Year 12 Mathematics (31 (9), 31 (6)). There were statistically significant associations between test scores and level of self-perception of mathematics proficiency and also likelihood of seeking help with mathematics (P < 0.05) but not with perceived importance of mathematics to SES. These findings reveal that the level of mathematics studied in high school is a critical factor determining the level of numeracy performance in SES students.

Multiple-choice examinations: adopting an evidence-based approach to exam technique.

PubMed

Hammond, E J; McIndoe, A K; Sansome, A J; Spargo, P M

1998-11-01

Negatively marked multiple-choice questions (MCQs) are part of the assessment process in both the Primary and Final examinations for the fellowship of the Royal College of Anaesthetists. It is said that candidates who guess will lose marks in the MCQ paper. We studied candidates attending a pre-examination revision course and have shown that an evaluation of examination technique is an important part of an individual's preparation. All candidates benefited substantially from backing their educated guesses while only 3 out of 27 lost marks from backing their wild guesses. Failure to appreciate the relationship between knowledge and technique may significantly affect a candidate's performance in the examination.

The Integrating and Segregating Effects of School Choice

ERIC Educational Resources Information Center

Koedel, Cory; Betts, Julian R.; Rice, Lorien A.; Zau, Andrew C.

2009-01-01

We evaluate the integrating and segregating effects of three distinct school choice programs in San Diego. We go beyond the traditional question of racial integration and examine the integration of students by test scores, parental education levels, and language status. In addition to measuring the net integrative effects of school choice, we also…

The Role of Empirical Research in Informing Debates about the Constitutionality of School Choice

ERIC Educational Resources Information Center

Saiger, Aaron

2006-01-01

Federal constitutional law currently permits choice programs that include religious schools only if they provide parents with "genuine and independent choice"; as the leading federal case demonstrates, whether this test is satisfied is an interesting and difficult empirical question. State doctrine regarding establishment of religion can be…

Questioning English Standards: Learner Attitudes and L2 Choices in Norway

ERIC Educational Resources Information Center

Rindal, Ulrikke

2014-01-01

This study investigates attitudes towards varieties of English among Norwegian adolescent learners and assesses the role of social evaluation for second language (L2) pronunciation choices by combining a verbal guise test with speaker commentary and reports of language choices. The results suggest that while American English is the most accessible…

Reducing the number of options on multiple-choice questions: response time, psychometrics and standard setting.

PubMed

Schneid, Stephen D; Armour, Chris; Park, Yoon Soo; Yudkowsky, Rachel; Bordage, Georges

2014-10-01

Despite significant evidence supporting the use of three-option multiple-choice questions (MCQs), these are rarely used in written examinations for health professions students. The purpose of this study was to examine the effects of reducing four- and five-option MCQs to three-option MCQs on response times, psychometric characteristics, and absolute standard setting judgements in a pharmacology examination administered to health professions students. We administered two versions of a computerised examination containing 98 MCQs to 38 Year 2 medical students and 39 Year 3 pharmacy students. Four- and five-option MCQs were converted into three-option MCQs to create two versions of the examination. Differences in response time, item difficulty and discrimination, and reliability were evaluated. Medical and pharmacy faculty judges provided three-level Angoff (TLA) ratings for all MCQs for both versions of the examination to allow the assessment of differences in cut scores. Students answered three-option MCQs an average of 5 seconds faster than they answered four- and five-option MCQs (36 seconds versus 41 seconds; p = 0.008). There were no significant differences in item difficulty and discrimination, or test reliability. Overall, the cut scores generated for three-option MCQs using the TLA ratings were 8 percentage points higher (p = 0.04). The use of three-option MCQs in a health professions examination resulted in a time saving equivalent to the completion of 16% more MCQs per 1-hour testing period, which may increase content validity and test score reliability, and minimise construct under-representation. The higher cut scores may result in higher failure rates if an absolute standard setting method, such as the TLA method, is used. The results from this study provide a cautious indication to health professions educators that using three-option MCQs does not threaten validity and may strengthen it by allowing additional MCQs to be tested in a fixed amount of testing time with no deleterious effect on the reliability of the test scores. © 2014 John Wiley & Sons Ltd.

A Randomized Educational Intervention Trial to Determine the Effect of Online Education on the Quality of Resident-Delivered Care.

PubMed

Dolan, Brigid M; Yialamas, Maria A; McMahon, Graham T

2015-09-01

There is limited research on whether online formative self-assessment and learning can change the behavior of medical professionals. We sought to determine if an adaptive longitudinal online curriculum in bone health would improve resident physicians' knowledge, and change their behavior regarding prevention of fragility fractures in women. We used a randomized control trial design in which 50 internal medicine resident physicians at a large academic practice were randomized to either receive a standard curriculum in bone health care alone, or to receive it augmented with an adaptive, longitudinal, online formative self-assessment curriculum delivered via multiple-choice questions. Outcomes were assessed 10 months after the start of the intervention. Knowledge outcomes were measured by a multiple-choice question examination. Clinical outcomes were measured by chart review, including bone density screening rate, calculation of the fracture risk assessment tool (FRAX) score, and rate of appropriate bisphosphonate prescription. Compared to the control group, residents participating in the intervention had higher scores on the knowledge test at the end of the study. Bone density screening rates and appropriate use of bisphosphonates were significantly higher in the intervention group compared with the control group. FRAX score reporting did not differ between the groups. Residents participating in a novel adaptive online curriculum outperformed peers in knowledge of fragility fracture prevention and care practices to prevent fracture. Online adaptive education can change behavior to improve patient care.

A Randomized Educational Intervention Trial to Determine the Effect of Online Education on the Quality of Resident-Delivered Care

PubMed Central

Dolan, Brigid M.; Yialamas, Maria A.; McMahon, Graham T.

2015-01-01

Background There is limited research on whether online formative self-assessment and learning can change the behavior of medical professionals. Objective We sought to determine if an adaptive longitudinal online curriculum in bone health would improve resident physicians' knowledge, and change their behavior regarding prevention of fragility fractures in women. Methods We used a randomized control trial design in which 50 internal medicine resident physicians at a large academic practice were randomized to either receive a standard curriculum in bone health care alone, or to receive it augmented with an adaptive, longitudinal, online formative self-assessment curriculum delivered via multiple-choice questions. Outcomes were assessed 10 months after the start of the intervention. Knowledge outcomes were measured by a multiple-choice question examination. Clinical outcomes were measured by chart review, including bone density screening rate, calculation of the fracture risk assessment tool (FRAX) score, and rate of appropriate bisphosphonate prescription. Results Compared to the control group, residents participating in the intervention had higher scores on the knowledge test at the end of the study. Bone density screening rates and appropriate use of bisphosphonates were significantly higher in the intervention group compared with the control group. FRAX score reporting did not differ between the groups. Conclusions Residents participating in a novel adaptive online curriculum outperformed peers in knowledge of fragility fracture prevention and care practices to prevent fracture. Online adaptive education can change behavior to improve patient care. PMID:26457142

Two layers LSTM with attention for multi-choice question answering in exams

NASA Astrophysics Data System (ADS)

Li, Yongbin

2018-03-01

Question Answering in Exams is typical question answering task that aims to test how accurately the model could answer the questions in exams. In this paper, we use general deep learning model to solve the multi-choice question answering task. Our approach is to build distributed word embedding of question and answers instead of manually extracting features or linguistic tools, meanwhile, for improving the accuracy, the external corpus is introduced. The framework uses a two layers LSTM with attention which get a significant result. By contrast, we introduce the simple long short-term memory (QA-LSTM) model and QA-LSTM-CNN model and QA-LSTM with attention model as the reference. Experiment demonstrate superior performance of two layers LSTM with attention compared to other models in question answering task.

Differences in gender performance on competitive physics selection tests

NASA Astrophysics Data System (ADS)

Wilson, Kate; Low, David; Verdon, Matthew; Verdon, Alix

2016-12-01

[This paper is part of the Focused Collection on Gender in Physics.] We have investigated gender differences in performance over the past eight years on the Australian Science Olympiad Exam (ASOE) for physics, which is taken by nearly 1000 high school students each year. The ASOE, run by Australian Science Innovations (ASI), is the initial stage of the process of selection of teams to represent Australia at the Asian and International Physics Olympiads. Students taking the exam are generally in their penultimate year of school and selected by teachers as being high performing in physics. Together with the overall differences in facility, we have investigated how the content and presentation of multiple-choice questions (MCQs) affects the particular answers selected by male and female students. Differences in the patterns of responses by male and female students indicate that males and females might be modeling situations in different ways. Some strong patterns were found in the gender gaps when the questions were categorized in five broad dimensions: content, process required, difficulty, presentation, and context. Almost all questions saw male students performing better, although gender differences were relatively small for questions with a more abstract context. Male students performed significantly better on most questions with a concrete context, although notable exceptions were found, including two such questions where female students performed better. Other categories that showed consistently large gaps favoring male students include questions with projectile motion and other two-dimensional motion or forces content, and processes involving interpreting diagrams. Our results have important implications, suggesting that we should be able to reduce the gender gaps in performance on MCQ tests by changing the way information is presented and setting questions in contexts that are less likely to favor males over females. This is important as MCQ tests are frequently used as diagnostic tests and aptitude tests as well as to assess learning.

Expanding the FCI to Eevaluate Conceptual Mastery of Energy, Momentum, and Rotational Dynamics

NASA Astrophysics Data System (ADS)

Chediak, Alex; Hay, Katrina

2010-03-01

Normalized gain on the Force Concept Inventory (FCI) has deservedly become a widely accepted assessment tool to evaluate conceptual mastery in a high school, college, or university-level mechanics course. Left out of this assessment, however, are important physics concepts typically presented in the same course. Conservation of energy and momentum as well as rotational motion receive scant (if any) coverage on the FCI (or, for that matter, the Mechanics Baseline Test). Yet these concepts are foundational for popular majors such as mechanical engineering, where high failure rates are often a concern. A revised assessment tool is presented, one that incorporates the strengths of the FCI (and preserves the straightforward multiple choice format), but assesses these other mechanics-related concepts. Ten additional questions are included, inspired in part by material from the Physics Education Group at the University of Washington and in part by the authors' own experiences with common student misperceptions. The questions are given as pre- and post tests at the authors' institutions, California Baptist University and Pacific Lutheran University, in both calculus-based and algebra-based mechanics courses, exploring breadth of applicability for our findings. We present normalized gain data for the traditional thirty FCI questions and for our ten additional questions.

Development of multiple choice pictorial test for measuring the dimensions of knowledge

NASA Astrophysics Data System (ADS)

Nahadi, Siswaningsih, Wiwi; Erna

2017-05-01

This study aims to develop a multiple choice pictorial test as a tool to measure dimension of knowledge in chemical equilibrium subject. The method used is Research and Development and validation that was conducted in the preliminary studies and model development. The product is multiple choice pictorial test. The test was developed by 22 items and tested to 64 high school students in XII grade. The quality of test was determined by value of validity, reliability, difficulty index, discrimination power, and distractor effectiveness. The validity of test was determined by CVR calculation using 8 validators (4 university teachers and 4 high school teachers) with average CVR value 0,89. The reliability of test has very high category with value 0,87. Discrimination power of items with a very good category is 32%, 59% as good category, and 20% as sufficient category. This test has a varying level of difficulty, item with difficult category is 23%, the medium category is 50%, and the easy category is 27%. The distractor effectiveness of items with a very poor category is 1%, poor category is 1%, medium category is 4%, good category is 39%, and very good category is 55%. The dimension of knowledge that was measured consist of factual knowledge, conceptual knowledge, and procedural knowledge. Based on the questionnaire, students responded quite well to the developed test and most of the students like this kind of multiple choice pictorial test that include picture as evaluation tool compared to the naration tests was dominated by text.

Performance on large-scale science tests: Item attributes that may impact achievement scores

NASA Astrophysics Data System (ADS)

Gordon, Janet Victoria

Significant differences in achievement among ethnic groups persist on the eighth-grade science Washington Assessment of Student Learning (WASL). The WASL measures academic performance in science using both scenario and stand-alone question types. Previous research suggests that presenting target items connected to an authentic context, like scenario question types, can increase science achievement scores especially in underrepresented groups and thus help to close the achievement gap. The purpose of this study was to identify significant differences in performance between gender and ethnic subgroups by question type on the 2005 eighth-grade science WASL. MANOVA and ANOVA were used to examine relationships between gender and ethnic subgroups as independent variables with achievement scores on scenario and stand-alone question types as dependent variables. MANOVA revealed no significant effects for gender, suggesting that the 2005 eighth-grade science WASL was gender neutral. However, there were significant effects for ethnicity. ANOVA revealed significant effects for ethnicity and ethnicity by gender interaction in both question types. Effect sizes were negligible for the ethnicity by gender interaction. Large effect sizes between ethnicities on scenario question types became moderate to small effect sizes on stand-alone question types. This indicates the score advantage the higher performing subgroups had over the lower performing subgroups was not as large on stand-alone question types compared to scenario question types. A further comparison examined performance on multiple-choice items only within both question types. Similar achievement patterns between ethnicities emerged; however, achievement patterns between genders changed in boys' favor. Scenario question types appeared to register differences between ethnic groups to a greater degree than stand-alone question types. These differences may be attributable to individual differences in cognition, characteristics of test items themselves and/or opportunities to learn. Suggestions for future research are made.

Pick-N Multiple Choice-Exams: A Comparison of Scoring Algorithms

ERIC Educational Resources Information Center

Bauer, Daniel; Holzer, Matthias; Kopp, Veronika; Fischer, Martin R.

2011-01-01

To compare different scoring algorithms for Pick-N multiple correct answer multiple-choice (MC) exams regarding test reliability, student performance, total item discrimination and item difficulty. Data from six 3rd year medical students' end of term exams in internal medicine from 2005 to 2008 at Munich University were analysed (1,255 students,…

Dynamic Testing of Analogical Reasoning in 5- to 6-Year-Olds: Multiple-Choice versus Constructed-Response Training Items

ERIC Educational Resources Information Center

Stevenson, Claire E.; Heiser, Willem J.; Resing, Wilma C. M.

2016-01-01

Multiple-choice (MC) analogy items are often used in cognitive assessment. However, in dynamic testing, where the aim is to provide insight into potential for learning and the learning process, constructed-response (CR) items may be of benefit. This study investigated whether training with CR or MC items leads to differences in the strategy…

Methods & Strategies: Deep Assessment

ERIC Educational Resources Information Center

Haas, Alison; Hollimon, Shameka; Lee, Okhee

2015-01-01

The "Next Generation Science Standards" ("NGSS") push students to have "a deeper understanding of content" (NGSS Lead States 2013, Appendix A, p. 4). However, with the reality of high-stakes assessments that rely primarily on multiple-choice questions, how can a science teacher analyze students' written responses…

Immediate detailed feedback to test-enhanced learning: an effective online educational tool.

PubMed

Wojcikowski, Ken; Kirk, Leslie

2013-11-01

Test-enhanced learning has gained popularity because it is an effective way to increase retention of knowledge; provided the student receives the correct answer soon after the test is taken. To determine whether detailed feedback provided to test-enhanced learning questions is an effective online educational tool for improving performance on complex biomedical information exams. A series of online multiple choice tests were developed to test knowledge of biomedical information that students were expected to know after each patient-case. Following submission of the student answers, one cohort (n = 52) received answers only while the following year, a second cohort (n = 51) received the answers with detailed feedback explaining why each answer was correct or incorrect. Students in both groups progressed through the series of online tests with little assessor intervention. Students receiving the answers along with the explanations within their feedback performed significantly better in the final biomedical information exam than those students receiving correct answers only. This pilot study found that the detailed feedback to test-enhanced learning questions is an important online learning tool. The increase in student performance in the complex biomedical information exam in this study suggests that detailed feedback should be investigated not only for increasing knowledge, but also be investigated for its effect on retention and application of knowledge.

Do Students Know What They Know and What They Don't Know? Using a Four-Tier Diagnostic Test to Assess the Nature of Students' Alternative Conceptions

ERIC Educational Resources Information Center

Caleon, Imelda S.; Subramaniam, R.

2010-01-01

This study reports on the development and application of a four-tier multiple-choice (4TMC) diagnostic instrument, which has not been reported in the literature. It is an enhanced version of the two-tier multiple-choice (2TMC) test. As in 2TMC tests, its answer and reason tiers measure students' content knowledge and explanatory knowledge,…

Development of a research ethics knowledge and analytical skills assessment tool.

PubMed

Taylor, Holly A; Kass, Nancy E; Ali, Joseph; Sisson, Stephen; Bertram, Amanda; Bhan, Anant

2012-04-01

The goal of this project was to develop and validate a new tool to evaluate learners' knowledge and skills related to research ethics. A core set of 50 questions from existing computer-based online teaching modules were identified, refined and supplemented to create a set of 74 multiple-choice, true/false and short answer questions. The questions were pilot-tested and item discrimination was calculated for each question. Poorly performing items were eliminated or refined. Two comparable assessment tools were created. These assessment tools were administered as a pre-test and post-test to a cohort of 58 Indian junior health research investigators before and after exposure to a new course on research ethics. Half of the investigators were exposed to the course online, the other half in person. Item discrimination was calculated for each question and Cronbach's α for each assessment tool. A final version of the assessment tool that incorporated the best questions from the pre-/post-test phase was used to assess retention of research ethics knowledge and skills 3 months after course delivery. The final version of the REKASA includes 41 items and had a Cronbach's α of 0.837. The results illustrate, in one sample of learners, the successful, systematic development and use of a knowledge and skills assessment tool in research ethics capable of not only measuring basic knowledge in research ethics and oversight but also assessing learners' ability to apply ethics knowledge to the analytical task of reasoning through research ethics cases, without reliance on essay or discussion-based examination. These promising preliminary findings should be confirmed with additional groups of learners.

Anatomical knowledge gain through a clay-modeling exercise compared to live and video observations.

PubMed

Kooloos, Jan G M; Schepens-Franke, Annelieke N; Bergman, Esther M; Donders, Rogier A R T; Vorstenbosch, Marc A T M

2014-01-01

Clay modeling is increasingly used as a teaching method other than dissection. The haptic experience during clay modeling is supposed to correspond to the learning effect of manipulations during exercises in the dissection room involving tissues and organs. We questioned this assumption in two pretest-post-test experiments. In these experiments, the learning effects of clay modeling were compared to either live observations (Experiment I) or video observations (Experiment II) of the clay-modeling exercise. The effects of learning were measured with multiple choice questions, extended matching questions, and recognition of structures on illustrations of cross-sections. Analysis of covariance with pretest scores as the covariate was used to elaborate the results. Experiment I showed a significantly higher post-test score for the observers, whereas Experiment II showed a significantly higher post-test score for the clay modelers. This study shows that (1) students who perform clay-modeling exercises show less gain in anatomical knowledge than students who attentively observe the same exercise being carried out and (2) performing a clay-modeling exercise is better in anatomical knowledge gain compared to the study of a video of the recorded exercise. The most important learning effect seems to be the engagement in the exercise, focusing attention and stimulating time on task. © 2014 American Association of Anatomists.

Cost analysis for computer supported multiple-choice paper examinations

PubMed Central

Mandel, Alexander; Hörnlein, Alexander; Ifland, Marianus; Lüneburg, Edeltraud; Deckert, Jürgen; Puppe, Frank

2011-01-01

Introduction: Multiple-choice-examinations are still fundamental for assessment in medical degree programs. In addition to content related research, the optimization of the technical procedure is an important question. Medical examiners face three options: paper-based examinations with or without computer support or completely electronic examinations. Critical aspects are the effort for formatting, the logistic effort during the actual examination, quality, promptness and effort of the correction, the time for making the documents available for inspection by the students, and the statistical analysis of the examination results. Methods: Since three semesters a computer program for input and formatting of MC-questions in medical and other paper-based examinations is used and continuously improved at Wuerzburg University. In the winter semester (WS) 2009/10 eleven, in the summer semester (SS) 2010 twelve and in WS 2010/11 thirteen medical examinations were accomplished with the program and automatically evaluated. For the last two semesters the remaining manual workload was recorded. Results: The cost of the formatting and the subsequent analysis including adjustments of the analysis of an average examination with about 140 participants and about 35 questions was 5-7 hours for exams without complications in the winter semester 2009/2010, about 2 hours in SS 2010 and about 1.5 hours in the winter semester 2010/11. Including exams with complications, the average time was about 3 hours per exam in SS 2010 and 2.67 hours for the WS 10/11. Discussion: For conventional multiple-choice exams the computer-based formatting and evaluation of paper-based exams offers a significant time reduction for lecturers in comparison with the manual correction of paper-based exams and compared to purely electronically conducted exams it needs a much simpler technological infrastructure and fewer staff during the exam. PMID:22205913

Cost analysis for computer supported multiple-choice paper examinations.

PubMed

Mandel, Alexander; Hörnlein, Alexander; Ifland, Marianus; Lüneburg, Edeltraud; Deckert, Jürgen; Puppe, Frank

2011-01-01

Multiple-choice-examinations are still fundamental for assessment in medical degree programs. In addition to content related research, the optimization of the technical procedure is an important question. Medical examiners face three options: paper-based examinations with or without computer support or completely electronic examinations. Critical aspects are the effort for formatting, the logistic effort during the actual examination, quality, promptness and effort of the correction, the time for making the documents available for inspection by the students, and the statistical analysis of the examination results. Since three semesters a computer program for input and formatting of MC-questions in medical and other paper-based examinations is used and continuously improved at Wuerzburg University. In the winter semester (WS) 2009/10 eleven, in the summer semester (SS) 2010 twelve and in WS 2010/11 thirteen medical examinations were accomplished with the program and automatically evaluated. For the last two semesters the remaining manual workload was recorded. The cost of the formatting and the subsequent analysis including adjustments of the analysis of an average examination with about 140 participants and about 35 questions was 5-7 hours for exams without complications in the winter semester 2009/2010, about 2 hours in SS 2010 and about 1.5 hours in the winter semester 2010/11. Including exams with complications, the average time was about 3 hours per exam in SS 2010 and 2.67 hours for the WS 10/11. For conventional multiple-choice exams the computer-based formatting and evaluation of paper-based exams offers a significant time reduction for lecturers in comparison with the manual correction of paper-based exams and compared to purely electronically conducted exams it needs a much simpler technological infrastructure and fewer staff during the exam.

Context effects and the temporal stability of stated preferences.

PubMed

Liebe, Ulf; Hundeshagen, Cordula; Beyer, Heiko; Cramon-Taubadel, Stephan von

2016-11-01

In stated preference studies it is assumed that individuals' answers reflect true preferences and are stable over time. We test these two assumptions of validity and reliability using as an example a choice experiment study on ethical consumption that measures preferences for a Peace Product jointly produced by Israeli and Palestinian producers as well as for organic products. In a web survey conducted in Germany, we investigate the validity assumption by manipulating the question context and presenting one group of respondents with questions on anti-Semitic and anti-Arabic attitudes before the choice tasks, and presenting another group with these questions after the choice tasks. In order to test the assumption of temporal stability, the same experimental set-up was repeated in a second survey based on a new sample ten months after the first. However, prior to the second survey an external event, a major violent dispute between Israelis and the Palestinians occurred. Overall, we find evidence for a context effect but not for temporal instability. In both surveys, the placement of the attitudinal questions before the choice tasks has a positive effect on the valuation of products from Israel, Palestinian products and the Peace Product (i.e. a directional context effect). The respondents seem to act according to an anti-discrimination norm. In line with this reasoning, we find an attention shift caused by the attitudinal questions. Organic products are valued much less positively if discriminatory attitudes are surveyed before the choice tasks. Furthermore, despite the violent dispute, stated preferences are very stable over time. This indicates high reliability of stated preference studies and encourages the use of study results by private and public decision makers. Copyright © 2016 Elsevier Inc. All rights reserved.

PubMed Central

Marotta, C.; Ventura, G.; Casuccio, N.; Dieli, F.; D'Angelo, C.; Restivo, V.; Costantino, C.; Vitale, F.; Casuccio, A.

2017-01-01

Summary Introduction. Vaccination coverages threaten to decrease because of false beliefs in their unsafety and inefficacy. Therefore formation of future health-care workers on this topic is fundamental to deal with any doubt and to promote active immunization among general population. Methods. In order to assess health-care students' knowledge about vaccination before an integrated seminar on this topic, and to evaluate their improvement after the educational intervention, an integrated educational intervention was held by a multidisciplinary team. Before and after the seminar, 118 students of medicine and biology schools at Palermo University were asked to answer 10 multiple-choice questions regarding vaccine history, mechanism of action, side effects, composition, use and nowadays issues (hesitancy). Two more questions investigating possible changes on students' attitudes towards vaccination and the usefulness of the formative intervention, were added at the post-test phase of the survey. Results. Eighty-one out of 118 students (68.6%) answered to both pre- and post-test questions. 97.6% and 81.5% of the participating group also completed the two additional questions about their improvement in knowledge (question 11) and attitudes (question 12) towards vaccinations. The post-test results showed a significant improvement for all questions administered, except for number 3 (about a specific immunological content), with an overall percentage of correct answers increasing from 38.8% to 77.6% (p©< 0.001). Conclusions. The present explorative study put the basis for future studies, stronger in the methodology, and highlights the importance of educating health-care professions students by integrated extra-curricular intervention to be held early in their degree curricula and in order to improve knowledge and attitudes towards vaccinations and to prepare them to promote vaccines among the general population. PMID:28900348

Common Student Misconceptions in Exercise Physiology and Biochemistry

ERIC Educational Resources Information Center

Morton, James P.; Doran, Dominic A.; MacLaren, Don P. M.

2008-01-01

The present study represents a preliminary investigation designed to identify common misconceptions in students' understanding of physiological and biochemical topics within the academic domain of sport and exercise sciences. A specifically designed misconception inventory (consisting of 10 multiple-choice questions) was administered to a cohort…

"Essential Principles of Economics:" A Hypermedia Textbook.

ERIC Educational Resources Information Center

McCain, Roger A.

2000-01-01

Discusses an electronic textbook called "Essential Principles of Economics." Explains that economic concepts are found by following links from the table of contents, while each chapter includes both expository information and interactive material including online multiple-choice drill questions. States that the textbook is a "work…

Designing Templates for Interactive Tasks in CALL Tutorials.

ERIC Educational Resources Information Center

Ruhlmann, Felicitas

The development of templates for computer-assisted language learning (CALL) is discussed, based on experiences with primarily linear multimedia tutorial programs. Design of templates for multiple-choice questions and interactive tasks in a prototype module is described. Possibilities of enhancing interactivity by introducing problem-oriented…

Visitors Abroad.

ERIC Educational Resources Information Center

Lepke, Helen; And Others

Seven German-American cross-cultural social situations are presented in English, each followed by a multiple-choice question concerning the etiquette or cultural implications of the actions. An answer key provides comments to stimulate class discussion. These everyday situations occur in Germany or the United States, and include eating at a…

Learning to Write about Mathematics

ERIC Educational Resources Information Center

Parker, Renee; Breyfogle, M. Lynn

2011-01-01

Beginning in third grade, Pennsylvania students are required to take the Pennsylvania State Standardized Assessment (PSSA), which presents multiple-choice mathematics questions and open-ended mathematics problems. Consistent with the Communication Standard of the National Council of Teachers of Mathematics, while solving the open-ended problems,…

An international comparison study of pharmacy students' achievement goals and their relationship to assessment type and scores.

PubMed

Alrakaf, Saleh; Anderson, Claire; Coulman, Sion A; John, Dai N; Tordoff, June; Sainsbury, Erica; Rose, Grenville; Smith, Lorraine

2015-04-25

To identify pharmacy students' preferred achievement goals in a multi-national undergraduate population, to investigate achievement goal preferences across comparable degree programs, and to identify relationships between achievement goals, academic performance, and assessment type. The Achievement Goal Questionnaire was administered to second year students in 4 universities in Australia, New Zealand, England, and Wales. Academic performance was measured using total scores, multiple-choice questions, and written answers (short essay). Four hundred eighty-six second year students participated. Students showed an overall preference for the mastery-approach goal orientation across all sites. The predicted relationships between goal orientation and multiple-choice questions, and written answers scores, were significant. This study is the first of its kind to examine pharmacy students' achievement goals at a multi-national level and to differentiate between assessment type and measures of achievement motivation. Students adopting a mastery-approach goal are more likely to gain high scores in assessments that measure understanding and depth of knowledge.

An International Comparison Study of Pharmacy Students’ Achievement Goals and their Relationship to Assessment Type and Scores

PubMed Central

Anderson, Claire; Coulman, Sion A.; John, Dai N.; Tordoff, June; Sainsbury, Erica; Rose, Grenville; Smith, Lorraine

2015-01-01

Objective: To identify pharmacy students’ preferred achievement goals in a multi-national undergraduate population, to investigate achievement goal preferences across comparable degree programs, and to identify relationships between achievement goals, academic performance, and assessment type. Methods: The Achievement Goal Questionnaire was administered to second year students in 4 universities in Australia, New Zealand, England, and Wales. Academic performance was measured using total scores, multiple-choice questions, and written answers (short essay). Results: Four hundred eighty-six second year students participated. Students showed an overall preference for the mastery-approach goal orientation across all sites. The predicted relationships between goal orientation and multiple-choice questions, and written answers scores, were significant. Conclusion: This study is the first of its kind to examine pharmacy students’ achievement goals at a multi-national level and to differentiate between assessment type and measures of achievement motivation. Students adopting a mastery-approach goal are more likely to gain high scores in assessments that measure understanding and depth of knowledge. PMID:25995510

Comparing Item Performance on Three- Versus Four-Option Multiple Choice Questions in a Veterinary Toxicology Course.

PubMed

Royal, Kenneth; Dorman, David

2018-06-09

The number of answer options is an important element of multiple-choice questions (MCQs). Many MCQs contain four or more options despite the limited literature suggesting that there is little to no benefit beyond three options. The purpose of this study was to evaluate item performance on 3-option versus 4-option MCQs used in a core curriculum course in veterinary toxicology at a large veterinary medical school in the United States. A quasi-experimental, crossover design was used in which students in each class were randomly assigned to take one of two versions (A or B) of two major exams. Both the 3-option and 4-option MCQs resulted in similar psychometric properties. The findings of our study support earlier research in other medical disciplines and settings that likewise concluded there was no significant change in the psychometric properties of three option MCQs when compared to the traditional MCQs with four or more options.

Gender and Ethnicity Differences in Multiple-Choice Testing. Effects of Self-Assessment and Risk-Taking Propensity

DTIC Science & Technology

1993-05-01

correctness of the response provides I some advantages. They are: i 1. Increased reliability of the test; 2. Examinees pay more attention to the multiple...their choice 3 of test date. Each sign up sheet was divided into four cells: Non-Hispanic males and females and Hispanic males and females. 3 I I I...certain prestige and financial rewards; or entering a conservatory of music for advanced training with a well-known pianist . Mr. H realizes that even

Web-Based Evaluation System to Measure Learning Effectiveness in Kampo Medicine

PubMed Central

Usuku, Koichiro; Segawa, Makoto; Wang, Yue; Ogashiwa, Kahori; Fujita, Yusuke; Ogihara, Hiroyuki; Tazuma, Susumu

2016-01-01

Measuring the learning effectiveness of Kampo Medicine (KM) education is challenging. The aim of this study was to develop a web-based test to measure the learning effectiveness of KM education among medical students (MSs). We used an open-source Moodle platform to test 30 multiple-choice questions classified into 8-type fields (eight basic concepts of KM) including “qi-blood-fluid” and “five-element” theories, on 117 fourth-year MSs. The mean (±standard deviation [SD]) score on the web-based test was 30.2 ± 11.9 (/100). The correct answer rate ranged from 17% to 36%. A pattern-based portfolio enabled these rates to be individualized in terms of KM proficiency. MSs with scores higher (n = 19) or lower (n = 14) than mean ± 1SD were defined as high or low achievers, respectively. Cluster analysis using the correct answer rates for the 8-type field questions revealed clear divisions between high and low achievers. Interestingly, each high achiever had a different proficiency pattern. In contrast, three major clusters were evident among low achievers, all of whom responded with a low percentage of or no correct answers. In addition, a combination of three questions accurately classified high and low achievers. These findings suggest that our web-based test allows individual quantitative assessment of the learning effectiveness of KM education among MSs. PMID:27738440

Web-Based Evaluation System to Measure Learning Effectiveness in Kampo Medicine.

PubMed

Iizuka, Norio; Usuku, Koichiro; Nakae, Hajime; Segawa, Makoto; Wang, Yue; Ogashiwa, Kahori; Fujita, Yusuke; Ogihara, Hiroyuki; Tazuma, Susumu; Hamamoto, Yoshihiko

2016-01-01

Measuring the learning effectiveness of Kampo Medicine (KM) education is challenging. The aim of this study was to develop a web-based test to measure the learning effectiveness of KM education among medical students (MSs). We used an open-source Moodle platform to test 30 multiple-choice questions classified into 8-type fields (eight basic concepts of KM) including "qi-blood-fluid" and "five-element" theories, on 117 fourth-year MSs. The mean (±standard deviation [SD]) score on the web-based test was 30.2 ± 11.9 (/100). The correct answer rate ranged from 17% to 36%. A pattern-based portfolio enabled these rates to be individualized in terms of KM proficiency. MSs with scores higher ( n = 19) or lower ( n = 14) than mean ± 1SD were defined as high or low achievers, respectively. Cluster analysis using the correct answer rates for the 8-type field questions revealed clear divisions between high and low achievers. Interestingly, each high achiever had a different proficiency pattern. In contrast, three major clusters were evident among low achievers, all of whom responded with a low percentage of or no correct answers. In addition, a combination of three questions accurately classified high and low achievers. These findings suggest that our web-based test allows individual quantitative assessment of the learning effectiveness of KM education among MSs.

Alignment of learning objectives and assessments in therapeutics courses to foster higher-order thinking.

PubMed

FitzPatrick, Beverly; Hawboldt, John; Doyle, Daniel; Genge, Terri

2015-02-17

To determine whether national educational outcomes, course objectives, and classroom assessments for 2 therapeutics courses were aligned for curricular content and cognitive processes, and if they included higher-order thinking. Document analysis and student focus groups were used. Outcomes, objectives, and assessment tasks were matched for specific therapeutics content and cognitive processes. Anderson and Krathwohl's Taxonomy was used to define higher-order thinking. Students discussed whether assessments tested objectives and described their thinking when responding to assessments. There were 7 outcomes, 31 objectives, and 412 assessment tasks. The alignment for content and cognitive processes was not satisfactory. Twelve students participated in the focus groups. Students thought more short-answer questions than multiple choice questions matched the objectives for content and required higher-order thinking. The alignment analysis provided data that could be used to reveal and strengthen the enacted curriculum and improve student learning.

Development of a testlet generator in re-engineering the Indonesian physics national-exams

NASA Astrophysics Data System (ADS)

Mindyarto, Budi Naini; Mardapi, Djemari; Bastari

2017-08-01

The Indonesian Physics national-exams are end-of-course summative assessments that could be utilized to support the assessment for learning in physics educations. This paper discusses the development and evaluation of a testlet generator based on a re-engineering of Indonesian physics national exams. The exam problems were dissected and decomposed into testlets revealing the deeper understanding of the underlying physical concepts by inserting a qualitative question and its scientific reasoning question. A template-based generator was built to facilitate teachers in generating testlet variants that would be more conform to students' scientific attitude development than their original simple multiple-choice formats. The testlet generator was built using open source software technologies and was evaluated focusing on the black-box testing by exploring the generator's execution, inputs and outputs. The results showed the correctly-performed functionalities of the developed testlet generator in validating inputs, generating testlet variants, and accommodating polytomous item characteristics.

Effectiveness of computer-assisted interactive videodisc instruction in teaching rheumatology to physical and occupational therapy students.

PubMed

Sanford, M K; Hazelwood, S E; Bridges, A J; Cutts, J H; Mitchell, J A; Reid, J C; Sharp, G

1996-01-01

A computer-assisted interactive videodisc instructional program, HP-RHEUM was designed to teach clinical findings in arthritis to occupational and physical therapy students. Using the Rheumatology Image Library videodisc produced by the National Library of Medicine, HP-RHEUM consists of instructional modules which employ advance organizers, examples/nonexamples, summaries, and immediate feedback. To see if HP-RHEUM would be as effective as traditional classroom instruction, control data were collected in 1991 from 52 OT and PT students. Treatment data were collected from 61 students in 1992 when HP-RHEUM entirely replaced lectures. Identical pre- and post-tests consisted of 70 multiple choice questions, with 24 matched to slides. On the slide questions the HP-RHEUM group had significantly higher scores. Otherwise, there was no significant difference in performance between groups. HP-RHEUM provided an independent learning method and enhanced visual comprehension of rheumatologic disease concepts.

A cross-cultural comparison of high school students' responses to a science centre show on the physics of sound in South Africa.

PubMed

Fish, Derek; Allie, Saalih; Pelaez, Nancy; Anderson, Trevor

2017-10-01

We report on the attitudes and ideas developed by students from three distinct school groups to a science show about sound. We addressed two research questions: (1) How do the students compare with respect to their (a) attitudes to the sound show and to science in general and (b) changes in conceptual understanding as a result of the show and (2) what changes could be made to the show, and to science shows in general, that would be sensitive to the cultural and language differences of the groups? These were addressed by multiple-choice, pre- and post-tests comprising both attitudinal and conceptual questions. Our results pointed to a common enjoyment of the show but a different understanding of concepts and consequent learning, which suggest that science shows (and science teaching) need to be adjusted to accommodate different cultural groups for maximum impact.

Pneumatics.

ERIC Educational Resources Information Center

Engelbrecht, Nancy; And Others

This unit on pneumatics, for use in postsecondary programs, is organized in eight sections. Each section consists of information sheets with line drawings and multiple-choice questions for each topic in the sections. Answers are provided at the back of the book. The following topics are covered: (1) introduction--pressure, principles of gases,…

Investigating Students' Understanding of the Dissolving Process

ERIC Educational Resources Information Center

Naah, Basil M.; Sanger, Michael J.

2013-01-01

In a previous study, the authors identified several student misconceptions regarding the process of dissolving ionic compounds in water. The present study used multiple-choice questions whose distractors were derived from these misconceptions to assess students' understanding of the dissolving process at the symbolic and particulate levels. The…

Directions in Rehabilitation Counseling, 1993.

ERIC Educational Resources Information Center

Directions in Rehabilitation Counseling, 1993

1993-01-01

This volume of 12 lessons--each written by either a medical or a mental health professional--provides expert information on a variety of medical and psychological issues in rehabilitative counseling. The lessons, each of which concludes with a few multiple-choice questions, are: (1) "Geriatric Alcoholism: Identification and Elder-Specific…

Academic Performance in Introductory Accounting: Do Learning Styles Matter?

ERIC Educational Resources Information Center

Tan, Lin Mei; Laswad, Fawzi

2015-01-01

This study examines the impact of learning styles on academic performance using major assessment methods (examinations and assignments including multiple-choice and constructed response questions (CRQs)) in an introductory accounting course. Students' learning styles were assessed using Kolb's Learning Style Inventory Version 3.1. The results…

What We've Learned about Assessing Hands-On Science.

ERIC Educational Resources Information Center

Shavelson, Richard J.; Baxter, Gail P.

1992-01-01

A recent study compared hands-on scientific inquiry assessment to assessments involving lab notebooks, computer simulations, short-answer paper-and-pencil problems, and multiple-choice questions. Creating high quality performance assessments is a costly, time-consuming process requiring considerable scientific and technological know-how. Improved…

MO-DE-BRA-01: Flipped Physics Courses Within a Radiologic Technologist Program: Video Production and Long Term Outcomes

DOE Office of Scientific and Technical Information (OSTI.GOV)

Oshiro, T; Donaghy, M; Slechta, A

Purpose: To determine if the flipped class format has an effect on examination results for a radiologic technologist (RT) program and discuss benefits from creating video resources. Methods: From 2001–2015, students had taken both a radiological physics and quality control (QC) class as a part of their didactic training. In 2005/2006, the creation of videos of didactic lectures and QC test demonstrations allowed for a flip where content was studied at home while exercises and reviews were done in-class. Final examinations were retrospectively reviewed from this timeframe. 12 multiple choice physics questions (MCP) and 5 short answer QC questions (SAQC)more » were common to pre and post flip exams. The RT program’s ARRT exam scores were also obtained and compared to national averages. Results: In total, 36 lecture videos and 65 quality control videos were created for the flipped content. Data was ∼2.4GB and distributed to students via USB or CD media. For MCP questions, scores improved by 7.9% with the flipped format and significance (Student’s t-test, p<0.05) was found for 3 of the 12 questions. SAQC questions showed improvement by 14.6% and significance was found for 2 of the 5 questions. Student enrollment increased from ∼14 (2001–2004) to ∼23 students (2005–15). Content was continuously added post-flip due to the efficiency of delivery. The QC class in 2003 covered 45 test setups in-class while 65 were covered with video segments in 2014. Flipped materials are currently being repurposed. In 2015, this video content was restructured into an ARRT exam review guide and in 2016, the content was reorganized for fluoroscopy training for physicians. Conclusion: We believe that flipped classes can improve efficiency of content delivery and improve student performance even with an increase in class size. This format allows for flexibility in learning as well as re-use in multiple applications.« less

Does the Concept of the "Flipped Classroom" Extend to the Emergency Medicine Clinical Clerkship?

PubMed

Heitz, Corey; Prusakowski, Melanie; Willis, George; Franck, Christopher

2015-11-01

Linking educational objectives and clinical learning during clerkships can be difficult. Clinical shifts during emergency medicine (EM) clerkships provide a wide variety of experiences, some of which may not be relevant to recommended educational objectives. Students can be directed to standardize their clinical experiences, and this improves performance on examinations. We hypothesized that applying a "flipped classroom" model to the clinical clerkship would improve performance on multiple-choice testing when compared to standard learning. Students at two institutions were randomized to complete two of four selected EM clerkship topics in a "flipped fashion," and two others in a standard fashion. For flipped topics, students were directed to complete chief complaint-based asynchronous modules prior to a shift, during which they were directed to focus on the chief complaint. For the other two topics, modules were to be performed at the students' discretion, and shifts would not have a theme. At the end of the four-week clerkship, a 40-question multiple-choice examination was administered with 10 questions per topic. We compared performance on flipped topics with those performed in standard fashion. Students were surveyed on perceived effectiveness, ability to follow the protocol, and willingness of preceptors to allow a chief-complaint focus. Sixty-nine students participated; examination scores for 56 were available for analysis. For the primary outcome, no difference was seen between the flipped method and standard (p=0.494.) A mixed model approach showed no effect of flipped status, protocol adherence, or site of rotation on the primary outcome of exam scores. Students rated the concept of the flipped clerkship highly (3.48/5). Almost one third (31.1%) of students stated that they were unable to adhere to the protocol. Preparation for a clinical shift with pre-assigned, web-based learning modules followed by an attempt at chief-complaint-focused learning during a shift did not result in improvements in performance on a multiple-choice assessment of knowledge; however, one third of participants did not adhere strictly to the protocol. Future investigations should ensure performance of pre-assigned learning as well as clinical experiences, and consider alternate measures of knowledge.

Digital lectures for learning gross anatomy: a study of their efficacy.

PubMed

Singh, Anudeep; Min, Aung Ko Ko

2017-03-01

The current study investigates the level of students' learning and attitudes towards the teaching and learning process when using digital lectures to teach gross anatomy to year 1 medical students. The study sampled year 1 medical students of cohorts 2013 and 2014. The year 1 medical students in 2013 were taught gross anatomy of the heart by didactic classroom lectures while those in 2014 were taught with digital lectures using the same content. A review session was conducted for the 2014 cohort. A 19-item survey was distributed amongst students to investigate their attitudes and feedback. The data were analysed using SPSS software. The 2014 cohort had a mean score of 47.65 for short essay questions and 51.19 for multiple choice questions, while the 2013 cohort scored an average of 36.80 for short essay questions and 49.22 for multiple choice questions. The difference in scores for each type of question was found to be significant. Using a 5-point Likert scale, students gave an average of 4.11 when asked if they liked the teaching and learning process and would like it to be applied further. The results of the study provide strong evidence that the digital teaching and learning process was well received by students and could also lead to improved performance. Digital lectures can provide a satisfactory substitute for classroom lectures to teach gross anatomy, thus providing flexibility in learning and efficient learning, whilst also freeing lecture slots to promote mastery learning.

Webinar Software: A Tool for Developing More Effective Lectures (Online or In-Person)

PubMed Central

Mayorga, Eduardo P.; Bekerman, Jesica G.; Palis, Ana G.

2014-01-01

Purpose: To describe the use of online seminars (webinars) to improve learning experience for medical residents and fostering critical thinking. Materials and Methods: Sixty-one online seminars (webinars) for residents were developed from April 2012 to February 2013. Residents attended the lectures in the same room as the presenter or from distant locations. Residents interacted with the presenter using their personal computers, tablets, or smartphones. They were able to ask questions and answer the instructor's multiple choice or open-ended questions. The lecture dynamics consisted of: (1) The presentation of a clinical case by an expert on the clinical topic; (2) the instructor asked open-ended and multiple-choice questions about the problem-resolution process; (3) participants responded questions individually; (4) participants received feedback on their answers; (5) a brief conference was given on the learning objectives and the content, also fostering interactive participation; (6) lectures were complemented with work documents. Results: This method allowed for exploration of learning of scientific knowledge and the acquisition of other medical competences (such as patient care, interpersonal and communication skills, and professionalism). The question-and-answer activity and immediate feedback gave attendees the chance to participate actively in the conference, reflect on the topic, correct conceptual errors, and exercise critical thinking. All these factors are necessary for learning. Conclusions: This modality, which facilitates interaction, active participation, and immediate feedback, could allow learners to acquire knowledge more effectively. PMID:24791102

Improving the identification accuracy of senior witnesses: do prelineup questions and sequential testing help?

PubMed

Memon, Amina; Gabbert, Fiona

2003-04-01

Eyewitness research has identified sequential lineup testing as a way of reducing false lineup choices while maintaining accurate identifications. The authors examined the usefulness of this procedure for reducing false choices in older adults. Young and senior witnesses viewed a crime video and were later presented with target present orabsent lineups in a simultaneous or sequential format. In addition, some participants received prelineup questions about their memory for a perpetrator's face and about their confidence in their ability to identify the culprit or to correctly reject the lineup. The sequential lineup reduced false choosing rates among young and older adults in target-absent conditions. In target-present conditions, sequential testing significantly reduced the correct identification rate in both age groups.

Preference index supported by motivation tests in Nile tilapia

PubMed Central

2017-01-01

The identification of animal preferences is assumed to provide better rearing environments for the animals in question. Preference tests focus on the frequency of approaches or the time an animal spends in proximity to each item of the investigated resource during a multiple-choice trial. Recently, a preference index (PI) was proposed to differentiate animal preferences from momentary responses (Sci Rep, 2016, 6:28328, DOI: 10.1038/srep28328). This index also quantifies the degree of preference for each item. Each choice response is also weighted, with the most recent responses weighted more heavily, but the index includes the entire bank of tests, and thus represents a history-based approach. In this study, we compared this PI to motivation tests, which consider how much effort is expended to access a resource. We performed choice tests over 7 consecutive days for 34 Nile tilapia fish that presented with different colored compartments in each test. We first detected the preferred and non-preferred colors of each fish using the PI and then tested their motivation to reach these compartments. We found that fish preferences varied individually, but the results were consistent with the motivation profiles, as individual fish were more motivated (the number of touches made on transparent, hinged doors that prevented access to the resource) to access their preferred items. On average, most of the 34 fish avoided the color yellow and showed less motivation to reach yellow and red colors. The fish also exhibited greater motivation to access blue and green colors (the most preferred colors). These results corroborate the PI as a reliable tool for the identification of animal preferences. We recommend this index to animal keepers and researchers to identify an animal’s preferred conditions. PMID:28426689

Preference index supported by motivation tests in Nile tilapia.

PubMed

Maia, Caroline Marques; Volpato, Gilson Luiz

2017-01-01

The identification of animal preferences is assumed to provide better rearing environments for the animals in question. Preference tests focus on the frequency of approaches or the time an animal spends in proximity to each item of the investigated resource during a multiple-choice trial. Recently, a preference index (PI) was proposed to differentiate animal preferences from momentary responses (Sci Rep, 2016, 6:28328, DOI: 10.1038/srep28328). This index also quantifies the degree of preference for each item. Each choice response is also weighted, with the most recent responses weighted more heavily, but the index includes the entire bank of tests, and thus represents a history-based approach. In this study, we compared this PI to motivation tests, which consider how much effort is expended to access a resource. We performed choice tests over 7 consecutive days for 34 Nile tilapia fish that presented with different colored compartments in each test. We first detected the preferred and non-preferred colors of each fish using the PI and then tested their motivation to reach these compartments. We found that fish preferences varied individually, but the results were consistent with the motivation profiles, as individual fish were more motivated (the number of touches made on transparent, hinged doors that prevented access to the resource) to access their preferred items. On average, most of the 34 fish avoided the color yellow and showed less motivation to reach yellow and red colors. The fish also exhibited greater motivation to access blue and green colors (the most preferred colors). These results corroborate the PI as a reliable tool for the identification of animal preferences. We recommend this index to animal keepers and researchers to identify an animal's preferred conditions.

Augmenting Fellow Education Through Spaced Multiple-Choice Questions.

PubMed

Barsoumian, Alice E; Yun, Heather C

2018-01-01

The San Antonio Uniformed Services Health Education Consortium Infectious Disease Fellowship program historically included a monthly short-answer and multiple-choice quiz. The intent was to ensure medical knowledge in relevant content areas that may not be addressed through clinical rotations, such as operationally relevant infectious disease. After completion, it was discussed in a small group with faculty. Over time, faculty noted increasing dissatisfaction with the activity. Spaced interval education is useful in retention of medical knowledge and skills by medical students and residents. Its use in infectious disease fellow education has not been described. To improve the quiz experience, we assessed the introduction of spaced education curriculum in our program. A pre-intervention survey was distributed to assess the monthly quiz with Likert scale and open-ended questions. A multiple-choice question spaced education curriculum was created using the Qstream(R) platform in 2011. Faculty development on question writing was conducted. Two questions were delivered every 2 d. Incorrectly and correctly answered questions were repeated after 7 and 13 d, respectively. Questions needed to be answered correctly twice to be retired. Fellow satisfaction was assessed at semi-annual fellowship reviews over 5 yr and by a one-time repeat survey. Pre-intervention survey of six fellows indicated dissatisfaction with the time commitment of the monthly quiz (median Likert score of 2, mean 6.5 h to complete), neutral in perceived utility, but satisfaction with knowledge retention (Likert score 4). Eighteen fellows over 5 yr participated in the spaced education curriculum. Three quizzes with 20, 39, and 48 questions were designed. Seventeen percentage of questions addressed operationally relevant topics. Fifty-nine percentage of questions were answered correctly on first attempt, improving to 93% correct answer rate at the end of the analysis. Questions were attempted 2,999 times. Fellows consistently indicated that the platform was "highly enjoyed," "beneficial," a "fun format," and "completely satisfied." Fellows additionally commented that they desired more questions and considered the platform helpful in board preparation. Formal survey data post-intervention found that the fellows were satisfied with the new approach, found it to be useful in board preparation, overall educational value, and in-line with their personal learning style (median Likert score of 4 for all queries). Fellows were satisfied with time commitment, spending a mean of 47 min on the spaced education curriculum questions per month. Introduction of a spaced education curriculum resulted in a sustained positive learner experience for >5 yr with demonstrated mastery of material. Spaced education learning is a viable addition to augment training experience, especially in areas of curricular gaps such as operational medicine. Correct answer data may also be useful to perform Accreditation Council for Graduate Medical Education-required objective assessment of knowledge. Published by Oxford University Press on behalf of the Association of Military Surgeons of the United States 2017. This work is written by (a) US Government employee(s) and is in the public domain in the US.

Directions in Rehabilitation Counseling, 1991.

ERIC Educational Resources Information Center

Directions in Rehabilitation Counseling, 1991

1991-01-01

This volume of 12 lessons--each one written by either a medical or a mental health professional--provides expert information on a variety of medical and psychological issues in rehabilitative counseling. The lessons, each of which concludes with a few multiple-choice questions, are as follows: (1) "An Update on Post-Traumatic Stress…

Case Study of a Computer Based Examination System

ERIC Educational Resources Information Center

Fluck, Andrew; Pullen, Darren; Harper, Colleen

2009-01-01

Electronic supported assessment or e-Assessment is a field of growing importance, but it has yet to make a significant impact in the Australian higher education sector (Byrnes & Ellis, 2006). Current computer based assessment models focus on the assessment of knowledge rather than deeper understandings, using multiple choice type questions,…

Using Data Analysis Problems in a Large General Microbiology Course.

ERIC Educational Resources Information Center

Deutch, Charles E.

1997-01-01

Argues that data analysis problems can be used successfully in large introductory microbiology courses, even when exams consist entirely of multiple-choice questions and out-of-class contact with the instructor is limited. Discusses course organization, problem structure, student performance and response, advantages of using data analysis…

Beyond Word Processing: Rhetorical Invention with Computers.

ERIC Educational Resources Information Center

Strickland, James

In the area of composition, computer assisted instruction (CAI) must move beyond the limited concerns of the current-traditional rhetoric to address the larger issues of writing, become process-centered, and involve active writing rather than answering multiple-choice questions. Researchers cite four major types of interactive CAI, the last of…

Supporting Students' Learning: The Use of Formative Online Assessments

ERIC Educational Resources Information Center

Einig, Sandra

2013-01-01

This paper investigates the impact of online multiple choice questions (MCQs) on students' learning in an undergraduate Accounting module at a British university. The impact is considered from three perspectives: an analysis of how students use the MCQs; students' perceptions expressed in a questionnaire survey; and an investigation of the…

AN ADJUNCT TO SELF STUDY.

ERIC Educational Resources Information Center

MEYER, DONALD E.

TWO EXPERIMENTS WERE CONDUCTED TO EVALUATE A SELF-STUDY PROGRAM PREPARED WITHIN TWO OPERATIONAL FIGHTER-INTERCEPTOR SQUADRONS. BASED ON A TECHNICAL ORDER MANUAL, IT CONSISTED OF SEQUENCED MULTIPLE-CHOICE QUESTIONS WITH A SIMPLE PUNCHBOARD AS A CONFIRMING DEVICE. IN EXPERIMENT ONE, TWO OPERATIONAL SQUADRONS WERE DIVIDED INTO TWO GROUPS JUDGED TO BE…

Item Readability and Science Achievement in TIMSS 2003 in South Africa

ERIC Educational Resources Information Center

Dempster, Edith R.; Reddy, Vijay

2007-01-01

This study investigated the relationship between readability of 73 text-only multiple-choice questions from Trends in International Mathematics and Science Study (TIMSS) 2003 and performance of two groups of South African learners: those with limited English-language proficiency (learners attending African schools) and those with better…

Directions in Rehabilitation Counseling, 1994.

ERIC Educational Resources Information Center

Directions in Rehabilitation Counseling, 1994

1994-01-01

This volume of 12 lessons--each one written by either a medical or mental health professional--provides expert information on a variety of medical and psychological issues in rehabilitative counseling. The lessons, each of which concludes with a few multiple-choice questions, are as follows: (1) "Behavioral Techniques for Treatment of…

Directions in Rehabilitation Counseling, 1990.

ERIC Educational Resources Information Center

Directions in Rehabilitation Counseling, 1990

1990-01-01

This volume of 12 lessons--each one written by either a medical or a mental health professional--provides expert information on a variety of medical and psychological issues in rehabilitative counseling. The lessons, each of which concludes with a few multiple-choice questions, are as follows: (1) "Rehabilitation of the Seriously Mentally…

Exploring the Use of Audience Response Systems in Secondary School Science Classrooms

ERIC Educational Resources Information Center

Kay, Robin; Knaack, Liesel

2009-01-01

An audience response systems (ARS) allows students to respond to multiple choice questions using remote control devices. Once the feedback is collected and displayed, the teacher and students discuss misconceptions and difficulties experienced. ARSs have been extremely popular and effective in higher education science classrooms, although almost…

Primary Teachers' Literacy and Attitudes on Education for Sustainable Development

ERIC Educational Resources Information Center

Spiropoulou, Dimitra; Antonakaki, Triantafyllia; Kontaxaki, Sophia; Bouras, Sarantis

2007-01-01

This paper reports on research concerning Greek in-service Primary teachers' perceptions about environmental issues and attitudes towards Education for Sustainable Development. A questionnaire with multiple-choice and open-ended questions was used in order to gain more comprehensive understanding of their thoughts. The analysis of data revealed…

Heubach Smoking Habits and Attitudes Questionnaire.

ERIC Educational Resources Information Center

Heubach, Philip Gilbert

This Questionnaire, consisting of 74 yes/no, multiple choice, and completion items, is designed to assess smoking practices and attitudes toward smoking in high school students. Questions pertain to personal data, family smoking practices and attitudes, personal smoking habits, reasons for smoking or not smoking, and opinions on smoking. Detailed…

Employment Status and Adjustment in Mothers Who Delay Parenthood.

ERIC Educational Resources Information Center

Dienstag, Esther L.

Examined were differences in satisfaction with parenthood among 125 primiparous, middle-class, working and non-working women above and below 30 years of age who completed multiple-choice questionnaires within 6 months of the birth of their child. Questions pertained to responsive attitudes toward infants, attitudes toward women's roles, and…

Michigan High School Student Drug Attitudes and Behavior Questionnaire.

ERIC Educational Resources Information Center

Bogg, Richard A.; And Others

This questionnaire assesses drug use practices and attitudes toward drugs in high school students. The instrument has 59 items (multiple choice or completion), some with several parts. The question pertain to aspirations for the future, general attitudes and opinions, biographic and demographic data, family background and relationships, alcohol…

An Inventory on Rotational Kinematics of a Particle: Unravelling Misconceptions and Pitfalls in Reasoning

ERIC Educational Resources Information Center

Mashood, K. K.; Singh, Vijay A.

2012-01-01

Student difficulties regarding the angular velocity ([image omitted]) and angular acceleration ([image omitted]) of a particle have remained relatively unexplored in contrast to their linear counterparts. We present an inventory comprising multiple choice questions aimed at probing misconceptions and eliciting ill-suited reasoning patterns. The…

Directions in Rehabilitation Counseling, 1992.

ERIC Educational Resources Information Center

Directions in Rehabilitation Counseling, 1992

1992-01-01

This volume of 12 separate lessons--each written by either a medical or mental health professional--provides expert information on a wide variety of medical and psychological issues in rehabilitative counseling. The lessons, each of which concludes with a few multiple-choice questions, are as follows: (1) "Adaptive Styles in the Etiology of…

Advanced Placement Economics. Teacher Resource Manual.

ERIC Educational Resources Information Center

Morton, John S.

This book, in conjunction with the student activities books for macroeconomics and microeconomics, is designed for teaching the Advanced Placement Economics course. The book contains five units for the microeconomic portion and six units for the macroeconomic portion of the text. Along with the many activities are sample multiple-choice questions,…

Conceptual Level of Understanding about Sound Concept: Sample of Fifth Grade Students

ERIC Educational Resources Information Center

Bostan Sarioglan, Ayberk

2016-01-01

In this study, students' conceptual change processes related to the sound concept were examined. Study group was comprises of 325 fifth grade middle school students. Three multiple-choice questions were used as the data collection tool. At the data analysis process "scientific response", "scientifically unacceptable response"…

Calculus Student Understanding of Continuity

ERIC Educational Resources Information Center

Wangle, Jayleen Lillian

2013-01-01

Continuity is a central concept in calculus. Yet very few students seem to understand the nature of continuity. The research described was conducted in two stages. Students were asked questions in multiple choice and true/false format regarding function, limit and continuity. These results were used to identify participants as strong, weak or…

Alternative Assessment Techniques for Blended and Online Courses

ERIC Educational Resources Information Center

Litchfield, Brenda C.; Dempsey, John V.

2013-01-01

Alternative assessment techniques are essential for increasing student learning in blended and online courses. Rather than simply answer multiple-choice questions, students can choose activities in an academic contract. By using a contract, students will be active participants in their own learning. Contracts add a dimension of authenticity to…

Continuing Environmental Health Education for Environmental Health Personnel, Lesson Six.

ERIC Educational Resources Information Center

Journal of Environmental Health, 1981

1981-01-01

Presents the sixth and final lesson on general environmental health, appearing since January, 1977 in this journal. Twenty-five multiple choice questions appear dealing with environmental health topics such as food sanitation, milk sanitation, vector control, public health housing, institutional environmental health, waste disposal, air pollution,…

Harnessing Technology to Improve Formative Assessment of Student Conceptions in STEM: Forging a National Network

ERIC Educational Resources Information Center

Haudek, Kevin C.; Kaplan, Jennifer J.; Knight, Jennifer; Long, Tammy; Merrill, John; Munn, Alan; Nehm, Ross; Smith, Michelle; Urban-Lurain, Mark

2011-01-01

Concept inventories, consisting of multiple-choice questions designed around common student misconceptions, are designed to reveal student thinking. However, students often have complex, heterogeneous ideas about scientific concepts. Constructed-response assessments, in which students must create their own answer, may better reveal students'…

Development and Validation of the Homeostasis Concept Inventory

ERIC Educational Resources Information Center

McFarland, Jenny L.; Price, Rebecca M.; Wenderoth, Mary Pat; Martinková, Patrícia; Cliff, William; Michael, Joel; Modell, Harold; Wright, Ann

2017-01-01

We present the Homeostasis Concept Inventory (HCI), a 20-item multiple-choice instrument that assesses how well undergraduates understand this critical physiological concept. We used an iterative process to develop a set of questions based on elements in the Homeostasis Concept Framework. This process involved faculty experts and undergraduate…

Will a Short Training Session Improve Multiple-Choice Item-Writing Quality by Dental School Faculty? A Pilot Study.

PubMed

Dellinges, Mark A; Curtis, Donald A

2017-08-01

Faculty members are expected to write high-quality multiple-choice questions (MCQs) in order to accurately assess dental students' achievement. However, most dental school faculty members are not trained to write MCQs. Extensive faculty development programs have been used to help educators write better test items. The aim of this pilot study was to determine if a short workshop would result in improved MCQ item-writing by dental school faculty at one U.S. dental school. A total of 24 dental school faculty members who had previously written MCQs were randomized into a no-intervention group and an intervention group in 2015. Six previously written MCQs were randomly selected from each of the faculty members and given an item quality score. The intervention group participated in a training session of one-hour duration that focused on reviewing standard item-writing guidelines to improve in-house MCQs. The no-intervention group did not receive any training but did receive encouragement and an explanation of why good MCQ writing was important. The faculty members were then asked to revise their previously written questions, and these were given an item quality score. The item quality scores for each faculty member were averaged, and the difference from pre-training to post-training scores was evaluated. The results showed a significant difference between pre-training and post-training MCQ difference scores for the intervention group (p=0.04). This pilot study provides evidence that the training session of short duration was effective in improving the quality of in-house MCQs.

Demand Characteristics of Multiple-Choice Items.

ERIC Educational Resources Information Center

Diamond, James J.; Williams, David V.

Thirteen graduate students were asked to indicate for each of 24 multiple-choice items whether the item tested "recall of specific information," a "higher order skill," or "don't know." The students were also asked to state their general basis for judging the items. The 24 items had been previously classified according to Bloom's cognitive-skills…

Examining the Prediction of Reading Comprehension on Different Multiple-Choice Tests

ERIC Educational Resources Information Center

Andreassen, Rune; Braten, Ivar

2010-01-01

In this study, 180 Norwegian fifth-grade students with a mean age of 10.5 years were administered measures of word recognition skills, strategic text processing, reading motivation and working memory. Six months later, the same students were given three different multiple-choice reading comprehension measures. Based on three forced-order…

A Better Benchmark Assessment: Multiple-Choice versus Project-Based

ERIC Educational Resources Information Center

Peariso, Jamon F.

2006-01-01

The purpose of this literature review and Ex Post Facto descriptive study was to determine which type of benchmark assessment, multiple-choice or project-based, provides the best indication of general success on the history portion of the CST (California Standards Tests). The result of the study indicates that although the project-based benchmark…