Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing.

PubMed

Butler, Andrew C; Roediger, Henry L

2008-04-01

Multiple-choice tests are used frequently in higher education without much consideration of the impact this form of assessment has on learning. Multiple-choice testing enhances retention of the material tested (the testing effect); however, unlike other tests, multiple-choice can also be detrimental because it exposes students to misinformation in the form of lures. The selection of lures can lead students to acquire false knowledge (Roediger & Marsh, 2005). The present research investigated whether feedback could be used to boost the positive effects and reduce the negative effects of multiple-choice testing. Subjects studied passages and then received a multiple-choice test with immediate feedback, delayed feedback, or no feedback. In comparison with the no-feedback condition, both immediate and delayed feedback increased the proportion of correct responses and reduced the proportion of intrusions (i.e., lure responses from the initial multiple-choice test) on a delayed cued recall test. Educators should provide feedback when using multiple-choice tests.

Asymmetry in Student Achievement on Multiple-Choice and Constructed-Response Items in Reversible Mathematics Processes

ERIC Educational Resources Information Center

Sangwin, Christopher J.; Jones, Ian

2017-01-01

In this paper we report the results of an experiment designed to test the hypothesis that when faced with a question involving the inverse direction of a reversible mathematical process, students solve a multiple-choice version by verifying the answers presented to them by the direct method, not by undertaking the actual inverse calculation.…

Manipulations of Choice Familiarity in Multiple-Choice Testing Support a Retrieval Practice Account of the Testing Effect

ERIC Educational Resources Information Center

Jang, Yoonhee; Pashler, Hal; Huber, David E.

2014-01-01

We performed 4 experiments assessing the learning that occurs when taking a test. Our experiments used multiple-choice tests because the processes deployed during testing can be manipulated by varying the nature of the choice alternatives. Previous research revealed that a multiple-choice test that includes "none of the above" (NOTA)…

Effects of Test Expectation on Multiple-Choice Performance and Subjective Ratings

ERIC Educational Resources Information Center

Balch, William R.

2007-01-01

Undergraduates studied the definitions of 16 psychology terms, expecting either a multiple-choice (n = 132) or short-answer (n = 122) test. All students then received the same multiple-choice test, requiring them to recognize the definitions as well as novel examples of the terms. Compared to students expecting a multiple-choice test, those…

Multiple Choice Testing and the Retrieval Hypothesis of the Testing Effect

ERIC Educational Resources Information Center

Sensenig, Amanda E.

2010-01-01

Taking a test often leads to enhanced later memory for the tested information, a phenomenon known as the "testing effect". This memory advantage has been reliably demonstrated with recall tests but not multiple choice tests. One potential explanation for this finding is that multiple choice tests do not rely on retrieval processes to the same…

The Positive and Negative Consequences of Multiple-Choice Testing

ERIC Educational Resources Information Center

Roediger, Henry L.; Marsh, Elizabeth J.

2005-01-01

Multiple-choice tests are commonly used in educational settings but with unknown effects on students' knowledge. The authors examined the consequences of taking a multiple-choice test on a later general knowledge test in which students were warned not to guess. A large positive testing effect was obtained: Prior testing of facts aided final…

Students' Geographic Knowledge and Skills in Different Kinds of Tests: Multiple-Choice versus Performance Assessment.

ERIC Educational Resources Information Center

Kon, Jane Heckley; Martin-Kniep, Giselle O.

1992-01-01

Describes a case study to determine whether performance tests are a feasible alternative to multiple-choice tests. Examines the difficulties of administering and scoring performance assessments. Explains that the study employed three performance tests and one multiple-choice test. Concludes that performance test administration and scoring was no…

An Empirical Comparison of Five Linear Equating Methods for the NEAT Design

ERIC Educational Resources Information Center

Suh, Youngsuk; Mroch, Andrew A.; Kane, Michael T.; Ripkey, Douglas R.

2009-01-01

In this study, a data base containing the responses of 40,000 candidates to 90 multiple-choice questions was used to mimic data sets for 50-item tests under the "nonequivalent groups with anchor test" (NEAT) design. Using these smaller data sets, we evaluated the performance of five linear equating methods for the NEAT design with five levels of…

COMPUTER TECHNIQUES FOR WEEKLY MULTIPLE-CHOICE TESTING.

ERIC Educational Resources Information Center

BROYLES, DAVID

TO ENCOURAGE POLITICAL SCIENCE STUDENTS TO READ PROPERLY AND CONTINUOUSLY, THE AUTHOR GIVES FREQUENT SHORT QUIZZES BASED ON THE ASSIGNED READINGS. FOR EASE IN ADMINISTRATION AND SCORING, HE USES MARK-SENSE CARDS, ON WHICH THE STUDENT MARKS DESIGNATED AREAS TO INDICATE HIS NUMBER AND HIS CHOICE OF ANSWERS. TO EMPHASIZE THE VALUE OF CONTINUED HIGH…

Meatcutting Testbook, Part 2.

ERIC Educational Resources Information Center

California State Dept. of Education, Sacramento. Bureau of Publications.

This document contains objective tests for each topic in the Meatcutting Workbook, Part 2, which is designed for apprenticeship meatcutting programs in California. Each of the 30 tests consists of from 5 to 65 multiple-choice items with most tests containing approximately 10 items. The tests are grouped according to the eight units of the…

"None of the above" as a correct and incorrect alternative on a multiple-choice test: implications for the testing effect.

PubMed

Odegard, Timothy N; Koen, Joshua D

2007-11-01

Both positive and negative testing effects have been demonstrated with a variety of materials and paradigms (Roediger & Karpicke, 2006b). The present series of experiments replicate and extend the research of Roediger and Marsh (2005) with the addition of a "none-of-the-above" response option. Participants (n=32 in both experiments) read a set of passages, took an initial multiple-choice test, completed a filler task, and then completed a final cued-recall test (Experiment 1) or multiple-choice test (Experiment 2). Questions were manipulated on the initial multiple-choice test by adding a "none-of-the-above" response alternative (choice "E") that was incorrect ("E" Incorrect) or correct ("E" Correct). The results from both experiments demonstrated that the positive testing effect was negated when the "none-of-the-above" alternative was the correct response on the initial multiple-choice test, but was still present when the "none-of-the-above" alternative was an incorrect response.

Physics Achievement Test.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

This document is an evaluation instrument developed as a part of Harvard Project Physics (HPP). It consists of a 36-item, multiple choice (five options) Physics Achievement Test (PAT) designed to measure general knowledge of physics as well as the material emphasized in HPP. (PEB)

Measures of Partial Knowledge and Unexpected Responses in Multiple-Choice Tests

ERIC Educational Resources Information Center

Chang, Shao-Hua; Lin, Pei-Chun; Lin, Zih-Chuan

2007-01-01

This study investigates differences in the partial scoring performance of examinees in elimination testing and conventional dichotomous scoring of multiple-choice tests implemented on a computer-based system. Elimination testing that uses the same set of multiple-choice items rewards examinees with partial knowledge over those who are simply…

Solomon design analysis of multiple-choice Rorschach animal content.

PubMed

Feigenbaum, D; Costello, R M

1975-10-01

The Solomon four-group design was used to study the effects of a persuasive message on a selected multiple-choice Rorschach index--animal content. The independent variable elicited behavior in a predictable manner. Pretesting as a main effect was not significant, but as an interactional effect obviated the effect of the persuasive message. Although knowledge of test rationale can elicit behavior that conforms to experimental demand characteristics, some subjects nonetheless acted in defiance of such information. A condition for defiance in this experimental arrangement, however, was pretesting. Other possibilities regarding the study of compliance behavior and the use of pathognomonic indicators were suggested. Ethical issues were raised.

The Design and Development of a Context-Rich, Photo-Based Online Testing to Assess Students' Science Learning

ERIC Educational Resources Information Center

Lin, Min-Jin; Guo, Chorng-Jee; Hsu, Chia-Er

2011-01-01

This study designed and developed a CP-MCT (content-rich, photo-based multiple choice online test) to assess whether college students can apply the basic light concept to interpret daily light phenomena. One hundred college students volunteered to take the CP-MCT, and the results were statistically analyzed by applying t-test or ANOVA (Analysis of…

Geography Students Assess Their Learning Using Computer-Marked Tests.

ERIC Educational Resources Information Center

Hogg, Jim

1997-01-01

Reports on a pilot study designed to assess the potential of computer-marked tests for allowing students to monitor their learning. Students' answers to multiple choice tests were fed into a computer that provided a full analysis of their strengths and weaknesses. Students responded favorably to the feedback. (MJP)

Meatcutting Testbook, Part I.

ERIC Educational Resources Information Center

Strazicich, Mirko, Ed.

This document contains objective tests for each lesson in the Meatcutting Workbook, Part I (see note), which is designed for apprenticeship programs in meatcutting in California. Each of the 36 tests contains from 10 to 45 multiple-choice items. The tests are grouped according to the eight units of the workbook: the apprentice meatcutter; applied…

Rigorously testing multialternative decision field theory against random utility models.

PubMed

Berkowitsch, Nicolas A J; Scheibehenne, Benjamin; Rieskamp, Jörg

2014-06-01

Cognitive models of decision making aim to explain the process underlying observed choices. Here, we test a sequential sampling model of decision making, multialternative decision field theory (MDFT; Roe, Busemeyer, & Townsend, 2001), on empirical grounds and compare it against 2 established random utility models of choice: the probit and the logit model. Using a within-subject experimental design, participants in 2 studies repeatedly choose among sets of options (consumer products) described on several attributes. The results of Study 1 showed that all models predicted participants' choices equally well. In Study 2, in which the choice sets were explicitly designed to distinguish the models, MDFT had an advantage in predicting the observed choices. Study 2 further revealed the occurrence of multiple context effects within single participants, indicating an interdependent evaluation of choice options and correlations between different context effects. In sum, the results indicate that sequential sampling models can provide relevant insights into the cognitive process underlying preferential choices and thus can lead to better choice predictions. PsycINFO Database Record (c) 2014 APA, all rights reserved.

A Strategy for Replacing Sum Scoring

ERIC Educational Resources Information Center

Ramsay, James O.; Wiberg, Marie

2017-01-01

This article promotes the use of modern test theory in testing situations where sum scores for binary responses are now used. It directly compares the efficiencies and biases of classical and modern test analyses and finds an improvement in the root mean squared error of ability estimates of about 5% for two designed multiple-choice tests and…

Reducing the Need for Guesswork in Multiple-Choice Tests

ERIC Educational Resources Information Center

Bush, Martin

2015-01-01

The humble multiple-choice test is very widely used within education at all levels, but its susceptibility to guesswork makes it a suboptimal assessment tool. The reliability of a multiple-choice test is partly governed by the number of items it contains; however, longer tests are more time consuming to take, and for some subject areas, it can be…

The Impact of Escape Alternative Position Change in Multiple-Choice Test on the Psychometric Properties of a Test and Its Items Parameters

ERIC Educational Resources Information Center

Hamadneh, Iyad Mohammed

2015-01-01

This study aimed at investigating the impact changing of escape alternative position in multiple-choice test on the psychometric properties of a test and it's items parameters (difficulty, discrimination & guessing), and estimation of examinee ability. To achieve the study objectives, a 4-alternative multiple choice type achievement test…

Optimizing multiple-choice tests as tools for learning.

PubMed

Little, Jeri L; Bjork, Elizabeth Ligon

2015-01-01

Answering multiple-choice questions with competitive alternatives can enhance performance on a later test, not only on questions about the information previously tested, but also on questions about related information not previously tested-in particular, on questions about information pertaining to the previously incorrect alternatives. In the present research, we assessed a possible explanation for this pattern: When multiple-choice questions contain competitive incorrect alternatives, test-takers are led to retrieve previously studied information pertaining to all of the alternatives in order to discriminate among them and select an answer, with such processing strengthening later access to information associated with both the correct and incorrect alternatives. Supporting this hypothesis, we found enhanced performance on a later cued-recall test for previously nontested questions when their answers had previously appeared as competitive incorrect alternatives in the initial multiple-choice test, but not when they had previously appeared as noncompetitive alternatives. Importantly, however, competitive alternatives were not more likely than noncompetitive alternatives to be intruded as incorrect responses, indicating that a general increased accessibility for previously presented incorrect alternatives could not be the explanation for these results. The present findings, replicated across two experiments (one in which corrective feedback was provided during the initial multiple-choice testing, and one in which it was not), thus strongly suggest that competitive multiple-choice questions can trigger beneficial retrieval processes for both tested and related information, and the results have implications for the effective use of multiple-choice tests as tools for learning.

Macros for Educational Research.

ERIC Educational Resources Information Center

Woodrow, Janice E. J.

1988-01-01

Describes the design and operation of two macros written in the programming language of Microsoft's EXCEL for educational research applications. The first macro determines the frequency of responses to a Likert-type questionnaire or multiple-choice test; the second performs a one-way analysis of variance test. (Author/LRW)

Are Multiple Choice Tests Fair to Medical Students with Specific Learning Disabilities?

ERIC Educational Resources Information Center

Ricketts, Chris; Brice, Julie; Coombes, Lee

2010-01-01

The purpose of multiple choice tests of medical knowledge is to estimate as accurately as possible a candidate's level of knowledge. However, concern is sometimes expressed that multiple choice tests may also discriminate in undesirable and irrelevant ways, such as between minority ethnic groups or by sex of candidates. There is little literature…

The Effects of Clinically Relevant Multiple-Choice Items on the Statistical Discrimination of Physician Clinical Competence.

ERIC Educational Resources Information Center

Downing, Steven M.; Maatsch, Jack L.

To test the effect of clinically relevant multiple-choice item content on the validity of statistical discriminations of physicians' clinical competence, data were collected from a field test of the Emergency Medicine Examination, test items for the certification of specialists in emergency medicine. Two 91-item multiple-choice subscales were…

Developing, Analyzing, and Using Distractors for Multiple-Choice Tests in Education: A Comprehensive Review

ERIC Educational Resources Information Center

Gierl, Mark J.; Bulut, Okan; Guo, Qi; Zhang, Xinxin

2017-01-01

Multiple-choice testing is considered one of the most effective and enduring forms of educational assessment that remains in practice today. This study presents a comprehensive review of the literature on multiple-choice testing in education focused, specifically, on the development, analysis, and use of the incorrect options, which are also…

Test Design Project: Studies in Test Bias. Annual Report.

ERIC Educational Resources Information Center

McArthur, David

Item bias in a multiple-choice test can be detected by appropriate analyses of the persons x items scoring matrix. This permits comparison of groups of examinees tested with the same instrument. The test may be biased if it is not measuring the same thing in comparable groups, if groups are responding to different aspects of the test items, or if…

Learning From Tests: Facilitation of Delayed Recall by Initial Recognition Alternatives.

ERIC Educational Resources Information Center

Whitten, William B., II; Leonard, Janet Mauriello

1980-01-01

Two experiments were designed to determine the effects of multiple-choice recognition test alternatives on subsequent memory for the correct answers. Results of both experiments are interpreted as demonstrations of the principle that long-term retention is facilitated such that memory evaluation occurs during initial recognition tests. (Author/RD)

Electronics. Criterion-Referenced Test (CRT) Item Bank.

ERIC Educational Resources Information Center

Davis, Diane, Ed.

This document contains 519 criterion-referenced multiple choice and true or false test items for a course in electronics. The test item bank is designed to work with both the Vocational Instructional Management System (VIMS) and the Vocational Administrative Management System (VAMS) in Missouri. The items are grouped into 15 units covering the…

Auto Mechanics. Criterion-Referenced Test (CRT) Item Bank.

ERIC Educational Resources Information Center

Tannehill, Dana, Ed.

This document contains 546 criterion-referenced multiple choice and true or false test items for a course in auto mechanics. The test item bank is designed to work with both the Vocational Instructional Management System (VIMS) and Vocational Administrative Management System (VAMS) in Missouri. The items are grouped into 35 units covering the…

A Comparison of Methods for Transforming Sentences into Test Questions for Instructional Materials. Technical Report #1.

ERIC Educational Resources Information Center

Roid, Gale; And Others

Several measurement theorists have convincingly argued that methods of writing test questions, particularly for criterion-referenced tests, should be based on operationally defined rules. This study was designed to examine and further refine a method for objectively generating multiple-choice questions for prose instructional materials. Important…

Nursing Diagnoses and Caring for Patients with Sickle Cell Disease.

ERIC Educational Resources Information Center

London, Fran

1990-01-01

This continuing education article is designed to teach nurses to describe sickle cell anemia, identify complications, specify signs and symptoms, and describe nursing interventions. It concludes with a multiple-choice test. (SK)

Can Multiple-Choice Testing Induce Desirable Difficulties? Evidence from the Laboratory and the Classroom.

PubMed

Bjork, Elizabeth Ligon; Soderstrom, Nicholas C; Little, Jeri L

2015-01-01

The term desirable difficulties (Bjork, 1994) refers to conditions of learning that, though often appearing to cause difficulties for the learner and to slow down the process of acquisition, actually improve long-term retention and transfer. One known desirable difficulty is testing (as compared with restudy), although typically it is tests that clearly involve retrieval--such as free and cued recall tests--that are thought to induce these learning benefits and not multiple-choice tests. Nonetheless, multiple-choice testing is ubiquitous in educational settings and many other high-stakes situations. In this article, we discuss research, in both the laboratory and the classroom, exploring whether multiple-choice testing can also be fashioned to promote the type of retrieval processes known to improve learning, and we speculate about the necessary properties that multiple-choice questions must possess, as well as the metacognitive strategy students need to use in answering such questions, to achieve this goal.

All of the above: When multiple correct response options enhance the testing effect.

PubMed

Bishara, Anthony J; Lanzo, Lauren A

2015-01-01

Previous research has shown that multiple choice tests often improve memory retention. However, the presence of incorrect lures often attenuates this memory benefit. The current research examined the effects of "all of the above" (AOTA) options. When such options are correct, no incorrect lures are present. In the first three experiments, a correct AOTA option on an initial test led to a larger memory benefit than no test and standard multiple choice test conditions. The benefits of a correct AOTA option occurred even without feedback on the initial test; for both 5-minute and 48-hour retention delays; and for both cued recall and multiple choice final test formats. In the final experiment, an AOTA question led to better memory retention than did a control condition that had identical timing and exposure to response options. However, the benefits relative to this control condition were similar regardless of the type of multiple choice test (AOTA or not). Results suggest that retrieval contributes to multiple choice testing effects. However, the extra testing effect from a correct AOTA option, rather than being due to more retrieval, might be due simply to more exposure to correct information.

Effects of Mayfield's Four Questions (M4Q) on Nursing Students' Self-Efficacy and Multiple-Choice Test Scores

ERIC Educational Resources Information Center

Mayfield, Linda Riggs

2010-01-01

This study examined the effects of being taught the Mayfield's Four Questions multiple-choice test-taking strategy on the perceived self-efficacy and multiple-choice test scores of nursing students in a two-year associate degree program. Experimental and control groups were chosen by stratified random sampling. Subjects completed the 10-statement…

Tailoring auditory training to patient needs with single and multiple talkers: transfer-appropriate gains on a four-choice discrimination test.

PubMed

Barcroft, Joe; Sommers, Mitchell S; Tye-Murray, Nancy; Mauzé, Elizabeth; Schroy, Catherine; Spehar, Brent

2011-11-01

Our long-term objective is to develop an auditory training program that will enhance speech recognition in those situations where patients most want improvement. As a first step, the current investigation trained participants using either a single talker or multiple talkers to determine if auditory training leads to transfer-appropriate gains. The experiment implemented a 2 × 2 × 2 mixed design, with training condition as a between-participants variable and testing interval and test version as repeated-measures variables. Participants completed a computerized six-week auditory training program wherein they heard either the speech of a single talker or the speech of six talkers. Training gains were assessed with single-talker and multi-talker versions of the Four-choice discrimination test. Participants in both groups were tested on both versions. Sixty-nine adult hearing-aid users were randomly assigned to either single-talker or multi-talker auditory training. Both groups showed significant gains on both test versions. Participants who trained with multiple talkers showed greater improvement on the multi-talker version whereas participants who trained with a single talker showed greater improvement on the single-talker version. Transfer-appropriate gains occurred following auditory training, suggesting that auditory training can be designed to target specific patient needs.

An Instrument to Predict Job Performance of Home Health Aides--Testing the Reliability and Validity.

ERIC Educational Resources Information Center

Sturges, Jack; Quina, Patricia

The development of four paper-and-pencil tests, useful in assessing the effectiveness of inservice training provided to either nurses aides or home health aides, was described. These tests were designed for utilization in employment selection and case assignment. Two tests of 37 multiple-choice items and two tests of 10 matching items were…

On the Equivalence of Constructed-Response and Multiple-Choice Tests.

ERIC Educational Resources Information Center

Traub, Ross E.; Fisher, Charles W.

Two sets of mathematical reasoning and two sets of verbal comprehension items were cast into each of three formats--constructed response, standard multiple-choice, and Coombs multiple-choice--in order to assess whether tests with indentical content but different formats measure the same attribute, except for possible differences in error variance…

Criterion Referenced Inventory. Grade 7 Skill Clusters, Objectives, and Illustrations.

ERIC Educational Resources Information Center

Montgomery County Public Schools, Rockville, MD.

Part of a series of competency-based test materials for grades six through ten, this test booklet for seventh graders contains multiple-choice questions designed to aid in the evaluation of the pupils' library skills. Accompanied by a separate booklet of illustrations which are to be used in conjunction with the questions, the test covers the…

The Validity of the Major Field Test in Psychology as a Programme Assessment Tool

ERIC Educational Resources Information Center

Gallagher, Shawn P.; Cook, Shaun P.

2013-01-01

The Major Field Test in Psychology (MFT) is a standardised test designed to assess subject mastery at the conclusion of an undergraduate career. Eighty-one graduating majors completed the MFT and 56 of them also took a multiple-choice exam of questions drawn randomly from an introductory psychology test bank. Like the MFT, the constructed exam was…

Understanding Test-Takers' Perceptions of Difficulty in EAP Vocabulary Tests: The Role of Experiential Factors

ERIC Educational Resources Information Center

Oruç Ertürk, Nesrin; Mumford, Simon E.

2017-01-01

This study, conducted by two researchers who were also multiple-choice question (MCQ) test item writers at a private English-medium university in an English as a foreign language (EFL) context, was designed to shed light on the factors that influence test-takers' perceptions of difficulty in English for academic purposes (EAP) vocabulary, with the…

Comparison of paragraph comprehension test scores with reading versus listening-reading and multiple-choice versus nominal recall administration techniques: justification for the bypass approach.

PubMed

Weinberg, W A; McLean, A; Snider, R L; Rintelmann, J W; Brumback, R A

1989-12-01

Eight groups of learning disabled children (N = 100), categorized by the clinical Lexical Paradigm as good readers or poor readers, were individually administered the Gilmore Oral Reading Test, Form D, by one of four input/retrieval methods: (1) the standardized method of administration in which the child reads each paragraph aloud and then answers five questions relating to the paragraph [read/recall method]; (2) the child reads each paragraph aloud and then for each question selects the correct answer from among three choices read by the examiner [read/choice method]; (3) the examiner reads each paragraph aloud and reads each of the five questions to the child to answer [listen/recall method]; and (4) the examiner reads each paragraph aloud and then for each question reads three multiple-choice answers from which the child selects the correct answer [listen/choice method]. The major difference in scores was between the groups tested by the recall versus the orally read multiple-choice methods. This study indicated that poor readers who listened to the material and were tested by orally read multiple-choice format could perform as well as good readers. The performance of good readers was not affected by listening or by the method of testing. The multiple-choice testing improved the performance of poor readers independent of the input method. This supports the arguments made previously that a "bypass approach" to education of poor readers in which testing is accomplished using an orally read multiple-choice format can enhance the child's school performance on reading-related tasks. Using a listening while reading input method may further enhance performance.

Testing Collective Memory: Representing the Soviet Union on Multiple-Choice Questions

ERIC Educational Resources Information Center

Reich, Gabriel A.

2011-01-01

This article tests the assumption that state-mandated multiple-choice history exams are a cultural tool for disseminating an "official" collective memory. Findings from a qualitative study of a collection of multiple-choice questions that relate to the history of the Soviet Union are presented. The 263 questions all come from New York…

The Relationship of Deep and Surface Study Approaches on Factual and Applied Test-Bank Multiple-Choice Question Performance

ERIC Educational Resources Information Center

Yonker, Julie E.

2011-01-01

With the advent of online test banks and large introductory classes, instructors have often turned to textbook publisher-generated multiple-choice question (MCQ) exams in their courses. Multiple-choice questions are often divided into categories of factual or applied, thereby implicating levels of cognitive processing. This investigation examined…

Mechanical Waves Conceptual Survey: Its Modification and Conversion to a Standard Multiple-Choice Test

ERIC Educational Resources Information Center

Barniol, Pablo; Zavala, Genaro

2016-01-01

In this article we present several modifications of the mechanical waves conceptual survey, the most important test to date that has been designed to evaluate university students' understanding of four main topics in mechanical waves: propagation, superposition, reflection, and standing waves. The most significant changes are (i) modification of…

Selected Test Items in American History. Bulletin Number 6, Fifth Edition.

ERIC Educational Resources Information Center

Anderson, Howard R.; Lindquist, E. F.

Designed for high school students, this bulletin provides an extensive file of 1,062 multiple-choice questions in American history. Taken largely from the Iowa Every-Pupil Program and the Cooperative Test Service standardized examinations, the questions are chronologically divided into 16 topic areas. They include exploration and discovery;…

Developing and Validating Proof Comprehension Tests in Undergraduate Mathematics

ERIC Educational Resources Information Center

Mejía-Ramos, Juan Pablo; Lew, Kristen; de la Torre, Jimmy; Weber, Keith

2017-01-01

In this article, we describe and illustrate the process by which we developed and validated short, multiple-choice, reliable tests to assess undergraduate students' comprehension of three mathematical proofs. We discuss the purpose for each stage and how it benefited the design of our instruments. We also suggest ways in which this process could…

Force, Velocity, and Work: The Effects of Different Contexts on Students' Understanding of Vector Concepts Using Isomorphic Problems

ERIC Educational Resources Information Center

Barniol, Pablo; Zavala, Genaro

2014-01-01

In this article we compare students' understanding of vector concepts in problems with no physical context, and with three mechanics contexts: force, velocity, and work. Based on our "Test of Understanding of Vectors," a multiple-choice test presented elsewhere, we designed two isomorphic shorter versions of 12 items each: a test with no…

Applied Reading Test--Forms A and B, Interim Manual, and Answer Sheets.

ERIC Educational Resources Information Center

Australian Council for Educational Research, Hawthorn.

Designed for use in the selection of apprentices, trainees, technical and trade personnel, and any other persons who need to read and understand text of a technical nature, this Applied Reading Test specimen set contains six passages and 32 items, has a 30-minute time limit, and is presented in a reusable multiple choice test booklet. The specimen…

Immediate vs. Delayed Feedback in a Computer-Managed Test: Effects on Long-Term Retention. Technical Report, March 1976-August 1976.

ERIC Educational Resources Information Center

Sturges, Persis T.

This experiment was designed to test the effect of immediate and delayed feedback on retention of learning in an educational situation. Four groups of college undergraduates took a multiple-choice computer-managed test. Three of these groups received informative feedback (the entire item with the correct answer identified) either: (1) immediately…

Multiple-choice pretesting potentiates learning of related information.

PubMed

Little, Jeri L; Bjork, Elizabeth Ligon

2016-10-01

Although the testing effect has received a substantial amount of empirical attention, such research has largely focused on the effects of tests given after study. The present research examines the effect of using tests prior to study (i.e., as pretests), focusing particularly on how pretesting influences the subsequent learning of information that is not itself pretested but that is related to the pretested information. In Experiment 1, we found that multiple-choice pretesting was better for the learning of such related information than was cued-recall pretesting or a pre-fact-study control condition. In Experiment 2, we found that the increased learning of non-pretested related information following multiple-choice testing could not be attributed to increased time allocated to that information during subsequent study. Last, in Experiment 3, we showed that the benefits of multiple-choice pretesting over cued-recall pretesting for the learning of related information persist over 48 hours, thus demonstrating the promise of multiple-choice pretesting to potentiate learning in educational contexts. A possible explanation for the observed benefits of multiple-choice pretesting for enhancing the effectiveness with which related nontested information is learned during subsequent study is discussed.

Evidence-based point-of-care tests and device designs for disaster preparedness.

PubMed

Brock, T Keith; Mecozzi, Daniel M; Sumner, Stephanie; Kost, Gerald J

2010-01-01

To define pathogen tests and device specifications needed for emerging point-of-care (POC) technologies used in disasters. Surveys included multiple-choice and ranking questions. Multiple-choice questions were analyzed with the chi2 test for goodness-of-fit and the binomial distribution test. Rankings were scored and compared using analysis of variance and Tukey's multiple comparison test. Disaster care experts on the editorial boards of the American Journal of Disaster Medicine and the Disaster Medicine and Public Health Preparedness, and the readers of the POC Journal. Vibrio cholera and Staphylococcus aureus were top-ranked pathogens for testing in disaster settings. Respondents felt that disaster response teams should be equipped with pandemic infectious disease tests for novel 2009 H1N1 and avian H5N1 influenza (disaster care, p < 0.05; POC, p < 0.01). In disaster settings, respondents preferred self-contained test cassettes (disaster care, p < 0.05; POC, p < 0.001) for direct blood sampling (POC, p < 0.01) and disposal of biological waste (disaster care, p < 0.05; POC, p < 0.001). Multiplex testing performed at the POC was preferred in urgent care and emergency room settings. Evidence-based needs assessment identifies pathogen detection priorities in disaster care scenarios, in which Vibrio cholera, methicillin-sensitive and methicillin-resistant Staphylococcus aureus, and Escherichia coli ranked the highest. POC testing should incorporate setting-specific design criteria such as safe disposable cassettes and direct blood sampling at the site of care.

Test of understanding of vectors: A reliable multiple-choice vector concept test

NASA Astrophysics Data System (ADS)

Barniol, Pablo; Zavala, Genaro

2014-06-01

In this article we discuss the findings of our research on students' understanding of vector concepts in problems without physical context. First, we develop a complete taxonomy of the most frequent errors made by university students when learning vector concepts. This study is based on the results of several test administrations of open-ended problems in which a total of 2067 students participated. Using this taxonomy, we then designed a 20-item multiple-choice test [Test of understanding of vectors (TUV)] and administered it in English to 423 students who were completing the required sequence of introductory physics courses at a large private Mexican university. We evaluated the test's content validity, reliability, and discriminatory power. The results indicate that the TUV is a reliable assessment tool. We also conducted a detailed analysis of the students' understanding of the vector concepts evaluated in the test. The TUV is included in the Supplemental Material as a resource for other researchers studying vector learning, as well as instructors teaching the material.

To Show or Not to Show: The Effects of Item Stems and Answer Options on Performance on a Multiple-Choice Listening Comprehension Test

ERIC Educational Resources Information Center

Yanagawa, Kozo; Green, Anthony

2008-01-01

The purpose of this study is to examine whether the choice between three multiple-choice listening comprehension test formats results in any difference in listening comprehension test performance. The three formats entail (a) allowing test takers to preview both the question stem and answer options prior to listening; (b) allowing test takers to…

Detecting a Gender-Related Differential Item Functioning Using Transformed Item Difficulty

ERIC Educational Resources Information Center

Abedalaziz, Nabeel; Leng, Chin Hai; Alahmadi, Ahlam

2014-01-01

The purpose of the study was to examine gender differences in performance on multiple-choice mathematical ability test, administered within the context of high school graduation test that was designed to match eleventh grade curriculum. The transformed item difficulty (TID) was used to detect a gender related DIF. A random sample of 1400 eleventh…

Industrial Arts Test Development, Book III. Resource Items for Graphics Technology, Power Technology, Production Technology.

ERIC Educational Resources Information Center

New York State Education Dept., Albany.

This booklet is designed to assist teachers in developing examinations for classroom use. It is a collection of 955 objective test questions, mostly multiple choice, for industrial arts students in the three areas of graphics technology, power technology, and production technology. Scoring keys are provided. There are no copyright restrictions,…

Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper

PubMed Central

Palmer, Edward J; Devitt, Peter G

2007-01-01

Background Reliable and valid written tests of higher cognitive function are difficult to produce, particularly for the assessment of clinical problem solving. Modified Essay Questions (MEQs) are often used to assess these higher order abilities in preference to other forms of assessment, including multiple-choice questions (MCQs). MEQs often form a vital component of end-of-course assessments in higher education. It is not clear how effectively these questions assess higher order cognitive skills. This study was designed to assess the effectiveness of the MEQ to measure higher-order cognitive skills in an undergraduate institution. Methods An analysis of multiple-choice questions and modified essay questions (MEQs) used for summative assessment in a clinical undergraduate curriculum was undertaken. A total of 50 MCQs and 139 stages of MEQs were examined, which came from three exams run over two years. The effectiveness of the questions was determined by two assessors and was defined by the questions ability to measure higher cognitive skills, as determined by a modification of Bloom's taxonomy, and its quality as determined by the presence of item writing flaws. Results Over 50% of all of the MEQs tested factual recall. This was similar to the percentage of MCQs testing factual recall. The modified essay question failed in its role of consistently assessing higher cognitive skills whereas the MCQ frequently tested more than mere recall of knowledge. Conclusion Construction of MEQs, which will assess higher order cognitive skills cannot be assumed to be a simple task. Well-constructed MCQs should be considered a satisfactory replacement for MEQs if the MEQs cannot be designed to adequately test higher order skills. Such MCQs are capable of withstanding the intellectual and statistical scrutiny imposed by a high stakes exit examination. PMID:18045500

Low-cost and high-speed optical mark reader based on an intelligent line camera

NASA Astrophysics Data System (ADS)

Hussmann, Stephan; Chan, Leona; Fung, Celine; Albrecht, Martin

2003-08-01

Optical Mark Recognition (OMR) is thoroughly reliable and highly efficient provided that high standards are maintained at both the planning and implementation stages. It is necessary to ensure that OMR forms are designed with due attention to data integrity checks, the best use is made of features built into the OMR, used data integrity is checked before the data is processed and data is validated before it is processed. This paper describes the design and implementation of an OMR prototype system for marking multiple-choice tests automatically. Parameter testing is carried out before the platform and the multiple-choice answer sheet has been designed. Position recognition and position verification methods have been developed and implemented in an intelligent line scan camera. The position recognition process is implemented into a Field Programmable Gate Array (FPGA), whereas the verification process is implemented into a micro-controller. The verified results are then sent to the Graphical User Interface (GUI) for answers checking and statistical analysis. At the end of the paper the proposed OMR system will be compared with commercially available system on the market.

Wrong Answers on Multiple-Choice Achievement Tests: Blind Guesses or Systematic Choices?.

ERIC Educational Resources Information Center

Powell, J. C.

A multi-faceted model for the selection of answers for multiple-choice tests was developed from the findings of a series of exploratory studies. This model implies that answer selection should be curvilinear. A series of models were tested for fit using the chi square procedure. Data were collected from 359 elementary school students ages 9-12.…

Multiple Choice Items: How to Gain the Most out of Them.

ERIC Educational Resources Information Center

Talmir, Pinchas

1991-01-01

Describes how multiple-choice items can be designed and used as an effective diagnostic tool by avoiding their pitfalls and by taking advantage of their potential benefits. The following issues are discussed: correct' versus best answers; construction of diagnostic multiple-choice items; the problem of guessing; the use of justifications of…

Difficulty and Discriminability of Introductory Psychology Test Items.

ERIC Educational Resources Information Center

Scialfa, Charles; Legare, Connie; Wenger, Larry; Dingley, Louis

2001-01-01

Analyzes multiple-choice questions provided in test banks for introductory psychology textbooks. Study 1 offered a consistent picture of the objective difficulty of multiple-choice tests for introductory psychology students, while both studies 1 and 2 indicated that test items taken from commercial test banks have poor psychometric properties.…

Optimizing Multiple-Choice Tests as Learning Events

ERIC Educational Resources Information Center

Little, Jeri Lynn

2011-01-01

Although generally used for assessment, tests can also serve as tools for learning--but different test formats may not be equally beneficial. Specifically, research has shown multiple-choice tests to be less effective than cued-recall tests in improving the later retention of the tested information (e.g., see meta-analysis by Hamaker, 1986),…

Mechanical waves conceptual survey: Its modification and conversion to a standard multiple-choice test

NASA Astrophysics Data System (ADS)

Barniol, Pablo; Zavala, Genaro

2016-06-01

In this article we present several modifications of the mechanical waves conceptual survey, the most important test to date that has been designed to evaluate university students' understanding of four main topics in mechanical waves: propagation, superposition, reflection, and standing waves. The most significant changes are (i) modification of several test questions that had some problems in their original design, (ii) standardization of the number of options for each question to five, (iii) conversion of the two-tier questions to multiple-choice questions, and (iv) modification of some questions to make them independent of others. To obtain a final version of the test, we administered both the original and modified versions several times to students at a large private university in Mexico. These students were completing a course that covers the topics tested by the survey. The final modified version of the test was administered to 234 students. In this study we present the modifications for each question, and discuss the reasons behind them. We also analyze the results obtained by the final modified version and offer a comparison between the original and modified versions. In the Supplemental Material we present the final modified version of the test. It can be used by teachers and researchers to assess students' understanding of, and learning about, mechanical waves.

Set of Criteria for Efficiency of the Process Forming the Answers to Multiple-Choice Test Items

ERIC Educational Resources Information Center

Rybanov, Alexander Aleksandrovich

2013-01-01

Is offered the set of criteria for assessing efficiency of the process forming the answers to multiple-choice test items. To increase accuracy of computer-assisted testing results, it is suggested to assess dynamics of the process of forming the final answer using the following factors: loss of time factor and correct choice factor. The model…

Step by Step: Biology Undergraduates' Problem-Solving Procedures during Multiple-Choice Assessment

ERIC Educational Resources Information Center

Prevost, Luanna B.; Lemons, Paula P.

2016-01-01

This study uses the theoretical framework of domain-specific problem solving to explore the procedures students use to solve multiple-choice problems about biology concepts. We designed several multiple-choice problems and administered them on four exams. We trained students to produce written descriptions of how they solved the problem, and this…

Introducing Standardized EFL/ESL Exams

ERIC Educational Resources Information Center

Laborda, Jesus Garcia

2007-01-01

This article presents the features, and a brief comparison, of some of the most well-known high-stakes exams. They are classified in the following fashion: tests that only include multiple-choice questions, tests that include writing and multiple-choice questions, and tests that include speaking questions. The tests reviewed are: BULATS, IELTS,…

Criterion Referenced Assessment Bank. Grade 6 Skill Clusters, Objectives, and Illustrations.

ERIC Educational Resources Information Center

Montgomery County Public Schools, Rockville, MD.

Part of a series of competency-based test materials for grades six through ten, this set of nine test booklets for sixth graders contains multiple-choice questions designed to aid in the evaluation of the pupils' library skills. Accompanied by a separate, tenth booklet of illustrations which are to be used in conjunction with the questions, the…

catcher: A Software Program to Detect Answer Copying in Multiple-Choice Tests Based on Nominal Response Model

ERIC Educational Resources Information Center

Kalender, Ilker

2012-01-01

catcher is a software program designed to compute the [omega] index, a common statistical index for the identification of collusions (cheating) among examinees taking an educational or psychological test. It requires (a) responses and (b) ability estimations of individuals, and (c) item parameters to make computations and outputs the results of…

Evaluation of an Intervention Instructional Program to Facilitate Understanding of Basic Particle Concepts among Students Enrolled in Several Levels of Study

ERIC Educational Resources Information Center

Treagust, David F.; Chandrasegaran, A. L.; Zain, Ahmad N. M.; Ong, Eng Tek; Karpudewan, Mageswary; Halim, Lilia

2011-01-01

The efficacy of an intervention instructional program was evaluated to facilitate understanding of particle theory concepts among students (N = 190) using a diagnostic instrument consisting of eleven two-tier multiple-choice items in a pre-test--post-test design. The students involved were high school students, undergraduates and postgraduates…

Measuring more than we know? An examination of the motivational and situational influences in science achievement

NASA Astrophysics Data System (ADS)

Haydel, Angela Michelle

The purpose of this dissertation was to advance theoretical understanding about fit between the personal resources of individuals and the characteristics of science achievement tasks. Testing continues to be pervasive in schools, yet we know little about how students perceive tests and what they think and feel while they are actually working on test items. This study focused on both the personal (cognitive and motivational) and situational factors that may contribute to individual differences in achievement-related outcomes. 387 eighth grade students first completed a survey including measures of science achievement goals, capability beliefs, efficacy related to multiple-choice items and performance assessments, validity beliefs about multiple-choice items and performance assessments, and other perceptions of these item formats. Students then completed science achievement tests including multiple-choice items and two performance assessments. A sample of students was asked to verbalize both thoughts and feelings as they worked through the test items. These think-alouds were transcribed and coded for evidence of cognitive, metacognitive and motivational engagement. Following each test, all students completed measures of effort, mood, energy level and strategy use during testing. Students reported that performance assessments were more challenging, authentic, interesting and valid than multiple-choice tests. They also believed that comparisons between students were easier using multiple-choice items. Overall, students tried harder, felt better, had higher levels of energy and used more strategies while working on performance assessments. Findings suggested that performance assessments might be more congruent with a mastery achievement goal orientation, while multiple-choice tests might be more congruent with a performance achievement goal orientation. A variable-centered analytic approach including regression analyses provided information about how students, on average, who differed in terms of their teachers' ratings of their science ability, achievement goals, capability beliefs and experiences with science achievement tasks perceived, engaged in, and performed on multiple-choice items and performance assessments. Person-centered analyses provided information about the perceptions, engagement and performance of subgroups of individuals who had different motivational characteristics. Generally, students' personal goals and capability beliefs related more strongly to test perceptions, but not performance, while teacher ratings of ability and test-specific beliefs related to performance.

Validation and structural analysis of the kinematics concept test

NASA Astrophysics Data System (ADS)

Lichtenberger, A.; Wagner, C.; Hofer, S. I.; Stern, E.; Vaterlaus, A.

2017-06-01

The kinematics concept test (KCT) is a multiple-choice test designed to evaluate students' conceptual understanding of kinematics at the high school level. The test comprises 49 multiple-choice items about velocity and acceleration, which are based on seven kinematic concepts and which make use of three different representations. In the first part of this article we describe the development and the validation process of the KCT. We applied the KCT to 338 Swiss high school students who attended traditional teaching in kinematics. We analyzed the response data to provide the psychometric properties of the test. In the second part we present the results of a structural analysis of the test. An exploratory factor analysis of 664 student answers finally uncovered the seven kinematics concepts as factors. However, the analysis revealed a hierarchical structure of concepts. At the higher level, mathematical concepts group together, and then split up into physics concepts at the lower level. Furthermore, students who seem to understand a concept in one representation have difficulties transferring the concept to similar problems in another representation. Both results have implications for teaching kinematics. First, teaching mathematical concepts beforehand might be beneficial for learning kinematics. Second, instructions have to be designed to teach students the change between different representations.

Multiple-Choice Test Bias Due to Answering Strategy Variation.

ERIC Educational Resources Information Center

Frary, Robert B.; Giles, Mary B.

This paper describes the development and investigation of a new approach to determining the existence of bias in multiple-choice test scores. Previous work in this area has concentrated almost exclusively on bias attributable to specific test items or to differences in test score distributions across racial or ethnic groups. In contrast, the…

Evaluation of Performance and Perceptions of Electronic vs. Paper Multiple-Choice Exams

ERIC Educational Resources Information Center

Washburn, Shannon; Herman, James; Stewart, Randolph

2017-01-01

In the veterinary professional curriculum, methods of examination in many courses are transitioning from the traditional paper-based exams to electronic-based exams. Therefore, a controlled trial to evaluate the impact of testing methodology on examination performance in a veterinary physiology course was designed and implemented. Formalized…

Assessing Pupils' Skills in Experimentation

ERIC Educational Resources Information Center

Hammann, Marcus; Phan, Thi Thanh Hoi; Ehmer, Maike; Grimm, Tobias

2008-01-01

This study is concerned with different forms of assessment of pupils' skills in experimentation. The findings of three studies are reported. Study 1 investigates whether it is possible to develop reliable multiple-choice tests for the skills of forming hypotheses, designing experiments and analysing experimental data. Study 2 compares scores from…

E-Assessment as a Service

ERIC Educational Resources Information Center

Amelung, M.; Krieger, K.; Rosner, D.

2011-01-01

Assessment is an essential element in learning processes. It is therefore not unsurprising that almost all learning management systems (LMSs) offer support for assessment, e.g., for the creation, execution, and evaluation of multiple choice tests. We have designed and implemented generic support for assessment that is based on assignments that…

Cognitive Diagnostic Models for Tests with Multiple-Choice and Constructed-Response Items

ERIC Educational Resources Information Center

Kuo, Bor-Chen; Chen, Chun-Hua; Yang, Chih-Wei; Mok, Magdalena Mo Ching

2016-01-01

Traditionally, teachers evaluate students' abilities via their total test scores. Recently, cognitive diagnostic models (CDMs) have begun to provide information about the presence or absence of students' skills or misconceptions. Nevertheless, CDMs are typically applied to tests with multiple-choice (MC) items, which provide less diagnostic…

Samejima Items in Multiple-Choice Tests: Identification and Implications

ERIC Educational Resources Information Center

Rahman, Nazia

2013-01-01

Samejima hypothesized that non-monotonically increasing item response functions (IRFs) of ability might occur for multiple-choice items (referred to here as "Samejima items") if low ability test takers with some, though incomplete, knowledge or skill are drawn to a particularly attractive distractor, while very low ability test takers…

Making the Most of Multiple Choice

ERIC Educational Resources Information Center

Brookhart, Susan M.

2015-01-01

Multiple-choice questions draw criticism because many people perceive they test only recall or atomistic, surface-level objectives and do not require students to think. Although this can be the case, it does not have to be that way. Susan M. Brookhart suggests that multiple-choice questions are a useful part of any teacher's questioning repertoire…

Using Multiple-Choice Questions to Evaluate In-Depth Learning of Economics

ERIC Educational Resources Information Center

Buckles, Stephen; Siegfried, John J.

2006-01-01

Multiple-choice questions are the basis of a significant portion of assessment in introductory economics courses. However, these questions, as found in course assessments, test banks, and textbooks, often fail to evaluate students' abilities to use and apply economic analysis. The authors conclude that multiple-choice questions can be used to…

Emergency Medical Care. A Manual for the Paramedic in the Field--Workbook.

ERIC Educational Resources Information Center

National Highway Traffic Safety Administration (DOT), Washington, DC.

This workbook is designed to accompany the text of the same name and to serve as an aid to both learning and review during the course of study. The workbook consists of 15 module self-tests and vocabulary lists that follow the modules of the text. Tests consist of objective questions (multiple choice, fill-in-the-blank, short answers, and…

Format of Options in Multiple Choice Test vis-a-vis Test Performance

ERIC Educational Resources Information Center

Bendulo, Hermabeth O.; Tibus, Erlinda D.; Bande, Rhodora A.; Oyzon, Voltaire Q.; Milla, Norberto E.; Macalinao, Myrna L.

2017-01-01

Testing or evaluation in an educational context is primarily used to measure or evaluate and authenticate the academic readiness, learning advancement, acquisition of skills, or instructional needs of learners. This study tried to determine whether the varied combinations of arrangements of options and letter cases in a Multiple-Choice Test (MCT)…

Some Effects of Changes in Question Structure and Sequence on Performance in a Multiple Choice Chemistry Test.

ERIC Educational Resources Information Center

Hodson, D.

1984-01-01

Investigated the effect on student performance of changes in question structure and sequence on a GCE 0-level multiple-choice chemistry test. One finding noted is that there was virtually no change in test reliability on reducing the number of options (from five to per test item). (JN)

Sex Differences in the Tendency to Omit Items on Multiple-Choice Tests: 1980-2000

ERIC Educational Resources Information Center

von Schrader, Sarah; Ansley, Timothy

2006-01-01

Much has been written concerning the potential group differences in responding to multiple-choice achievement test items. This discussion has included references to possible disparities in tendency to omit such test items. When test scores are used for high-stakes decision making, even small differences in scores and rankings that arise from male…

Equal Opportunity in the Classroom: Test Construction in a Diversity-Sensitive Environment.

ERIC Educational Resources Information Center

Ghorpade, Jai; Lackritz, James R.

1998-01-01

Two multiple-choice tests and one essay test were taken by 231 students (50/50 male/female, 192 White, 39 East Asian, Black, Mexican American, or Middle Eastern). Multiple-choice tests showed no significant differences in equal employment opportunity terms; women and men scored about the same on essays, but minority students had significantly…

Measuring the Consistency in Change in Hepatitis B Knowledge among Three Different Types of Tests: True/False, Multiple Choice, and Fill in the Blanks Tests.

ERIC Educational Resources Information Center

Sahai, Vic; Demeyere, Petra; Poirier, Sheila; Piro, Felice

1998-01-01

The recall of information about Hepatitis B demonstrated by 180 seventh graders was tested with three test types: (1) short-answer; (2) true/false; and (3) multiple-choice. Short answer testing was the most reliable. Suggestions are made for the use of short-answer tests in evaluating student knowledge. (SLD)

Quality Multiple-Choice Test Questions: Item-Writing Guidelines and an Analysis of Auditing Testbanks.

ERIC Educational Resources Information Center

Hansen, James D.; Dexter, Lee

1997-01-01

Analysis of test item banks in 10 auditing textbooks found that 75% of questions violated one or more guidelines for multiple-choice items. In comparison, 70% of a certified public accounting exam bank had no violations. (SK)

Assessing Multiple Choice Question (MCQ) Tests--A Mathematical Perspective

ERIC Educational Resources Information Center

Scharf, Eric M.; Baldwin, Lynne P.

2007-01-01

The reasoning behind popular methods for analysing the raw data generated by multiple choice question (MCQ) tests is not always appreciated, occasionally with disastrous results. This article discusses and analyses three options for processing the raw data produced by MCQ tests. The article shows that one extreme option is not to penalize a…

Piloting a Polychotomous Partial-Credit Scoring Procedure in a Multiple-Choice Test

ERIC Educational Resources Information Center

Tsopanoglou, Antonios; Ypsilandis, George S.; Mouti, Anna

2014-01-01

Multiple-choice (MC) tests are frequently used to measure language competence because they are quick, economical and straightforward to score. While degrees of correctness have been investigated for partially correct responses in combined-response MC tests, degrees of incorrectness in distractors and the role they play in determining the…

Preliminary Findings on the Computer-Administered Multiple-Choice Online Causal Comprehension Assessment, a Diagnostic Reading Comprehension Test

ERIC Educational Resources Information Center

Davison, Mark L.; Biancarosa, Gina; Carlson, Sarah E.; Seipel, Ben; Liu, Bowen

2018-01-01

The computer-administered Multiple-Choice Online Causal Comprehension Assessment (MOCCA) for Grades 3 to 5 has an innovative, 40-item multiple-choice structure in which each distractor corresponds to a comprehension process upon which poor comprehenders have been shown to rely. This structure requires revised thinking about measurement issues…

Aligning Items and Achievement Levels: A Study Comparing Expert Judgments

ERIC Educational Resources Information Center

Kaliski, Pamela; Huff, Kristen; Barry, Carol

2011-01-01

For educational achievement tests that employ multiple-choice (MC) items and aim to reliably classify students into performance categories, it is critical to design MC items that are capable of discriminating student performance according to the stated achievement levels. This is accomplished, in part, by clearly understanding how item design…

Assessing Community College Student Knowledge in the Liberal Arts

ERIC Educational Resources Information Center

Cohen, Arthur M.; Schuetz, Pam; Chang, June C.; Plecha, Michelle

2003-01-01

The General Academic Learning Experience (GALE) is an assessment of community college student knowledge in the liberal arts. The study involved the design and administration of an instrument, which included a demographic survey and a multiple-choice content test. In total, over 2,500 students from 10 colleges in Southern California participated.…

A New Internet Tool for Automatic Evaluation in Control Systems and Programming

ERIC Educational Resources Information Center

Munoz de la Pena, D.; Gomez-Estern, F.; Dormido, S.

2012-01-01

In this paper we present a web-based innovative education tool designed for automating the collection, evaluation and error detection in practical exercises assigned to computer programming and control engineering students. By using a student/instructor code-fusion architecture, the conceptual limits of multiple-choice tests are overcome by far.…

Development and Validation of the Conceptual Assessment of Natural Selection (CANS)

ERIC Educational Resources Information Center

Kalinowski, Steven T.; Leonard, Mary J.; Taper, Mark L.

2016-01-01

We developed and validated the Conceptual Assessment of Natural Selection (CANS), a multiple-choice test designed to assess how well college students understand the central principles of natural selection. The expert panel that reviewed the CANS concluded its questions were relevant to natural selection and generally did a good job sampling the…

Post-Graduate Student Performance in "Supervised In-Class" vs. "Unsupervised Online" Multiple Choice Tests: Implications for Cheating and Test Security

ERIC Educational Resources Information Center

Ladyshewsky, Richard K.

2015-01-01

This research explores differences in multiple choice test (MCT) scores in a cohort of post-graduate students enrolled in a management and leadership course. A total of 250 students completed the MCT in either a supervised in-class paper and pencil test or an unsupervised online test. The only statistically significant difference between the nine…

Do Sequentially-Presented Answer Options Prevent the Use of Testwiseness Cues on Continuing Medical Education Tests?

ERIC Educational Resources Information Center

Willing, Sonja; Ostapczuk, Martin; Musch, Jochen

2015-01-01

Testwiseness--that is, the ability to find subtle cues towards the solution by the simultaneous comparison of the available answer options--threatens the validity of multiple-choice (MC) tests. Discrete-option multiple-choice (DOMC) has recently been proposed as a computerized alternative testing format for MC tests, and presumably allows for a…

Developing Achievement Test: A Research for Assessment of 5th Grade Biology Subject

ERIC Educational Resources Information Center

Sener, Nilay; Tas, Erol

2017-01-01

The purpose of this study is to prepare a multiple-choice achievement test with high reliability and validity for the "Let's Solve the Puzzle of Our Body" unit. For this purpose, a multiple choice achievement test consisting of 46 items was applied to 178 fifth grade students in total. As a result of the test and material analysis…

Backwash Effects of Language-Testing in Primary and Secondary Education.

ERIC Educational Resources Information Center

Wesdorp, H.

A debate has been carried on in Dutch educational circles about the widespread use of multiple-choice tests, and a number of objections have been raised against the use of such tests. This paper reports on research into the validity of the objections, in particular with respect to the possible effect of multiple-choice tests on the teaching of…

Second Language Reading Topic Familiarity and Test Score: Test-Taking Strategies for Multiple-Choice Comprehension Questions

ERIC Educational Resources Information Center

Lee, Jia-Ying

2011-01-01

The main purpose of this study was to compare the strategies used by Chinese-speaking students when confronted with familiar versus unfamiliar topics in a multiple-choice format reading comprehension test. The focus was on describing what students do when they are taking reading comprehension tests by asking students to verbalize their thoughts.…

The weighted priors approach for combining expert opinions in logistic regression experiments

DOE PAGES

Quinlan, Kevin R.; Anderson-Cook, Christine M.; Myers, Kary L.

2017-04-24

When modeling the reliability of a system or component, it is not uncommon for more than one expert to provide very different prior estimates of the expected reliability as a function of an explanatory variable such as age or temperature. Our goal in this paper is to incorporate all information from the experts when choosing a design about which units to test. Bayesian design of experiments has been shown to be very successful for generalized linear models, including logistic regression models. We use this approach to develop methodology for the case where there are several potentially non-overlapping priors under consideration.more » While multiple priors have been used for analysis in the past, they have never been used in a design context. The Weighted Priors method performs well for a broad range of true underlying model parameter choices and is more robust when compared to other reasonable design choices. Finally, we illustrate the method through multiple scenarios and a motivating example. Additional figures for this article are available in the online supplementary information.« less

The weighted priors approach for combining expert opinions in logistic regression experiments

DOE Office of Scientific and Technical Information (OSTI.GOV)

Quinlan, Kevin R.; Anderson-Cook, Christine M.; Myers, Kary L.

When modeling the reliability of a system or component, it is not uncommon for more than one expert to provide very different prior estimates of the expected reliability as a function of an explanatory variable such as age or temperature. Our goal in this paper is to incorporate all information from the experts when choosing a design about which units to test. Bayesian design of experiments has been shown to be very successful for generalized linear models, including logistic regression models. We use this approach to develop methodology for the case where there are several potentially non-overlapping priors under consideration.more » While multiple priors have been used for analysis in the past, they have never been used in a design context. The Weighted Priors method performs well for a broad range of true underlying model parameter choices and is more robust when compared to other reasonable design choices. Finally, we illustrate the method through multiple scenarios and a motivating example. Additional figures for this article are available in the online supplementary information.« less

Comparing Assessments of Students' Knowledge by Computerized Open-Ended and Multiple-Choice Tests.

ERIC Educational Resources Information Center

Anbar, Michael

1991-01-01

Interactive computerized tests accepting unrestricted natural-language input were used to assess knowledge of clinical biophysics at the State University of New York at Buffalo. Comparison of responses to open-ended sequential questions and multiple-choice questions on the same material found the two formats test different aspects of competence.…

Multiple-Choice versus Constructed-Response Tests in the Assessment of Mathematics Computation Skills.

ERIC Educational Resources Information Center

Gadalla, Tahany M.

The equivalence of multiple-choice (MC) and constructed response (discrete) (CR-D) response formats as applied to mathematics computation at grade levels two to six was tested. The difference between total scores from the two response formats was tested for statistical significance, and the factor structure of items in both response formats was…

Developing Multiple Choice Tests: Tips & Techniques

ERIC Educational Resources Information Center

McCowan, Richard J.

1999-01-01

Item writing is a major responsibility of trainers. Too often, qualified staff who prepare lessons carefully and teach conscientiously use inadequate tests that do not validly reflect the true level of trainee achievement. This monograph describes techniques for constructing multiple-choice items that measure student performance accurately. It…

No Computer Left Behind

ERIC Educational Resources Information Center

Cohen, Daniel J.; Rosenzweig, Roy

2006-01-01

The combination of the Web and the cell phone forecasts the end of the inexpensive technologies of multiple-choice tests and grading machines. These technological developments are likely to bring the multiple-choice test to the verge of obsolescence, mounting a substantial challenge to the presentation of history and other disciplines.

Performance of Men and Women on Multiple-Choice and Constructed-Response Tests for Beginning Teachers. Research Report. ETS RR-04-48

ERIC Educational Resources Information Center

Livingston, Samuel A.; Rupp, Stacie L.

2004-01-01

Some previous research results imply that women tend to perform better, relative to men, on constructed-response (CR) tests than on multiple-choice (MC) tests in the same subjects. An analysis of data from several tests used in the licensing of beginning teachers supported this hypothesis, to varying degrees, in most of the tests investigated. The…

Of Small Beauties and Large Beasts: The Quality of Distractors on Multiple-Choice Tests Is More Important than Their Quantity

ERIC Educational Resources Information Center

Papenberg, Martin; Musch, Jochen

2017-01-01

In multiple-choice tests, the quality of distractors may be more important than their number. We therefore examined the joint influence of distractor quality and quantity on test functioning by providing a sample of 5,793 participants with five parallel test sets consisting of items that differed in the number and quality of distractors.…

Does the Position of Response Options in Multiple-Choice Tests Matter?

ERIC Educational Resources Information Center

Hohensinn, Christine; Baghaei, Purya

2017-01-01

In large scale multiple-choice (MC) tests alternate forms of a test may be developed to prevent cheating by changing the order of items or by changing the position of the response options. The assumption is that since the content of the test forms are the same the order of items or the positions of the response options do not have any effect on…

Valuing Assessment in Teacher Education - Multiple-Choice Competency Testing

ERIC Educational Resources Information Center

Martin, Dona L.; Itter, Diane

2014-01-01

When our focus is on assessment educators should work to value the nature of assessment. This paper presents a new approach to multiple-choice competency testing in mathematics education. The instrument discussed here reflects student competence, encourages self-regulatory learning behaviours and links content with current curriculum documents and…

How to Assess Student Performance in Science: Going beyond Multiple-Choice Tests. Third Edition

ERIC Educational Resources Information Center

Butler, Susan M.; McColskey, Wendy; O'Sullivan, Rita

2005-01-01

Educational systems promote student growth in a variety of dimensions. Basic content knowledge can be effectively assessed with multiple-choice and completion tests. However educational reforms have become more concerned with higher-order cognitive dimensions (problem-solving, creativity), social dimensions (communication skills, ability to work…

"I Don't Know" and Multiple Choice Analysis of Pre- and Post-Tests

ERIC Educational Resources Information Center

Spears, Karen; Wilson, Mary

2010-01-01

Evaluation is an essential component of any Extension education program. One tool, the pre- and post-test, provides measurable evaluation data. Yet often the answer "I don't know" or all possible answers to a multiple choice question are not included in the repeated measure analysis. Because more than two answers are offered, the test of marginal…

Predictive Validity of a Multiple-Choice Test for Placement in a Community College

ERIC Educational Resources Information Center

Verbout, Mary F.

2013-01-01

Multiple-choice tests of punctuation and usage are used throughout the United States to assess the writing skills of new community college students in order to place them in either a basic writing course or first-year composition. To determine whether using the COMPASS Writing Test (CWT) is a valid placement at a community college, student test…

Teaching Composition Skills with Weekly Multiple Choice Tests in Lieu of Theme Writing. Final Report.

ERIC Educational Resources Information Center

Scannell, Dale P.; Haugh, Oscar M.

The purpose of the study was to compare the effectiveness with which composition skills could be taught by the traditional theme-assignment approach and by an experimental method using weekly multiple-choice composition tests in lieu of theme writing. The weekly tests were based on original but typical first-draft compositions and covered problems…

Grade 12 Diploma Examination, English 33. Part B: Reading. (Multiple Choice). Readings Booklet. 1988 Edition.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

Intended for students taking Grade 12 Diploma Examinations in English 33 in Alberta, Canada, this reading test is designed to be administered with a questions booklet. The following short selections taken from fiction, nonfiction, poetry, drama, and day-to-day functional materials are included: (1) "M is for Mother" (Marjorie Riddle);…

A Quantitative Assessment of the Cultural Knowledge, Attitudes, and Experiences of Junior and Senior Dietetics Students

ERIC Educational Resources Information Center

McArthur, Laura H.; Greathouse, Karen R.; Smith, Erskine R.; Holbert, Donald

2011-01-01

Objective: To assess the cultural competence of dietetics majors. Design: Self-administered questionnaire. Setting: Classrooms at 7 universities. Participants: Two hundred eighty-three students--98 juniors (34.6%) and 185 seniors (65.4%)--recruited during class time. Main Outcome Measures: Knowledge was measured using a multiple-choice test,…

Engine Tune-up Service. Unit 6: Emission Control Systems. Posttests. Automotive Mechanics Curriculum.

ERIC Educational Resources Information Center

Morse, David T.; May, Theodore R.

This book of posttests is designed to accompany the Engine Tune-Up Service Student Guide for Unit 6, Emission Control Systems, available separately as CE 031 220. Focus of the posttests is inspecting, testing, and servicing emission control systems. One multiple choice posttest is provided that covers the seven performance objectives contained in…

Engine Tune-Up Service. Unit 4: Secondary Circuit. Posttests. Automotive Mechanics Curriculum.

ERIC Educational Resources Information Center

Morse, David T.

This book of posttests is designed to accompany the Engine Tune-Up Service Student Guide for Unit 4, Secondary Circuit, available separately as CE 031 214. Focus of the posttests is testing and servicing the secondary ignition circuit. One multiple choice posttest is provided that covers the seven performance objectives contained in the unit. (No…

Development and Evaluation of a Questionnaire to Assess Physical Educators' Knowledge of Student Assessment

ERIC Educational Resources Information Center

Emmanouilidou, Kyriaki; Derri, Vassiliki; Aggelousis, Nicolaos; Vassiliadou, Olga

2012-01-01

The purpose of this pilot study was to develop and evaluate an instrument for measuring Greek elementary physical educators' knowledge of student assessment. A multiple-choice questionnaire comprised of items about concepts, methods, tools, and types of student assessment in physical education was designed and tested. The initial 35-item…

Assessing the Validity of Multiple-Choice Questions in Measuring Fourth Graders' Ability to Interpret Graphs about Motion and Temperature

ERIC Educational Resources Information Center

Dulger, Mehmet; Deniz, Hasan

2017-01-01

The purpose of this paper is to assess the validity of multiple-choice questions in measuring fourth grade students' ability to interpret graphs related to physical science topics such as motion and temperature. We administered a test including 6 multiple-choice questions to 28 fourth grade students. Students were asked to explain their thinking…

A gatekeeping procedure to test a primary and a secondary endpoint in a group sequential design with multiple interim looks.

PubMed

Tamhane, Ajit C; Gou, Jiangtao; Jennison, Christopher; Mehta, Cyrus R; Curto, Teresa

2018-03-01

Glimm et al. (2010) and Tamhane et al. (2010) studied the problem of testing a primary and a secondary endpoint, subject to a gatekeeping constraint, using a group sequential design (GSD) with K=2 looks. In this article, we greatly extend the previous results to multiple (K>2) looks. If the familywise error rate (FWER) is to be controlled at a preassigned α level then it is clear that the primary boundary must be of level α. We show under what conditions one α-level primary boundary is uniformly more powerful than another. Based on this result, we recommend the choice of the O'Brien and Fleming (1979) boundary over the Pocock (1977) boundary for the primary endpoint. For the secondary endpoint the choice of the boundary is more complicated since under certain conditions the secondary boundary can be refined to have a nominal level α'>α, while still controlling the FWER at level α, thus boosting the secondary power. We carry out secondary power comparisons via simulation between different choices of primary-secondary boundary combinations. The methodology is applied to the data from the RALES study (Pitt et al., 1999; Wittes et al., 2001). An R library package gsrsb to implement the proposed methodology is made available on CRAN. © 2017, The International Biometric Society.

Models for Scoring Missing Responses to Multiple-Choice Items. Program Statistics Research Technical Report No. 94-1.

ERIC Educational Resources Information Center

Longford, Nicholas T.

This study is a critical evaluation of the roles for coding and scoring of missing responses to multiple-choice items in educational tests. The focus is on tests in which the test-takers have little or no motivation; in such tests omitting and not reaching (as classified by the currently adopted operational rules) is quite frequent. Data from the…

Performance Evaluation of a Database System in a Multiple Backend Configurations,

DTIC Science & Technology

1984-10-01

leaving a systemn process , the * internal performance measuremnents of MMSD have been carried out. Mathodo lo.- gies for constructing test databases...access d i rectory data via the AT, EDIT, and CDT. In designing the test database, one of the key concepts is the choice of the directory attributes in...internal timing. These requests are selected since they retrieve the seIaI lest portion of the test database and the processing time for each request is

A Multiple Choice Version of the Sentence Completion Method

ERIC Educational Resources Information Center

Shouval, Ron; And Others

1975-01-01

It was concluded that a multiple choice form corresponding to a sentence completion measure, test clearly defined personality areas (such as autonomy) could be a reasonable alternative for many purposes. (Author/DEP)

Violating Conventional Wisdom in Multiple Choice Test Construction

ERIC Educational Resources Information Center

Taylor, Annette Kujawski

2005-01-01

This research examined 2 elements of multiple-choice test construction, balancing the key and optimal number of options. In Experiment 1 the 3 conditions included a balanced key, overrepresentation of a and b responses, and overrepresentation of c and d responses. The results showed that error-patterns were independent of the key, reflecting…

Multiple-Choice Tests with Correction Allowed in Autism: An Excel Applet

ERIC Educational Resources Information Center

Martinez, Elisabetta Monari

2010-01-01

The valuation of academic achievements in students with severe language impairment is problematic if they also have difficulties in sustaining attention and in praxic skills. In severe autism all of these difficulties may occur together. Multiple-choice tests offer the advantage that simple praxic skills are required, allowing the tasks to be…

Application of a Multidimensional Nested Logit Model to Multiple-Choice Test Items

ERIC Educational Resources Information Center

Bolt, Daniel M.; Wollack, James A.; Suh, Youngsuk

2012-01-01

Nested logit models have been presented as an alternative to multinomial logistic models for multiple-choice test items (Suh and Bolt in "Psychometrika" 75:454-473, 2010) and possess a mathematical structure that naturally lends itself to evaluating the incremental information provided by attending to distractor selection in scoring. One potential…

Semantic Similarity Measures for the Generation of Science Tests in Basque

ERIC Educational Resources Information Center

Aldabe, Itziar; Maritxalar, Montse

2014-01-01

The work we present in this paper aims to help teachers create multiple-choice science tests. We focus on a scientific vocabulary-learning scenario taking place in a Basque-language educational environment. In this particular scenario, we explore the option of automatically generating Multiple-Choice Questions (MCQ) by means of Natural Language…

English 30, Part B: Reading. Questions Booklet. Grade 12 Diploma Examination, January 1997.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton. Student Evaluation Branch.

Intended for students taking the Grade 12 Diploma Examinations in English 30, this "questions booklet" presents 70 multiple choice test items based on 8 reading selections in the accompanying readings booklet. After instructions for students, the booklet presents the multiple choice items which test students' comprehension of the poetry,…

The Relationship of Expert-System Scored Constrained Free-Response Items to Multiple-Choice and Open-Ended Items.

ERIC Educational Resources Information Center

Bennett, Randy Elliot; And Others

1990-01-01

The relationship of an expert-system-scored constrained free-response item type to multiple-choice and free-response items was studied using data for 614 students on the College Board's Advanced Placement Computer Science (APCS) Examination. Implications for testing and the APCS test are discussed. (SLD)

Score Increase and Partial-Credit Validity When Administering Multiple-Choice Tests Using an Answer-Until-Correct Format

ERIC Educational Resources Information Center

Slepkov, Aaron D.; Vreugdenhil, Andrew J.; Shiell, Ralph C.

2016-01-01

There are numerous benefits to answer-until-correct (AUC) approaches to multiple-choice testing, not the least of which is the straightforward allotment of partial credit. However, the benefits of granting partial credit can be tempered by the inevitable increase in test scores and by fears that such increases are further contaminated by a large…

A multi-instructor, team-based, active-learning exercise to integrate basic and clinical sciences content.

PubMed

Kolluru, Srikanth; Roesch, Darren M; Akhtar de la Fuente, Ayesha

2012-03-12

To introduce a multiple-instructor, team-based, active-learning exercise to promote the integration of basic sciences (pathophysiology, pharmacology, and medicinal chemistry) and clinical sciences in a doctor of pharmacy curriculum. A team-based learning activity that involved pre-class reading assignments, individual-and team-answered multiple-choice questions, and evaluation and discussion of a clinical case, was designed, implemented, and moderated by 3 faculty members from the pharmaceutical sciences and pharmacy practice departments. Student performance was assessed using a multiple-choice examination, an individual readiness assurance test (IRAT), a team readiness assurance test (TRAT), and a subjective, objective, assessment, and plan (SOAP) note. Student attitudes were assessed using a pre- and post-exercise survey instrument. Students' understanding of possible correct treatment strategies for depression improved. Students were appreciative of this true integration of basic sciences knowledge in a pharmacotherapy course and to have faculty members from both disciplines present to answer questions. Mean student score on the on depression module for the examination was 80.4%, indicating mastery of the content. An exercise led by multiple instructors improved student perceptions of the importance of team-based teaching. Integrated teaching and learning may be achieved when instructors from multiple disciplines work together in the classroom using proven team-based, active-learning exercises.

Development and Preliminary Testing of the Food Choice Priorities Survey (FCPS): Assessing the Importance of Multiple Factors on College Students' Food Choices.

PubMed

Vilaro, Melissa J; Zhou, Wenjun; Colby, Sarah E; Byrd-Bredbenner, Carol; Riggsbee, Kristin; Olfert, Melissa D; Barnett, Tracey E; Mathews, Anne E

2017-12-01

Understanding factors that influence food choice may help improve diet quality. Factors that commonly affect adults' food choices have been described, but measures that identify and assess food choice factors specific to college students are lacking. This study developed and tested the Food Choice Priorities Survey (FCPS) among college students. Thirty-seven undergraduates participated in two focus groups ( n = 19; 11 in the male-only group, 8 in the female-only group) and interviews ( n = 18) regarding typical influences on food choice. Qualitative data informed the development of survey items with a 5-point Likert-type scale (1 = not important, 5 = extremely important). An expert panel rated FCPS items for clarity, relevance, representativeness, and coverage using a content validity form. To establish test-retest reliability, 109 first-year college students completed the 14-item FCPS at two time points, 0-48 days apart ( M = 13.99, SD = 7.44). Using Cohen's weighted κ for responses within 20 days, 11 items demonstrated moderate agreement and 3 items had substantial agreement. Factor analysis revealed a three-factor structure (9 items). The FCPS is designed for college students and provides a way to determine the factors of greatest importance regarding food choices among this population. From a public health perspective, practical applications include using the FCPS to tailor health communications and behavior change interventions to factors most salient for food choices of college students.

Evaluation of EIT system performance.

PubMed

Yasin, Mamatjan; Böhm, Stephan; Gaggero, Pascal O; Adler, Andy

2011-07-01

An electrical impedance tomography (EIT) system images internal conductivity from surface electrical stimulation and measurement. Such systems necessarily comprise multiple design choices from cables and hardware design to calibration and image reconstruction. In order to compare EIT systems and study the consequences of changes in system performance, this paper describes a systematic approach to evaluate the performance of the EIT systems. The system to be tested is connected to a saline phantom in which calibrated contrasting test objects are systematically positioned using a position controller. A set of evaluation parameters are proposed which characterize (i) data and image noise, (ii) data accuracy, (iii) detectability of single contrasts and distinguishability of multiple contrasts, and (iv) accuracy of reconstructed image (amplitude, resolution, position and ringing). Using this approach, we evaluate three different EIT systems and illustrate the use of these tools to evaluate and compare performance. In order to facilitate the use of this approach, all details of the phantom, test objects and position controller design are made publicly available including the source code of the evaluation and reporting software.

The "None of the Above" Option in Multiple-Choice Testing: An Experimental Study

ERIC Educational Resources Information Center

DiBattista, David; Sinnige-Egger, Jo-Anne; Fortuna, Glenda

2014-01-01

The authors assessed the effects of using "none of the above" as an option in a 40-item, general-knowledge multiple-choice test administered to undergraduate students. Examinees who selected "none of the above" were given an incentive to write the correct answer to the question posed. Using "none of the above" as the…

Multiple-Choice Exams: An Obstacle for Higher-Level Thinking in Introductory Science Classes

ERIC Educational Resources Information Center

Stanger-Hall, Kathrin F.

2012-01-01

Learning science requires higher-level (critical) thinking skills that need to be practiced in science classes. This study tested the effect of exam format on critical-thinking skills. Multiple-choice (MC) testing is common in introductory science courses, and students in these classes tend to associate memorization with MC questions and may not…

Multiple-Choice Question Tests: A Convenient, Flexible and Effective Learning Tool? A Case Study

ERIC Educational Resources Information Center

Douglas, Mercedes; Wilson, Juliette; Ennis, Sean

2012-01-01

The research presented in this paper is part of a project investigating assessment practices, funded by the Scottish Funding Council. Using established principles of good assessment and feedback, the use of online formative and summative multiple choice tests (MCT's) was piloted to support independent and self-directed learning and improve…

The Effect of the Multiple-Choice Item Format on the Measurement of Knowledge of Language Structure

ERIC Educational Resources Information Center

Currie, Michael; Chiramanee, Thanyapa

2010-01-01

Noting the widespread use of multiple-choice items in tests in English language education in Thailand, this study compared their effect against that of constructed-response items. One hundred and fifty-two university undergraduates took a test of English structure first in constructed-response format, and later in three, stem-equivalent…

Identifying Key Features of Student Performance in Educational Video Games and Simulations through Cluster Analysis

ERIC Educational Resources Information Center

Kerr, Deirdre; Chung, Gregory K. W. K.

2012-01-01

The assessment cycle of "evidence-centered design" (ECD) provides a framework for treating an educational video game or simulation as an assessment. One of the main steps in the assessment cycle of ECD is the identification of the key features of student performance. While this process is relatively simple for multiple choice tests, when…

Evaluation of Students' Views about the Use of SCORM (Sharable Content Object Reference Model)-Compatible Materials in Physics Teaching

ERIC Educational Resources Information Center

Gonen, Selahattin; Basaran, Bulent

2013-01-01

In the present study, a web site including instructional materials such as Whiteboard Movies (WBM), simulations and animations and testing materials such as true-false, fill-in-the-blanks, puzzles, open-ended questions and multiple-choice questions was designed. The study was carried out with 76 students attending Dicle College (DC), Diyarbakir…

Engine Tune-up Service. Unit 2: Charging System. Posttests. Automotive Mechanics Curriculum.

ERIC Educational Resources Information Center

Richardson, Roger L.; Bacon, E. Miles

This book of posttests is designed to accompany the Engine Tune-Up Service Student Guide for Unit 2, Charging System, available separately as CE 031 208. Focus of the posttest is on the testing of the charging system. One multiple choice posttest is provided, that covers the three performance objectives contained in the unit. (No answer key is…

New Multiple-Choice Measures of Historical Thinking: An Investigation of Cognitive Validity

ERIC Educational Resources Information Center

Smith, Mark D.

2018-01-01

History education scholars have recognized the need for test validity research in recent years and have called for empirical studies that explore how to best measure historical thinking processes. The present study was designed to help answer this call and to provide a model that others can adapt to carry this line of research forward. It employed…

Development of a Microcomputer-Based Adaptive Testing System. Phase I. Specification of Requirements and Preliminary Design.

DTIC Science & Technology

1982-06-30

treatments, and cure (or kill ) a patient. Administratively, the items were in a multiple-choice format and the simulation proceeded by branching...Discs: dual 5 1/4 inch floppies (IM) Bus: N/A Operating System: CP/M, MmmOST Price: $3,495 -14 ~-174- - ’i~ Model 820 Xerox 1341 West Mockingbird Lane

Is It Working? Distractor Analysis Results from the Test Of Astronomy STandards (TOAST) Assessment Instrument

NASA Astrophysics Data System (ADS)

Slater, Stephanie

2009-05-01

The Test Of Astronomy STandards (TOAST) assessment instrument is a multiple-choice survey tightly aligned to the consensus learning goals stated by the American Astronomical Society - Chair's Conference on ASTRO 101, the American Association of the Advancement of Science's Project 2061 Benchmarks, and the National Research Council's National Science Education Standards. Researchers from the Cognition in Astronomy, Physics and Earth sciences Research (CAPER) Team at the University of Wyoming's Science and Math Teaching Center (UWYO SMTC) have been conducting a question-by-question distractor analysis procedure to determine the sensitivity and effectiveness of each item. In brief, the frequency each possible answer choice, known as a foil or distractor on a multiple-choice test, is determined and compared to the existing literature on the teaching and learning of astronomy. In addition to having statistical difficulty and discrimination values, a well functioning assessment item will show students selecting distractors in the relative proportions to how we expect them to respond based on known misconceptions and reasoning difficulties. In all cases, our distractor analysis suggests that all items are functioning as expected. These results add weight to the validity of the Test Of Astronomy STandards (TOAST) assessment instrument, which is designed to help instructors and researchers measure the impact of course-length duration instructional strategies for undergraduate science survey courses with learning goals tightly aligned to the consensus goals of the astronomy education community.

Should essays and other "open-ended"-type questions retain a place in written summative assessment in clinical medicine?

PubMed

Hift, Richard J

2014-11-28

Written assessments fall into two classes: constructed-response or open-ended questions, such as the essay and a number of variants of the short-answer question, and selected-response or closed-ended questions; typically in the form of multiple-choice. It is widely believed that constructed response written questions test higher order cognitive processes in a manner that multiple-choice questions cannot, and consequently have higher validity. An extensive review of the literature suggests that in summative assessment neither premise is evidence-based. Well-structured open-ended and multiple-choice questions appear equivalent in their ability to assess higher cognitive functions, and performance in multiple-choice assessments may correlate more highly than the open-ended format with competence demonstrated in clinical practice following graduation. Studies of construct validity suggest that both formats measure essentially the same dimension, at least in mathematics, the physical sciences, biology and medicine. The persistence of the open-ended format in summative assessment may be due to the intuitive appeal of the belief that synthesising an answer to an open-ended question must be both more cognitively taxing and similar to actual experience than is selecting a correct response. I suggest that cognitive-constructivist learning theory would predict that a well-constructed context-rich multiple-choice item represents a complex problem-solving exercise which activates a sequence of cognitive processes which closely parallel those required in clinical practice, hence explaining the high validity of the multiple-choice format. The evidence does not support the proposition that the open-ended assessment format is superior to the multiple-choice format, at least in exit-level summative assessment, in terms of either its ability to test higher-order cognitive functioning or its validity. This is explicable using a theory of mental models, which might predict that the multiple-choice format will have higher validity, a statement for which some empiric support exists. Given the superior reliability and cost-effectiveness of the multiple-choice format consideration should be given to phasing out open-ended format questions in summative assessment. Whether the same applies to non-exit-level assessment and formative assessment is a question which remains to be answered; particularly in terms of the educational effect of testing, an area which deserves intensive study.

I can't wait: Methods for measuring and moderating individual differences in impulsive choice.

PubMed

Peterson, Jennifer R; Hill, Catherine C; Marshall, Andrew T; Stuebing, Sarah L; Kirkpatrick, Kimberly

2015-01-01

Impulsive choice behavior occurs when individuals make choices without regard for future consequences. This behavior is often maladaptive and is a common symptom in many disorders, including drug abuse, compulsive gambling, and obesity. Several proposed mechanisms may influence impulsive choice behavior. These mechanisms provide a variety of pathways that may provide the basis for individual differences that are often evident when measuring choice behavior. This review provides an overview of these different pathways to impulsive choice, and the behavioral intervention strategies being developed to moderate impulsive choice. Because of the compelling link between impulsive choice behavior and the near-epidemic pervasiveness of obesity in the United States, we focus on the relationship between impulsive choice behavior and obesity as a test case for application of the multiple pathways approach. Choosing immediate gratification over healthier long term food choices is a contributing factor to the obesity crisis. Behavioral interventions can lead to more self controlled choices in a rat pre-clinical model, suggesting a possible gateway for translation to human populations. Designing and implementing effective impulsive choice interventions is crucial to improving the overall health and well-being of impulsive individuals.

I can't wait: Methods for measuring and moderating individual differences in impulsive choice

PubMed Central

Peterson, Jennifer R.; Hill, Catherine C.; Marshall, Andrew T.; Stuebing, Sarah L.; Kirkpatrick, Kimberly

2016-01-01

Impulsive choice behavior occurs when individuals make choices without regard for future consequences. This behavior is often maladaptive and is a common symptom in many disorders, including drug abuse, compulsive gambling, and obesity. Several proposed mechanisms may influence impulsive choice behavior. These mechanisms provide a variety of pathways that may provide the basis for individual differences that are often evident when measuring choice behavior. This review provides an overview of these different pathways to impulsive choice, and the behavioral intervention strategies being developed to moderate impulsive choice. Because of the compelling link between impulsive choice behavior and the near-epidemic pervasiveness of obesity in the United States, we focus on the relationship between impulsive choice behavior and obesity as a test case for application of the multiple pathways approach. Choosing immediate gratification over healthier long term food choices is a contributing factor to the obesity crisis. Behavioral interventions can lead to more self controlled choices in a rat pre-clinical model, suggesting a possible gateway for translation to human populations. Designing and implementing effective impulsive choice interventions is crucial to improving the overall health and well-being of impulsive individuals. PMID:27695664

Multiple-Choice Testing Using Immediate Feedback--Assessment Technique (IF AT®) Forms: Second-Chance Guessing vs. Second-Chance Learning?

ERIC Educational Resources Information Center

Merrel, Jeremy D.; Cirillo, Pier F.; Schwartz, Pauline M.; Webb, Jeffrey A.

2015-01-01

Multiple choice testing is a common but often ineffective method for evaluating learning. A newer approach, however, using Immediate Feedback Assessment Technique (IF AT®, Epstein Educational Enterprise, Inc.) forms, offers several advantages. In particular, a student learns immediately if his or her answer is correct and, in the case of an…

Accommodations for Multiple Choice Tests

ERIC Educational Resources Information Center

Trammell, Jack

2011-01-01

Students with learning or learning-related disabilities frequently struggle with multiple choice assessments due to difficulty discriminating between items, filtering out distracters, and framing a mental best answer. This Practice Brief suggests accommodations and strategies that disability service providers can utilize in conjunction with…

Which form of assessment provides the best information about student performance in chemistry examinations?

NASA Astrophysics Data System (ADS)

Hudson, Ross D.; Treagust, David F.

2013-04-01

Background . This study developed from observations of apparent achievement differences between male and female chemistry performances in a state university entrance examination. Male students performed more strongly than female students, especially in higher scores. Apart from the gender of the students, two other important factors that might influence student performance were format of questions (short-answer or multiple-choice) and type of questions (recall or application). Purpose The research question addressed in this study was: Is there a relationship between performance in state university entrance examinations in chemistry and school chemistry examinations and student gender, format of questions - multiple-choice or short-answer, and conceptual level - recall or application? Sample The two sources of data were: (1) secondary analyses of five consecutive years' data published by the examining authority of chemistry examinations, and (2) tests conducted with 192 students which provided information about all aspects of the three variables (question format, question type and gender) under consideration. Design and methods Both sources of data were analysed using ANOVA to compare means for the variables under consideration and the statistical significance of any differences. The data from the tests were also analysed using Rasch analysis to determine differences in gender performance. Results When overall mean data are considered, both male and female students performed better on multiple-choice questions and recall questions than on short-answer questions and application questions, respectively. When overall mean data are considered, male students outperformed female students in both the university entrance and school tests, particularly in the higher scores. When data were analysed with Rasch, there was no statistically significant difference in performance between males and females of equal ability. Conclusions Both male and female students generally perform better on multiple-choice questions than they do on short-answer questions. However, when the questions are matched in terms of difficulty (using Rasch analysis), the differences in performance between multiple-choice and short-answer are quite small. Rasch analysis showed that there was little difference in performance between males and females of equal ability. This study shows that a simple face-value score analysis of relative student performance - in this case, in chemistry - can be deceptive unless the actual abilities of the students concerned, as measured by a tool such as Rasch, are taken into consideration before reaching any conclusion.

Engine Tune-Up Service. Unit 1: Battery and Cranking System. Posttests. Automotive Mechanics Curriculum.

ERIC Educational Resources Information Center

Goodson-Roberts, Ludy; And Others

This book of posttests is designed to accompany the Engine Tune-Up Service Student Guide for Unit 1, Battery and Cranking System. Focus of the posttests is the testing of the battery and cranking system. Four multiple choice posttests are provided, one for each of the performance objectives contained in the unit. (No answer keys are provided.)…

Designing Adaptive Instructional Environments: Insights from Empirical Evidence

DTIC Science & Technology

2011-10-01

theorems. Cohen’s f effect size for pretest to posttest gain, averaged across different problems = 0.46. 7 Basis for Adaptation Ability of...problems and took a posttest . Measures of Learning 26-item multiple choice pretest and posttest . Effect size on posttest scores as measured by...solving algebraic equations. Measures of Learning Pretest and posttest using rapid diagnostic testing procedure: Student had to provide their

New Stuff in I/O (In-Baskets and Orals). The Development, Administration and Scoring of In-Baskets and Orals for the New York State Correction Captain Examination.

ERIC Educational Resources Information Center

Kaiser, Paul D.; Brull, Harry

The design, administration, scoring, and results of the 1993 New York State Correctional Captain Examination are described. The examination was administered to 405 candidates. As in previous Sergeant and Lieutenant examinations, candidates also completed latent image written simulation problems and open/closed book multiple choice test components.…

Using a Classroom Response System to Improve Multiple-Choice Performance in AP® Physics

NASA Astrophysics Data System (ADS)

Bertrand, Peggy

2009-04-01

Participation in rigorous high school courses such as Advanced Placement (AP®) Physics increases the likelihood of college success, especially for students who are traditionally underserved. Tackling difficult multiple-choice exams should be part of any AP program because well-constructed multiple-choice questions, such as those on AP exams and on the Force Concept Inventory,2 are particularly good at rooting out common and persisting student misconceptions. Additionally, there are barriers to multiple-choice performance that have little to do with content mastery. For example, a student might fail to read the question thoroughly, forget to apply a reasonableness test to the answer, or simply work too slowly.

Comparison of individual answer and group answer with and without structured peer assessment

NASA Astrophysics Data System (ADS)

Kablan, Zeynel

2014-09-01

Background:Cooperative learning activities provide active participation of students leading to better learning. The literature suggests that cooperative learning activities need to be structured for a more effective and productive interaction. Purpose: This study aimed to test the differences among three instructional conditions in terms of science achievement. Sample:A total of 79 fifth-grade students, 42 males (53%) and 37 females (47%), participated in the study. Design and Methods:In the first condition, students answered the teacher's questions individually by raising hands. In the second condition, students discussed the answer in groups and came up with a single group answer. In this condition, the teacher provided only verbal directions to the groups without using any strategy or material. In the third condition, students used a 'peer assessment form' before giving the group answer. A pre-/post-test experimental design was used. Multiple-choice and open-ended tests were used for data collection. One-way analysis of variance (ANOVA) was conducted to test the differences in the test scores between the three groups (individual answer, unstructured group answer and structured group answer). Results:Results showed that there were no significant differences among the three learning conditions in terms of their multiple-choice test scores. In terms of the open-ended test scores, students in the structured group answer condition scored significantly higher than the students in the individual answer condition. Conclusions:Structuring the group work through peer assessment helped to monitor the group discussion, provided a better learning compared to the individual answer condition, and helped students to participate in the activity equally.

Development of multiple choice pictorial test for measuring the dimensions of knowledge

NASA Astrophysics Data System (ADS)

Nahadi, Siswaningsih, Wiwi; Erna

2017-05-01

This study aims to develop a multiple choice pictorial test as a tool to measure dimension of knowledge in chemical equilibrium subject. The method used is Research and Development and validation that was conducted in the preliminary studies and model development. The product is multiple choice pictorial test. The test was developed by 22 items and tested to 64 high school students in XII grade. The quality of test was determined by value of validity, reliability, difficulty index, discrimination power, and distractor effectiveness. The validity of test was determined by CVR calculation using 8 validators (4 university teachers and 4 high school teachers) with average CVR value 0,89. The reliability of test has very high category with value 0,87. Discrimination power of items with a very good category is 32%, 59% as good category, and 20% as sufficient category. This test has a varying level of difficulty, item with difficult category is 23%, the medium category is 50%, and the easy category is 27%. The distractor effectiveness of items with a very poor category is 1%, poor category is 1%, medium category is 4%, good category is 39%, and very good category is 55%. The dimension of knowledge that was measured consist of factual knowledge, conceptual knowledge, and procedural knowledge. Based on the questionnaire, students responded quite well to the developed test and most of the students like this kind of multiple choice pictorial test that include picture as evaluation tool compared to the naration tests was dominated by text.

Pick-N Multiple Choice-Exams: A Comparison of Scoring Algorithms

ERIC Educational Resources Information Center

Bauer, Daniel; Holzer, Matthias; Kopp, Veronika; Fischer, Martin R.

2011-01-01

To compare different scoring algorithms for Pick-N multiple correct answer multiple-choice (MC) exams regarding test reliability, student performance, total item discrimination and item difficulty. Data from six 3rd year medical students' end of term exams in internal medicine from 2005 to 2008 at Munich University were analysed (1,255 students,…

Dynamic Testing of Analogical Reasoning in 5- to 6-Year-Olds: Multiple-Choice versus Constructed-Response Training Items

ERIC Educational Resources Information Center

Stevenson, Claire E.; Heiser, Willem J.; Resing, Wilma C. M.

2016-01-01

Multiple-choice (MC) analogy items are often used in cognitive assessment. However, in dynamic testing, where the aim is to provide insight into potential for learning and the learning process, constructed-response (CR) items may be of benefit. This study investigated whether training with CR or MC items leads to differences in the strategy…

Do Students Know What They Know and What They Don't Know? Using a Four-Tier Diagnostic Test to Assess the Nature of Students' Alternative Conceptions

ERIC Educational Resources Information Center

Caleon, Imelda S.; Subramaniam, R.

2010-01-01

This study reports on the development and application of a four-tier multiple-choice (4TMC) diagnostic instrument, which has not been reported in the literature. It is an enhanced version of the two-tier multiple-choice (2TMC) test. As in 2TMC tests, its answer and reason tiers measure students' content knowledge and explanatory knowledge,…

Increasing Choice Making in Students with Intellectual Disability

ERIC Educational Resources Information Center

Sparks, Shannon Lynn; Pierce, Tom; Higgins, Kyle; Miller, Susan; Tandy, Richard

2016-01-01

The purpose of this study was to examine the effectiveness of choice-making training with six high school students with intellectual disability. A multiple probe design with one replication was used to evaluate the efficacy of the choice-making training. The results suggest participants increased and maintained their choice-making abilities.…

Social Choice in a Computer-Assisted Simulation

ERIC Educational Resources Information Center

Thavikulwat, Precha

2009-01-01

Pursuing a line of inquiry suggested by Crookall, Martin, Saunders, and Coote, the author applied, within the framework of design science, an optimal-design approach to incorporate into a computer-assisted simulation two innovative social choice processes: the multiple period double auction and continuous voting. Expectations that the…

Strategies for Coping in a Complex World: Adherence Behavior Among Older Adults with Chronic Illness

PubMed Central

Ross-Degnan, Dennis; Adams, Alyce S.; Safran, Dana Gelb; Soumerai, Stephen B.

2007-01-01

Background Increasing numbers of medicines increase nonadherence. Little is known about how older adults manage multiple medicines for multiple illnesses. Objectives To explore how older adults with multiple illnesses make choices about medicines. Design Semistructured interviews with older adults taking several medications. Accounts of respondents’ medicine-taking behavior were collected. Participants Twenty community-dwelling seniors with health insurance, in Eastern Massachusetts, aged 67–90, (4–12 medicines, 3–9 comorbidities). Approach Qualitative analysis using constant comparison to explain real choices made about medicines in the past (“historical”) and hypothetical (“future”) choices. Results Respondents reported both past (“historical”) choices and hypothetical (“future”) choices between medicines. Although people discussed effectiveness and future risk of the disease when prompted to prioritize their medicines (future choices), key factors leading to nonadherence (historical choices) were costs and side effects. Specific choices were generally dominated by 1 factor, and respondents rarely reported making explicit trade-offs between different factors. Factors affecting 1 choice were not necessarily the same as those affecting another choice in the same person. There was no evidence of “adherent” personalities. Conclusion Prescribing a new medicine, a change in provider or copayment can provoke new choices about both new and existing medications in older adults with multiple morbidities. PMID:17406952

The Effectiveness of Problem-Based Learning Approach Based on Multiple Intelligences in Terms of Student’s Achievement, Mathematical Connection Ability, and Self-Esteem

NASA Astrophysics Data System (ADS)

Kartikasari, A.; Widjajanti, D. B.

2017-02-01

The aim of this study is to explore the effectiveness of learning approach using problem-based learning based on multiple intelligences in developing student’s achievement, mathematical connection ability, and self-esteem. This study is experimental research with research sample was 30 of Grade X students of MIA III MAN Yogyakarta III. Learning materials that were implemented consisting of trigonometry and geometry. For the purpose of this study, researchers designed an achievement test made up of 44 multiple choice questions with respectively 24 questions on the concept of trigonometry and 20 questions for geometry. The researcher also designed a connection mathematical test and self-esteem questionnaire that consisted of 7 essay questions on mathematical connection test and 30 items of self-esteem questionnaire. The learning approach said that to be effective if the proportion of students who achieved KKM on achievement test, the proportion of students who achieved a minimum score of high category on the results of both mathematical connection test and self-esteem questionnaire were greater than or equal to 70%. Based on the hypothesis testing at the significance level of 5%, it can be concluded that the learning approach using problem-based learning based on multiple intelligences was effective in terms of student’s achievement, mathematical connection ability, and self-esteem.

Gender and Ethnicity Differences in Multiple-Choice Testing. Effects of Self-Assessment and Risk-Taking Propensity

DTIC Science & Technology

1993-05-01

correctness of the response provides I some advantages. They are: i 1. Increased reliability of the test; 2. Examinees pay more attention to the multiple...their choice 3 of test date. Each sign up sheet was divided into four cells: Non-Hispanic males and females and Hispanic males and females. 3 I I I...certain prestige and financial rewards; or entering a conservatory of music for advanced training with a well-known pianist . Mr. H realizes that even

Application of a Utility Analysis to Evaluate a Novel Assessment Tool for Clinically Oriented Physiology and Pharmacology

ERIC Educational Resources Information Center

Cramer, Nicholas; Asmar, Abdo; Gorman, Laurel; Gros, Bernard; Harris, David; Howard, Thomas; Hussain, Mujtaba; Salazar, Sergio; Kibble, Jonathan D.

2016-01-01

Multiple-choice questions are a gold-standard tool in medical school for assessment of knowledge and are the mainstay of licensing examinations. However, multiple-choice questions items can be criticized for lacking the ability to test higher-order learning or integrative thinking across multiple disciplines. Our objective was to develop a novel…

The Development and Validation of a Two-Tiered Multiple-Choice Instrument to Identify Alternative Conceptions in Earth Science

ERIC Educational Resources Information Center

Mangione, Katherine Anna

2010-01-01

This study was to determine reliability and validity for a two-tiered, multiple- choice instrument designed to identify alternative conceptions in earth science. Additionally, this study sought to identify alternative conceptions in earth science held by preservice teachers, to investigate relationships between self-reported confidence scores and…

Multiple objective optimization in reliability demonstration test

DOE PAGES

Lu, Lu; Anderson-Cook, Christine Michaela; Li, Mingyang

2016-10-01

Reliability demonstration tests are usually performed in product design or validation processes to demonstrate whether a product meets specified requirements on reliability. For binomial demonstration tests, the zero-failure test has been most commonly used due to its simplicity and use of minimum sample size to achieve an acceptable consumer’s risk level. However, this test can often result in unacceptably high risk for producers as well as a low probability of passing the test even when the product has good reliability. This paper explicitly explores the interrelationship between multiple objectives that are commonly of interest when planning a demonstration test andmore » proposes structured decision-making procedures using a Pareto front approach for selecting an optimal test plan based on simultaneously balancing multiple criteria. Different strategies are suggested for scenarios with different user priorities and graphical tools are developed to help quantify the trade-offs between choices and to facilitate informed decision making. As a result, potential impacts of some subjective user inputs on the final decision are studied to offer insights and useful guidance for general applications.« less

Further Support for Changing Multiple-Choice Answers.

ERIC Educational Resources Information Center

Fabrey, Lawrence J.; Case, Susan M.

1985-01-01

The effect on test scores of changing answers to multiple-choice questions was studied and compared to earlier research. The current setting was a nationally administered, in-training, specialty examination for medical residents in obstetrics and gynecology. Both low and high scorers improved their scores when they changed answers. (SW)

High School Students' Concepts of Acids and Bases.

ERIC Educational Resources Information Center

Ross, Bertram H. B.

An investigation of Ontario high school students' understanding of acids and bases with quantitative and qualitative methods revealed misconceptions. A concept map, based on the objectives of the Chemistry Curriculum Guideline, generated multiple-choice items and interview questions. The multiple-choice test was administered to 34 grade 12…

MDTri: robust and efficient global mixed integer search of spaces of multiple ternary alloys: A DIRECT-inspired optimization algorithm for experimentally accessible computational material design

DOE PAGES

Graf, Peter A.; Billups, Stephen

2017-07-24

Computational materials design has suffered from a lack of algorithms formulated in terms of experimentally accessible variables. Here we formulate the problem of (ternary) alloy optimization at the level of choice of atoms and their composition that is normal for synthesists. Mathematically, this is a mixed integer problem where a candidate solution consists of a choice of three elements, and how much of each of them to use. This space has the natural structure of a set of equilateral triangles. We solve this problem by introducing a novel version of the DIRECT algorithm that (1) operates on equilateral triangles insteadmore » of rectangles and (2) works across multiple triangles. We demonstrate on a test case that the algorithm is both robust and efficient. Lastly, we offer an explanation of the efficacy of DIRECT -- specifically, its balance of global and local search -- by showing that 'potentially optimal rectangles' of the original algorithm are akin to the Pareto front of the 'multi-component optimization' of global and local search.« less

MDTri: robust and efficient global mixed integer search of spaces of multiple ternary alloys: A DIRECT-inspired optimization algorithm for experimentally accessible computational material design

DOE Office of Scientific and Technical Information (OSTI.GOV)

Graf, Peter A.; Billups, Stephen

Computational materials design has suffered from a lack of algorithms formulated in terms of experimentally accessible variables. Here we formulate the problem of (ternary) alloy optimization at the level of choice of atoms and their composition that is normal for synthesists. Mathematically, this is a mixed integer problem where a candidate solution consists of a choice of three elements, and how much of each of them to use. This space has the natural structure of a set of equilateral triangles. We solve this problem by introducing a novel version of the DIRECT algorithm that (1) operates on equilateral triangles insteadmore » of rectangles and (2) works across multiple triangles. We demonstrate on a test case that the algorithm is both robust and efficient. Lastly, we offer an explanation of the efficacy of DIRECT -- specifically, its balance of global and local search -- by showing that 'potentially optimal rectangles' of the original algorithm are akin to the Pareto front of the 'multi-component optimization' of global and local search.« less

Early Design Choices: Capture, Model, Integrate, Analyze, Simulate

NASA Technical Reports Server (NTRS)

Malin, Jane T.

2004-01-01

I. Designs are constructed incrementally to meet requirements and solve problems: a) Requirements types: objectives, scenarios, constraints, ilities. etc. b) Problem/issue types: risk/safety, cost/difficulty, interaction, conflict, etc. II. Capture requirements, problems and solutions: a) Collect design and analysis products and make them accessible for integration and analysis; b) Link changes in design requirements, problems and solutions; and c) Harvest design data for design models and choice structures. III. System designs are constructed by multiple groups designing interacting subsystems a) Diverse problems, choice criteria, analysis methods and point solutions. IV. Support integration and global analysis of repercussions: a) System implications of point solutions; b) Broad analysis of interactions beyond totals of mass, cost, etc.

Delayed, but not immediate, feedback after multiple-choice questions increases performance on a subsequent short-answer, but not multiple-choice, exam: evidence for the dual-process theory of memory.

PubMed

Sinha, Neha; Glass, Arnold Lewis

2015-01-01

Three experiments, two performed in the laboratory and one embedded in a college psychology lecture course, investigated the effects of immediate versus delayed feedback following a multiple-choice exam on subsequent short answer and multiple-choice exams. Performance on the subsequent multiple-choice exam was not affected by the timing of the feedback on the prior exam; however, performance on the subsequent short answer exam was better following delayed than following immediate feedback. This was true regardless of the order in which immediate versus delayed feedback was given. Furthermore, delayed feedback only had a greater effect than immediate feedback on subsequent short answer performance following correct, confident responses on the prior exam. These results indicate that delayed feedback cues a student's prior response and increases subsequent recollection of that response. The practical implication is that delayed feedback is better than immediate feedback during academic testing.

Demand Characteristics of Multiple-Choice Items.

ERIC Educational Resources Information Center

Diamond, James J.; Williams, David V.

Thirteen graduate students were asked to indicate for each of 24 multiple-choice items whether the item tested "recall of specific information," a "higher order skill," or "don't know." The students were also asked to state their general basis for judging the items. The 24 items had been previously classified according to Bloom's cognitive-skills…

Examining the Prediction of Reading Comprehension on Different Multiple-Choice Tests

ERIC Educational Resources Information Center

Andreassen, Rune; Braten, Ivar

2010-01-01

In this study, 180 Norwegian fifth-grade students with a mean age of 10.5 years were administered measures of word recognition skills, strategic text processing, reading motivation and working memory. Six months later, the same students were given three different multiple-choice reading comprehension measures. Based on three forced-order…

A Better Benchmark Assessment: Multiple-Choice versus Project-Based

ERIC Educational Resources Information Center

Peariso, Jamon F.

2006-01-01

The purpose of this literature review and Ex Post Facto descriptive study was to determine which type of benchmark assessment, multiple-choice or project-based, provides the best indication of general success on the history portion of the CST (California Standards Tests). The result of the study indicates that although the project-based benchmark…

Automatic Scoring of Paper-and-Pencil Figural Responses. Research Report.

ERIC Educational Resources Information Center

Martinez, Michael E.; And Others

Large-scale testing is dominated by the multiple-choice question format. Widespread use of the format is due, in part, to the ease with which multiple-choice items can be scored automatically. This paper examines automatic scoring procedures for an alternative item type: figural response. Figural response items call for the completion or…

A Two-Tier Multiple Choice Questions to Diagnose Thermodynamic Misconception of Thai and Laos Students

NASA Astrophysics Data System (ADS)

Kamcharean, Chanwit; Wattanakasiwich, Pornrat

The objective of this study was to diagnose misconceptions of Thai and Lao students in thermodynamics by using a two-tier multiple-choice test. Two-tier multiple choice questions consist of the first tier, a content-based question and the second tier, a reasoning-based question. Data of student understanding was collected by using 10 two-tier multiple-choice questions. Thai participants were the first-year students (N = 57) taking a fundamental physics course at Chiang Mai University in 2012. Lao participants were high school students in Grade 11 (N = 57) and Grade 12 (N = 83) at Muengnern high school in Xayaboury province, Lao PDR. As results, most students answered content-tier questions correctly but chose incorrect answers for reason-tier questions. When further investigating their incorrect reasons, we found similar misconceptions as reported in previous studies such as incorrectly relating pressure with temperature when presenting with multiple variables.

A Handbook for Alcohol and Drug Control Officers. Volume II. Appendices.

DTIC Science & Technology

1975-02-01

informed respondent is regarding drug/alcohol side - effects , what respondent has learned from a given program or experience, etc.). There are a number...appearance, can list side effects of each and can score Z% on a multiple-choice test concerning federal and state laws and Armed Services Regulations...at least X% on a multiple- choice test regarding the major side effects of substance abuse. * The number of enlisted men found unfit for duty because

Diagramming the Never Ending Story: Student-generated diagrammatic stories integrate and retain science concepts improving science literacy

NASA Astrophysics Data System (ADS)

Pillsbury, Ralph T.

This research examined an instructional strategy called Diagramming the Never Ending Story: A method called diagramming was taught to sixth grade students via an outdoor science inquiry ecology unit. Students generated diagrams of the new ecology concepts they encountered, creating explanatory 'captions' for their newly drawn diagrams while connecting them in a memorable story. The diagramming process culminates in 20-30 meter-long murals called the Never Ending Story: Months of science instruction are constructed as pictorial scrolls, making sense of all new science concepts they encounter. This method was taught at a North Carolina "Public" Charter School, Children's Community School, to measure its efficacy in helping students comprehend scientific concepts and retain them thereby increasing science literacy. There were four demographically similar classes of 20 students each. Two 'treatment' classes, randomly chosen from the four classes, generated their own Never Ending Stories after being taught the diagramming method. A Solomon Four-Group Design was employed: Two Classes (one control, one treatment) were administered pre- and post; two classes received post tests only. The tests were comprised of multiple choice, fill-in and extended response (open-ended) sections. Multiple choice and fill-in test data were not statistically significant whereas extended response test data confirm that treatment classes made statistically significant gains.

The effects of a test-taking strategy intervention for high school students with test anxiety in advanced placement science courses

NASA Astrophysics Data System (ADS)

Markus, Doron J.

Test anxiety is one of the most debilitating and disruptive factors associated with underachievement and failure in schools (Birenbaum, Menucha, Nasser, & Fadia, 1994; Tobias, 1985). Researchers have suggested that interventions that combine multiple test-anxiety reduction techniques are most effective at reducing test anxiety levels (Ergene, 2003). For the current study, involving 62 public high school students enrolled in advanced placement science courses, the researcher designed a multimodal intervention designed to reduce test anxiety. Analyses were conducted to assess the relationships among test anxiety levels, unit examination scores, and irregular multiple-choice error patterns (error clumping), as well as changes in these measures after the intervention. Results indicate significant, positive relationships between some measures of test anxiety and error clumping, as well as significant, negative relationships between test anxiety levels and student achievement. In addition, results show significant decreases in holistic measures of test anxiety among students with low anxiety levels, as well as decreases in Emotionality subscores of test anxiety among students with high levels of test anxiety. There were no significant changes over time in the Worry subscores of test anxiety. Suggestions for further research include further confirmation of the existence of error clumping, and its causal relationship with test anxiety.

Diagnostic grand rounds: a new teaching concept to train diagnostic reasoning.

PubMed

Stieger, Stefan; Praschinger, Andrea; Kletter, Kurt; Kainberger, Franz

2011-06-01

Diagnostic reasoning is a core skill in teaching and learning in undergraduate curricula. Diagnostic grand rounds (DGRs) as a subform of grand rounds are intended to train the students' skills in the selection of appropriate tests and in the interpretation of test results. The aim of this study was to test DGRs for their ability to improve diagnostic reasoning by using a pre-post-test design. During one winter term, all 398 fifth-year students (36.1% male, 63.9% female) solved 23 clinical cases presented in 8 DGRs. In an online questionnaire, a Diagnostic Thinking Inventory (DTI) with 41 items was evaluated for flexibility in thinking and structure of knowledge in memory. Results were correlated with those from a summative multiple-choice knowledge test and of the learning objectives in a logbook. The students' DTI scores in the post-test were significantly higher than those reported in the pre-test. DTI scores at either testing time did not correlate with medical knowledge as assessed by a multiple-choice knowledge test. Abilities acquired during clinical clerkships as documented in a logbook could only account for a small proportion of the increase in the flexibility subscale score. This effect still remained significant after accounting for potential confounders. Establishing DGRs proofed to be an effective way of successfully improving both students' diagnostic reasoning and the ability to select the appropriate test method in routine clinical practice. Copyright © 2009 Elsevier Ireland Ltd. All rights reserved.

What influences participation in genetic carrier testing? Results from a discrete choice experiment.

PubMed

Hall, Jane; Fiebig, Denzil G; King, Madeleine T; Hossain, Ishrat; Louviere, Jordan J

2006-05-01

This study explores factors that influence participation in genetic testing programs and the acceptance of multiple tests. Tay Sachs and cystic fibrosis are both genetically determined recessive disorders with differing severity, treatment availability, and prevalence in different population groups. We used a discrete choice experiment with a general community and an Ashkenazi Jewish sample; data were analysed using multinomial logit with random coefficients. Although Jewish respondents were more likely to be tested, both groups seem to be making very similar tradeoffs across attributes when they make genetic testing choices.

Effects of a History of Differential Reinforcement on Preference for Choice

ERIC Educational Resources Information Center

Karsina, Allen; Thompson, Rachel H.; Rodriguez, Nicole M.

2011-01-01

The effects of a history of differential reinforcement for selecting a free-choice versus a restricted-choice stimulus arrangement on the subsequent responding of 7 undergraduates in a computer-based game of chance were examined using a concurrent-chains arrangement and a multiple-baseline-across-participants design. In the free-choice…

An Empirical Comparison of DDF Detection Methods for Understanding the Causes of DIF in Multiple-Choice Items

ERIC Educational Resources Information Center

Suh, Youngsuk; Talley, Anna E.

2015-01-01

This study compared and illustrated four differential distractor functioning (DDF) detection methods for analyzing multiple-choice items. The log-linear approach, two item response theory-model-based approaches with likelihood ratio tests, and the odds ratio approach were compared to examine the congruence among the four DDF detection methods.…

The Influence of a Response Format Test Accommodation for College Students with and without Disabilities

ERIC Educational Resources Information Center

Potter, Kyle; Lewandowski, Lawrence; Spenceley, Laura

2016-01-01

Standardised and other multiple-choice examinations often require the use of an answer sheet with fill-in bubbles (i.e. "bubble" or Scantron sheet). Students with disabilities causing impairments in attention, learning and/or visual-motor skill may have difficulties with multiple-choice examinations that employ such a response style.…

Multiple Choice Test Bias Uncovered by Use of an "I Don't Know" Alternative.

ERIC Educational Resources Information Center

Sherman, Susan W.

The multiple-choice science exercises used by the National Assessment of Educational Progress include an "I Don't Know" (IDK) alternative to estimate more accurately knowledge of groups of respondents. Group percentages of IDK responses were examined and compared with correct responses to see if the IDK introduces bias. Variance common…

The Effect of Images on Item Statistics in Multiple Choice Anatomy Examinations

ERIC Educational Resources Information Center

Notebaert, Andrew J.

2017-01-01

Although multiple choice examinations are often used to test anatomical knowledge, these often forgo the use of images in favor of text-based questions and answers. Because anatomy is reliant on visual resources, examinations using images should be used when appropriate. This study was a retrospective analysis of examination items that were text…

Fast Assessments with Digital Tools Using Multiple-Choice Questions

ERIC Educational Resources Information Center

Howell, Dusti D.; Tseng, Daphne ChingYu; Colorado-Resa, Jozenia T.

2017-01-01

Multiple Choice Questions (MCQs) have come a long way since they were used in "The Kansas Silent Reading Test" in 1915. After over 100 years of MCQs, new innovative digital tools using this form of assessment can help foster interactivity in today's classrooms. This article describes three free online MCQ tools that are relatively quick…

Piloting a Geoscience Literacy Exam for Assessing Students' Understanding of Earth, Climate, Atmospheric and Ocean Science Concepts

NASA Astrophysics Data System (ADS)

Steer, D. N.; Iverson, E. A.; Manduca, C. A.

2013-12-01

This research seeks to develop valid and reliable questions that faculty can use to assess geoscience literacy across the curriculum. We are particularly interested on effects of curricula developed to teach Earth, Climate, Atmospheric, and Ocean Science concepts in the context of societal issues across the disciplines. This effort is part of the InTeGrate project designed to create a population of college graduates who are poised to use geoscience knowledge in developing solutions to current and future environmental and resource challenges. Details concerning the project are found at http://serc.carleton.edu/integrate/index.html. The Geoscience Literacy Exam (GLE) under development presently includes 90 questions. Each big idea from each literacy document can be probed using one or more of three independent questions: 1) a single answer, multiple choice question aimed at basic understanding or application of key concepts, 2) a multiple correct answer, multiple choice question targeting the analyzing to analysis levels and 3) a short essay question that tests analysis or evaluation cognitive levels. We anticipate multiple-choice scores and the detail and sophistication of essay responses will increase as students engage with the curriculum. As part of the field testing of InTeGrate curricula, faculty collected student responses from classes that involved over 700 students. These responses included eight pre- and post-test multiple-choice questions that covered various concepts across the four literacies. Discrimination indices calculated from the data suggest that the eight tested questions provide a valid measure of literacy within the scope of the concepts covered. Student normalized gains across an academic term with limited InTeGrate exposure (typically two or fewer weeks of InTeGrate curriculum out of 14 weeks) were found to average 16% gain. A small set of control data (250 students in classes from one institution where no InTeGrate curricula were used) was also collected from a larger bank of test questions. Discrimination indices across the full bank showed variation and additional work is underway to refine and field test in other settings these questions in the absence of InTeGrate curricula. When complete, faculty will be able to assemble sets of questions to track progress toward meeting literacy goals. In addition to covering geoscience content knowledge and understanding, a complementary attitudinal pre/post survey was also developed with the intent to probe InTeGrate students' ability and motivation to use their geoscience expertise to address problems of environmental sustainability. The final instruments will be made available to the geoscience education community as an assessment to be used in conjunction with InTeGrate teaching materials or as a stand-alone tool for departments to measure student learning and attitudinal gains across the major.

Multiple choice questions can be designed or revised to challenge learners' critical thinking.

PubMed

Tractenberg, Rochelle E; Gushta, Matthew M; Mulroney, Susan E; Weissinger, Peggy A

2013-12-01

Multiple choice (MC) questions from a graduate physiology course were evaluated by cognitive-psychology (but not physiology) experts, and analyzed statistically, in order to test the independence of content expertise and cognitive complexity ratings of MC items. Integration of higher order thinking into MC exams is important, but widely known to be challenging-perhaps especially when content experts must think like novices. Expertise in the domain (content) may actually impede the creation of higher-complexity items. Three cognitive psychology experts independently rated cognitive complexity for 252 multiple-choice physiology items using a six-level cognitive complexity matrix that was synthesized from the literature. Rasch modeling estimated item difficulties. The complexity ratings and difficulty estimates were then analyzed together to determine the relative contributions (and independence) of complexity and difficulty to the likelihood of correct answers on each item. Cognitive complexity was found to be statistically independent of difficulty estimates for 88 % of items. Using the complexity matrix, modifications were identified to increase some item complexities by one level, without affecting the item's difficulty. Cognitive complexity can effectively be rated by non-content experts. The six-level complexity matrix, if applied by faculty peer groups trained in cognitive complexity and without domain-specific expertise, could lead to improvements in the complexity targeted with item writing and revision. Targeting higher order thinking with MC questions can be achieved without changing item difficulties or other test characteristics, but this may be less likely if the content expert is left to assess items within their domain of expertise.

Effectiveness of an audience response system in teaching pharmacology to baccalaureate nursing students.

PubMed

Vana, Kimberly D; Silva, Graciela E; Muzyka, Diann; Hirani, Lorraine M

2011-06-01

It has been proposed that students' use of an audience response system, commonly called clickers, may promote comprehension and retention of didactic material. Whether this method actually improves students' grades, however, is still not determined. The purpose of this study was to evaluate whether a lecture format utilizing multiple-choice PowerPoint slides and an audience response system was more effective than a lecture format using only multiple-choice PowerPoint slides in the comprehension and retention of pharmacological knowledge in baccalaureate nursing students. The study also assessed whether the additional use of clickers positively affected students' satisfaction with their learning. Results from 78 students who attended lecture classes with multiple-choice PowerPoint slides plus clickers were compared with those of 55 students who utilized multiple-choice PowerPoint slides only. Test scores between these two groups were not significantly different. A satisfaction questionnaire showed that 72.2% of the control students did not desire the opportunity to use clickers. Of the group utilizing the clickers, 92.3% recommend the use of this system in future courses. The use of multiple-choice PowerPoint slides and an audience response system did not seem to improve the students' comprehension or retention of pharmacological knowledge as compared with those who used solely multiple-choice PowerPoint slides.

Exploring Equity Properties in Equating Using AP® Examinations. Research Report No. 2012-4

ERIC Educational Resources Information Center

Lee, Eunjung; Lee, Won-Chan; Brennan, Robert L.

2012-01-01

In almost all high-stakes testing programs, test equating is necessary to ensure that test scores across multiple test administrations are equivalent and can be used interchangeably. Test equating becomes even more challenging in mixed-format tests, such as Advanced Placement Program® (AP®) Exams, that contain both multiple-choice and constructed…

Effect of response format on cognitive reflection: Validating a two- and four-option multiple choice question version of the Cognitive Reflection Test.

PubMed

Sirota, Miroslav; Juanchich, Marie

2018-03-27

The Cognitive Reflection Test, measuring intuition inhibition and cognitive reflection, has become extremely popular because it reliably predicts reasoning performance, decision-making, and beliefs. Across studies, the response format of CRT items sometimes differs, based on the assumed construct equivalence of tests with open-ended versus multiple-choice items (the equivalence hypothesis). Evidence and theoretical reasons, however, suggest that the cognitive processes measured by these response formats and their associated performances might differ (the nonequivalence hypothesis). We tested the two hypotheses experimentally by assessing the performance in tests with different response formats and by comparing their predictive and construct validity. In a between-subjects experiment (n = 452), participants answered stem-equivalent CRT items in an open-ended, a two-option, or a four-option response format and then completed tasks on belief bias, denominator neglect, and paranormal beliefs (benchmark indicators of predictive validity), as well as on actively open-minded thinking and numeracy (benchmark indicators of construct validity). We found no significant differences between the three response formats in the numbers of correct responses, the numbers of intuitive responses (with the exception of the two-option version, which had a higher number than the other tests), and the correlational patterns of the indicators of predictive and construct validity. All three test versions were similarly reliable, but the multiple-choice formats were completed more quickly. We speculate that the specific nature of the CRT items helps build construct equivalence among the different response formats. We recommend using the validated multiple-choice version of the CRT presented here, particularly the four-option CRT, for practical and methodological reasons. Supplementary materials and data are available at https://osf.io/mzhyc/ .

Student certainty answering misconception question: study of Three-Tier Multiple-Choice Diagnostic Test in Acid-Base and Solubility Equilibrium

NASA Astrophysics Data System (ADS)

Ardiansah; Masykuri, M.; Rahardjo, S. B.

2018-04-01

Students’ concept comprehension in three-tier multiple-choice diagnostic test related to student confidence level. The confidence level related to certainty and student’s self-efficacy. The purpose of this research was to find out students’ certainty in misconception test. This research was quantitative-qualitative research method counting students’ confidence level. The research participants were 484 students that were studying acid-base and equilibrium solubility subject. Data was collected using three-tier multiple-choice (3TMC) with thirty questions and students’ questionnaire. The findings showed that #6 item gives the highest misconception percentage and high student confidence about the counting of ultra-dilute solution’s pH. Other findings were that 1) the student tendency chosen the misconception answer is to increase over item number, 2) student certainty decreased in terms of answering the 3TMC, and 3) student self-efficacy and achievement were related each other in the research. The findings suggest some implications and limitations for further research.

Evaluation of a preschool nutrition education program based on the theory of multiple intelligences.

PubMed

Cason, K L

2001-01-01

This report describes the evaluation of a preschool nutrition education program based on the theory of multiple intelligences. Forty-six nutrition educators provided a series of 12 lessons to 6102 preschool-age children. The program was evaluated using a pretest/post-test design to assess differences in fruit and vegetable identification, healthy snack choices, willingness to taste foods, and eating behaviors. Subjects showed significant improvement in food identification and recognition, healthy snack identification, willingness to taste foods, and frequency of fruit, vegetable, meat, and dairy consumption. The evaluation indicates that the program was an effective approach for educating preschool children about nutrition.

The impact of two multiple-choice question formats on the problem-solving strategies used by novices and experts.

PubMed

Coderre, Sylvain P; Harasym, Peter; Mandin, Henry; Fick, Gordon

2004-11-05

Pencil-and-paper examination formats, and specifically the standard, five-option multiple-choice question, have often been questioned as a means for assessing higher-order clinical reasoning or problem solving. This study firstly investigated whether two paper formats with differing number of alternatives (standard five-option and extended-matching questions) can test problem-solving abilities. Secondly, the impact of the alternatives number on psychometrics and problem-solving strategies was examined. Think-aloud protocols were collected to determine the problem-solving strategy used by experts and non-experts in answering Gastroenterology questions, across the two pencil-and-paper formats. The two formats demonstrated equal ability in testing problem-solving abilities, while the number of alternatives did not significantly impact psychometrics or problem-solving strategies utilized. These results support the notion that well-constructed multiple-choice questions can in fact test higher order clinical reasoning. Furthermore, it can be concluded that in testing clinical reasoning, the question stem, or content, remains more important than the number of alternatives.

The Effect of SSM Grading on Reliability When Residual Items Have No Discriminating Power.

ERIC Educational Resources Information Center

Kane, Michael T.; Moloney, James M.

Gilman and Ferry have shown that when the student's score on a multiple choice test is the total number of responses necessary to get all items correct, substantial increases in reliability can occur. In contrast, similar procedures giving partial credit on multiple choice items have resulted in relatively small gains in reliability. The analysis…

Improving Measures via Examining the Behavior of Distractors in Multiple-Choice Tests: Assessment and Remediation

ERIC Educational Resources Information Center

Sideridis, Georgios; Tsaousis, Ioannis; Al Harbi, Khaleel

2017-01-01

The purpose of the present article was to illustrate, using an example from a national assessment, the value from analyzing the behavior of distractors in measures that engage the multiple-choice format. A secondary purpose of the present article was to illustrate four remedial actions that can potentially improve the measurement of the…

A Systematic Assessment of "None of the Above" on Multiple Choice Tests in a First Year Psychology Classroom

ERIC Educational Resources Information Center

Pachai, Matthew V.; DiBattista, David; Kim, Joseph A.

2015-01-01

Multiple choice writing guidelines are decidedly split on the use of "none of the above" (NOTA), with some authors discouraging and others advocating its use. Moreover, empirical studies of NOTA have produced mixed results. Generally, these studies have utilized NOTA as either the correct response or a distractor and assessed its effect…

Polytomous versus Dichotomous Scoring on Multiple-Choice Examinations: Development of a Rubric for Rating Partial Credit

ERIC Educational Resources Information Center

Grunert, Megan L.; Raker, Jeffrey R.; Murphy, Kristen L.; Holme, Thomas A.

2013-01-01

The concept of assigning partial credit on multiple-choice test items is considered for items from ACS Exams. Because the items on these exams, particularly the quantitative items, use common student errors to define incorrect answers, it is possible to assign partial credits to some of these incorrect responses. To do so, however, it becomes…

The Development of Multiple-Choice Items Consistent with the AP Chemistry Curriculum Framework to More Accurately Assess Deeper Understanding

ERIC Educational Resources Information Center

Domyancich, John M.

2014-01-01

Multiple-choice questions are an important part of large-scale summative assessments, such as the advanced placement (AP) chemistry exam. However, past AP chemistry exam items often lacked the ability to test conceptual understanding and higher-order cognitive skills. The redesigned AP chemistry exam shows a distinctive shift in item types toward…

The effects of field dependent/independent style awareness on learning strategies and outcomes in an instructional hypermedia module

NASA Astrophysics Data System (ADS)

Fyle, Clifford Omodele

The purpose of this study was to examine whether field-dependent/independent style awareness affects learning outcomes and learning strategies used in a hypermedia instructional module. Field-dependent/independent style was measured using the Global Embedded Figures Test. Style awareness meant that students were provided with information and explanations about their individual cognitive styles and the learning strategies that accommodate those styles. The study entailed examining students' achievement in a multiple-choice test and performance in a design task, and also their navigation patterns as they studied a science-oriented Webquest. The sample consisted of 149 eighth-grade students in 10 sections of a science class taught by two teachers in a public middle school. A two-group posttest-only design on one factor (style awareness) was used. Sixty-eight students in five sections of the class were assigned to the treatment group (field dependent/independent style awareness) while the other 81 students in five sections were assigned to the control group (no field dependent/independent style awareness). The study took place over a period of 6 days. On the first day, students in the treatment group were first tested and debriefed on their individual styles. Next, all students in both the treatment and control groups studied the hypermedia instructional module (Webquest) over a period of two days. On the fourth and fifth days students worked on the performance tasks, and on the sixth day students took the multiple-choice test and students in the control group were tested and debriefed on their individual styles. The findings indicate that style awareness significantly influenced the learning strategies of field-dependent students as they studied and carried out learning tasks in the Webquest. Field-dependent students with style awareness used hypertext links and navigated the menu sequentially a greater number of times than their counterparts with no style awareness. Correspondingly, there were no significant findings for field-independent students of the effects of style awareness on learning strategies. The findings also revealed significant differences in terms of style awareness and its interactions with achievement on the multiple-choice test. Both field-dependent and field-independent students with style awareness achieved higher scores than their counterparts who received no style awareness. There were however no significant findings with respect to the effects of style awareness on performance on the design task. Overall this study demonstrated that providing middle-school students with cognitive-style awareness training can improve both their academic performance as well as enable them to adopt more effective learning strategies when learning in hypermedia environments.

Investigating the potential influence of established multiple-choice test-taking cues on item response in a pharmacotherapy board certification examination preparatory manual: a pilot study.

PubMed

Gettig, Jacob P

2006-04-01

To determine the prevalence of established multiple-choice test-taking correct and incorrect answer cues in the American College of Clinical Pharmacy's Updates in Therapeutics: The Pharmacotherapy Preparatory Course, 2005 Edition, as an equal or lesser surrogate indication of the prevalence of such cues in the Pharmacotherapy board certification examination. All self-assessment and patient case question-and-answer sets were assessed individually to determine if they were subject to selected correct and incorrect answer cues commonly seen in multiple-choice question writing. If the question was considered evaluable, correct answer cues-longest answer, mid-range number, one of two similar choices, and one of two opposite choices-were tallied. In addition, incorrect answer cues- inclusionary language and grammatical mismatch-were also tallied. Each cue was counted if it did what was expected or did the opposite of what was expected. Multiple cues could be identified in each question. A total of 237 (47.7%) of 497 questions in the manual were deemed evaluable. A total of 325 correct answer cues and 35 incorrect answer cues were identified in the 237 evaluable questions. Most evaluable questions contained one to two correct and/or incorrect answer cue(s). Longest answer was the most frequently identified correct answer cue; however, it was the least likely to identify the correct answer. Inclusionary language was the most frequently identified incorrect answer cue. Incorrect answer cues were considerably more likely to identify incorrect answer choices than correct answer cues were able to identify correct answer choices. The use of established multiple-choice test-taking cues is unlikely to be of significant help when taking the Pharmacotherapy board certification examination, primarily because of the lack of questions subject to such cues and the inability of correct answer cues to accurately identify correct answers. Incorrect answer cues, especially the use of inclusionary language, almost always will accurately identify an incorrect answer choice. Assuming that questions in the preparatory course manual were equal or lesser surrogates of those in the board certification examination, it is unlikely that intuition alone can replace adequate preparation and studying as the sole determinant of examination success.

The Effects of Study Tasks in a Computer-Based Chemistry Learning Environment

NASA Astrophysics Data System (ADS)

Urhahne, Detlef; Nick, Sabine; Poepping, Anna Christin; Schulz, Sarah Jayne

2013-12-01

The present study examines the effects of different study tasks on the acquisition of knowledge about acids and bases in a computer-based learning environment. Three different task formats were selected to create three treatment conditions: learning with gap-fill and matching tasks, learning with multiple-choice tasks, and learning only from text and figures without any additional tasks. Participants were 196 ninth-grade students who learned with a self-developed multimedia program in a pretest-posttest control group design. Research results reveal that gap-fill and matching tasks were most effective in promoting knowledge acquisition, followed by multiple-choice tasks, and no tasks at all. The findings are in line with previous research on this topic. The effects can possibly be explained by the generation-recognition model, which predicts that gap-fill and matching tasks trigger more encompassing learning processes than multiple-choice tasks. It is concluded that instructional designers should incorporate more challenging study tasks for enhancing the effectiveness of computer-based learning environments.

A Framework for Instrument Development of a Choice Experiment: An Application to Type 2 Diabetes.

PubMed

Janssen, Ellen M; Segal, Jodi B; Bridges, John F P

2016-10-01

Choice experiments are increasingly used to obtain patient preference information for regulatory benefit-risk assessments. Despite the importance of instrument design, there remains a paucity of literature applying good research principles. We applied a novel framework for instrument development of a choice experiment to measure type 2 diabetes mellitus treatment preferences. Applying the framework, we used evidence synthesis, expert consultation, stakeholder engagement, pretest interviews, and pilot testing to develop a best-worst scaling (BWS) and discrete choice experiment (DCE). We synthesized attributes from published DCEs for type 2 diabetes, consulted clinical experts, engaged a national advisory board, conducted local cognitive interviews, and pilot tested a national survey. From published DCEs (n = 17), ten attribute categories were extracted with cost (n = 11) having the highest relative attribute importance (RAI) (range 6-10). Clinical consultation and stakeholder engagement identified six attributes for inclusion. Cognitive pretesting with local diabetes patients (n = 25) ensured comprehension of the choice experiment. Pilot testing with patients from a national sample (n = 50) identified nausea as most important (RAI for DCE: 10 [95 % CI 8.5-11.5]; RAI for BWS: 10 [95 % CI 8.9-11.1]). The developed choice experiment contained five attributes (A1c decrease, blood glucose stability, low blood glucose, nausea, additional medicine, and cost). The framework for instrument development of a choice experiment included five stages of development and incorporated multiple stakeholder perspectives. Further comparisons of instrument development approaches are needed to identify best practices. To facilitate comparisons, researchers need to be encouraged to publish or discuss their instrument development strategies and findings.

Psychometrics of Multiple Choice Questions with Non-Functioning Distracters: Implications to Medical Education.

PubMed

Deepak, Kishore K; Al-Umran, Khalid Umran; AI-Sheikh, Mona H; Dkoli, B V; Al-Rubaish, Abdullah

2015-01-01

The functionality of distracters in a multiple choice question plays a very important role. We examined the frequency and impact of functioning and non-functioning distracters on psychometric properties of 5-option items in clinical disciplines. We analyzed item statistics of 1115 multiple choice questions from 15 summative assessments of undergraduate medical students and classified the items into five groups by their number of non-functioning distracters. We analyzed the effect of varying degree of non-functionality ranging from 0 to 4, on test reliability, difficulty index, discrimination index and point biserial correlation. The non-functionality of distracters inversely affected the test reliability and quality of items in a predictable manner. The non-functioning distracters made the items easier and lowered the discrimination index significantly. Three non-functional distracters in a 5-option MCQ significantly affected all psychometric properties (p < 0.5). The corrected point biserial correlation revealed that the items with 3 functional options were psychometrically as effective as 5-option items. Our study reveals that a multiple choice question with 3 functional options provides lower most limit of item format that has adequate psychometric property. The test containing items with less number of functioning options have significantly lower reliability. The distracter function analysis and revision of nonfunctioning distracters can serve as important methods to improve the psychometrics and reliability of assessment.

Assessing Scientific Practices Using Machine-Learning Methods: How Closely Do They Match Clinical Interview Performance?

NASA Astrophysics Data System (ADS)

Beggrow, Elizabeth P.; Ha, Minsu; Nehm, Ross H.; Pearl, Dennis; Boone, William J.

2014-02-01

The landscape of science education is being transformed by the new Framework for Science Education (National Research Council, A framework for K-12 science education: practices, crosscutting concepts, and core ideas. The National Academies Press, Washington, DC, 2012), which emphasizes the centrality of scientific practices—such as explanation, argumentation, and communication—in science teaching, learning, and assessment. A major challenge facing the field of science education is developing assessment tools that are capable of validly and efficiently evaluating these practices. Our study examined the efficacy of a free, open-source machine-learning tool for evaluating the quality of students' written explanations of the causes of evolutionary change relative to three other approaches: (1) human-scored written explanations, (2) a multiple-choice test, and (3) clinical oral interviews. A large sample of undergraduates (n = 104) exposed to varying amounts of evolution content completed all three assessments: a clinical oral interview, a written open-response assessment, and a multiple-choice test. Rasch analysis was used to compute linear person measures and linear item measures on a single logit scale. We found that the multiple-choice test displayed poor person and item fit (mean square outfit >1.3), while both oral interview measures and computer-generated written response measures exhibited acceptable fit (average mean square outfit for interview: person 0.97, item 0.97; computer: person 1.03, item 1.06). Multiple-choice test measures were more weakly associated with interview measures (r = 0.35) than the computer-scored explanation measures (r = 0.63). Overall, Rasch analysis indicated that computer-scored written explanation measures (1) have the strongest correspondence to oral interview measures; (2) are capable of capturing students' normative scientific and naive ideas as accurately as human-scored explanations, and (3) more validly detect understanding than the multiple-choice assessment. These findings demonstrate the great potential of machine-learning tools for assessing key scientific practices highlighted in the new Framework for Science Education.

Simulation-Based Educational Module Improves Intern and Medical Student Performance of Closed Reduction and Percutaneous Pinning of Pediatric Supracondylar Humeral Fractures.

PubMed

Butler, Bennet A; Lawton, Cort D; Burgess, Jamie; Balderama, Earvin S; Barsness, Katherine A; Sarwark, John F

2017-12-06

Simulation-based education has been integrated into many orthopaedic residency programs to augment traditional teaching models. Here we describe the development and implementation of a combined didactic and simulation-based course for teaching medical students and interns how to properly perform a closed reduction and percutaneous pinning of a pediatric supracondylar humeral fracture. Subjects included in the study were either orthopaedic surgery interns or subinterns at our institution. Subjects all completed a combined didactic and simulation-based course on pediatric supracondylar humeral fractures. The first part of this course was an electronic (e)-learning module that the subjects could complete at home in approximately 40 minutes. The second part of the course was a 20-minute simulation-based skills learning session completed in the simulation center. Subject knowledge of closed reduction and percutaneous pinning of supracondylar humeral fractures was tested using a 30-question, multiple-choice, written test. Surgical skills were tested in the operating room or in a simulated operating room. Subject pre-intervention and post-intervention scores were compared to determine if and how much they had improved. A total of 21 subjects were tested. These subjects significantly improved their scores on both the written, multiple-choice test and skills test after completing the combined didactic and simulation module. Prior to the module, intern and subintern multiple-choice test scores were significantly worse than postgraduate year (PGY)-2 to PGY-5 resident scores (p < 0.01); after completion of the module, there was no significant difference in the multiple-choice test scores. After completing the module, there was no significant difference in skills test scores between interns and PGY-2 to PGY-5 residents. Both tests were validated using the scores obtained from PGY-2 to PGY-5 residents. Our combined didactic and simulation course significantly improved intern and subintern understanding of supracondylar humeral fractures and their ability to perform a closed reduction and percutaneous pinning of these fractures.

Multiple-choice tests stabilize access to marginal knowledge.

PubMed

Cantor, Allison D; Eslick, Andrea N; Marsh, Elizabeth J; Bjork, Robert A; Bjork, Elizabeth Ligon

2015-02-01

Marginal knowledge refers to knowledge that is stored in memory, but is not accessible at a given moment. For example, one might struggle to remember who wrote The Call of the Wild, even if that knowledge is stored in memory. Knowing how best to stabilize access to marginal knowledge is important, given that new learning often requires accessing and building on prior knowledge. While even a single opportunity to restudy marginal knowledge boosts its later accessibility (Berger, Hall, & Bahrick, 1999), in many situations explicit relearning opportunities are not available. Our question is whether multiple-choice tests (which by definition expose the learner to the correct answers) can also serve this function and, if so, how testing compares to restudying given that tests can be particularly powerful learning devices (Roediger & Karpicke, 2006). In four experiments, we found that multiple-choice testing had the power to stabilize access to marginal knowledge, and to do so for at least up to a week. Importantly, such tests did not need to be paired with feedback, although testing was no more powerful than studying. Overall, the results support the idea that one's knowledge base is unstable, with individual pieces of information coming in and out of reach. The present findings have implications for a key educational challenge: ensuring that students have continuing access to information they have learned.

Assessing reservoir operations risk under climate change

USGS Publications Warehouse

Brekke, L.D.; Maurer, E.P.; Anderson, J.D.; Dettinger, M.D.; Townsley, E.S.; Harrison, A.; Pruitt, T.

2009-01-01

Risk-based planning offers a robust way to identify strategies that permit adaptive water resources management under climate change. This paper presents a flexible methodology for conducting climate change risk assessments involving reservoir operations. Decision makers can apply this methodology to their systems by selecting future periods and risk metrics relevant to their planning questions and by collectively evaluating system impacts relative to an ensemble of climate projection scenarios (weighted or not). This paper shows multiple applications of this methodology in a case study involving California's Central Valley Project and State Water Project systems. Multiple applications were conducted to show how choices made in conducting the risk assessment, choices known as analytical design decisions, can affect assessed risk. Specifically, risk was reanalyzed for every choice combination of two design decisions: (1) whether to assume climate change will influence flood-control constraints on water supply operations (and how), and (2) whether to weight climate change scenarios (and how). Results show that assessed risk would motivate different planning pathways depending on decision-maker attitudes toward risk (e.g., risk neutral versus risk averse). Results also show that assessed risk at a given risk attitude is sensitive to the analytical design choices listed above, with the choice of whether to adjust flood-control rules under climate change having considerably more influence than the choice on whether to weight climate scenarios. Copyright 2009 by the American Geophysical Union.

Calibrating the Medical Council of Canada's Qualifying Examination Part I using an integrated item response theory framework: a comparison of models and designs.

PubMed

De Champlain, Andre F; Boulais, Andre-Philippe; Dallas, Andrew

2016-01-01

The aim of this research was to compare different methods of calibrating multiple choice question (MCQ) and clinical decision making (CDM) components for the Medical Council of Canada's Qualifying Examination Part I (MCCQEI) based on item response theory. Our data consisted of test results from 8,213 first time applicants to MCCQEI in spring and fall 2010 and 2011 test administrations. The data set contained several thousand multiple choice items and several hundred CDM cases. Four dichotomous calibrations were run using BILOG-MG 3.0. All 3 mixed item format (dichotomous MCQ responses and polytomous CDM case scores) calibrations were conducted using PARSCALE 4. The 2-PL model had identical numbers of items with chi-square values at or below a Type I error rate of 0.01 (83/3,499 or 0.02). In all 3 polytomous models, whether the MCQs were either anchored or concurrently run with the CDM cases, results suggest very poor fit. All IRT abilities estimated from dichotomous calibration designs correlated very highly with each other. IRT-based pass-fail rates were extremely similar, not only across calibration designs and methods, but also with regard to the actual reported decision to candidates. The largest difference noted in pass rates was 4.78%, which occurred between the mixed format concurrent 2-PL graded response model (pass rate= 80.43%) and the dichotomous anchored 1-PL calibrations (pass rate= 85.21%). Simpler calibration designs with dichotomized items should be implemented. The dichotomous calibrations provided better fit of the item response matrix than more complex, polytomous calibrations.

Evaluating the long-term impact of the Trauma Team Training course in Guyana: an explanatory mixed-methods approach.

PubMed

Pemberton, Julia; Rambaran, Madan; Cameron, Brian H

2013-02-01

We evaluated the retention of trauma knowledge and skills after an interprofessional Trauma Team Training (TTT) course in Guyana and explored the course impact on participants. A mixed-methods design evaluated knowledge using a multiple-choice quiz test, skills and trauma moulage simulation with checklists, and course impact with qualitative interviews. Participants were evaluated at 3 time points; before, after, and 4 months after TTT. Forty-seven course participants included 20 physicians, 17 nurses, and 10 paramedical providers. All participants had improved multiple-choice quiz test scores after the course and retained knowledge after 4 months, with nonphysicians showing the most improved scores. Trauma skill and moulage scores declined slightly after 4 months, with the greatest decline observed in complex skills. Qualitatively, course participants self-reported impact of the TTT course included improved empowerment, knowledge, teamwork, and patient care. Interprofessional team-based training led to the retention of trauma knowledge and skills as well as the empowerment of nonphysicians. The decline in performance of some trauma skills indicates the need for a regular trauma update course. Copyright © 2013 Elsevier Inc. All rights reserved.

Risk of error estimated from Palestine pharmacists' knowledge and certainty on the adverse effects and contraindications of active pharmaceutical ingredients and excipients.

PubMed

Shawahna, Ramzi; Al-Rjoub, Mohammed; Al-Horoub, Mohammed M; Al-Hroub, Wasif; Al-Rjoub, Bisan; Al-Nabi, Bashaaer Abd

2016-01-01

This study aimed to investigate community pharmacists' knowledge and certainty of adverse effects and contraindications of pharmaceutical products to estimate the risk of error. Factors influencing their knowledge and certainty were also investigated. The knowledge of community pharmacists was assessed in a cross-sectional design using a multiple-choice questions test on the adverse effects and contraindications of active pharmaceutical ingredients and excipients from May 2014 to March 2015. Self-rated certainty scores were also recorded for each question. Knowledge and certainty scores were combined to estimate the risk of error. Out of 315 subjects, 129 community pharmacists (41.0%) completed the 30 multiple-choice questions test on active ingredients and excipients. Knowledge on active ingredients was associated with the year of graduation and obtaining a licence to practice pharmacy. Knowledge on excipients was associated with the degree obtained. There was higher risk of error in items on excipients than those on ingredients (P<0.01). The knowledge of community pharmacists in Palestine was insufficient with high risk of errors. Knowledge of community pharmacists on the safety issues of active ingredients and excipients need to be improved.

Examen en Vue du Diplome Douzieme Annee. Langue et Litterature 30. Partie B: Lecture (Choix Multiples). Livret de Questions (Examination for the Twelfth Grade Diploma, Language and Literature 30. Part B: Reading--Multiple Choice. Questions Booklet).

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

As part of an examination required by the Alberta (Canada) Department of Education in order for 12th grade students to receive a diploma in French, this booklet contains the 80 multiple choice questions portion of Part B, the language and literature component of the January 1987 tests. Representing the genres of poetry, short story, the novel, and…

Examen en Vue du Diplome Douzieme Annee. Langue et Litterature 30. Partie B: Lecture (Choix Multiples). Livret de Questions (Examination for the Twelfth Grade Diploma, Language and Literature 30. Part B: Reading--Multiple Choice. Questions Booklet.)

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

As part of an examination required by the Alberta (Canada) Department of Education in order for 12th grade students to receive a diploma in French, this booklet contains the 80 multiple choice questions portion of Part B, the language and literature component of the January 1988 tests. Representing the genres of poetry, short story, novel, and…

Impact of an engineering design-based curriculum compared to an inquiry-based curriculum on fifth graders' content learning of simple machines

NASA Astrophysics Data System (ADS)

Marulcu, Ismail; Barnett, Michael

2016-01-01

Background: Elementary Science Education is struggling with multiple challenges. National and State test results confirm the need for deeper understanding in elementary science education. Moreover, national policy statements and researchers call for increased exposure to engineering and technology in elementary science education. The basic motivation of this study is to suggest a solution to both improving elementary science education and increasing exposure to engineering and technology in it. Purpose/Hypothesis: This mixed-method study examined the impact of an engineering design-based curriculum compared to an inquiry-based curriculum on fifth graders' content learning of simple machines. We hypothesize that the LEGO-engineering design unit is as successful as the inquiry-based unit in terms of students' science content learning of simple machines. Design/Method: We used a mixed-methods approach to investigate our research questions; we compared the control and the experimental groups' scores from the tests and interviews by using Analysis of Covariance (ANCOVA) and compared each group's pre- and post-scores by using paired t-tests. Results: Our findings from the paired t-tests show that both the experimental and comparison groups significantly improved their scores from the pre-test to post-test on the multiple-choice, open-ended, and interview items. Moreover, ANCOVA results show that students in the experimental group, who learned simple machines with the design-based unit, performed significantly better on the interview questions. Conclusions: Our analyses revealed that the design-based Design a people mover: Simple machines unit was, if not better, as successful as the inquiry-based FOSS Levers and pulleys unit in terms of students' science content learning.

Estimating Preferences for Complex Health Technologies: Lessons Learned and Implications for Personalized Medicine.

PubMed

Marshall, Deborah A; Gonzalez, Juan Marcos; MacDonald, Karen V; Johnson, F Reed

2017-01-01

We examine key study design challenges of using stated-preference methods to estimate the value of whole-genome sequencing (WGS) as a specific example of genomic testing. Assessing the value of WGS is complex because WGS provides multiple findings, some of which can be incidental in nature and unrelated to the specific health concerns that motivated the test. In addition, WGS results can include actionable findings (variants considered to be clinically useful and can be acted on), findings for which evidence for best clinical action is not available (variants considered clinically valid but do not meet as high of a standard for clinical usefulness), and findings of unknown significance. We consider three key challenges encountered in designing our national study on the value of WGS-layers of uncertainty, potential downstream consequences with endogenous aspects, and both positive and negative utility associated with testing information-and potential solutions as strategies to address these challenges. We conceptualized the decision to acquire WGS information as a series of sequential choices that are resolved separately. To determine the value of WGS information at the initial decision to undergo WGS, we used contingent valuation questions, and to elicit respondent preferences for reducing risks of health problems and the consequences of taking the steps to reduce these risks, we used a discrete-choice experiment. We conclude by considering the implications for evaluating the value of other complex health technologies that involve multiple forms of uncertainty. Copyright © 2017 International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. All rights reserved.

Assessment of representational competence in kinematics

NASA Astrophysics Data System (ADS)

Klein, P.; Müller, A.; Kuhn, J.

2017-06-01

A two-tier instrument for representational competence in the field of kinematics (KiRC) is presented, designed for a standard (1st year) calculus-based introductory mechanics course. It comprises 11 multiple choice (MC) and 7 multiple true-false (MTF) questions involving multiple representational formats, such as graphs, pictures, and formal (mathematical) expressions (1st tier). Furthermore, students express their answer confidence for selected items, providing additional information (2nd tier). Measurement characteristics of KiRC were assessed in a validation sample (pre- and post-test, N =83 and N =46 , respectively), including usefulness for measuring learning gain. Validity is checked by interviews and by benchmarking KiRC against related measures. Values for item difficulty, discrimination, and consistency are in the desired ranges; in particular, a good reliability was obtained (KR 20 =0.86 ). Confidence intervals were computed and a replication study yielded values within the latter. For practical and research purposes, KiRC as a diagnostic tool goes beyond related extant instruments both for the representational formats (e.g., mathematical expressions) and for the scope of content covered (e.g., choice of coordinate systems). Together with the satisfactory psychometric properties it appears a versatile and reliable tool for assessing students' representational competency in kinematics (and of its potential change). Confidence judgments add further information to the diagnostic potential of the test, in particular for representational misconceptions. Moreover, we present an analytic result for the question—arising from guessing correction or educational considerations—of how the total effect size (Cohen's d ) varies upon combination of two test components with known individual effect sizes, and then discuss the results in the case of KiRC (MC and MTF combination). The introduced method of test combination analysis can be applied to any test comprising two components for the purpose of finding effect size ranges.

Order of Presentation of Dimensions Does Not Systematically Bias Utility Weights from a Discrete Choice Experiment.

PubMed

Norman, Richard; Kemmler, Georg; Viney, Rosalie; Pickard, A Simon; Gamper, Eva; Holzner, Bernhard; Nerich, Virginie; King, Madeleine

2016-12-01

Discrete choice experiments (DCEs) are increasingly used to value aspects of health. An issue with their adoption is that results may be sensitive to the order in which dimensions of health are presented in the valuation task. Findings in the literature regarding order effects are discordant at present. To quantify the magnitude of order effect of quality-of-life (QOL) dimensions within the context of a DCE designed to produce country-specific value sets for the EORTC Quality of Life Utility Measure-Core 10 dimensions (QLU-C10D), a new utility instrument derived from the widely used cancer-specific QOL questionnaire, the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire-Core 30. The DCE comprised 960 choice sets, divided into 60 versions of 16 choice sets, with each respondent assigned to a version. Within each version, the order of QLU-C10D QOL dimensions was randomized, followed by life duration in the last position. The DCE was completed online by 2053 individuals in France and Germany. We analyzed the data with a series of conditional logit models, adjusted for repeated choices within respondent. We used F tests to assess order effects, correcting for multiple hypothesis testing. Each F test failed to reject the null hypothesis of no position effect: 1) all QOL order positions considered jointly; 2) last QOL position only; 3) first QOL position only. Furthermore, the order coefficients were small relative to those of the QLU-C10D QOL dimension levels. The order of presentation of QOL dimensions within a DCE designed to provide utility weights for the QLU-C10D had little effect on level coefficients of those QOL dimensions. Copyright © 2016 International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. All rights reserved.

Resistance of Collard Green Genotypes to Bemisia tabaci Biotype B: Characterization of Antixenosis.

PubMed

Domingos, G M; Baldin, E L L; Canassa, V F; Silva, I F; Lourenção, A L

2018-08-01

Bemisia tabaci (Genn.) biotype B (Hemiptera: Aleyrodidae) is an important pest of vegetable crops, including collard greens Brassica oleracea var. acephala (Brassicaceae). The use of resistant genotypes is an interesting option to reduce insect populations and can be used as an important tool for integrated pest management (IPM). This study evaluated 32 genotypes of collard greens against the attack of silver leaf whitefly, with the aim to characterize antixenosis. Initially, a multiple-choice trial was conducted using all genotypes, in which the adult attractiveness was assessed on two leaves per genotype at 24 and 48 h after infestation. After 48 h, one leaf of each genotype was randomly selected for the determination of the number of eggs per square centimeter. From the results of the multiple-choice trial, 13 genotypes were selected for a no-choice oviposition test, following the same method of the previous test. Colorimetric analyses were also performed to establish possible correlations between leaf color and insect colonization. Genotypes HS-20, OE, and VA were less attractive, demonstrating antixenosis. Genotypes LG, VE, J, MG, MOP, HS-20, VA, and MT had less oviposition in the multiple-choice test, which indicated expression of antixenosis. In the no-choice test, genotypes VE, P1C, CCB, RI-919, H, and J had less oviposition, which also characterized antixenosis. Therefore, genotypes VE and J showed the highest resistance stability because both had less oviposition in both test modalities. Thus, the resistance to B. tabaci biotype B indicates the genotypes HS-20, OE, VA, VE, and J are promising for use in breeding programs to develop resistance to whitefly.

The effects of computer simulation versus hands-on dissection and the placement of computer simulation within the learning cycle on student achievement and attitude

NASA Astrophysics Data System (ADS)

Hopkins, Kathryn Susan

The value of dissection as an instructional strategy has been debated, but not evidenced in research literature. The purpose of this study was to examine the efficacy of using computer simulated frog dissection as a substitute for traditional hands-on frog dissection and to examine the possible enhancement of achievement by combining the two strategies in a specific sequence. In this study, 134 biology students at two Central Texas schools were divided into the five following treatment groups: computer simulation of frog dissection, computer simulation before dissection, traditional hands-on frog dissection, dissection before computer simulation, and textual worksheet materials. The effects on achievement were evaluated by labeling 10 structures on three diagrams, identifying 11 pinned structures on a prosected frog, and answering 9 multiple-choice questions over the dissection process. Attitude was evaluated using a thirty item survey with a five-point Likert scale. The quasi-experimental design was pretest/post-test/post-test nonequivalent group for both control and experimental groups, a 2 x 2 x 5 completely randomized factorial design (gender, school, five treatments). The pretest/post-test design was incorporated to control for prior knowledge using analysis of covariance. The dissection only group evidenced a significantly higher performance than all other treatments except dissection-then-computer on the post-test segment requiring students to label pinned anatomical parts on a prosected frog. Interactions between treatment and school in addition to interaction between treatment and gender were found to be significant. The diagram and attitude post-tests evidenced no significant difference. Results on the nine multiple-choice questions about dissection procedures indicated a significant difference between schools. The interaction between treatment and school was also found to be significant. On a delayed post-test, a significant difference in gender was found on the diagram labeling segment of the post-test. Males were reported to have the higher score. Since existing research conflicts with this study's results, additional research using authentic assessment is recommended. Instruction should be aligned with dissection content and process objectives for each treatment group, and the teacher variable should be controlled.

Teaching habitat and animal classification to fourth graders using an engineering-design model

NASA Astrophysics Data System (ADS)

Marulcu, Ismail

2014-05-01

Background: The motivation for this work is built upon the premise that there is a need for research-based materials for design-based science instruction. In this paper, a small portion of our work investigating the impact of a LEGOTM engineering unit on fourth grade students' preconceptions and understanding of animals is presented. Purpose: The driving questions for our work are: (1) What is the impact of an engineering-design-based curricular module on students' understanding of habitat and animal classification? (2) What are students' misconceptions regarding animal classification and habitat? Sample: The study was conducted in an inner-city K-8 school in the northeastern region of the United States. There were two fourth grade classrooms in the school. The first classroom included seven girls and nine boys, whereas the other classroom included eight girls and eight boys. All fourth grade students participated in the study. Design and methods: In answering the research questions mixed-method approaches are used. Data collection methods included pre- and post-tests, pre- and post-interviews, student journals, and classroom observations. Identical pre- and post-tests were administered to measure students' understanding of animals. They included four multiple-choice and six open-ended questions. Identical pre- and post-interviews were administered to explore students' in-depth understanding of animals. Results: Our results show that students significantly increased their performance after instruction on both the multiple-choice questions (t = -3.586, p = .001) and the open-ended questions (t = -5.04, p = .000). They performed better on the post interviews as well. Also, it is found that design-based instruction helped students comprehend core concepts of a life science subject, animals. Conclusions: Based on these results, the main argument of the study is that engineering design is a useful framework for teaching not only physical science-related subjects, but also life science subjects in elementary science classrooms.

Solving Geometric Problems by Using Algebraic Representation for Junior High School Level 3 in Van Hiele at Geometric Thinking Level

ERIC Educational Resources Information Center

Suwito, Abi; Yuwono, Ipung; Parta, I. Nengah; Irawati, Santi; Oktavianingtyas, Ervin

2016-01-01

This study aims to determine the ability of algebra students who have 3 levels van Hiele levels. Follow its framework Dindyal framework (2007). Students are required to do 10 algebra shaped multiple choice, then students work 15 about the geometry of the van Hiele level in the form of multiple choice questions. The question has been tested levels…

Assessment of item-writing flaws in multiple-choice questions.

PubMed

Nedeau-Cayo, Rosemarie; Laughlin, Deborah; Rus, Linda; Hall, John

2013-01-01

This study evaluated the quality of multiple-choice questions used in a hospital's e-learning system. Constructing well-written questions is fraught with difficulty, and item-writing flaws are common. Study results revealed that most items contained flaws and were written at the knowledge/comprehension level. Few items had linked objectives, and no association was found between the presence of objectives and flaws. Recommendations include education for writing test questions.

An Item Response Theory Analysis of Palmore's Facts on Aging Quiz (FAQ) Using the Three Parameter Model.

ERIC Educational Resources Information Center

Obiekwe, Jerry C.

Palmore's Facts on Aging Quiz (FAQ) (E. Palmore, 1977) is an instrument that is used to educate, to measure learning, to test knowledge, to measure attitudes toward aging, and in research. A comparative analysis was performed between the FAQ I and its multiple choice version and the FAQ II and its multiple choice version in terms of their item…

Helping physics teacher-candidates develop questioning skills through innovative technology use

NASA Astrophysics Data System (ADS)

Milner-Bolotin, Marina

2015-12-01

Peer Instruction has been used successfully in undergraduate classrooms for decades. Its success depends largely on the quality of multiple-choice questions. Yet it is still rare in secondary schools because of teachers' lack of experience in designing, evaluating, and implementing conceptual questions. Research-based multiple-choice conceptual questions are also underutilized in physics teacher education. This study explores the implementation of Peer Instruction enhanced by PeerWise collaborative online system, in a physics methods course in a physics teacher education program.

Writing Multiple Choice Outcome Questions to Assess Knowledge and Competence.

PubMed

Brady, Erik D

2015-11-01

Few articles contemplate the need for good guidance in question item-writing in the continuing education (CE) space. Although many of the core principles of sound item design translate to the CE health education team, the need exists for specific examples for nurse educators that clearly describe how to measure changes in competence and knowledge using multiple choice items. In this article, some keys points and specific examples for nursing CE providers are shared. Copyright 2015, SLACK Incorporated.

The Effectiveness of learning materials based on multiple intelligence on the understanding of global warming

NASA Astrophysics Data System (ADS)

Liliawati, W.; Purwanto; Zulfikar, A.; Kamal, R. N.

2018-05-01

This study aims to examine the effectiveness of the use of teaching materials based on multiple intelligences on the understanding of high school students’ material on the theme of global warming. The research method used is static-group pretest-posttest design. Participants of the study were 60 high school students of XI class in one of the high schools in Bandung. Participants were divided into two classes of 30 students each for the experimental class and control class. The experimental class uses compound-based teaching materials while the experimental class does not use a compound intelligence-based teaching material. The instrument used is a test of understanding of the concept of global warming with multiple choices form amounted to 15 questions and 5 essay items. The test is given before and after it is applied to both classes. Data analysis using N-gain and effect size. The results obtained that the N-gain for both classes is in the medium category and the effectiveness of the use of teaching materials based on the results of effect-size test results obtained in the high category.

Preference as a Function of Active Interresponse Times: A Test of the Active Time Model

ERIC Educational Resources Information Center

Misak, Paul; Cleaveland, J. Mark

2011-01-01

In this article, we describe a test of the active time model for concurrent variable interval (VI) choice. The active time model (ATM) suggests that the time since the most recent response is one of the variables controlling choice in concurrent VI VI schedules of reinforcement. In our experiment, pigeons were trained in a multiple concurrent…

Feedback-related brain activity predicts learning from feedback in multiple-choice testing.

PubMed

Ernst, Benjamin; Steinhauser, Marco

2012-06-01

Different event-related potentials (ERPs) have been shown to correlate with learning from feedback in decision-making tasks and with learning in explicit memory tasks. In the present study, we investigated which ERPs predict learning from corrective feedback in a multiple-choice test, which combines elements from both paradigms. Participants worked through sets of multiple-choice items of a Swahili-German vocabulary task. Whereas the initial presentation of an item required the participants to guess the answer, corrective feedback could be used to learn the correct response. Initial analyses revealed that corrective feedback elicited components related to reinforcement learning (FRN), as well as to explicit memory processing (P300) and attention (early frontal positivity). However, only the P300 and early frontal positivity were positively correlated with successful learning from corrective feedback, whereas the FRN was even larger when learning failed. These results suggest that learning from corrective feedback crucially relies on explicit memory processing and attentional orienting to corrective feedback, rather than on reinforcement learning.

Mixed-Format Test Score Equating: Effect of Item-Type Multidimensionality, Length and Composition of Common-Item Set, and Group Ability Difference

ERIC Educational Resources Information Center

Wang, Wei

2013-01-01

Mixed-format tests containing both multiple-choice (MC) items and constructed-response (CR) items are now widely used in many testing programs. Mixed-format tests often are considered to be superior to tests containing only MC items although the use of multiple item formats leads to measurement challenges in the context of equating conducted under…

The impact of interpretive and reductive front-of-pack labels on food choice and willingness to pay.

PubMed

Talati, Zenobia; Norman, Richard; Pettigrew, Simone; Neal, Bruce; Kelly, Bridget; Dixon, Helen; Ball, Kylie; Miller, Caroline; Shilton, Trevor

2017-12-19

This study examined how front-of-pack labels and product healthfulness affect choice and willingness to pay across a range of foods. It was hypothesized that: (i) product choice and (ii) willingness to pay would be more aligned with product healthfulness when healthfulness was expressed through the Health Star Rating, followed by the Multiple Traffic Light, then the Daily Intake Guide, and (iii) the Nutrition Facts Panel would be viewed infrequently. Adults and children aged 10+ years (n = 2069) completed an online discrete choice task involving mock food packages. A 4 food type (cookies, corn flakes, pizza, yoghurt) × 2 front-of-pack label presence (present, absent) × 3 front-of-pack label type (Daily Intake Guide, Multiple Traffic Light, Health Star Rating) × 3 price (cheap, moderate, expensive) × 3 healthfulness (less healthy, moderately healthy, healthier) design was used. A 30 s time limit was imposed for each choice. Of the three front-of-pack labels tested, the Health Star Rating produced the largest differences in choices, with 40% (95% CIs: 38%-42%) of respondents selecting the healthier variant, 33% selecting the moderately healthy variant (95% CIs: 31%-35%), and 23% (95% CIs: 21%-24%) selecting the less healthy variant of the four products included in the study. The Multiple Traffic Light led to significant differences in choices between healthier (35%, 95% CIs: 33%-37%) and less healthy products (29%, 95% CIs: 27%-31%), but not moderately healthy products (32%, 95% CIs: 30%-34%). No significant differences in choices were observed by product healthfulness when the Daily Intake Guide was present. Only the Health Star Rating resulted in a significantly greater willingness to pay for healthier versus less healthy products. The Nutrition Facts Panel was viewed for only 7% of all mock packages. Front-of-pack labels that are more interpretive, such as the Health Star Rating, can be more effective at directing consumers towards healthier choices than reductive front-of-pack labels such as the Daily Intake Guide. The study results provide policy makers with clear guidance on the types of front-of-pack labels that are most likely to achieve positive health outcomes at a population level.

Resistance Mechanisms in Hepatitis C Virus: implications for Direct-Acting Antiviral Use.

PubMed

Bagaglio, Sabrina; Uberti-Foppa, Caterina; Morsica, Giulia

2017-07-01

Multiple direct-acting antiviral (DAA)-based regimens are currently approved that provide one or more interferon-free treatment options for hepatitis C virus (HCV) genotypes (G) 1-6. The choice of a DAA regimen, duration of therapy, and use of ribavirin depends on multiple viral and host factors, including HCV genotype, the detection of resistance-associated amino acid (aa) substitutions (RASs), prior treatment experience, and presence of cirrhosis. In regard to viral factors that may guide the treatment choice, the most important is the infecting genotype because a number of DAAs are genotype-designed. The potency and the genetic barrier may also impact the choice of treatment. One important and debated possible virologic factor that may negatively influence the response to DAAs is the presence of baseline RASs. Baseline resistance testing is currently not routinely considered or recommended for initiating HCV treatment, due to the overall high response rates (sustained virological response >90%) obtained. Exceptions are patients infected by HCV G1a when initiating treatment with simeprevir and elbasvir/grazoprevir or in those with cirrhosis prior to daclatasvir/sofosbuvir treatment because of natural polymorphisms demonstrated in sites of resistance. On the basis of these observations, first-line strategies should be optimized to overcome treatment failure due to HCV resistance.

Comprehension of confidence intervals - development and piloting of patient information materials for people with multiple sclerosis: qualitative study and pilot randomised controlled trial.

PubMed

Rahn, Anne C; Backhus, Imke; Fuest, Franz; Riemann-Lorenz, Karin; Köpke, Sascha; van de Roemer, Adrianus; Mühlhauser, Ingrid; Heesen, Christoph

2016-09-20

Presentation of confidence intervals alongside information about treatment effects can support informed treatment choices in people with multiple sclerosis. We aimed to develop and pilot-test different written patient information materials explaining confidence intervals in people with relapsing-remitting multiple sclerosis. Further, a questionnaire on comprehension of confidence intervals was developed and piloted. We developed different patient information versions aiming to explain confidence intervals. We used an illustrative example to test three different approaches: (1) short version, (2) "average weight" version and (3) "worm prophylaxis" version. Interviews were conducted using think-aloud and teach-back approaches to test feasibility and analysed using qualitative content analysis. To assess comprehension of confidence intervals, a six-item multiple choice questionnaire was developed and tested in a pilot randomised controlled trial using the online survey software UNIPARK. Here, the average weight version (intervention group) was tested against a standard patient information version on confidence intervals (control group). People with multiple sclerosis were invited to take part using existing mailing-lists of people with multiple sclerosis in Germany and were randomised using the UNIPARK algorithm. Participants were blinded towards group allocation. Primary endpoint was comprehension of confidence intervals, assessed with the six-item multiple choice questionnaire with six points representing perfect knowledge. Feasibility of the patient information versions was tested with 16 people with multiple sclerosis. For the pilot randomised controlled trial, 64 people with multiple sclerosis were randomised (intervention group: n = 36; control group: n = 28). More questions were answered correctly in the intervention group compared to the control group (mean 4.8 vs 3.8, mean difference 1.1 (95 % CI 0.42-1.69), p = 0.002). The questionnaire's internal consistency was moderate (Cronbach's alpha = 0.56). The pilot-phase shows promising results concerning acceptability and feasibility. Pilot randomised controlled trial results indicate that the patient information is well understood and that knowledge gain on confidence intervals can be assessed with a set of six questions. German Clinical Trials Register: DRKS00008561 . Registered 8th of June 2015.

Retrieval practice with short-answer, multiple-choice, and hybrid tests.

PubMed

Smith, Megan A; Karpicke, Jeffrey D

2014-01-01

Retrieval practice improves meaningful learning, and the most frequent way of implementing retrieval practice in classrooms is to have students answer questions. In four experiments (N=372) we investigated the effects of different question formats on learning. Students read educational texts and practised retrieval by answering short-answer, multiple-choice, or hybrid questions. In hybrid conditions students first attempted to recall answers in short-answer format, then identified answers in multiple-choice format. We measured learning 1 week later using a final assessment with two types of questions: those that could be answered by recalling information verbatim from the texts and those that required inferences. Practising retrieval in all format conditions enhanced retention, relative to a study-only control condition, on both verbatim and inference questions. However, there were little or no advantages of answering short-answer or hybrid format questions over multiple-choice questions in three experiments. In Experiment 4, when retrieval success was improved under initial short-answer conditions, there was an advantage of answering short-answer or hybrid questions over multiple-choice questions. The results challenge the simple conclusion that short-answer questions always produce the best learning, due to increased retrieval effort or difficulty, and demonstrate the importance of retrieval success for retrieval-based learning activities.

Embedding Intervention Targets into Caregiving Routines and Other Activities of the Families Choice.

ERIC Educational Resources Information Center

Hollingshead, Lorie; Harris, Kristy; Stremel, Kathleen

This training module on embedding intervention targets into caregiving routines and other activities of the families' choice is from the Mississippi Early Education Program for Children with Multiple Disabilities, a program designed to train Individuals with Disabilities Education Act Part H service coordinators and service providers to use family…

Assessing secondary science students' knowledge of molecule movement, concentration gradients, and equilibrium through multiple contexts

NASA Astrophysics Data System (ADS)

Raven, Sara

2015-09-01

Background: Studies have shown that students' knowledge of osmosis and diffusion and the concepts associated with these processes is often inaccurate. This is important to address, as these concepts not only provide the foundation for more advanced topics in biology and chemistry, but are also threaded throughout both state and national science standards. Purpose: In this study, designed to determine the completeness and accuracy of three specific students' knowledge of molecule movement, concentration gradients, and equilibrium, I sought to address the following question: Using multiple evaluative methods, how can students' knowledge of molecule movement, concentration gradients, and equilibrium be characterized? Sample: This study focuses on data gathered from three students - Emma, Henry, and Riley - all of whom were gifted/honors ninth-grade biology students at a suburban high school in the southeast United States. Design and Methods: Using various qualitative data analysis techniques, I analyzed multiple sources of data from the three students, including multiple-choice test results, written free-response answers, think-aloud interview responses, and student drawings. Results: Results of the analysis showed that students maintained misconceptions about molecule movement, concentration gradients, and equilibrium. The conceptual knowledge students demonstrated differed depending on the assessment method, with the most distinct differences appearing on the multiple-choice versus the free-response questions, and in verbal versus written formats. Conclusions: Multiple levels of assessment may be required to obtain an accurate picture of content knowledge, as free-response and illustrative tasks made it difficult for students to conceal any misconceptions. Using a variety of assessment methods within a section of the curriculum can arguably help to provide a deeper understanding of student knowledge and learning, as well as illuminate misconceptions that may have remained unknown if only one assessment method was used. Furthermore, beyond simply evaluating past learning, multiple assessment methods may aid in student comprehension of key concepts.

Modeling Polytomous Item Responses Using Simultaneously Estimated Multinomial Logistic Regression Models

ERIC Educational Resources Information Center

Anderson, Carolyn J.; Verkuilen, Jay; Peyton, Buddy L.

2010-01-01

Survey items with multiple response categories and multiple-choice test questions are ubiquitous in psychological and educational research. We illustrate the use of log-multiplicative association (LMA) models that are extensions of the well-known multinomial logistic regression model for multiple dependent outcome variables to reanalyze a set of…

Experimental problem solving: An instructional improvement field experiment

NASA Astrophysics Data System (ADS)

Ross, John A.; Maynes, Florence J.

An instructional program based on expert-novice differences in experimental problem-solving performance was taught to grade 6 students (N = 265). Classes of students were randomly assigned to conditions in a delayed treatment design. Performance was assessed with multiple-choice and open-ended measures of specific transfer. Between group comparisons using pretest scores as a covariate showed that treatment condition students consistently outperformed controls; similar results were revealed in the within group comparisons. The achievement of the early treatment group did not decline in tests administered one month after the posttest.

Virtual test: A student-centered software to measure student's critical thinking on human disease

NASA Astrophysics Data System (ADS)

Rusyati, Lilit; Firman, Harry

2016-02-01

The study "Virtual Test: A Student-Centered Software to Measure Student's Critical Thinking on Human Disease" is descriptive research. The background is importance of computer-based test that use element and sub element of critical thinking. Aim of this study is development of multiple choices to measure critical thinking that made by student-centered software. Instruments to collect data are (1) construct validity sheet by expert judge (lecturer and medical doctor) and professional judge (science teacher); and (2) test legibility sheet by science teacher and junior high school student. Participants consisted of science teacher, lecturer, and medical doctor as validator; and the students as respondent. Result of this study are describe about characteristic of virtual test that use to measure student's critical thinking on human disease, analyze result of legibility test by students and science teachers, analyze result of expert judgment by science teachers and medical doctor, and analyze result of trial test of virtual test at junior high school. Generally, result analysis shown characteristic of multiple choices to measure critical thinking was made by eight elements and 26 sub elements that developed by Inch et al.; complete by relevant information; and have validity and reliability more than "enough". Furthermore, specific characteristic of multiple choices to measure critical thinking are information in form science comic, table, figure, article, and video; correct structure of language; add source of citation; and question can guide student to critical thinking logically.

Effects of Simplifying Choice Tasks on Estimates of Taste Heterogeneity in Stated-Choice Surveys

PubMed Central

Johnson, F. Reed; Ozdemir, Semra; Phillips, Kathryn A

2011-01-01

Researchers usually employ orthogonal arrays or D-optimal designs with little or no attribute overlap in stated-choice surveys. The challenge is to balance statistical efficiency and respondent burden to minimize the overall error in the survey responses. This study examined whether simplifying the choice task, by using a design with more overlap, provides advantages over standard minimum-overlap methods. We administered two designs for eliciting HIV test preferences to split samples. Surveys were undertaken at four HIV testing locations in San Francisco, California. Personal characteristics had different effects on willingness to pay for the two treatments, and gains in statistical efficiency in the minimal-overlap version more than compensated for possible imprecision from increased measurement error. PMID:19880234

[Working as a clinician-scientist in psychosomatic medicine: status, skills and research productivity].

PubMed

Hartmann, Mechthild; Wild, Beate; Herzog, Wolfgang; Nikendei, Christoph; Zipfel, Stephan; Henningsen, Peter; Löwe, Bernd

2008-06-01

Even though there is a high need of clinical research for the medical and psychotherapeutic practice in Germany, the interest in clinical research seems to be decreasing. The aim of this study was to assess the circumstances under which clinical research in psychosocial medicine is performed and to identify opportunities for improvement. n = 53 residents of the departments for Psychosomatic Medicine of the University Hospitals of Heidelberg and Tübingen and of the Technical University of Munich were asked about their research activities, their subjective research skills, and their productivity in clinical psychosocial research. In addition, objective research knowledge was investigated using a multiple-choice test. Both, subjective research skills and objective research knowledge were relatively low. The percentage of correct answers in the multiple choice test was 33 %. Subjective problems were predominately stated regarding "biostatistics" and "study design". In terms of research productivity, 33 % of residents had published as first authors of an original journal article, and 12 % had submitted a successful grant proposal. Altogether, there is a high need of training in the field of clinical psychosomatic research. We are presenting a training model that is adapted to the conditions of young clinicians and that addresses both general clinical research and specific psychosocial clinical research.

Nutritional ecology of the formosan subterranean termite (Isoptera: Rhinotermitidae): feeding response to commercial wood species.

PubMed

Morales-Ramos, J A; Rojas, M G

2001-04-01

The feeding preferences of the Formosan subterranean termite, Coptotermes formosanus Shiraki, were tested in three separate experiments on 28 different wood species. Experiment 1 was a multiple-choice test designed to test relative preferences among 24 wood species commercially available in New Orleans, LA. Experiment 2 was a similar study designed to test relative preferences among 21 wood species shown or reported to be unpalatable to the Formosan subterranean termite. Experiment 3 was a no-choice test to examine the feeding deterrence of the 10 least preferred wood species. Preference was determined by consumption rates. Birch (Betula alleghaniensis Britton), red gum (Liquidambar styraciflua L.), Parana pine [Araucaria angustifolia (Bert.) 1, sugar maple (Acer saccharum Marsh.), pecan (Carya illinoensis Wangenh.), and northern red oak (Quercus rubra L.) were the most preferred species by C. formosanus in order of consumption rate. All of these species were significantly more preferred than southern yellow pine (Pinus taeda L.), widely used for monitoring. Sinker cypress [ = old growth bald cypress, Taxodium distichum (L.)], western red cedar (Thuja plicata Donn), Alaskan yellow cedar (Chamaecyparis nootkatensis D. Don), eastern red cedar (Juniperus virginiana L.), sassafras [Sassafras albidum (Nutt.)], Spanish cedar (Cedrella odorata L.), Honduras mahogany (Swietenia macrophyla King), Indian rosewood (Dalbergia latifolia Roxb.), Honduras rosewood (D. stevensonii Standl.), and morado (Machaerium sp.) induced significant feeding deterrence and mortality to C. formosanus. The last eight species produced 100% mortality after 3 mo.

Multiple Choice: How Public School Leaders in New Orleans' Saturated Market View Private School Competitors

ERIC Educational Resources Information Center

Jabbar, Huriya; Li, Dongmei M.

2016-01-01

School choice policies, such as charter schools and vouchers, are in part designed to induce competition between schools. While several studies have examined the impact of private school competition on public schools, few studies have explored school leaders' perceptions of private school competitors. This study examines the extent to which public…

Proverb comprehension in individuals with agenesis of the corpus callosum.

PubMed

Rehmel, Jamie L; Brown, Warren S; Paul, Lynn K

2016-09-01

Comprehension of non-literal language involves multiple neural systems likely involving callosal connections. We describe proverb comprehension impairments in individuals with isolated agenesis of the corpus callosum (AgCC) and normal-range general intelligence. Experiment 1 compared Gorham Proverb Test (Gorham, 1956) performance in 19 adults with AgCC and 33 neurotypical control participants of similar age, sex, and intelligence. Experiment 2 used the Proverbs subtest of the Delis-Kaplan Executive Function System (D-KEFS, 2001) to compare 19 adults with AgCC and 17 control participants with similar age, sex, and intelligence. Gorham Proverbs performance was impaired in the AgCC group for both the free-response and multiple-choice tasks. On the D-KEFS proverbs test, the AgCC group performed significantly worse on the free-response task (and all derivative scores) despite normal levels of performance on the multiple-choice task. Covarying verbal intelligence did not alter these outcomes. However, covarying a measure of non-literal language comprehension considerably reduced group differences in proverb comprehension on the Gorham test, but had little effect on the D-KEFS group differences. The difference between groups seemed to be greatest when participants had to generate their own interpretation (free response), or in the multiple choice format when the test included many proverbs that were likely to be less familiar. Taken together, the results of this study clearly show that proverb comprehension is diminished in individuals with AgCC compared to their peers. Copyright © 2016 Elsevier Inc. All rights reserved.

Redesigning the Preexam Review Session

ERIC Educational Resources Information Center

King, Daniel

2010-01-01

In a large enrollment, multiple-section course, review sessions enable efficient and consistent delivery of information to all students. A redesigned review session has been implemented to increase attendance and improve effectiveness. The new design involves a practice quiz consisting of about 20 multiple-choice questions that students answer…

Assessing understanding of relative clauses: a comparison of multiple-choice comprehension versus sentence repetition.

PubMed

Frizelle, Pauline; O'Neill, Clodagh; Bishop, Dorothy V M

2017-11-01

Although sentence repetition is considered a reliable measure of children's grammatical knowledge, few studies have directly compared children's sentence repetition performance with their understanding of grammatical structures. The current study aimed to compare children's performance on these two assessment measures, using a multiple-choice picture-matching sentence comprehension task and a sentence repetition task. Thirty-three typically developing children completed both assessments, which included relative clauses representing a range of syntactic roles. Results revealed a similar order of difficulty of constructions on both measures but little agreement between them when evaluating individual differences. Interestingly, repetition was the easier of the two measures, with children showing the ability to repeat sentences they did not understand. This discrepancy is primarily attributed to the additional processing load resulting from the design of multiple-choice comprehension tasks, and highlights the fact that these assessments are invoking skills beyond those of linguistic competence.

Evolution of A Distributed Live, Virtual, Constructive Environment for Human in the Loop Unmanned Aircraft Testing

NASA Technical Reports Server (NTRS)

Murphy, James R.; Otto, Neil M.

2017-01-01

NASA's Unmanned Aircraft Systems Integration in the National Airspace System Project is conducting human in the loop simulations and flight testing intended to reduce barriers associated with enabling routine airspace access for unmanned aircraft. The primary focus of these tests is interaction of the unmanned aircraft pilot with the display of detect and avoid alerting and guidance information. The project's integrated test and evaluation team was charged with developing the test infrastructure. As with any development effort, compromises in the underlying system architecture and design were made to allow for the rapid prototyping and open-ended nature of the research. In order to accommodate these design choices, a distributed test environment was developed incorporating Live, Virtual, Constructive, (LVC) concepts. The LVC components form the core infrastructure support simulation of UAS operations by integrating live and virtual aircraft in a realistic air traffic environment. This LVC infrastructure enables efficient testing by leveraging the use of existing assets distributed across multiple NASA Centers. Using standard LVC concepts enable future integration with existing simulation infrastructure.

Evolution of A Distributed Live, Virtual, Constructive Environment for Human in the Loop Unmanned Aircraft Testing

NASA Technical Reports Server (NTRS)

Murphy, Jim; Otto, Neil

2017-01-01

NASA's Unmanned Aircraft Systems Integration in the National Airspace System Project is conducting human in the loop simulations and flight testing intended to reduce barriers associated with enabling routine airspace access for unmanned aircraft. The primary focus of these tests is interaction of the unmanned aircraft pilot with the display of detect and avoid alerting and guidance information. The projects integrated test and evaluation team was charged with developing the test infrastructure. As with any development effort, compromises in the underlying system architecture and design were made to allow for the rapid prototyping and open-ended nature of the research. In order to accommodate these design choices, a distributed test environment was developed incorporating Live, Virtual, Constructive, (LVC) concepts. The LVC components form the core infrastructure support simulation of UAS operations by integrating live and virtual aircraft in a realistic air traffic environment. This LVC infrastructure enables efficient testing by leveraging the use of existing assets distributed across multiple NASA Centers. Using standard LVC concepts enable future integration with existing simulation infrastructure.

Insights into Students' Conceptual Understanding Using Textual Analysis: A Case Study in Signal Processing

ERIC Educational Resources Information Center

Goncher, Andrea M.; Jayalath, Dhammika; Boles, Wageeh

2016-01-01

Concept inventory tests are one method to evaluate conceptual understanding and identify possible misconceptions. The multiple-choice question format, offering a choice between a correct selection and common misconceptions, can provide an assessment of students' conceptual understanding in various dimensions. Misconceptions of some engineering…

Characterization of Antixenosis in Soybean Genotypes to Bemisia tabaci (Hemiptera: Aleyrodidae) Biotype B.

PubMed

Baldin, E L L; Cruz, P L; Morando, R; Silva, I F; Bentivenha, J P F; Tozin, L R S; Rodrigues, T M

2017-08-01

Bemisia tabaci biotype B (Gennadius) is one of the most important soybean pest worldwide. Herein, 15 soybean genotypes were evaluated, to characterize the occurrence of antixenosis to B. tabaci biotype B. Initially, a multiple-choice test with all genotypes was carried out, evaluating the settling and oviposition preference at 3 d after infestation, and the colonization by nymphs after 48 d of infestation. Subsequently, a no-choice test, using 14 genotypes, was conducted with infested plants individually, and the number of eggs was counted after 72 h. Then, 10 genotypes were selected (indicative of resistance and susceptibility), which were evaluated for whitefly settling 24, 48, and 72 h after infestation and for oviposition 72 h after infestation. The trichomes of the leaflets were characterized for density, size, and inclination to establish possible correlations with the settling and oviposition in the genotypes. In the first multiple-choice test, involving 15 genotypes, 'IAC-17,' 'IAC-19,' and UX-2569-159 expressed antixenosis against B. tabaci. 'Jackson,' 'P98Y11,' and PI-229358 exhibited the same behavior in the no-choice test. In the multiple-choice test, 'Jackson,' 'P98Y11,' and 'TMG1176 RR' were the least attractive and least used for oviposition. The antixenosis shown by 'Jackson,' 'P98Y11,' and PI-229358 may be related to the characteristics of the trichomes (lower density and inclined). Based on the experiments carried out, 'IAC-17,' 'IAC-19,' 'Jackson,' 'P98Y11,' PI-229358, TMG1176 RR, and UX-2569-159 are considered promising for resistance to B. tabaci biotype B and may be exploited in soybean breeding programs for resistance to insects. © The Authors 2017. Published by Oxford University Press on behalf of Entomological Society of America. All rights reserved. For Permissions, please email: journals.permissions@oup.com.

Refining a brief decision aid in stable CAD: cognitive interviews.

PubMed

Kelly-Blake, Karen; Clark, Stacie; Dontje, Katherine; Olomu, Adesuwa; Henry, Rebecca C; Rovner, David R; Rothert, Marilyn L; Holmes-Rovner, Margaret

2014-02-13

We describe the results of cognitive interviews to refine the "Making Choices©" Decision Aid (DA) for shared decision-making (SDM) about stress testing in patients with stable coronary artery disease (CAD). We conducted a systematic development process to design a DA consistent with International Patient Decision Aid Standards (IPDAS) focused on Alpha testing criteria. Cognitive interviews were conducted with ten stable CAD patients using the "think aloud" interview technique to assess the clarity, usefulness, and design of each page of the DA. Participants identified three main messages: 1) patients have multiple options based on stress tests and they should be discussed with a physician, 2) take care of yourself, 3) the stress test is the gold standard for determining the severity of your heart disease. Revisions corrected the inaccurate assumption of item number three. Cognitive interviews proved critical for engaging patients in the development process and highlighted the necessity of clear message development and use of design principles that make decision materials easy to read and easy to use. Cognitive interviews appear to contribute critical information from the patient perspective to the overall systematic development process for designing decision aids.

Make Better Choices (MBC): Study design of a randomized controlled trial testing optimal technology-supported change in multiple diet and physical activity risk behaviors

PubMed Central

2010-01-01

Background Suboptimal diet and physical inactivity are prevalent, co-occurring chronic disease risk factors, yet little is known about how to maximize multiple risk behavior change. Make Better Choices, a randomized controlled trial, tests competing hypotheses about the optimal way to promote healthy change in four bundled risk behaviors: high saturated fat intake, low fruit and vegetable intake, low physical activity, and high sedentary leisure screen time. The study aim is to determine which combination of two behavior change goals - one dietary, one activity - yields greatest overall healthy lifestyle change. Methods/Design Adults (n = 200) with poor quality diet and sedentary lifestyle will be recruited and screened for study eligibility. Participants will be trained to record their diet and activities onto a personal data assistant, and use it to complete two weeks of baseline. Those who continue to show all four risk behaviors after baseline recording will be randomized to one of four behavior change prescriptions: 1) increase fruits and vegetables and increase physical activity, 2) decrease saturated fat and increase physical activity, 3) increase fruits and vegetable and decrease saturated fat, or 4) decrease saturated fat and decrease sedentary activity. They will use decision support feedback on the personal digital assistant and receive counseling from a coach to alter their diet and activity during a 3-week prescription period when payment is contingent upon meeting behavior change goals. They will continue recording on an intermittent schedule during a 4.5-month maintenance period when payment is not contingent upon goal attainment. The primary outcome is overall healthy lifestyle change, aggregated across all four risk behaviors. Discussion The Make Better Choices trial tests a disseminable lifestyle intervention supported by handheld technology. Findings will fill a gap in knowledge about optimal goal prescription to facilitate simultaneous diet and activity change. Results will shed light on which goal prescription maximizes healthful lifestyle change. Trial Registration Clinical Trials Gov. Identifier NCT00113672 PMID:20920275

The Impact of the Tree Prior on Molecular Dating of Data Sets Containing a Mixture of Inter- and Intraspecies Sampling.

PubMed

Ritchie, Andrew M; Lo, Nathan; Ho, Simon Y W

2017-05-01

In Bayesian phylogenetic analyses of genetic data, prior probability distributions need to be specified for the model parameters, including the tree. When Bayesian methods are used for molecular dating, available tree priors include those designed for species-level data, such as the pure-birth and birth-death priors, and coalescent-based priors designed for population-level data. However, molecular dating methods are frequently applied to data sets that include multiple individuals across multiple species. Such data sets violate the assumptions of both the speciation and coalescent-based tree priors, making it unclear which should be chosen and whether this choice can affect the estimation of node times. To investigate this problem, we used a simulation approach to produce data sets with different proportions of within- and between-species sampling under the multispecies coalescent model. These data sets were then analyzed under pure-birth, birth-death, constant-size coalescent, and skyline coalescent tree priors. We also explored the ability of Bayesian model testing to select the best-performing priors. We confirmed the applicability of our results to empirical data sets from cetaceans, phocids, and coregonid whitefish. Estimates of node times were generally robust to the choice of tree prior, but some combinations of tree priors and sampling schemes led to large differences in the age estimates. In particular, the pure-birth tree prior frequently led to inaccurate estimates for data sets containing a mixture of inter- and intraspecific sampling, whereas the birth-death and skyline coalescent priors produced stable results across all scenarios. Model testing provided an adequate means of rejecting inappropriate tree priors. Our results suggest that tree priors do not strongly affect Bayesian molecular dating results in most cases, even when severely misspecified. However, the choice of tree prior can be significant for the accuracy of dating results in the case of data sets with mixed inter- and intraspecies sampling. [Bayesian phylogenetic methods; model testing; molecular dating; node time; tree prior.]. © The authors 2016. Published by Oxford University Press, on behalf of the Society of Systematic Biologists. All rights reserved. For permissions, please e-mail: journals.permission@oup.com.

Motion and force control of multiple robotic manipulators

NASA Technical Reports Server (NTRS)

Wen, John T.; Kreutz-Delgado, Kenneth

1992-01-01

This paper addresses the motion and force control problem of multiple robot arms manipulating a cooperatively held object. A general control paradigm is introduced which decouples the motion and force control problems. For motion control, different control strategies are constructed based on the variables used as the control input in the controller design. There are three natural choices; acceleration of a generalized coordinate, arm tip force vectors, and the joint torques. The first two choices require full model information but produce simple models for the control design problem. The last choice results in a class of relatively model independent control laws by exploiting the Hamiltonian structure of the open loop system. The motion control only determines the joint torque to within a manifold, due to the multiple-arm kinematic constraint. To resolve the nonuniqueness of the joint torques, two methods are introduced. If the arm and object models are available, an optimization can be performed to best allocate the desired and effector control force to the joint actuators. The other possibility is to control the internal force about some set point. It is shown that effective force regulation can be achieved even if little model information is available.

Memorial Consequences of Answering SAT II Questions

ERIC Educational Resources Information Center

Marsh, Elizabeth J.; Agarwal, Pooja K.; Roediger, Henry L., III

2009-01-01

Many thousands of students take standardized tests every year. In the current research, we asked whether answering standardized test questions affects students' later test performance. Prior research has shown both positive and negative effects of multiple-choice testing on later tests, with negative effects arising from students selecting…

Time-Series Analysis: Assessing the Effects of Multiple Educational Interventions in a Small-Enrollment Course

NASA Astrophysics Data System (ADS)

Warren, Aaron R.

2009-11-01

Time-series designs are an alternative to pretest-posttest methods that are able to identify and measure the impacts of multiple educational interventions, even for small student populations. Here, we use an instrument employing standard multiple-choice conceptual questions to collect data from students at regular intervals. The questions are modified by asking students to distribute 100 Confidence Points among the options in order to indicate the perceived likelihood of each answer option being the correct one. Tracking the class-averaged ratings for each option produces a set of time-series. ARIMA (autoregressive integrated moving average) analysis is then used to test for, and measure, changes in each series. In particular, it is possible to discern which educational interventions produce significant changes in class performance. Cluster analysis can also identify groups of students whose ratings evolve in similar ways. A brief overview of our methods and an example are presented.

Teaching and Evaluation Materials Utilizing Multiple Representations in Mechanics

ERIC Educational Resources Information Center

Savinainen, A.; Nieminen, P.; Makynen, A.; Viiri, J.

2013-01-01

In this paper, we present materials and teaching ideas utilizing multiple representations in the contexts of kinematics and the force concept. These ideas and materials are substantiated by evidence and can be readily used in teaching with no special training. In addition, we briefly discuss two multiple-choice tests based on physics education…

A simple test of choice stepping reaction time for assessing fall risk in people with multiple sclerosis.

PubMed

Tijsma, Mylou; Vister, Eva; Hoang, Phu; Lord, Stephen R

2017-03-01

Purpose To determine (a) the discriminant validity for established fall risk factors and (b) the predictive validity for falls of a simple test of choice stepping reaction time (CSRT) in people with multiple sclerosis (MS). Method People with MS (n = 210, 21-74y) performed the CSRT, sensorimotor, balance and neuropsychological tests in a single session. They were then followed up for falls using monthly fall diaries for 6 months. Results The CSRT test had excellent discriminant validity with respect to established fall risk factors. Frequent fallers (≥3 falls) performed significantly worse in the CSRT test than non-frequent fallers (0-2 falls). With the odds of suffering frequent falls increasing 69% with each SD increase in CSRT (OR = 1.69, 95% CI: 1.27-2.26, p = <0.001). In regression analysis, CSRT was best explained by sway, time to complete the 9-Hole Peg test, knee extension strength of the weaker leg, proprioception and the time to complete the Trails B test (multiple R 2   =   0.449, p < 0.001). Conclusions A simple low tech CSRT test has excellent discriminative and predictive validity in relation to falls in people with MS. This test may prove useful in documenting longitudinal changes in fall risk in relation to MS disease progression and effects of interventions. Implications for rehabilitation Good choice stepping reaction time (CSRT) is required for maintaining balance. A simple low-tech CSRT test has excellent discriminative and predictive validity in relation to falls in people with MS. This test may prove useful documenting longitudinal changes in fall risk in relation to MS disease progression and effects of interventions.

Trends in computer applications in science assessment

NASA Astrophysics Data System (ADS)

Kumar, David D.; Helgeson, Stanley L.

1995-03-01

Seven computer applications to science assessment are reviewed. Conventional test administration includes record keeping, grading, and managing test banks. Multiple-choice testing involves forced selection of an answer from a menu, whereas constructed-response testing involves options for students to present their answers within a set standard deviation. Adaptive testing attempts to individualize the test to minimize the number of items and time needed to assess a student's knowledge. Figurai response testing assesses science proficiency in pictorial or graphic mode and requires the student to construct a mental image rather than selecting a response from a multiple choice menu. Simulations have been found useful for performance assessment on a large-scale basis in part because they make it possible to independently specify different aspects of a real experiment. An emerging approach to performance assessment is solution pathway analysis, which permits the analysis of the steps a student takes in solving a problem. Virtually all computer-based testing systems improve the quality and efficiency of record keeping and data analysis.

An item response curves analysis of the Force Concept Inventory

NASA Astrophysics Data System (ADS)

Morris, Gary A.; Harshman, Nathan; Branum-Martin, Lee; Mazur, Eric; Mzoughi, Taha; Baker, Stephen D.

2012-09-01

Several years ago, we introduced the idea of item response curves (IRC), a simplistic form of item response theory (IRT), to the physics education research community as a way to examine item performance on diagnostic instruments such as the Force Concept Inventory (FCI). We noted that a full-blown analysis using IRT would be a next logical step, which several authors have since taken. In this paper, we show that our simple approach not only yields similar conclusions in the analysis of the performance of items on the FCI to the more sophisticated and complex IRT analyses but also permits additional insights by characterizing both the correct and incorrect answer choices. Our IRC approach can be applied to a variety of multiple-choice assessments but, as applied to a carefully designed instrument such as the FCI, allows us to probe student understanding as a function of ability level through an examination of each answer choice. We imagine that physics teachers could use IRC analysis to identify prominent misconceptions and tailor their instruction to combat those misconceptions, fulfilling the FCI authors' original intentions for its use. Furthermore, the IRC analysis can assist test designers to improve their assessments by identifying nonfunctioning distractors that can be replaced with distractors attractive to students at various ability levels.

Evaluation of five guidelines for option development in multiple-choice item-writing.

PubMed

Martínez, Rafael J; Moreno, Rafael; Martín, Irene; Trigo, M Eva

2009-05-01

This paper evaluates certain guidelines for writing multiple-choice test items. The analysis of the responses of 5013 subjects to 630 items from 21 university classroom achievement tests suggests that an option should not differ in terms of heterogeneous content because such error has a slight but harmful effect on item discrimination. This also occurs with the "None of the above" option when it is the correct one. In contrast, results do not show the supposedly negative effects of a different-length option, the use of specific determiners, or the use of the "All of the above" option, which not only decreases difficulty but also improves discrimination when it is the correct option.

Getting Lucky: How Guessing Threatens the Validity of Performance Classifications

ERIC Educational Resources Information Center

Foley, Brett P.

2016-01-01

There is always a chance that examinees will answer multiple choice (MC) items correctly by guessing. Design choices in some modern exams have created situations where guessing at random through the full exam--rather than only for a subset of items where the examinee does not know the answer--can be an effective strategy to pass the exam. This…

The establisment of an achievement test for determination of primary teachers’ knowledge level of earthquake

DOE Office of Scientific and Technical Information (OSTI.GOV)

Aydin, Süleyman, E-mail: yupul@hotmail.com; Haşiloğlu, M. Akif, E-mail: mehmet.hasiloglu@hotmail.com; Kunduraci, Ayşe, E-mail: ayse-kndrc@hotmail.com

In this study it was aimed to improve an academic achievement test to establish the students’ knowledge about the earthquake and the ways of protection from earthquakes. In the method of this study, the steps that Webb (1994) was created to improve an academic achievement test for a unit were followed. In the developmental process of multiple choice test having 25 questions, was prepared to measure the pre-service teachers’ knowledge levels about the earthquake and the ways of protection from earthquakes. The multiple choice test was presented to view of six academics (one of them was from geographic field andmore » five of them were science educator) and two expert teachers in science Prepared test was applied to 93 pre-service teachers studying in elementary education department in 2014-2015 academic years. As a result of validity and reliability of the study, the test was composed of 20 items. As a result of these applications, Pearson Moments Multiplication half-reliability coefficient was found to be 0.94. When this value is adjusted according to Spearman Brown reliability coefficient the reliability coefficient was set at 0.97.« less

Common Student Misconceptions in Exercise Physiology and Biochemistry

ERIC Educational Resources Information Center

Morton, James P.; Doran, Dominic A.; MacLaren, Don P. M.

2008-01-01

The present study represents a preliminary investigation designed to identify common misconceptions in students' understanding of physiological and biochemical topics within the academic domain of sport and exercise sciences. A specifically designed misconception inventory (consisting of 10 multiple-choice questions) was administered to a cohort…

Designing Templates for Interactive Tasks in CALL Tutorials.

ERIC Educational Resources Information Center

Ruhlmann, Felicitas

The development of templates for computer-assisted language learning (CALL) is discussed, based on experiences with primarily linear multimedia tutorial programs. Design of templates for multiple-choice questions and interactive tasks in a prototype module is described. Possibilities of enhancing interactivity by introducing problem-oriented…

An Alternative Method for Teaching and Testing Reading Comprehension.

ERIC Educational Resources Information Center

Courchene, Robert

1995-01-01

The summary cloze technique offers an alternative to multiple choice. Summary cloze exercises are prepared by summarizing the content of the original text. The shortened text is transformed into a rational cloze exercise. The learner completes the summary text using the list of choices provided. This technique is a good measure of reading…

Fixed or mixed: a comparison of three, four and mixed-option multiple-choice tests in a Fetal Surveillance Education Program

PubMed Central

2013-01-01

Background Despite the widespread use of multiple-choice assessments in medical education assessment, current practice and published advice concerning the number of response options remains equivocal. This article describes an empirical study contrasting the quality of three 60 item multiple-choice test forms within the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG) Fetal Surveillance Education Program (FSEP). The three forms are described below. Methods The first form featured four response options per item. The second form featured three response options, having removed the least functioning option from each item in the four-option counterpart. The third test form was constructed by retaining the best performing version of each item from the first two test forms. It contained both three and four option items. Results Psychometric and educational factors were taken into account in formulating an approach to test construction for the FSEP. The four-option test performed better than the three-option test overall, but some items were improved by the removal of options. The mixed-option test demonstrated better measurement properties than the fixed-option tests, and has become the preferred test format in the FSEP program. The criteria used were reliability, errors of measurement and fit to the item response model. Conclusions The position taken is that decisions about the number of response options be made at the item level, with plausible options being added to complete each item on both psychometric and educational grounds rather than complying with a uniform policy. The point is to construct the better performing item in providing the best psychometric and educational information. PMID:23453056

Fixed or mixed: a comparison of three, four and mixed-option multiple-choice tests in a Fetal Surveillance Education Program.

PubMed

Zoanetti, Nathan; Beaves, Mark; Griffin, Patrick; Wallace, Euan M

2013-03-04

Despite the widespread use of multiple-choice assessments in medical education assessment, current practice and published advice concerning the number of response options remains equivocal. This article describes an empirical study contrasting the quality of three 60 item multiple-choice test forms within the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG) Fetal Surveillance Education Program (FSEP). The three forms are described below. The first form featured four response options per item. The second form featured three response options, having removed the least functioning option from each item in the four-option counterpart. The third test form was constructed by retaining the best performing version of each item from the first two test forms. It contained both three and four option items. Psychometric and educational factors were taken into account in formulating an approach to test construction for the FSEP. The four-option test performed better than the three-option test overall, but some items were improved by the removal of options. The mixed-option test demonstrated better measurement properties than the fixed-option tests, and has become the preferred test format in the FSEP program. The criteria used were reliability, errors of measurement and fit to the item response model. The position taken is that decisions about the number of response options be made at the item level, with plausible options being added to complete each item on both psychometric and educational grounds rather than complying with a uniform policy. The point is to construct the better performing item in providing the best psychometric and educational information.

40 CFR 158.500 - Toxicology data requirements table.

Code of Federal Regulations, 2011 CFR

2011-07-01

... that apply to an individual test and include specific conditions, qualifications, or exceptions to the designated test in the table are listed in paragraph (e) of this section. (b) Use patterns. (1) Food use...; PAIRA=Pure active ingredient radio-labeled; Choice=Choice of several test substances depending on study...

40 CFR 158.500 - Toxicology data requirements table.

Code of Federal Regulations, 2012 CFR

2012-07-01

... that apply to an individual test and include specific conditions, qualifications, or exceptions to the designated test in the table are listed in paragraph (e) of this section. (b) Use patterns. (1) Food use...; PAIRA=Pure active ingredient radio-labeled; Choice=Choice of several test substances depending on study...

40 CFR 158.500 - Toxicology data requirements table.

Code of Federal Regulations, 2010 CFR

2010-07-01

... that apply to an individual test and include specific conditions, qualifications, or exceptions to the designated test in the table are listed in paragraph (e) of this section. (b) Use patterns. (1) Food use...; PAIRA=Pure active ingredient radio-labeled; Choice=Choice of several test substances depending on study...

40 CFR 158.500 - Toxicology data requirements table.

Code of Federal Regulations, 2014 CFR

2014-07-01

... that apply to an individual test and include specific conditions, qualifications, or exceptions to the designated test in the table are listed in paragraph (e) of this section. (b) Use patterns. (1) Food use...; PAIRA=Pure active ingredient radio-labeled; Choice=Choice of several test substances depending on study...

Motives underlying food choice: dentists, porters and dietary health promotion.

PubMed

Crossley, M L; Khan, S N

2001-08-25

Differences in dental decay and disease amongst socioeconomic groups are thought to derive, in part, from variations in dietary practices and differences in education. The aim of this exploratory study was to examine whether differences in motivating factors affecting food choice could be found in a comparison of two groups at very different ends of the social spectrum: dentists and porters/cleaners. A convenience sample of 100 people (51 porters/cleaners and 49 dentists) working in the dental school at a university in the North West of England were approached to interview face-to-face and complete the Food Choice Questionnaire (FCQ), a previously validated measure designed to assess nine main factors relevant to peoples' food choices. A sample size of 100 was chosen because it was adequate to test validity (using a two-group Chi-square test with a 0.050 two sided significance). Findings were analysed using independent sample t-test and multiple linear regression. Results indicated significant differences between porters/cleaners and dentists in terms of their motives for food choice on six of the nine FCQ factors. These included convenience (p < 0.001), natural content (p < 0.05), price (p < 0.005), familiarity (p < .0001), mood (p < 0.03) and ethical concern (p < 0.01). Porters/cleaners tended to rate the factors covenience, price, mood and familiarity more highly, whereas dentists did the same for natural content and ethical concern. Awareness of the differences in motivational factors affecting food choice between different social groups is important to dental practitioners who are being taught to play an increasing role in health promotion. If dental practitioners are to partake meaningfully in such a role, it is necessary for them to be aware not only of their own motives in food selection, but also of the way in which those motives may differ from those of their clients.

Test Pool Questions, Area III.

ERIC Educational Resources Information Center

Sloan, Jamee Reid

This manual contains multiple choice questions to be used in testing students on nurse training objectives. Each test includes several questions covering each concept. The concepts in section A, medical surgical nursing, are diseases of the following systems: musculoskeletal; central nervous; cardiovascular; gastrointestinal; urinary and male…

A multiple choice testing program coupled with a year-long elective experience is associated with improved performance on the internal medicine in-training examination.

PubMed

Mathis, Bradley R; Warm, Eric J; Schauer, Daniel P; Holmboe, Eric; Rouan, Gregory W

2011-11-01

The Internal Medicine In-Training Exam (IM-ITE) assesses the content knowledge of internal medicine trainees. Many programs use the IM-ITE to counsel residents, to create individual remediation plans, and to make fundamental programmatic and curricular modifications. To assess the association between a multiple-choice testing program administered during 12 consecutive months of ambulatory and inpatient elective experience and IM-ITE percentile scores in third post-graduate year (PGY-3) categorical residents. Retrospective cohort study. One hundred and four categorical internal medicine residents. Forty-five residents in the 2008 and 2009 classes participated in the study group, and the 59 residents in the three classes that preceded the use of the testing program, 2005-2007, served as controls. A comprehensive, elective rotation specific, multiple-choice testing program and a separate board review program, both administered during a continuous long-block elective experience during the twelve months between the second post-graduate year (PGY-2) and PGY-3 in-training examinations. We analyzed the change in median individual percent correct and percentile scores between the PGY-1 and PGY-2 IM-ITE and between the PGY-2 and PGY-3 IM-ITE in both control and study cohorts. For our main outcome measure, we compared the change in median individual percentile rank between the control and study cohorts between the PGY-2 and the PGY-3 IM-ITE testing opportunities. After experiencing the educational intervention, the study group demonstrated a significant increase in median individual IM-ITE percentile score between PGY-2 and PGY-3 examinations of 8.5 percentile points (p < 0.01). This is significantly better than the increase of 1.0 percentile point seen in the control group between its PGY-2 and PGY-3 examination (p < 0.01). A comprehensive multiple-choice testing program aimed at PGY-2 residents during a 12-month continuous long-block elective experience is associated with improved PGY-3 IM-ITE performance.

Effective use of multimedia presentations to maximize learning within high school science classrooms

NASA Astrophysics Data System (ADS)

Rapp, Eric

This research used an evidenced-based experimental 2 x 2 factorial design General Linear Model with Repeated Measures Analysis of Covariance (RMANCOVA). For this analysis, time served as the within-subjects factor while treatment group (i.e., static and signaling, dynamic and signaling, static without signaling, and dynamic without signaling) served as the between-subject independent variable. Three dependent variables were used to assess learner outcomes: (a) a 14 multiple-choice pre and post-test to measure knowledge retention, (b) a pre and post-test concept map to measure synthesis and structure of knowledge, and (c) four questions based on a Likert scale asking students to rank the cognitive difficulty of understanding four aspects of the animation they engaged in. A mental rotations test was used in the pretest conditions to establish a control and used as a covariate. The treatment contained a four minute and 53 second animation that served as an introductory multimedia presentation explaining the gravitational effects of the moon and sun on the earth. These interactions occur at predictable times and are responsible for creating the tidal effects experienced on Earth. There were 99 volunteer high school participants enrolled in science classes randomly assigned to one of four treatment conditions. The research was conducted to determine how motion and the principle of signaling, established in The Cognitive Theory of Multimedia Learning affected precollege learners. The experiment controlled for modality, segmenting, temporal contiguity, redundancy, and navigational control. Results of the RMANCOVA indicated statistical significance for the within subjects effect: over time for all participants, with time and knowledge retention measured from the multiple-choice results, and in the category quality of concepts represented in the concept map analysis. However, there were no significant differences in the between groups analysis for knowledge retention based on the multiple-choice assessment, or among groups over time in the concept map variables number of concepts, levels, and quality of concepts. Additionally, when measuring cognitive difficulty when learning from the animations, no significant differences were measured.

Measuring University students' understanding of the greenhouse effect - a comparison of multiple-choice, short answer and concept sketch assessment tools with respect to students' mental models

NASA Astrophysics Data System (ADS)

Gold, A. U.; Harris, S. E.

2013-12-01

The greenhouse effect comes up in most discussions about climate and is a key concept related to climate change. Existing studies have shown that students and adults alike lack a detailed understanding of this important concept or might hold misconceptions. We studied the effectiveness of different interventions on University-level students' understanding of the greenhouse effect. Introductory level science students were tested for their pre-knowledge of the greenhouse effect using validated multiple-choice questions, short answers and concept sketches. All students participated in a common lesson about the greenhouse effect and were then randomly assigned to one of two lab groups. One group explored an existing simulation about the greenhouse effect (PhET-lesson) and the other group worked with absorption spectra of different greenhouse gases (Data-lesson) to deepen the understanding of the greenhouse effect. All students completed the same assessment including multiple choice, short answers and concept sketches after participation in their lab lesson. 164 students completed all the assessments, 76 completed the PhET lesson and 77 completed the data lesson. 11 students missed the contrasting lesson. In this presentation we show the comparison between the multiple-choice questions, short answer questions and the concept sketches of students. We explore how well each of these assessment types represents student's knowledge. We also identify items that are indicators of the level of understanding of the greenhouse effect as measured in correspondence of student answers to an expert mental model and expert responses. Preliminary data analysis shows that student who produce concept sketch drawings that come close to expert drawings also choose correct multiple-choice answers. However, correct multiple-choice answers are not necessarily an indicator that a student produces an expert-like correlating concept sketch items. Multiple-choice questions that require detailed knowledge of the greenhouse effect (e.g. direction of re-emission of infrared energy from greenhouse gas) are significantly more likely to be answered correctly by students who also produce expert-like concept sketch items than by students who don't include this aspect in their sketch and don't answer the multiple choice questions correctly. This difference is not as apparent for less technical multiple-choice questions (e.g. type of radiation emitted by Sun). Our findings explore the formation of student's mental models throughout different interventions and how well the different assessment techniques used in this study represent the student understanding of the overall concept.

Collaborative Testing: Cognitive and Interpersonal Processes Related to Enhanced Test Performance

ERIC Educational Resources Information Center

Kapitanoff, Susan H.

2009-01-01

Research has demonstrated that collaborative testing, working on tests in groups, leads to improved test scores but the mechanism by which this occurs has not been specified. Three factors were proposed as mediators: cognitive processes, interpersonal interactions and reduced test-anxiety. Thirty-three students completed a multiple-choice exam…

Retention of Prose Following Testing with Different Types of Tests.

ERIC Educational Resources Information Center

Duchastel, Philippe C.

1981-01-01

Taking a test on a passage one has just studied is known to enhance later retention. This effect was influenced by the type of initial test used. It was evident in the case of the initial short-answer test, but not in the case of multiple choice and free recall tests. (Author/RD)

A random approach of test macro generation for early detection of hotspots

NASA Astrophysics Data System (ADS)

Lee, Jong-hyun; Kim, Chin; Kang, Minsoo; Hwang, Sungwook; Yang, Jae-seok; Harb, Mohammed; Al-Imam, Mohamed; Madkour, Kareem; ElManhawy, Wael; Kwan, Joe

2016-03-01

Multiple-Patterning Technology (MPT) is still the preferred choice over EUV for the advanced technology nodes, starting the 20nm node. Down the way to 7nm and 5nm nodes, Self-Aligned Multiple Patterning (SAMP) appears to be one of the effective multiple patterning techniques in terms of achieving small pitch of printed lines on wafer, yet its yield is in question. Predicting and enhancing the yield in the early stages of technology development are some of the main objectives for creating test macros on test masks. While conventional yield ramp techniques for a new technology node have relied on using designs from previous technology nodes as a starting point to identify patterns for Design of Experiment (DoE) creation, these techniques are challenging to apply in the case of introducing an MPT technique like SAMP that did not exist in previous nodes. This paper presents a new strategy for generating test structures based on random placement of unit patterns that can construct more meaningful bigger patterns. Specifications governing the relationships between those unit patterns can be adjusted to generate layout clips that look like realistic SAMP designs. A via chain can be constructed to connect the random DoE of SAMP structures through a routing layer to external pads for electrical measurement. These clips are decomposed according to the decomposition rules of the technology into the appropriate mandrel and cut masks. The decomposed clips can be tested through simulations, or electrically on silicon to discover hotspots. The hotspots can be used in optimizing the fabrication process and models to fix them. They can also be used as learning patterns for DFM deck development. By expanding the size of the randomly generated test structures, more hotspots can be detected. This should provide a faster way to enhance the yield of a new technology node.

Optimization of a matched-filter receiver for frequency hopping code acquisition in jamming

NASA Astrophysics Data System (ADS)

Pawlowski, P. R.; Polydoros, A.

A matched-filter receiver for frequency hopping (FH) code acquisition is optimized when either partial-band tone jamming or partial-band Gaussian noise jamming is present. The receiver is matched to a segment of the FH code sequence, sums hard per-channel decisions to form a test, and uses multiple tests to verify acquisition. The length of the matched filter and the number of verification tests are fixed. Optimization is then choosing thresholds to maximize performance based upon the receiver's degree of knowledge about the jammer ('side-information'). Four levels of side-information are considered, ranging from none to complete. The latter level results in a constant-false-alarm-rate (CFAR) design. At each level, performance sensitivity to threshold choice is analyzed. Robust thresholds are chosen to maximize performance as the jammer varies its power distribution, resulting in simple design rules which aid threshold selection. Performance results, which show that optimum distributions for the jammer power over the total FH bandwidth exist, are presented.

The science achievement of various subgroups on alternative assessment formats

NASA Astrophysics Data System (ADS)

Lawrenz, Frances; Huffman, Douglas; Welch, Wayne

2001-05-01

The purpose of this study was to examine the science achievement outcomes for different subgroups of students using different assessment formats. A nationally representative sample of approximately 3,500 ninth grade science students from 13 high schools throughout the United States completed a series of science assessments designed to measure their level of achievement on the national science education standards. All of the schools were using a curriculum designed to meet the standards. The assessments included a multiple-choice test, a written open-ended test, a hands-on lab skills test, and a hands-on full investigation. The results show that the student outcomes on the different assessment formats are more highly correlated for higher achieving students than for lower achieving students. Patterns for different cultural groups also vary by assessment format. There were no differences found for sex. The results support the notion that different assessment formats assess different competencies and that the achievement of students from different subgroups varies by assessment format.

Wood Preference of Reticulitermes virginicus (Blattodea: Rhinotermitidae) Using No-, Two-, and Four-Choice Designs and Seven Different Measures of Wood Consumption.

PubMed

Lee, T-Y; Forschler, B T

2016-04-01

Three hundred Reticulitermes virginicus (Banks) workers were exposed to three 1-cm3 wood blocks of either Quercus sp. (Red Oak), Populus sp. (Poplar), Pinus sp. (Pine), or Sequoia sp. (Redwood) placed into one of the three bioassay designs (no-, two-, and four-choice) for 21 d. Termite wood consumption was measured by wood weight loss, resistance class, and visual rating. Wood consumption rates were determined using four formulas in addition to two standardized visual rating scales (American Society for Testing and Materials [ASTM] and American Wood Protection Association [AWPA]) and a preference ranking obtained for each measure. The wood consumption formula, rating scale, and preference rankings were compared by bioassay design. The overall preference ranking of the four wood types as determined by the combination of all three designs was—1) Pine, 2) Red Oak, 3) Redwood, and 4) Poplar. Results indicate that bioassay design influenced both wood consumption and preference rankings. A no-choice design can determine aversion; a four-choice design the most preferred wood; and a two-choice design can illuminate the fine details of comparative preference. The different formulas employed for calculation of consumption rate influenced preference ranking in the no- and four-choice designs but not the two-choice design.

The effectiveness of the 1996 Wood Magic Science Fair as an experiential field trip

NASA Astrophysics Data System (ADS)

Pearson, Robert Wayne

In response to the need for evaluating the effectiveness of the 1996 Wood Magic Science Fair (WMSF) at Mississippi State University's Forest and Wildlife Research Center/Forest Products Laboratory, two tests were developed. One test was designed for third-grade students and one for fourth-grade students who would attend the WMSF. Both tests have multiple choice answers and contained thirteen questions each. Five of the questions addressed general issues of the forest products industry that would be mentioned in an opening video but not stressed in the oral presentations of the WMSF. The students would have no active involvement in these presentations. These general issue questions represent passive involvement teaching. The eight remaining questions addressed specific information that would be stressed in the WMSF oral presentations and would allow students to actively participate in the presentations. The participation of the students in these presentations represents participatory teaching. The tests were given to the students (third and fourth grades) both before they attended the fair as a pre-test and after their attendance as a post-test with the only difference being that the multiple choice answers were arranged in a different order. Classroom teachers administered the tests to the students. The test results were evaluated for each class individually. Each question was evaluated and the results recorded in a table. The general information questions were grouped together as were the specific information questions. This grouping allowed a comparison to be made between passive and participatory learning. The results from the tests will help those making the science fair presentations evaluate their materials and methods. Statistical analysis of the results would indicate which questions had a significant change in the number of correct answers between the pre-test and post-test. This information will help the Fair planners and presenters in developing future programs.

Comparing narrative and multiple-choice formats in online communication skill assessment.

PubMed

Kim, Sara; Spielberg, Freya; Mauksch, Larry; Farber, Stu; Duong, Cuong; Fitch, Wes; Greer, Tom

2009-06-01

We compared multiple-choice and open-ended responses collected from a web-based tool designated 'Case for Change', which had been developed for assessing and teaching medical students in the skills involved in integrating sexual risk assessment and behaviour change discussions into patient-centred primary care visits. A total of 111 Year 3 students completed the web-based tool. A series of videos from one patient encounter illustrated how a clinician uses patient-centred communication and health behaviour change skills while caring for a patient presenting with a urinary tract infection. Each video clip was followed by a request for students to respond in two ways to the question: 'What would you do next?' Firstly, students typed their statements of what they would say to the patient. Secondly, students selected from a multiple-choice list the statements that most closely resembled their free text entries. These two modes of students' answers were analysed and compared. When articulating what they would say to the patient in a narrative format, students frequently used doctor-centred approaches that focused on premature diagnostic questioning or neglected to elicit patient perspectives. Despite the instruction to select a matching statement from the multiple-choice list, students tended to choose the most exemplary patient-centred statement, which was contrary to the doctor-centred approaches reflected in their narrative responses. Open-ended questions facilitate in-depth understanding of students' educational needs, although the scoring of narrative responses is time-consuming. Multiple-choice questions allow efficient scoring and individualised feedback associated with question items but do not fully elicit students' thought processes.

Exploring undergraduates' understanding of photosynthesis using diagnostic question clusters.

PubMed

Parker, Joyce M; Anderson, Charles W; Heidemann, Merle; Merrill, John; Merritt, Brett; Richmond, Gail; Urban-Lurain, Mark

2012-01-01

We present a diagnostic question cluster (DQC) that assesses undergraduates' thinking about photosynthesis. This assessment tool is not designed to identify individual misconceptions. Rather, it is focused on students' abilities to apply basic concepts about photosynthesis by reasoning with a coordinated set of practices based on a few scientific principles: conservation of matter, conservation of energy, and the hierarchical nature of biological systems. Data on students' responses to the cluster items and uses of some of the questions in multiple-choice, multiple-true/false, and essay formats are compared. A cross-over study indicates that the multiple-true/false format shows promise as a machine-gradable format that identifies students who have a mixture of accurate and inaccurate ideas. In addition, interviews with students about their choices on three multiple-choice questions reveal the fragility of students' understanding. Collectively, the data show that many undergraduates lack both a basic understanding of the role of photosynthesis in plant metabolism and the ability to reason with scientific principles when learning new content. Implications for instruction are discussed.

Exploring Undergraduates' Understanding of Photosynthesis Using Diagnostic Question Clusters

PubMed Central

Parker, Joyce M.; Anderson, Charles W.; Heidemann, Merle; Merrill, John; Merritt, Brett; Richmond, Gail; Urban-Lurain, Mark

2012-01-01

We present a diagnostic question cluster (DQC) that assesses undergraduates' thinking about photosynthesis. This assessment tool is not designed to identify individual misconceptions. Rather, it is focused on students' abilities to apply basic concepts about photosynthesis by reasoning with a coordinated set of practices based on a few scientific principles: conservation of matter, conservation of energy, and the hierarchical nature of biological systems. Data on students' responses to the cluster items and uses of some of the questions in multiple-choice, multiple-true/false, and essay formats are compared. A cross-over study indicates that the multiple-true/false format shows promise as a machine-gradable format that identifies students who have a mixture of accurate and inaccurate ideas. In addition, interviews with students about their choices on three multiple-choice questions reveal the fragility of students' understanding. Collectively, the data show that many undergraduates lack both a basic understanding of the role of photosynthesis in plant metabolism and the ability to reason with scientific principles when learning new content. Implications for instruction are discussed. PMID:22383617

Equating in Small-Scale Language Testing Programs

ERIC Educational Resources Information Center

LaFlair, Geoffrey T.; Isbell, Daniel; May, L. D. Nicolas; Gutierrez Arvizu, Maria Nelly; Jamieson, Joan

2017-01-01

Language programs need multiple test forms for secure administrations and effective placement decisions, but can they have confidence that scores on alternate test forms have the same meaning? In large-scale testing programs, various equating methods are available to ensure the comparability of forms. The choice of equating method is informed by…

Handbook for Driving Knowledge Testing.

ERIC Educational Resources Information Center

Pollock, William T.; McDole, Thomas L.

Materials intended for driving knowledge test development for use by operational licensing and education agencies are presented. A pool of 1,313 multiple choice test items is included, consisting of sets of specially developed and tested items covering principles of safe driving, legal regulations, and traffic control device knowledge pertinent to…

Using Tests as Learning Opportunities.

ERIC Educational Resources Information Center

Foos, Paul W.; Fisher, Ronald P.

1988-01-01

A study involving 105 undergraduates assessed the value of testing as a means of increasing, rather than simply monitoring, learning. Results indicate that fill-in-the-blank and items requiring student inferences were more effective, respectively, than multiple-choice tests and verbatim items in furthering student learning. (TJH)

Investigating High School Students' Understanding of Chemical Equilibrium Concepts

ERIC Educational Resources Information Center

Karpudewan, Mageswary; Treagust, David F.; Mocerino, Mauro; Won, Mihye; Chandrasegaran, A. L.

2015-01-01

This study investigated the year 12 students' (N = 56) understanding of chemical equilibrium concepts after instruction using two conceptual tests, the "Chemical Equilibrium Conceptual Test 1" ("CECT-1") consisting of nine two-tier multiple-choice items and the "Chemical Equilibrium Conceptual Test 2"…

Decision making under internal uncertainty: the case of multiple-choice tests with different scoring rules.

PubMed

Bereby-Meyer, Yoella; Meyer, Joachim; Budescu, David V

2003-02-01

This paper assesses framing effects on decision making with internal uncertainty, i.e., partial knowledge, by focusing on examinees' behavior in multiple-choice (MC) tests with different scoring rules. In two experiments participants answered a general-knowledge MC test that consisted of 34 solvable and 6 unsolvable items. Experiment 1 studied two scoring rules involving Positive (only gains) and Negative (only losses) scores. Although answering all items was the dominating strategy for both rules, the results revealed a greater tendency to answer under the Negative scoring rule. These results are in line with the predictions derived from Prospect Theory (PT) [Econometrica 47 (1979) 263]. The second experiment studied two scoring rules, which allowed respondents to exhibit partial knowledge. Under the Inclusion-scoring rule the respondents mark all answers that could be correct, and under the Exclusion-scoring rule they exclude all answers that might be incorrect. As predicted by PT, respondents took more risks under the Inclusion rule than under the Exclusion rule. The results illustrate that the basic process that underlies choice behavior under internal uncertainty and especially the effect of framing is similar to the process of choice under external uncertainty and can be described quite accurately by PT. Copyright 2002 Elsevier Science B.V.

Assessing Understanding of the Learning Cycle: The ULC

NASA Astrophysics Data System (ADS)

Marek, Edmund A.; Maier, Steven J.; McCann, Florence

2008-08-01

An 18-item, multiple choice, 2-tiered instrument designed to measure understanding of the learning cycle (ULC) was developed and field-tested from the learning cycle test (LCT) of Odom and Settlage ( Journal of Science Teacher Education, 7, 123 142, 1996). All question sets of the LCT were modified to some degree and 5 new sets were added, resulting in the ULC. The ULC measures (a) understandings and misunderstandings of the learning cycle, (b) the learning cycle’s association with Piaget’s ( Biology and knowledge theory: An essay on the relations between organic regulations and cognitive processes, 1975) theory of mental functioning, and (c) applications of the learning cycle. The resulting ULC instrument was evaluated for internal consistency with Cronbach’s alpha, yielding a coefficient of .791.

Effects of Repeated Testing on Short- and Long-Term Memory Performance across Different Test Formats

ERIC Educational Resources Information Center

Stenlund, Tova; Sundström, Anna; Jonsson, Bert

2016-01-01

This study examined whether practice testing with short-answer (SA) items benefits learning over time compared to practice testing with multiple-choice (MC) items, and rereading the material. More specifically, the aim was to test the hypotheses of "retrieval effort" and "transfer appropriate processing" by comparing retention…

Pursuing the Qualities of a "Good" Test

ERIC Educational Resources Information Center

Coniam, David

2014-01-01

This article examines the issue of the quality of teacher-produced tests, limiting itself in the current context to objective, multiple-choice tests. The article investigates a short, two-part 20-item English language test. After a brief overview of the key test qualities of reliability and validity, the article examines the two subtests in terms…

A Multiple-Choice Mushroom: Schools, Colleges Rely More than Ever on Standardized Tests.

ERIC Educational Resources Information Center

Hawkins, B. Denise

1995-01-01

This discussion of college entrance examinations reviews differences between the Scholastic Assessment Test (SAT) and the American College Test. It then focuses on the SAT, discussing numbers of students taking the tests, changes in test construction to recognize contributions of women and minorities, involvement of African Americans in…

Middle School Students' Responses to Two-Tier Tasks

ERIC Educational Resources Information Center

Haja, Shajahan; Clarke, David

2011-01-01

The structure of two-tier testing is such that the first tier consists of a multiple-choice question and the second tier requires justifications for choices of answers made in the first tier. This study aims to evaluate two-tier tasks in "proportion" in terms of students' capacity to write and select justifications and to examine the effect of…

Student Assessment System. Domain Referenced Tests. Transportation/Automotive Mechanics. Volume II: Theory. Georgia Vocational Education Program Articulation.

ERIC Educational Resources Information Center

Watkins, James F., Comp.

These written domain referenced tests (DRTs) for the area of transportation/automotive mechanics test cognitive abilities or knowledge of theory. Introductory materials describe domain referenced testing and test development. Each multiple choice test includes a domain statement, describing the behavior and content of the domain, and a test item…

Infiltration performance of engineered surfaces commonly used for distributed stormwater management

NASA Astrophysics Data System (ADS)

Valinski, Nicholas A.

Engineered porous media are commonly used in low impact development (LID) structures to mitigate excess stormwater in urban environments. Differences in infiltrability of these LID systems arise from the wide variety of materials used to create porous surfaces and subsequent maintenance, debris loading, and physical damage. In this study, infiltration capacity of six common materials was tested by multiple replicate experiments with automated mini-disk infiltrometers. The tested materials included porous asphalt, porous concrete, porous brick pavers, flexible porous pavement, engineered soils, and native soils. Porous asphalt, large porous brick pavers, and curb cutout rain gardens showed the greatest infiltration rates. Most engineered porous pavements and soils performed better than the native silt loam soils. Infiltration performance was found to be related more to site design and environmental factors than material choice. Sediment trap zones in both pavements and engineered soil rain gardens were found to be beneficial to the whole site performance. Winter chloride application had a large negative impact on poured in place concrete, making it a poor choice for heavily salted areas.

Infiltration performance of engineered surfaces commonly used for distributed stormwater management.

PubMed

Valinski, N A; Chandler, D G

2015-09-01

Engineered porous media are commonly used in low impact development (LID) structures to mitigate excess stormwater in urban environments. Differences in infiltrability of these LID systems arise from the wide variety of materials used to create porous surfaces and subsequent maintenance, debris loading, and physical damage. In this study, the infiltration capacity of six common materials was tested by multiple replicate experiments with automated mini-disk infiltrometers. The tested materials included porous asphalt, porous concrete, porous brick pavers, flexible porous pavement, engineered soils, and native soils. Porous asphalt, large porous brick pavers, and curb cutout rain gardens showed the greatest infiltration rates. Most engineered porous pavements and soils performed better than the native silt loam soils. Infiltration performance was found to be related more to site design and environmental factors than material choice. Sediment trap zones in both pavements and engineered soil rain gardens were found to be beneficial to the whole site performance. Winter chloride application had a large negative impact on poured in place concrete, making it a poor choice for heavily salted areas. Copyright © 2015 Elsevier Ltd. All rights reserved.

Fifth Graders' Learning About Simple Machines Through Engineering Design-Based Instruction Using LEGO™ Materials

NASA Astrophysics Data System (ADS)

Marulcu, Ismail; Barnett, Mike

2013-10-01

This study is part of a 5-year National Science Foundation-funded project, Transforming Elementary Science Learning Through LEGO™ Engineering Design. In this study, we report on the successes and challenges of implementing an engineering design-based and LEGO™-oriented unit in an urban classroom setting and we focus on the impact of the unit on students' content understanding of simple machines. The LEGO™ engineering-based simple machines module, which was developed for fifth graders by our research team, was implemented in an urban school in a large city in the Northeastern region of the USA. Thirty-three fifth grade students participated in the study, and they showed significant growth in content understanding. We measured students' content knowledge by using identical paper tests and semistructured interviews before and after instruction. Our paired t test analysis results showed that students significantly improved their test and interview scores (t = -3.62, p < 0.001 for multiple-choice items and t = -9.06, p < 0.000 for the open-ended items in the test and t = -12.11, p < 0.000 for the items in interviews). We also identified several alternative conceptions that are held by students on simple machines.

Harnessing Technology to Improve Formative Assessment of Student Conceptions in STEM: Forging a National Network

ERIC Educational Resources Information Center

Haudek, Kevin C.; Kaplan, Jennifer J.; Knight, Jennifer; Long, Tammy; Merrill, John; Munn, Alan; Nehm, Ross; Smith, Michelle; Urban-Lurain, Mark

2011-01-01

Concept inventories, consisting of multiple-choice questions designed around common student misconceptions, are designed to reveal student thinking. However, students often have complex, heterogeneous ideas about scientific concepts. Constructed-response assessments, in which students must create their own answer, may better reveal students'…

Adaptive x-ray threat detection using sequential hypotheses testing with fan-beam experimental data (Conference Presentation)

NASA Astrophysics Data System (ADS)

Thamvichai, Ratchaneekorn; Huang, Liang-Chih; Ashok, Amit; Gong, Qian; Coccarelli, David; Greenberg, Joel A.; Gehm, Michael E.; Neifeld, Mark A.

2017-05-01

We employ an adaptive measurement system, based on sequential hypotheses testing (SHT) framework, for detecting material-based threats using experimental data acquired on an X-ray experimental testbed system. This testbed employs 45-degree fan-beam geometry and 15 views over a 180-degree span to generate energy sensitive X-ray projection data. Using this testbed system, we acquire multiple view projection data for 200 bags. We consider an adaptive measurement design where the X-ray projection measurements are acquired in a sequential manner and the adaptation occurs through the choice of the optimal "next" source/view system parameter. Our analysis of such an adaptive measurement design using the experimental data demonstrates a 3x-7x reduction in the probability of error relative to a static measurement design. Here the static measurement design refers to the operational system baseline that corresponds to a sequential measurement using all the available sources/views. We also show that by using adaptive measurements it is possible to reduce the number of sources/views by nearly 50% compared a system that relies on static measurements.

Evaluating the Psychometric Characteristics of Generated Multiple-Choice Test Items

ERIC Educational Resources Information Center

Gierl, Mark J.; Lai, Hollis; Pugh, Debra; Touchie, Claire; Boulais, André-Philippe; De Champlain, André

2016-01-01

Item development is a time- and resource-intensive process. Automatic item generation integrates cognitive modeling with computer technology to systematically generate test items. To date, however, items generated using cognitive modeling procedures have received limited use in operational testing situations. As a result, the psychometric…

Item Analysis in Introductory Economics Testing.

ERIC Educational Resources Information Center

Tinari, Frank D.

1979-01-01

Computerized analysis of multiple choice test items is explained. Examples of item analysis applications in the introductory economics course are discussed with respect to three objectives: to evaluate learning; to improve test items; and to help improve classroom instruction. Problems, costs and benefits of the procedures are identified. (JMD)

Test-Wiseness Cues in the Options of Mathematics Items.

ERIC Educational Resources Information Center

Kuntz, Patricia

The quality of mathematics multiple choice items and their susceptibility to test wiseness were examined. Test wiseness was defined as "a subject's capacity to utilize the characteristics and formats of the test and/or test taking situation to receive a high score." The study used results of the Graduate Record Examinations Aptitude Test (GRE) and…

What makes a good home-based nocturnal seizure detector? A value sensitive design.

PubMed

van Andel, Judith; Leijten, Frans; van Delden, Hans; van Thiel, Ghislaine

2015-01-01

A device for the in-home detection of nocturnal seizures is currently being developed in the Netherlands, to improve care for patients with severe epilepsy. It is recognized that the design of medical technology is not value neutral: perspectives of users and developers are influential in design, and design choices influence these perspectives. However, during development processes, these influences are generally ignored and value-related choices remain implicit and poorly argued for. In the development process of the seizure detector we aimed to take values of all stakeholders into consideration. Therefore, we performed a parallel ethics study, using "value sensitive design." Analysis of stakeholder communication (in meetings and e-mail messages) identified five important values, namely, health, trust, autonomy, accessibility, and reliability. Stakeholders were then asked to give feedback on the choice of these values and how they should be interpreted. In a next step, the values were related to design choices relevant for the device, and then the consequences (risks and benefits) of these choices were investigated. Currently the process of design and testing of the device is still ongoing. The device will be validated in a trial in which the identified consequences of design choices are measured as secondary endpoints. Value sensitive design methodology is feasible for the development of new medical technology and can help designers substantiate the choices in their design.

Sequential effects in pigeon delayed matching-to-sample performance.

PubMed

Roitblat, H L; Scopatz, R A

1983-04-01

Pigeons were tested in a three-alternative delayed matching-to-sample task in which second-choices were permitted following first-choice errors. Sequences of responses both within and between trials were examined in three experiments. The first experiment demonstrates that the sample information contained in first-choice errors is not sufficient to account for the observed pattern of second choices. This result implies that second-choices following first-choice errors are based on a second examination of the contents of working memory. Proactive interference was found in the second experiment in the form of a dependency, beyond that expected on the basis of trial independent response bias, of first-choices from one trial on the first-choice emitted on the previous trial. Samples from the previous trial were not found to exert a significant influence on later trials. The magnitude of the intertrial association (Experiment 3) did not depend on the duration of the intertrial interval. In contrast, longer intertrial intervals and longer sample durations did facilitate choice accuracy, by strengthening the association between current samples and choices. These results are incompatible with a trace-decay and competition model; they suggest strongly that multiple influences act simultaneously and independently to control delayed matching-to-sample responding. These multiple influences include memory for the choice occurring on the previous trial, memory for the sample, and general effects of trial spacing.

Pairwise Multiple Comparisons in Single Group Repeated Measures Analysis.

ERIC Educational Resources Information Center

Barcikowski, Robert S.; Elliott, Ronald S.

Research was conducted to provide educational researchers with a choice of pairwise multiple comparison procedures (P-MCPs) to use with single group repeated measures designs. The following were studied through two Monte Carlo (MC) simulations: (1) The T procedure of J. W. Tukey (1953); (2) a modification of Tukey's T (G. Keppel, 1973); (3) the…

What is an Objective Structured Practical Examination in Anatomy?

ERIC Educational Resources Information Center

Yaqinuddin, Ahmed; Zafar, Muhammad; Ikram, Muhammad Faisal; Ganguly, Paul

2013-01-01

Assessing teaching-learning outcomes in anatomical knowledge is a complex task that requires the evaluation of multiple domains: theoretical, practical, and clinical knowledge. In general, theoretical knowledge is tested by a written examination system constituted by multiple choice questions (MCQs) and/or short answer questions (SAQ). The…

Effects of a history of differential reinforcement on preference for choice.

PubMed

Karsina, Allen; Thompson, Rachel H; Rodriguez, Nicole M

2011-03-01

The effects of a history of differential reinforcement for selecting a free-choice versus a restricted-choice stimulus arrangement on the subsequent responding of 7 undergraduates in a computer-based game of chance were examined using a concurrent-chains arrangement and a multiple-baseline-across-participants design. In the free-choice arrangement, participants selected three numbers, in any order, from an array of eight numbers presented on the computer screen. In the restricted-choice arrangement, participants selected the order of three numbers preselected from the array of eight by a computer program. In initial sessions, all participants demonstrated no consistent preference or preference for restricted choice. Differential reinforcement of free-choice selections resulted in increased preference for free choice immediately and in subsequent sessions in the absence of programmed differential outcomes. For 5 participants, changes in preference for choice were both robust and lasting, suggesting that a history of differential reinforcement for choice may affect preference for choice.

Effects of a History of Differential Reinforcement on Preference for Choice

PubMed Central

Karsina, Allen; Thompson, Rachel H; Rodriguez, Nicole M

2011-01-01

The effects of a history of differential reinforcement for selecting a free-choice versus a restricted-choice stimulus arrangement on the subsequent responding of 7 undergraduates in a computer-based game of chance were examined using a concurrent-chains arrangement and a multiple-baseline-across-participants design. In the free-choice arrangement, participants selected three numbers, in any order, from an array of eight numbers presented on the computer screen. In the restricted-choice arrangement, participants selected the order of three numbers preselected from the array of eight by a computer program. In initial sessions, all participants demonstrated no consistent preference or preference for restricted choice. Differential reinforcement of free-choice selections resulted in increased preference for free choice immediately and in subsequent sessions in the absence of programmed differential outcomes. For 5 participants, changes in preference for choice were both robust and lasting, suggesting that a history of differential reinforcement for choice may affect preference for choice. PMID:21541125

STRATOP: A Model for Designing Effective Product and Communication Strategies. Paper No. 470.

ERIC Educational Resources Information Center

Pessemier, Edgar A.

The STRATOP algorithm was developed to help planners and proponents find and test effectively designed choice objects and communication strategies. Choice objects can range from complex social, scientific, military, or educational alternatives to simple economic alternatives between assortments of branded convenience goods. Two classes of measured…

Testing to the Top: Everything But the Kitchen Sink?

ERIC Educational Resources Information Center

Dietel, Ron

2011-01-01

Two tests intended to measure student achievement of the Common Core State Standards will face intense scrutiny, but the test makers say they will include performance assessments and other items that are not multiple-choice questions. Incorporating performance items on this tests will bring up issues over scoring, costs, and validity.

ACER Chemistry Test Item Collection. ACER Chemtic Year 12.

ERIC Educational Resources Information Center

Australian Council for Educational Research, Hawthorn.

The chemistry test item banks contains 225 multiple-choice questions suitable for diagnostic and achievement testing; a three-page teacher's guide; answer key with item facilities; an answer sheet; and a 45-item sample achievement test. Although written for the new grade 12 chemistry course in Victoria, Australia, the items are widely applicable.…

Are Learning Disabled Students "Test-Wise?": An Inquiry into Reading Comprehension Test Items.

ERIC Educational Resources Information Center

Scruggs, Thomas E.; Lifson, Steve

The ability to correctly answer reading comprehension test items, without having read the accompanying reading passage, was compared for third grade learning disabled students and their peers from a regular classroom. In the first experiment, fourteen multiple choice items were selected from the Stanford Achievement Test. No reading passages were…

Passage Independence within Standardized Reading Comprehension Tests

ERIC Educational Resources Information Center

Roy-Charland, Annie; Colangelo, Gabrielle; Foglia, Victoria; Reguigui, Leïla

2017-01-01

In tests used to measure reading comprehension, validity is important in obtaining accurate results. Unfortunately, studies have shown that people can correctly answer some questions of these tests without reading the related passage. These findings bring forth the need to address whether this phenomenon is observed in multiple-choice only tests…

Food Service Supervisor. Dietetic Support Personnel Achievement Test.

ERIC Educational Resources Information Center

Oklahoma State Dept. of Vocational and Technical Education, Stillwater.

This guide contains a series of multiple-choice items and guidelines to assist instructors in composing criterion-referenced tests for use in the food service supervisor component of Oklahoma's Dietetic Support Personnel training program. Test items addressing each of the following occupational duty areas are provided: human relations; nutrient…

Food Production Worker. Dietetic Support Personnel Achievement Test.

ERIC Educational Resources Information Center

Oklahoma State Dept. of Vocational and Technical Education, Stillwater.

This guide contains a series of multiple-choice items and guidelines to assist instructors in composing criterion-referenced tests for use in the food production worker component of Oklahoma's Dietetic Support Personnel training program. Test items addressing each of the following occupational duty areas are provided: human relations; hygiene and…

Food Service Worker. Dietetic Support Personnel Achievement Test.

ERIC Educational Resources Information Center

Oklahoma State Dept. of Vocational and Technical Education, Stillwater.

This guide contains a series of multiple-choice items and guidelines to assist instructors in composing criterion-referenced tests for use in the food service worker component of Oklahoma's Dietetic Support Personnel training program. Test items addressing each of the following occupational duty areas are provided: human relations; personal…

Tilesetting Testbook.

ERIC Educational Resources Information Center

Strazicich, Mirko, Ed.

This testbook contains objective tests for each topic in the "Tilesetting Workbook" (see note) for use in the classroom portions of the union four-year apprentice tilesetting program in California. Tests are all multiple choice and are usually two pages in length. Tests are provided for each of the 44 topics covered in the five units of…

The Performance of IRT Model Selection Methods with Mixed-Format Tests

ERIC Educational Resources Information Center

Whittaker, Tiffany A.; Chang, Wanchen; Dodd, Barbara G.

2012-01-01

When tests consist of multiple-choice and constructed-response items, researchers are confronted with the question of which item response theory (IRT) model combination will appropriately represent the data collected from these mixed-format tests. This simulation study examined the performance of six model selection criteria, including the…

The influence of project-based learning on the student conception about kinematics and critical thinking skills

NASA Astrophysics Data System (ADS)

Handhika, J.; Cari, C.; Sunarno, W.; Suparmi, A.; Kurniadi, E.

2018-05-01

This research revealed the influence of project-based learning (PjBL) to increasing the level of the conception. The research method used the pre-experimental design with one group pre-test post-test. PjBL applied to students of physics education program of IKIP PGRI Madiun (23 Students). The test used to determine the level of conception is multiple choice tests and index of certainty. Activities on PjBL described. Obtained that the PjBL model can increase the level of conception and Critical thinking skills with the average normalized gain 0.49 and 0.57 (Medium category). It can be concluded that the PjBL could improve the level of conception and critical thinking ability of the students. Implementation of each model phase following learning objectives and needs analysis is the key to improve both.

Cardiopulmonary resuscitation knowledge and skills of registered nurses in Botswana.

PubMed

Rajeswaran, Lakshmi; Ehlers, Valerie J

2014-01-01

In Botswana nurses provide most health care in the primary, secondary and tertiary level clinics and hospitals. Trauma and medical emergencies are on the increase, and nurses should have cardiopulmonary resuscitation (CPR) knowledge and skills in order to be able to implement effective interventions in cardiac arrest situations. The objective of this descriptive study was to assess registered nurses’ CPR knowledge and skills. A pre-test, intervention and re-test time-series research design was adopted, and data were collected from 102 nurses from the 2 referral hospitals in Botswana. A multiple-choice questionnaire and checklist were used to collect data. All nurses failed the pre-test. Their knowledge and skills improved after training, but deteriorated over the three months until the post-test was conducted. The significantly low levels of registered nurses’ CPR skills in Botswana should be addressed by instituting country-wide CPR training and regular refresher courses

The Positive and Negative Effects of Science Concept Tests on Student Conceptual Understanding

ERIC Educational Resources Information Center

Chang, Chun-Yen; Yeh, Ting-Kuang; Barufaldi, James P.

2010-01-01

This study explored the phenomenon of testing effect during science concept assessments, including the mechanism behind it and its impact upon a learner's conceptual understanding. The participants consisted of 208 high school students, in either the 11th or 12th grade. Three types of tests (traditional multiple-choice test, correct concept test,…

Teaching in the Time of Testing: What Have You Lost?

ERIC Educational Resources Information Center

McCracken, Nancy Mellin; McCracken, Hugh Thomas

2001-01-01

Asks several teachers what they have lost from their teaching or their classroom since the growth in mandated, standardized testing. Considers the ill effects of mandated testing, and names some educational essentials at risk of being lost while testing rules. Discusses what is lost in high-stakes multiple-choice testing of new teachers. (SG)

Using Multigroup Confirmatory Factor Analysis to Test Measurement Invariance in Raters: A Clinical Skills Examination Application

ERIC Educational Resources Information Center

Kahraman, Nilufer; Brown, Crystal B.

2015-01-01

Psychometric models based on structural equation modeling framework are commonly used in many multiple-choice test settings to assess measurement invariance of test items across examinee subpopulations. The premise of the current article is that they may also be useful in the context of performance assessment tests to test measurement invariance…

The Effect of Device When Using Smartphones and Computers to Answer Multiple-Choice and Open-Response Questions in Distance Education

ERIC Educational Resources Information Center

Wilson, Thomas Royce

2017-01-01

Traditionally in higher education, online courses have been designed for computer users. However, the advent of mobile learning (m-learning) and the proliferation of smartphones have created two challenges for online students and instructional designers. First, instruction designed for a larger computer screen often loses its effectiveness when…

Decision making and preferences for acoustic signals in choice situations by female crickets.

PubMed

Gabel, Eileen; Kuntze, Janine; Hennig, R Matthias

2015-08-01

Multiple attributes usually have to be assessed when choosing a mate. Efficient choice of the best mate is complicated if the available cues are not positively correlated, as is often the case during acoustic communication. Because of varying distances of signalers, a female may be confronted with signals of diverse quality at different intensities. Here, we examined how available cues are weighted for a decision by female crickets. Two songs with different temporal patterns and/or sound intensities were presented in a choice paradigm and compared with female responses from a no-choice test. When both patterns were presented at equal intensity, preference functions became wider in choice situations compared with a no-choice paradigm. When the stimuli in two-choice tests were presented at different intensities, this effect was counteracted as preference functions became narrower compared with choice tests using stimuli of equal intensity. The weighting of intensity differences depended on pattern quality and was therefore non-linear. A simple computational model based on pattern and intensity cues reliably predicted female decisions. A comparison of processing schemes suggested that the computations for pattern recognition and directionality are performed in a network with parallel topology. However, the computational flow of information corresponded to serial processing. © 2015. Published by The Company of Biologists Ltd.

Pharmacotherapies for decreasing maladaptive choice in drug addiction: Targeting the behavior and the drug.

PubMed

Perkins, Frank N; Freeman, Kevin B

2018-01-01

Drug addiction can be conceptualized as a disorder of maladaptive decision making in which drugs are chosen at the expense of pro-social, nondrug alternatives. The study of decision making in drug addiction has focused largely on the role of impulsivity as a facilitator of addiction, in particular the tendency for drug abusers to choose small, immediate gains over larger but delayed outcomes (i.e., delay discounting). A parallel line of work, also focused on decision making in drug addiction, has focused on identifying the determinants underlying the choice to take drugs over nondrug alternatives (i.e., drug vs. nondrug choice). Both tracks of research have been valuable tools in the development of pharmacotherapies for treating maladaptive decision making in drug addiction, and a number of common drugs have been studied in both designs. However, we have observed that there is little uniformity in the administration regimens of potential treatments between the designs, which hinders congruence in the development of single treatment strategies to reduce both impulsive behavior and drug choice. The current review provides an overview of the drugs that have been tested in both delay-discounting and drug-choice designs, and focuses on drugs that reduced the maladaptive choice in both designs. Suggestions to enhance congruence between the findings in future studies are provided. Finally, we propose the use of a hybridized, experimental approach that may enable researchers to test the effectiveness of therapeutics at decreasing impulsive and drug choice in a single design. Published by Elsevier Inc.

First Results from the Test Of Astronomy STandards (TOAST) Assessment Instrument

NASA Astrophysics Data System (ADS)

Slater, Stephanie

2009-01-01

Considerable effort in the astronomy education research over the past several years has focused on developing assessment tools in the form of multiple-choice conceptual diagnostics and content knowledge surveys. This has been critically important in advancing astronomy as a sub-discipline of physics education research, allowing researchers to establish the initial knowledge state of students as well as to attempt to measure some of the impacts of innovative instructional interventions. Before now, few of the existing instruments were constructed upon a solid list of clearly articulated and widely agreed upon learning objectives. Moving beyond the 10-year old Astronomy Diagnostics Test, we have developed and validated a new assessment instrument that is tightly aligned to the consensus learning goals stated by the American Astronomical Society - Chair's Conference on ASTRO 101, the American Association of the Advancement of Science's Project 2061 Benchmarks, and the National Research Council's National Science Education Standards. Researchers from the Cognition in Astronomy, Physics and Earth sciences Research (CAPER) Team at the University of Wyoming's Science and Math Teaching Center (UWYO SMTC) designed a criterion-referenced assessment tool, called the Test Of Astronomy STandards (TOAST). Through iterative development, this multiple-choice instrument has a high degree of reliability and validity for instructors and researchers needing information on students’ initial knowledge state at the beginning of a course and can be used, in aggregate, to help measure the impact of course-length duration instructional strategies for undergraduate science survey courses with learning goals tightly aligned to the consensus goals of the astronomy education community.

Pattern recognition as a concept for multiple-choice questions in a national licensing exam.

PubMed

Freiwald, Tilo; Salimi, Madjid; Khaljani, Ehsan; Harendza, Sigrid

2014-11-14

Multiple-choice questions (MCQ) are still widely used in high stakes medical exams. We wanted to examine whether and to what extent a national licensing exam uses the concept of pattern recognition to test applied clinical knowledge. We categorized all 4,134 German National medical licensing exam questions between October 2006 and October 2012 by discipline, year, and type. We analyzed questions from the four largest disciplines: internal medicine (n = 931), neurology (n = 305), pediatrics (n = 281), and surgery (n = 233), with respect to the following question types: knowledge questions (KQ), pattern recognition questions (PRQ), inverse PRQ (IPRQ), and pseudo PRQ (PPRQ). A total 51.1% of all questions were of a higher taxonomical order (PRQ and IPRQ) with a significant decrease in the percentage of these questions (p <0.001) from 2006 (61.5%) to 2012 (41.6%). The proportion of PRQs and IPRQs was significantly lower (p <0.001) in internal medicine and surgery, compared to neurology and pediatrics. PRQs were mostly used in questions about diagnoses (71.7%). A significantly higher (p <0.05) percentage of PR/therapy questions was found for internal medicine compared with neurology and pediatrics. The concept of pattern recognition is used with different priorities and to various extents by the different disciplines in a high stakes exam to test applied clinical knowledge. Being aware of this concept may aid in the design and balance of MCQs in an exam with respect to testing clinical reasoning as a desired skill at the threshold of postgraduate medical education.

The Effect of English Language on Multiple Choice Question Scores of Thai Medical Students.

PubMed

Phisalprapa, Pochamana; Muangkaew, Wayuda; Assanasen, Jintana; Kunavisarut, Tada; Thongngarm, Torpong; Ruchutrakool, Theera; Kobwanthanakun, Surapon; Dejsomritrutai, Wanchai

2016-04-01

Universities in Thailand are preparing for Thailand's integration into the ASEAN Economic Community (AEC) by increasing the number of tests in English language. English language is not the native language of Thailand Differences in English language proficiency may affect scores among test-takers, even when subject knowledge among test-takers is comparable and may falsely represent the knowledge level of the test-taker. To study the impact of English language multiple choice test questions on test scores of medical students. The final examination of fourth-year medical students completing internal medicine rotation contains 120 multiple choice questions (MCQ). The languages used on the test are Thai and English at a ratio of 3:1. Individual scores of tests taken in both languages were collected and the effect of English language on MCQ was analyzed Individual MCQ scores were then compared with individual student English language proficiency and student grade point average (GPA). Two hundred ninety five fourth-year medical students were enrolled. The mean percentage of MCQ scores in Thai and English were significantly different (65.0 ± 8.4 and 56.5 ± 12.4, respectively, p < 0.001). The correlation between MCQ scores in Thai and English was fair (Spearman's correlation coefficient = 0.41, p < 0.001). Of 295 students, only 73 (24.7%) students scored higher when being tested in English than in Thai language. Students were classified into six grade categories (A, B+, B, C+, C, and D+), which cumulatively measured total internal medicine rotation performance score plus final examination score. MCQ scores from Thai language examination were more closely correlated with total course grades than were the scores from English language examination (Spearman's correlation coefficient = 0.73 (p < 0.001) and 0.53 (p < 0.001), respectively). The gap difference between MCQ scores in both languages was higher in borderline students than in the excellent student group (11.2 ± 11.2 and 7.1 ± 8.2, respectively, p < 0.001). Overall, average student English proficiency score was very high, at 3.71 ± 0.35 from a total of 4.00. Mean student GPA was 3.40 ± 0.33 from a possible 4.00. English language MCQ examination scores were more highly associated with GPA than with English language proficiency. The use of English language multiple choice question test may decrease scores of the fourth-year internal medicine post-rotation final examination, especially those of borderline students.

[Blended-learning in psychosomatics and psychotherapy - Increasing the satisfaction and knowledge of students with a web-based e-learning tool].

PubMed

Ferber, Julia; Schneider, Gudrun; Havlik, Linda; Heuft, Gereon; Friederichs, Hendrik; Schrewe, Franz-Bernhard; Schulz-Steinel, Andrea; Burgmer, Markus

2014-01-01

To improve the synergy of established methods of teaching, the Department of Psychosomatics and Psychotherapy, University Hospital Münster, developed a web-based elearning tool using video clips of standardized patients. The effect of this blended-learning approach was evaluated. A multiple-choice test was performed by a naive (without the e-learning tool) and an experimental (with the tool) cohort of medical students to test the groups' expertise in psychosomatics. In addition, participants' satisfaction with the new tool was evaluated (numeric rating scale of 0-10). The experimental cohort was more satisfied with the curriculum and more interested in psychosomatics. Furthermore, the experimental cohort scored significantly better in the multiple-choice test. The new tool proved to be an important addition to the classical curriculum as a blended-learning approach which improves students' satisfaction and knowledge in psychosomatics.

Duchenne Muscular Dystrophy: a Survey of Perspectives on Carrier Testing and Communication Within the Family.

PubMed

Hayes, Brenna; Hassed, Susan; Chaloner, Jae Lindsay; Aston, Christopher E; Guy, Carrie

2016-06-01

Carrier testing is widely available for multiple genetic conditions, and several professional organizations have created practice guidelines regarding appropriate clinical application and the testing of minors. Previous research has focused on carrier screening, predictive testing, and testing for X-linked conditions. However, family perspectives on carrier testing for X-linked lethal diseases have yet to be described. In this study, we explored communication within the family about carrier testing and the perspectives of mothers of sons with an X-linked lethal disease, Duchenne muscular dystrophy (DMD). Twenty-five mothers of sons with DMD participated in an anonymous online survey. Survey questions included multiple choice, Likert scale, and open ended, short answer questions. Analysis of the multiple choice and Likert scale questions revealed that most mothers preferred a gradual style of communication with their daughters regarding risk status. In addition, most participants reported having consulted with a genetic counselor and found it helpful. Comparisons between groups, analyzed using Fisher's exact tests, found no differences in preferred style due to mother's carrier status or having a daughter. Thematic analysis was conducted on responses to open ended questions. Themes identified included the impact of family implications, age and maturity, and a desire for autonomy regarding the decision to discuss and undergo carrier testing with at-risk daughters, particularly timing of these discussions. Implications for genetic counseling practice are discussed.

Science Library of Test Items. Volume Three. Mastery Testing Programme. Introduction and Manual.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

A set of short tests aimed at measuring student mastery of specific skills in the natural sciences are presented with a description of the mastery program's purposes, development, and methods. Mastery learning, criterion-referenced testing, and the scope of skills to be tested are defined. Each of the multiple choice tests for grades 7 through 10…

Test-Taking Strategies of Arab EFL Learners on Multiple Choice Tests

ERIC Educational Resources Information Center

Al Fraidan, Abdullah; Al-Khalaf, Khadija

2012-01-01

Many studies have focused on the function of learners' strategies in a variety of EFL domains. However, research on test-taking strategies (TTSs) has been limited, even though such strategies might influence test scores and, as a result, test validity. Motivated by this fact and in light of our own experience as EFL test-makers, this article will…

The CHOICE study: a "taste-test" of utilitarian vs. leisure walking among older adults.

PubMed

Hekler, Eric B; Castro, Cynthia M; Buman, Matthew P; King, Abby C

2012-01-01

Utilitarian walking (e.g., walking for transport) and leisure walking (e.g., walking for health/recreation) are encouraged to promote health, yet few studies have explored specific preferences for these two forms of physical activity or factors that impact such preferences. A quasi-experimental crossover design was used to evaluate how training underactive midlife and older adults in each type of walking impacted total steps taken and how it was linked to their subsequent choice of walking types. Participants (N = 16) were midlife and older adults (M age = 64 ± 8 years) who were mostly women (81%) and white (75%). To control for order effects, participants were randomized to instruction in either utilitarian or leisure walking for 2 weeks and then the other type for 2 weeks. Participants then entered a 2-week "free choice" phase in which they chose any mixture of the walking types. Outcome variables included walking via OMRON pedometer and the ratio of utilitarian versus leisure walking during the free-choice phase. Participants completed surveys about their neighborhood (NEWS) and daily travel to multiple locations. Instruction in leisure-only, utilitarian-only, and a freely chosen mixture of the two each resulted in significant increases in steps taken relative to baseline (ps < 0.05). Having to go to multiple locations daily and traveling greater distances to locations were associated with engagement in more utilitarian walking. In contrast, good walking paths, neighborhood aesthetics, easy access to exercise facilities, and perceiving easier access to neighborhood services were associated with more leisure walking. Results from this pilot study suggest that midlife and older adults may most easily meet guidelines through either leisure only or a mixture of leisure and utilitarian walking, and tailored suggestions based on the person's neighborhood may be useful.

The effect of reading assignments in guided inquiry learning on students’ critical thinking skills

NASA Astrophysics Data System (ADS)

Syarkowi, A.

2018-05-01

The purpose of this study was to determine the effect of reading assignment in guided inquiry learning on senior high school students’ critical thinking skills. The research method which was used in this research was quasi-experiment research method with reading task as the treatment. Topic of inquiry process was Kirchhoff law. The instrument was used for this research was 25 multiple choice interpretive exercises with justification. The multiple choice test was divided on 3 categories such as involve basic clarification, the bases for a decision and inference skills. The result of significance test proved the improvement of students’ critical thinking skills of experiment class was significantly higher when compared with the control class, so it could be concluded that reading assignment can improve students’ critical thinking skills.

Assessment of Stress Level among Dental School Students: An Indian Outlook

PubMed Central

Tangade, Pradeep S.; Mathur, Anmol; Gupta, Rahul; Chaudhary, Shikha

2011-01-01

Background: Evaluating the factors influence student perception, might allow programmatic changes designed to enhance students’ psychosocial well-being and academic performance. The purpose of this study was to investigate perceived sources of stress on dental students and the role of parents in their career choice decisions. Methods: In a cross-sectional design, a modified dental environmental stress questionnaire was administered to all dental students. The responses to the questionnaire were based on a four-point Lickert scale, not stressful to severely stressful. Means and standard deviations were determined for stress scores of individuals for each item and were used to compare the classes, genders, and choices. Student's t, one way ANOVA, and Newman Keuls multiple comparison post hoc tests were used to analysed the data at the significant level of P ≤ 0.05. Results: A total of 304 undergraduate students constituted the study population. Of the respondent, 60% were men and 40% were women. The final year students presented with higher stress scores. In gender comparison, men suffered more stress (62.9%) than women (60.1%). According to the career choice decision, students forced by parents had more perception of stress (69.0%) and the students staying in the hostel (62.7%) showed more perception of stress than day scholars (56.3%). Conclusion: The study indicates that the congenial environment needs to be created for dental education and the parents also need to be counselled against forcing their children to join an educational programme i.e. not of their choice. PMID:22013469

Developing Information Skills Test for Malaysian Youth Students Using Rasch Analysis

ERIC Educational Resources Information Center

Karim, Aidah Abdul; Shah, Parilah M.; Din, Rosseni; Ahmad, Mazalah; Lubis, Maimun Aqhsa

2014-01-01

This study explored the psychometric properties of a locally developed information skills test for youth students in Malaysia using Rasch analysis. The test was a combination of 24 structured and multiple choice items with a 4-point grading scale. The test was administered to 72 technical college students and 139 secondary school students. The…

Construction of Valid and Reliable Test for Assessment of Students

ERIC Educational Resources Information Center

Osadebe, P. U.

2015-01-01

The study was carried out to construct a valid and reliable test in Economics for secondary school students. Two research questions were drawn to guide the establishment of validity and reliability for the Economics Achievement Test (EAT). It is a multiple choice objective test of five options with 100 items. A sample of 1000 students was randomly…

Construction of Economics Achievement Test for Assessment of Students

ERIC Educational Resources Information Center

Osadebe, P. U.

2014-01-01

The study was carried out to construct a valid and reliable test in Economics for secondary school students. Two research questions were drawn to guide the establishment of validity and reliability for the Economics Achievement Test (EAT). It is a multiple choice objective test of five options with 100 items. A sample of 1000 students was randomly…

Test of Achievement in Quantitative Economics for Secondary Schools: Construction and Validation Using Item Response Theory

ERIC Educational Resources Information Center

Eleje, Lydia I.; Esomonu, Nkechi P. M.

2018-01-01

A Test to measure achievement in quantitative economics among secondary school students was developed and validated in this study. The test is made up 20 multiple choice test items constructed based on quantitative economics sub-skills. Six research questions guided the study. Preliminary validation was done by two experienced teachers in…

Two-Dimensional, Implicit Confidence Tests as a Tool for Recognizing Student Misconceptions

ERIC Educational Resources Information Center

Klymkowsky, Michael W.; Taylor, Linda B.; Spindler, Shana R.; Garvin-Doxas, R. Kathy

2006-01-01

The misconceptions that students bring with them, or that arise during instruction, are a critical barrier to learning. Implicit-confidence tests, a simple modification of the multiple-choice test, can be used as a strategy for recognizing student misconceptions. An important issue, however, is whether such tests are gender-neutral. We analyzed…

An Explanatory Item Response Theory Approach for a Computer-Based Case Simulation Test

ERIC Educational Resources Information Center

Kahraman, Nilüfer

2014-01-01

Problem: Practitioners working with multiple-choice tests have long utilized Item Response Theory (IRT) models to evaluate the performance of test items for quality assurance. The use of similar applications for performance tests, however, is often encumbered due to the challenges encountered in working with complicated data sets in which local…

American Sign Language Comprehension Test: A Tool for Sign Language Researchers

ERIC Educational Resources Information Center

Hauser, Peter C.; Paludneviciene, Raylene; Riddle, Wanda; Kurz, Kim B.; Emmorey, Karen; Contreras, Jessica

2016-01-01

The American Sign Language Comprehension Test (ASL-CT) is a 30-item multiple-choice test that measures ASL receptive skills and is administered through a website. This article describes the development and psychometric properties of the test based on a sample of 80 college students including deaf native signers, hearing native signers, deaf…

Force Concept Inventory-Based Multiple-Choice Test for Investigating Students' Representational Consistency

ERIC Educational Resources Information Center

Nieminen, Pasi; Savinainen, Antti; Viiri, Jouni

2010-01-01

This study investigates students' ability to interpret multiple representations consistently (i.e., representational consistency) in the context of the force concept. For this purpose we developed the Representational Variant of the Force Concept Inventory (R-FCI), which makes use of nine items from the 1995 version of the Force Concept Inventory…

Improvement of individual camouflage through background choice in ground-nesting birds.

PubMed

Stevens, Martin; Troscianko, Jolyon; Wilson-Aggarwal, Jared K; Spottiswoode, Claire N

2017-09-01

Animal camouflage is a longstanding example of adaptation. Much research has tested how camouflage prevents detection and recognition, largely focusing on changes to an animal's own appearance over evolution. However, animals could also substantially alter their camouflage by behaviourally choosing appropriate substrates. Recent studies suggest that individuals from several animal taxa could select backgrounds or positions to improve concealment. Here, we test whether individual wild animals choose backgrounds in complex environments, and whether this improves camouflage against predator vision. We studied nest site selection by nine species of ground-nesting birds (nightjars, plovers and coursers) in Zambia, and used image analysis and vision modeling to quantify egg and plumage camouflage to predator vision. Individual birds chose backgrounds that enhanced their camouflage, being better matched to their chosen backgrounds than to other potential backgrounds with respect to multiple aspects of camouflage. This occurred at all three spatial scales tested (a few cm and five meters from the nest, and compared to other sites chosen by conspecifics), and was the case for the eggs of all bird groups studied, and for adult nightjar plumage. Thus, individual wild animals improve their camouflage through active background choice, with choices highly refined across multiple spatial scales.

Improvement of individual camouflage through background choice in ground-nesting birds

PubMed Central

Stevens, Martin; Troscianko, Jolyon; Wilson-Aggarwal, Jared K.; Spottiswoode, Claire N.

2017-01-01

Animal camouflage is a longstanding example of adaptation. Much research has tested how camouflage prevents detection and recognition, largely focusing on changes to an animal's own appearance over evolution. However, animals could also substantially alter their camouflage by behaviourally choosing appropriate substrates. Recent studies suggest that individuals from several animal taxa could select backgrounds or positions to improve concealment. Here, we test whether individual wild animals choose backgrounds in complex environments, and whether this improves camouflage against predator vision. We studied nest site selection by nine species of ground-nesting birds (nightjars, plovers and coursers) in Zambia, and used image analysis and vision modeling to quantify egg and plumage camouflage to predator vision. Individual birds chose backgrounds that enhanced their camouflage, being better matched to their chosen backgrounds than to other potential backgrounds with respect to multiple aspects of camouflage. This occurred at all three spatial scales tested (a few cm and five meters from the nest, and compared to other sites chosen by conspecifics), and was the case for the eggs of all bird groups studied, and for adult nightjar plumage. Thus, individual wild animals improve their camouflage through active background choice, with choices highly refined across multiple spatial scales. PMID:28890937

Development and Validation of the Conceptual Assessment of Natural Selection (CANS)

PubMed Central

Kalinowski, Steven T.; Leonard, Mary J.; Taper, Mark L.

2016-01-01

We developed and validated the Conceptual Assessment of Natural Selection (CANS), a multiple-choice test designed to assess how well college students understand the central principles of natural selection. The expert panel that reviewed the CANS concluded its questions were relevant to natural selection and generally did a good job sampling the specific concepts they were intended to assess. Student interviews confirmed questions on the CANS provided accurate reflections of how students think about natural selection. And, finally, statistical analysis of student responses using item response theory showed that the CANS did a very good job of estimating how well students understood natural selection. The empirical reliability of the CANS was substantially higher than the Force Concept Inventory, a highly regarded test in physics that has a similar purpose. PMID:27856552

Will the "Real" Proficiency Standard Please Stand Up?

ERIC Educational Resources Information Center

Baron, Joan Boykoff; And Others

Connecticut's experience with four different standard-setting methods regarding multiple choice proficiency tests is described. The methods include Angoff, Nedelsky, Borderline Group, and Contrasting Groups Methods. All Connecticut ninth graders were administered proficiency tests in reading, language arts, and mathematics. As soon as final test…

The Prevalence of Multiple-Choice Testing in Registered Nurse Licensure-Qualifying Nursing Education Programs in New York State.

PubMed

Birkhead, Susan; Kelman, Glenda; Zittel, Barbara; Jatulis, Linnea

The aim of this study was to describe nurse educators' use of multiple-choice questions (MCQs) in testing in registered nurse licensure-qualifying nursing education programs in New York State. This study was a descriptive correlational analysis of data obtained from surveying 1,559 nurse educators; 297 educators from 61 institutions responded (response rate [RR] = 19 percent), yielding a final cohort of 200. MCQs were reported to comprise a mean of 81 percent of questions on a typical test. Baccalaureate program respondents were equally likely to use MCQs as associate degree program respondents (p > .05) but were more likely to report using other methods of assessing student achievement to construct course grades (p < .01). Both groups reported little use of alternate format-type questions. Respondent educators reported substantial reliance upon the use of MCQs, corroborating the limited data quantifying the prevalence of use of MCQ tests in licensure-qualifying nursing education programs.

Relationships among Testing Medium, Test Performance, and Testing Time of High School Students Who Are Visually Impaired

ERIC Educational Resources Information Center

Erin, Jane N.; Hong, Sunggye; Schoch, Christina; Kuo, YaJu

2006-01-01

This study compared the test scores and time required by high school students who are blind, sighted, or have low vision to complete tests administered in written and oral formats. The quantitative results showed that the blind students performed better on multiple-choice tests in braille and needed more time while taking tests in braille. The…

Identifying Students' Mathematical Skills from a Multiple-Choice Diagnostic Test Using an Iterative Technique to Minimise False Positives

ERIC Educational Resources Information Center

Manning, S.; Dix, A.

2008-01-01

There is anecdotal evidence that a significant number of students studying computing related courses at degree level have difficulty with sub-GCE mathematics. Testing of students' skills is often performed using diagnostic tests and a number of computer-based diagnostic tests exist, which work, essentially, by testing one specific diagnostic skill…

Test Performance Vis a Vis Test-Taking Strategies in Reading Comprehension of English as a Second Language.

ERIC Educational Resources Information Center

Roizen, Merle A.

A study of students' self-expressed test-taking strategies for reading comprehension tests in English as a second language (ESL) had as subjects 86 Hebrew-speaking university students in Israel. The students were given a reading comprehension test, half with open-ended questions and half with multiple choice. Half the responses were anonymous and…

Grade 9 Pilot Test. Mathematics. June 1988 = 9e Annee Test Pilote. Mathematiques. Juin 1988.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

This pilot test for ninth grade mathematics is written in both French and English. The test consists of 75 multiple-choice items. Students are given 90 minutes to complete the examination and the use of a calculator is highly recommended. The test content covers a wide range of mathematical topics including: decimals; exponents; arithmetic word…

ACER Chemistry Test Item Collection (ACER CHEMTIC Year 12 Supplement).

ERIC Educational Resources Information Center

Australian Council for Educational Research, Hawthorn.

This publication contains 317 multiple-choice chemistry test items related to topics covered in the Victorian (Australia) Year 12 chemistry course. It allows teachers access to a range of items suitable for diagnostic and achievement purposes, supplementing the ACER Chemistry Test Item Collection--Year 12 (CHEMTIC). The topics covered are: organic…

Appropriateness Measurement with Polychotomous Item Response Models and Standardized Indices. Measurement Series, 84-1.

ERIC Educational Resources Information Center

Drasgow, Fritz; And Others

The test scores of some examinees on a multiple-choice test may not provide adequate measures of their abilities. The goal of appropriateness measurement is to identify such individuals. Earlier theoretical and experimental work considered examinees answering all, or almost all, test items. This article reports research that extends…

Development of a State-Wide Competency Test for Marketing Education. Final Report.

ERIC Educational Resources Information Center

Smith, Clifton L.

A project was conducted to develop a valid, competency-referenced test on the core competencies identified for the Missouri Fundamentals of Marketing curriculum. During the project: (1) multiple-choice test items based on the core competencies in the Fundamentals of Marketing curriculum were developed; (2) instructions for onsite administration of…

The Effects of Item by Item Feedback Given during an Ability Test.

ERIC Educational Resources Information Center

Whetton, C.; Childs, R.

1981-01-01

Answer-until-correct (AUC) is a procedure for providing feedback during a multiple-choice test, giving an increased range of scores. The performance of secondary students on a verbal ability test using AUC procedures was compared with a group using conventional instructions. AUC scores considerably enhanced reliability but not validity.…

Investigation of Response Changes in the GRE Revised General Test

ERIC Educational Resources Information Center

Liu, Ou Lydia; Bridgeman, Brent; Gu, Lixiong; Xu, Jun; Kong, Nan

2015-01-01

Research on examinees' response changes on multiple-choice tests over the past 80 years has yielded some consistent findings, including that most examinees make score gains by changing answers. This study expands the research on response changes by focusing on a high-stakes admissions test--the Verbal Reasoning and Quantitative Reasoning measures…

Integrated Testlets: A New Form of Expert-Student Collaborative Testing

ERIC Educational Resources Information Center

Shiell, Ralph C.; Slepkov, Aaron D.

2015-01-01

Integrated testlets are a new assessment tool that encompass the procedural benefits of multiple-choice testing, the pedagogical advantages of free-response-based tests, and the collaborative aspects of a viva voce or defence examination format. The result is a robust assessment tool that provides a significant formative aspect for students.…

Testing with feedback improves recall of information in informed consent: A proof of concept study.

PubMed

Roberts, Katherine J; Revenson, Tracey A; Urken, Mark L; Fleszar, Sara; Cipollina, Rebecca; Rowe, Meghan E; Reis, Laura L Dos; Lepore, Stephen J

2016-08-01

This study investigates whether applying educational testing approaches to an informed consent video for a medical procedure can lead to greater recall of the information presented. Undergraduate students (n=120) were randomly assigned to watch a 20-min video on informed consent under one of three conditions: 1) tested using multiple-choice knowledge questions and provided with feedback on their answers after each 5-min segment; 2) tested with multiple choice knowledge questions but not provided feedback after each segment; or 3) watched the video without knowledge testing. Participants who were tested and provided feedback had significantly greater information recall compared to those who were tested but not provided feedback and to those not tested. The effect of condition was stronger for moderately difficult questions versus easy questions. Inserting knowledge tests and providing feedback about the responses at timed intervals in videos can be effective in improving recall of information. Providing informed consent information through a video not only standardizes the material, but using testing with feedback inserted within the video has the potential to increase recall and retention of this material. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

A learning progression based teaching module on the causes of seasons

NASA Astrophysics Data System (ADS)

Galano, S.

2016-03-01

In this paper, we report about designing and validating a teaching learning module based on a learning progression and focused on the causes of seasons. An initial learning progression about the Celestial Motion big idea -causes of seasons, lunar and solar eclipse and Moon phases- was developed and validated. Existing curricula, research studies on alternative conceptions about these phenomena, and students' answers to an open questionnaire were the starting point to develop initial learning progressions; then, a two-tier multiple-choice questionnaire was designed to validate and improve it. The questionnaire was submitted to about 300 secondary-school students whose answers were used to revise the hypothesized learning progressions. This improved version of the learning progression was used to design a module focused on the causes of seasons in which students were engaged in quantitative measurements with a photovoltaic panel to explain changes of the Sun rays' flow on the Earth's surface over the year. The efficacy of our module in improving students' understanding of the phenomenon of the seasons was tested using our questionnaire as pre- and post-test.

Do the Guideline Violations Influence Test Difficulty of High-Stake Test?: An Investigation on University Entrance Examination in Turkey

ERIC Educational Resources Information Center

Atalmis, Erkan Hasan

2016-01-01

Multiple-choice (MC) items are commonly used in high-stake tests. Thus, each item of such tests should be meticulously constructed to increase the accuracy of decisions based on test results. Haladyna and his colleagues (2002) addressed the valid item-writing guidelines to construct high quality MC items in order to increase test reliability and…

Inventory of Motive of Preference for Conventional Paper-and-Pencil Tests: A Study of Validity and Reliability

ERIC Educational Resources Information Center

Eser, Mehmet Taha; Dogan, Nuri

2017-01-01

Purpose: The objective of this study is to develop the Inventory of Motive of Preference for Conventional Paper-And-Pencil Tests and to evaluate students' motives for preferring written tests, short-answer tests, true/false tests or multiple-choice tests. This will add a measurement tool to the literature with valid and reliable results to help…

The effects of Georgia's Choice curricular reform model on third grade science scores on the Georgia Criterion Referenced Competency Test

NASA Astrophysics Data System (ADS)

Phemister, Art W.

The purpose of this study was to evaluate the effectiveness of the Georgia's Choice reading curriculum on third grade science scores on the Georgia Criterion Referenced Competency Test from 2002 to 2008. In assessing the effectiveness of the Georgia's Choice curriculum model this causal comparative study examined the 105 elementary schools that implemented Georgia's Choice and 105 randomly selected elementary schools that did not elect to use Georgia's Choice. The Georgia's Choice reading program used intensified instruction in an effort to increase reading levels for all students. The study used a non-equivalent control group with a pretest and posttest design to determine the effectiveness of the Georgia's Choice curriculum model. Findings indicated that third grade students in Non-Georgia's Choice schools outscored third grade students in Georgia's Choice schools across the span of the study.

V-TECS Criterion-Referenced Test Item Bank for Radiologic Technology Occupations.

ERIC Educational Resources Information Center

Reneau, Fred; And Others

This Vocational-Technical Education Consortium of States (V-TECS) criterion-referenced test item bank provides 696 multiple-choice items and 33 matching items for radiologic technology occupations. These job titles are included: radiologic technologist, chief; radiologic technologist; nuclear medicine technologist; radiation therapy technologist;…

Grading Multiple Choice Exams with Low-Cost and Portable Computer-Vision Techniques

NASA Astrophysics Data System (ADS)

Fisteus, Jesus Arias; Pardo, Abelardo; García, Norberto Fernández

2013-08-01

Although technology for automatic grading of multiple choice exams has existed for several decades, it is not yet as widely available or affordable as it should be. The main reasons preventing this adoption are the cost and the complexity of the setup procedures. In this paper, Eyegrade, a system for automatic grading of multiple choice exams is presented. While most current solutions are based on expensive scanners, Eyegrade offers a truly low-cost solution requiring only a regular off-the-shelf webcam. Additionally, Eyegrade performs both mark recognition as well as optical character recognition of handwritten student identification numbers, which avoids the use of bubbles in the answer sheet. When compared with similar webcam-based systems, the user interface in Eyegrade has been designed to provide a more efficient and error-free data collection procedure. The tool has been validated with a set of experiments that show the ease of use (both setup and operation), the reduction in grading time, and an increase in the reliability of the results when compared with conventional, more expensive systems.

Standard setting: comparison of two methods.

PubMed

George, Sanju; Haque, M Sayeed; Oyebode, Femi

2006-09-14

The outcome of assessments is determined by the standard-setting method used. There is a wide range of standard-setting methods and the two used most extensively in undergraduate medical education in the UK are the norm-reference and the criterion-reference methods. The aims of the study were to compare these two standard-setting methods for a multiple-choice question examination and to estimate the test-retest and inter-rater reliability of the modified Angoff method. The norm-reference method of standard-setting (mean minus 1 SD) was applied to the 'raw' scores of 78 4th-year medical students on a multiple-choice examination (MCQ). Two panels of raters also set the standard using the modified Angoff method for the same multiple-choice question paper on two occasions (6 months apart). We compared the pass/fail rates derived from the norm reference and the Angoff methods and also assessed the test-retest and inter-rater reliability of the modified Angoff method. The pass rate with the norm-reference method was 85% (66/78) and that by the Angoff method was 100% (78 out of 78). The percentage agreement between Angoff method and norm-reference was 78% (95% CI 69% - 87%). The modified Angoff method had an inter-rater reliability of 0.81-0.82 and a test-retest reliability of 0.59-0.74. There were significant differences in the outcomes of these two standard-setting methods, as shown by the difference in the proportion of candidates that passed and failed the assessment. The modified Angoff method was found to have good inter-rater reliability and moderate test-retest reliability.

The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments.

PubMed

Tarrant, Marie; Knierim, Aimee; Hayes, Sasha K; Ware, James

2006-12-01

Multiple-choice questions are a common assessment method in nursing examinations. Few nurse educators, however, have formal preparation in constructing multiple-choice questions. Consequently, questions used in baccalaureate nursing assessments often contain item-writing flaws, or violations to accepted item-writing guidelines. In one nursing department, 2770 MCQs were collected from tests and examinations administered over a five-year period from 2001 to 2005. Questions were evaluated for 19 frequently occurring item-writing flaws, for cognitive level, for question source, and for the distribution of correct answers. Results show that almost half (46.2%) of the questions contained violations of item-writing guidelines and over 90% were written at low cognitive levels. Only a small proportion of questions were teacher generated (14.1%), while 36.2% were taken from testbanks and almost half (49.4%) had no source identified. MCQs written at a lower cognitive level were significantly more likely to contain item-writing flaws. While there was no relationship between the source of the question and item-writing flaws, teacher-generated questions were more likely to be written at higher cognitive levels (p<0.001). Correct answers were evenly distributed across all four options and no bias was noted in the placement of correct options. Further training in item-writing is recommended for all faculty members who are responsible for developing tests. Pre-test review and quality assessment is also recommended to reduce the occurrence of item-writing flaws and to improve the quality of test questions.

Exploring problem solving strategies on multiple-choice science items: Comparing native Spanish-speaking English Language Learners and mainstream monolinguals

NASA Astrophysics Data System (ADS)

Kachchaf, Rachel Rae

The purpose of this study was to compare how English language learners (ELLs) and monolingual English speakers solved multiple-choice items administered with and without a new form of testing accommodation---vignette illustration (VI). By incorporating theories from second language acquisition, bilingualism, and sociolinguistics, this study was able to gain more accurate and comprehensive input into the ways students interacted with items. This mixed methods study used verbal protocols to elicit the thinking processes of thirty-six native Spanish-speaking English language learners (ELLs), and 36 native-English speaking non-ELLs when solving multiple-choice science items. Results from both qualitative and quantitative analyses show that ELLs used a wider variety of actions oriented to making sense of the items than non-ELLs. In contrast, non-ELLs used more problem solving strategies than ELLs. There were no statistically significant differences in student performance based on the interaction of presence of illustration and linguistic status or the main effect of presence of illustration. However, there were significant differences based on the main effect of linguistic status. An interaction between the characteristics of the students, the items, and the illustrations indicates considerable heterogeneity in the ways in which students from both linguistic groups think about and respond to science test items. The results of this study speak to the need for more research involving ELLs in the process of test development to create test items that do not require ELLs to carry out significantly more actions to make sense of the item than monolingual students.

Comparison between three option, four option and five option multiple choice question tests for quality parameters: A randomized study.

PubMed

Vegada, Bhavisha; Shukla, Apexa; Khilnani, Ajeetkumar; Charan, Jaykaran; Desai, Chetna

2016-01-01

Most of the academic teachers use four or five options per item of multiple choice question (MCQ) test as formative and summative assessment. Optimal number of options in MCQ item is a matter of considerable debate among academic teachers of various educational fields. There is a scarcity of the published literature regarding the optimum number of option in each item of MCQ in the field of medical education. To compare three options, four options, and five options MCQs test for the quality parameters - reliability, validity, item analysis, distracter analysis, and time analysis. Participants were 3 rd semester M.B.B.S. students. Students were divided randomly into three groups. Each group was given one set of MCQ test out of three options, four options, and five option randomly. Following the marking of the multiple choice tests, the participants' option selections were analyzed and comparisons were conducted of the mean marks, mean time, validity, reliability and facility value, discrimination index, point biserial value, distracter analysis of three different option formats. Students score more ( P = 0.000) and took less time ( P = 0.009) for the completion of three options as compared to four options and five options groups. Facility value was more ( P = 0.004) in three options group as compared to four and five options groups. There was no significant difference between three groups for the validity, reliability, and item discrimination. Nonfunctioning distracters were more in the four and five options group as compared to three option group. Assessment based on three option MCQs is can be preferred over four option and five option MCQs.

PERMANOVA-S: association test for microbial community composition that accommodates confounders and multiple distances.

PubMed

Tang, Zheng-Zheng; Chen, Guanhua; Alekseyenko, Alexander V

2016-09-01

Recent advances in sequencing technology have made it possible to obtain high-throughput data on the composition of microbial communities and to study the effects of dysbiosis on the human host. Analysis of pairwise intersample distances quantifies the association between the microbiome diversity and covariates of interest (e.g. environmental factors, clinical outcomes, treatment groups). In the design of these analyses, multiple choices for distance metrics are available. Most distance-based methods, however, use a single distance and are underpowered if the distance is poorly chosen. In addition, distance-based tests cannot flexibly handle confounding variables, which can result in excessive false-positive findings. We derive presence-weighted UniFrac to complement the existing UniFrac distances for more powerful detection of the variation in species richness. We develop PERMANOVA-S, a new distance-based method that tests the association of microbiome composition with any covariates of interest. PERMANOVA-S improves the commonly-used Permutation Multivariate Analysis of Variance (PERMANOVA) test by allowing flexible confounder adjustments and ensembling multiple distances. We conducted extensive simulation studies to evaluate the performance of different distances under various patterns of association. Our simulation studies demonstrate that the power of the test relies on how well the selected distance captures the nature of the association. The PERMANOVA-S unified test combines multiple distances and achieves good power regardless of the patterns of the underlying association. We demonstrate the usefulness of our approach by reanalyzing several real microbiome datasets. miProfile software is freely available at https://medschool.vanderbilt.edu/tang-lab/software/miProfile z.tang@vanderbilt.edu or g.chen@vanderbilt.edu Supplementary data are available at Bioinformatics online. © The Author 2016. Published by Oxford University Press.

PERMANOVA-S: association test for microbial community composition that accommodates confounders and multiple distances

PubMed Central

Tang, Zheng-Zheng; Chen, Guanhua; Alekseyenko, Alexander V.

2016-01-01

Motivation: Recent advances in sequencing technology have made it possible to obtain high-throughput data on the composition of microbial communities and to study the effects of dysbiosis on the human host. Analysis of pairwise intersample distances quantifies the association between the microbiome diversity and covariates of interest (e.g. environmental factors, clinical outcomes, treatment groups). In the design of these analyses, multiple choices for distance metrics are available. Most distance-based methods, however, use a single distance and are underpowered if the distance is poorly chosen. In addition, distance-based tests cannot flexibly handle confounding variables, which can result in excessive false-positive findings. Results: We derive presence-weighted UniFrac to complement the existing UniFrac distances for more powerful detection of the variation in species richness. We develop PERMANOVA-S, a new distance-based method that tests the association of microbiome composition with any covariates of interest. PERMANOVA-S improves the commonly-used Permutation Multivariate Analysis of Variance (PERMANOVA) test by allowing flexible confounder adjustments and ensembling multiple distances. We conducted extensive simulation studies to evaluate the performance of different distances under various patterns of association. Our simulation studies demonstrate that the power of the test relies on how well the selected distance captures the nature of the association. The PERMANOVA-S unified test combines multiple distances and achieves good power regardless of the patterns of the underlying association. We demonstrate the usefulness of our approach by reanalyzing several real microbiome datasets. Availability and Implementation: miProfile software is freely available at https://medschool.vanderbilt.edu/tang-lab/software/miProfile. Contact: z.tang@vanderbilt.edu or g.chen@vanderbilt.edu Supplementary information: Supplementary data are available at Bioinformatics online. PMID:27197815

Effects of socioscientific issues-based instruction on argumentation ability and biology concepts of upper secondary school students

NASA Astrophysics Data System (ADS)

Faelt, Surasak; Samiphak, Sara; Pattaradilokrat, Sittiporn

2018-01-01

Argumentation skill is an essential skill needed in students, and one of the competencies in scientific literacy. Through arguing on socioscientific issues, students may gain deeper conceptual understanding. The purpose of this research is to examine the efficacy of a socioscientific issues-based instruction compared with an inquirybased instruction. This is to determine which one is better in promoting 10th grade students' argumentation ability and biology concepts of digestive system and cellular respiration. The forty 10th grade students included in this study were from two mathematics-science program classes in a medium-sized secondary school located in a suburb of Buriram province, Thailand. The research utilizes a quasi-experimental design; pre-test post-test control group design. We developed and implemented 4 lesson plans for both socioscientific issues-based instruction and inquiry-based instruction. Ten weeks were used to collect the data. A paper-based questionnaire and informal interviews were designed to test students' argumentation ability, and the two-tier multiple-choice test was designed to test their biology concepts. This research explore qualitatively and quantitatively students' argumentation abilities and biology concepts, using arithmetic mean, mean of percentage, standard deviation and t-test. Results show that there is no significant difference between the two group regarding mean scores of the argumentation ability. However, there is significant difference between the two groups regarding mean scores of the biology concepts. This suggests that socioscientific issues-based instruction could be used to improve students' biology concepts.

Pressure scanning choices - Rotary vs electronic

NASA Astrophysics Data System (ADS)

Pemberton, Addison

The choices available for present-day pressure scanning applications are described. Typical pressure scanning applications include wind tunnels, flight testing, turbine engine testing, process control, and laboratory/bench testing. The Scanivalve concept is discussed and it is noted that their use eliminates the cost of multiple individual pressure transducers and their signal conditioners as well as associated wiring for each pressure to be measured. However, they are limited to a maximum acquisition speed of 20 ports/sec/scanner. The advantages of electronic pressure scanners include in-situ calibration on demand, fast data acquisition speed, and high reliability. On the other hand, they are three times more expensive than rotary Scanivalves.

Development of Achievement Test: Validity and Reliability Study for Achievement Test on Matter Changing

ERIC Educational Resources Information Center

Kara, Filiz; Celikler, Dilek

2015-01-01

For "Matter Changing" unit included in the Secondary School 5th Grade Science Program, it is intended to develop a test conforming the gains described in the program, and that can determine students' achievements. For this purpose, a multiple-choice test of 48 questions is arranged, consisting of 8 questions for each gain included in the…

Using Testbanking To Implement Classroom Management/Extension through the Use of Computers.

ERIC Educational Resources Information Center

Thommen, John D.

Testbanking provides teachers with an effective, low-cost, time-saving opportunity to improve the testing aspect of their classes. Testbanking, which involves the use of a testbank program and a computer, allows teachers to develop and generate tests and test-forms with a minimum of effort. Teachers who test using true and false, multiple choice,…

The influence of caloric information on cafeteria food choices

PubMed Central

Dubbert, Patricia M.; Johnson, William G.; Schlundt, David G.; Montague, Noni Ward

1984-01-01

We evaluated the effect of posting caloric information on food purchases at a cafeteria. Purchases of more than 14,300 entrees, vegetables, and salads by 6,970 customers were unobtrusively monitored via the cash register inventory control system during 15 evening observations. A quasi-multiple-baseline design across food groups was used to test the additive effect of labeling the three lowest caloric choices for vegetables, salads, and entrees. A linear logit analysis confirmed that labeling increased the probability of low calorie selections for vegetables and salads, but not for entrees. Observations of meals purchased by a subsample of 413 customers indicated labeling did not change the total caloric content of meals. The number of customers and total sales per evening were unaffected by the labeling intervention. The results suggest that manipulating environmental cues may be an effective method for changing food purchases in a cafeteria, but labeling individual items may not be the best way to decrease total calories purchased. PMID:16795673

Constructing objective tests

NASA Astrophysics Data System (ADS)

Aubrecht, Gordon J.; Aubrecht, Judith D.

1983-07-01

True-false or multiple-choice tests can be useful instruments for evaluating student progress. We examine strategies for planning objective tests which serve to test the material covered in science (physics) courses. We also examine strategies for writing questions for tests within a test blueprint. The statistical basis for judging the quality of test items are discussed. Reliability, difficulty, and discrimination indices are defined and examples presented. Our recommendation are rather easily put into practice.

Implementing Clinical Research Using Factorial Designs: A Primer.

PubMed

Baker, Timothy B; Smith, Stevens S; Bolt, Daniel M; Loh, Wei-Yin; Mermelstein, Robin; Fiore, Michael C; Piper, Megan E; Collins, Linda M

2017-07-01

Factorial experiments have rarely been used in the development or evaluation of clinical interventions. However, factorial designs offer advantages over randomized controlled trial designs, the latter being much more frequently used in such research. Factorial designs are highly efficient (permitting evaluation of multiple intervention components with good statistical power) and present the opportunity to detect interactions amongst intervention components. Such advantages have led methodologists to advocate for the greater use of factorial designs in research on clinical interventions (Collins, Dziak, & Li, 2009). However, researchers considering the use of such designs in clinical research face a series of choices that have consequential implications for the interpretability and value of the experimental results. These choices include: whether to use a factorial design, selection of the number and type of factors to include, how to address the compatibility of the different factors included, whether and how to avoid confounds between the type and number of interventions a participant receives, and how to interpret interactions. The use of factorial designs in clinical intervention research poses choices that differ from those typically considered in randomized clinical trial designs. However, the great information yield of the former encourages clinical researchers' increased and careful execution of such designs. Copyright © 2017. Published by Elsevier Ltd.

Tuning into YouTube in the Classroom: Improving Assessment Scores through Social Media

ERIC Educational Resources Information Center

Younger, Dylinda W.; Duncan, Jan E.; Hart, LaToya M.

2013-01-01

Despite the consistent tendencies of higher-education faculty to utilize single testing measures (i.e. essay or multiple choice), education research indicates effective assessment of student learning must incorporate multiple formats. With the surge of online courses, programs, and universities in the last 20 years, there is an increasing need to…

The Good, the Bad, and the Irrelevant: Neural Mechanisms of Learning Real and Hypothetical Rewards and Effort

PubMed Central

Kolling, Nils; Nelissen, Natalie; Wittmann, Marco K.; Harmer, Catherine J.; Rushworth, Matthew F. S.

2015-01-01

Natural environments are complex, and a single choice can lead to multiple outcomes. Agents should learn which outcomes are due to their choices and therefore relevant for future decisions and which are stochastic in ways common to all choices and therefore irrelevant for future decisions between options. We designed an experiment in which human participants learned the varying reward and effort magnitudes of two options and repeatedly chose between them. The reward associated with a choice was randomly real or hypothetical (i.e., participants only sometimes received the reward magnitude associated with the chosen option). The real/hypothetical nature of the reward on any one trial was, however, irrelevant for learning the longer-term values of the choices, and participants ought to have only focused on the informational content of the outcome and disregarded whether it was a real or hypothetical reward. However, we found that participants showed an irrational choice bias, preferring choices that had previously led, by chance, to a real reward in the last trial. Amygdala and ventromedial prefrontal activity was related to the way in which participants' choices were biased by real reward receipt. By contrast, activity in dorsal anterior cingulate cortex, frontal operculum/anterior insula, and especially lateral anterior prefrontal cortex was related to the degree to which participants resisted this bias and chose effectively in a manner guided by aspects of outcomes that had real and more sustained relationships with particular choices, suppressing irrelevant reward information for more optimal learning and decision making. SIGNIFICANCE STATEMENT In complex natural environments, a single choice can lead to multiple outcomes. Human agents should only learn from outcomes that are due to their choices, not from outcomes without such a relationship. We designed an experiment to measure learning about reward and effort magnitudes in an environment in which other features of the outcome were random and had no relationship with choice. We found that, although people could learn about reward magnitudes, they nevertheless were irrationally biased toward repeating certain choices as a function of the presence or absence of random reward features. Activity in different brain regions in the prefrontal cortex either reflected the bias or reflected resistance to the bias. PMID:26269633

Investigating Administered Essay and Multiple-Choice Tests in the English Department of Islamic Azad University, Hamedan Branch

ERIC Educational Resources Information Center

Karimi, Lotfollah; Mehrdad, Ali Gholami

2012-01-01

This study has attempted to investigate the administered written tests in the language department of Islamic Azad University of Hamedan, Iran from validity, practicality and reliability points of view. To this end two steps were taken. First, examining 112 tests, we knew that the face validity of 50 tests had been threatened, 9 tests lacked…

Practical Implications of Test Dimensionality for Item Response Theory Calibration of the Medical College Admission Test. MCAT Monograph.

ERIC Educational Resources Information Center

Childs, Ruth A.; Oppler, Scott H.

The use of item response theory (IRT) in the Medical College Admission Test (MCAT) testing program has been limited. This study provides a basis for future IRT analyses of the MCAT by exploring the dimensionality of each of the MCAT's three multiple-choice test sections (Verbal Reasoning, Physical Sciences, and Biological Sciences) and the…

Science Library of Test Items. Volume Nineteen. A Collection of Multiple Choice Test Items Relating Mainly to Geology.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Bilingual Test as a Test Accommodation to Determine the Mathematics Achievement of Mainstream Students with Limited English Proficiency

ERIC Educational Resources Information Center

Shanmugam, S. Kanageswari Suppiah; Lan, Ong Saw

2013-01-01

Purpose: This study aims to investigate the validity of using bilingual test to measure the mathematics achievement of students who have limited English proficiency (LEP). The bilingual test and the English-only test consist of 20 computation and 20 word problem multiple-choice questions (from TIMSS 2003 and 2007 released items. The bilingual test…

Science Library of Test Items. Volume Seventeen. A Collection of Multiple Choice Test Items Relating Mainly to Biology.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Science Library of Test Items. Volume Eighteen. A Collection of Multiple Choice Test Items Relating Mainly to Chemistry.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Predict-share-observe-explain learning activity for the Torricelli's tank experiment

NASA Astrophysics Data System (ADS)

Panich, Charunya; Puttharugsa, Chokchai; Khemmani, Supitch

2018-01-01

The purpose of this research was to study the students' scientific concept and achievement on fluid mechanics before and after the predict-share-observe-explain (PSOE) learning activity for the Torricelli's tank experiment. The 24 participants, who were selected by purposive sampling, were students at grade 12 at Nannakorn School, Nan province. A one group pre-test/post-test design was employed in the study. The research instruments were 1) the lesson plans using the PSOE learning activity and 2) two-tier multiple choice question and subjective tests. The results indicated that students had better scientific concept about Torricelli's tank experiment and the post-test mean score was significantly higher than the pre-test mean score at a 0.05 level of significance. Moreover, the students had retention of knowledge after the PSOE learning activity for 4 weeks at a 0.05 level of significance. The study showed that the PSOE learning activity is suitable for developing students' scientific concept and achievement.

Music lessons are associated with increased verbal memory in individuals with Williams syndrome.

PubMed

Dunning, Brittany A; Martens, Marilee A; Jungers, Melissa K

2014-11-16

Williams syndrome (WS) is a genetic disorder characterized by intellectual delay and an affinity for music. It has been previously shown that familiar music can enhance verbal memory in individuals with WS who have had music training. There is also evidence that unfamiliar, or novel, music may also improve cognitive recall. This study was designed to examine if a novel melody could also enhance verbal memory in individuals with WS, and to more fully characterize music training in this population. We presented spoken or sung sentences that described an animal and its group name to 44 individuals with WS, and then tested their immediate and delayed memory using both recall and multiple choice formats. Those with formal music training (average duration of training 4½ years) scored significantly higher on both the spoken and sung recall items, as well as on the spoken multiple choice items, than those with no music training. Music therapy, music enjoyment, age, and Verbal IQ did not impact performance on the memory tasks. These findings provide further evidence that formal music lessons may impact the neurological pathways associated with verbal memory in individuals with WS, consistent with findings in typically developing individuals. Copyright © 2014 Elsevier Ltd. All rights reserved.

Project Physics Tests 6, The Nucleus.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 6 are presented in this booklet. Included are 70 multiple-choice and 24 problem-and-essay questions. Nuclear physics fundamentals are examined with respect to the shell model, isotopes, neutrons, protons, nuclides, charge-to-mass ratios, alpha particles, Becquerel's discovery, gamma rays, cyclotrons,…

Do Multiple-Choice Options Inflate Estimates of Vocabulary Size on the VST?

ERIC Educational Resources Information Center

Stewart, Jeffrey

2014-01-01

Validated under a Rasch framework (Beglar, 2010), the Vocabulary Size Test (VST) (Nation & Beglar, 2007) is an increasingly popular measure of decontextualized written receptive vocabulary size in the field of second language acquisition. However, although the validation indicates that the test has high internal reliability, still unaddressed…

Ability Level Estimation of Students on Probability Unit via Computerized Adaptive Testing

ERIC Educational Resources Information Center

Özyurt, Hacer; Özyurt, Özcan

2015-01-01

Problem Statement: Learning-teaching activities bring along the need to determine whether they achieve their goals. Thus, multiple choice tests addressing the same set of questions to all are frequently used. However, this traditional assessment and evaluation form contrasts with modern education, where individual learning characteristics are…

Project Physics Tests 1, Concepts of Motion.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 1 are presented in this booklet, consisting of 70 multiple-choice and 20 problem-and-essay questions. Concepts of motion are examined with respect to velocities, acceleration, forces, vectors, Newton's laws, and circular motion. Suggestions are made for time consumption in answering some items. Besides…

Project Physics Tests 5, Models of the Atom.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 5 are presented in this booklet. Included are 70 multiple-choice and 23 problem-and-essay questions. Concepts of atomic model are examined on aspects of relativistic corrections, electron emission, photoelectric effects, Compton effect, quantum theories, electrolysis experiments, atomic number and mass,…

Project Physics Tests 4, Light and Electromagnetism.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 4 are presented in this booklet. Included are 70 multiple-choice and 22 problem-and-essay questions. Concepts of light and electromagnetism are examined on charges, reflection, electrostatic forces, electric potential, speed of light, electromagnetic waves and radiations, Oersted's and Faraday's work,…

Online Testing: The Dog Sat on My Keyboard.

ERIC Educational Resources Information Center

White, Jacci

This paper will highlight some advantages and disadvantages of several online models for student assessment. These models will include: live exams, multiple choice tests, essay exams, and student projects. In addition, real student responses and "problems" will be used as prompts to improve models of authentic online assessment in mathematics.…

A Database System for Course Administration.

ERIC Educational Resources Information Center

Benbasat, Izak; And Others

1982-01-01

Describes a computer-assisted testing system which produces multiple-choice examinations for a college course in business administration. The system uses SPIRES (Stanford Public Information REtrieval System) to manage a database of questions and related data, mark-sense cards for machine grading tests, and ACL (6) (Audit Command Language) to…

Building the BIKE: Development and Testing of the Biotechnology Instrument for Knowledge Elicitation (BIKE)

ERIC Educational Resources Information Center

Witzig, Stephen B.; Rebello, Carina M.; Siegel, Marcelle A.; Freyermuth, Sharyn K.; Izci, Kemal; McClure, Bruce

2014-01-01

Identifying students' conceptual scientific understanding is difficult if the appropriate tools are not available for educators. Concept inventories have become a popular tool to assess student understanding; however, traditionally, they are multiple choice tests. International science education standard documents advocate that assessments…

Fundamentals of Marketing Core Curriculum. Test Items and Assessment Techniques.

ERIC Educational Resources Information Center

Smith, Clifton L.; And Others

This document contains multiple choice test items and assessment techniques for Missouri's fundamentals of marketing core curriculum. The core curriculum is divided into these nine occupational duties: (1) communications in marketing; (2) economics and marketing; (3) employment and advancement; (4) human relations in marketing; (5) marketing…

Project Physics Tests 3, The Triumph of Mechanics.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 3 are presented in this booklet. Included are 70 multiple-choice and 20 problem-and-essay questions. Concepts of mechanics are examined on energy, momentum, kinetic theory of gases, pulse analyses, "heat death," water waves, power, conservation laws, normal distribution, thermodynamic laws, and…

Using skin carotenoids to assess dietary changes in students after one academic year of participating in the shaping healthy choices program

USDA-ARS?s Scientific Manuscript database

Objective: To determine whether 4th-grade students participating in the Shaping Healthy Choices Program (SHCP), a school-based nutrition intervention, change vegetable intake Design: quasi-experimental single group pre-test, post-test with a self-selected, convenience sample of students recruited at...

Preference of Students on the Format of Options in a Multiple-Choice Test

ERIC Educational Resources Information Center

Oyzon, Voltaire Q.; Bendulo, Hermabeth O.; Tibus, Erlinda D.; Bande, Rhodora A.; Macalinao, Myrna L.

2016-01-01

Schools in the Philippines, especially those that are offering teacher education programs, are advised to construct examinations that are Licensure Examination for Teachers (LET)-like test items. This is because "if any aspect of a test is unfamiliar to candidates, they are likely to perform less well than they would do otherwise on…

Test of Understanding of Vectors: A Reliable Multiple-Choice Vector Concept Test

ERIC Educational Resources Information Center

Barniol, Pablo; Zavala, Genaro

2014-01-01

In this article we discuss the findings of our research on students' understanding of vector concepts in problems without physical context. First, we develop a complete taxonomy of the most frequent errors made by university students when learning vector concepts. This study is based on the results of several test administrations of open-ended…

Test-Enhanced Learning in a Middle School Science Classroom: The Effects of Quiz Frequency and Placement

ERIC Educational Resources Information Center

McDaniel, Mark A.; Agarwal, Pooja K.; Huelser, Barbie J.; McDermott, Kathleen B.; Roediger, Henry L., III

2011-01-01

Typically, teachers use tests to evaluate students' knowledge acquisition. In a novel experimental study, we examined whether low-stakes testing ("quizzing") can be used to foster students' learning of course content in 8th grade science classes. Students received multiple-choice quizzes (with feedback); in the quizzes, some target…

Australian Chemistry Test Item Bank: Years 11 & 12. Volume 1.

ERIC Educational Resources Information Center

Commons, C., Ed.; Martin, P., Ed.

Volume 1 of the Australian Chemistry Test Item Bank, consisting of two volumes, contains nearly 2000 multiple-choice items related to the chemistry taught in Year 11 and Year 12 courses in Australia. Items which were written during 1979 and 1980 were initially published in the "ACER Chemistry Test Item Collection" and in the "ACER…

Validation of a Standardized Multiple-Choice Multicultural Competence Test: Implications for Training, Assessment, and Practice

ERIC Educational Resources Information Center

Gillem, Angela R.; Bartoli, Eleonora; Bertsch, Kristin N.; McCarthy, Maureen A.; Constant, Kerra; Marrero-Meisky, Sheila; Robbins, Steven J.; Bellamy, Scarlett

2016-01-01

The Multicultural Counseling and Psychotherapy Test (MCPT), a measure of multicultural counseling competence (MCC), was validated in 2 phases. In Phase 1, the authors administered 451 test items derived from multicultural guidelines in counseling and psychology to 32 multicultural experts and 30 nonexperts. In Phase 2, the authors administered the…

Correction for Guessing in the Framework of the 3PL Item Response Theory

ERIC Educational Resources Information Center

Chiu, Ting-Wei

2010-01-01

Guessing behavior is an important topic with regard to assessing proficiency on multiple choice tests, particularly for examinees at lower levels of proficiency due to greater the potential for systematic error or bias which that inflates observed test scores. Methods that incorporate a correction for guessing on high-stakes tests generally rely…

Memory-Context Effects of Screen Color in Multiple-Choice and Fill-In Tests

ERIC Educational Resources Information Center

Prestera, Gustavo E.; Clariana, Roy; Peck, Andrew

2005-01-01

In this experimental study, 44 undergraduates completed five computer-based instructional lessons and either two multiplechoice tests or two fill-in-the-blank tests. Color-coded borders were displayed during the lesson, adjacent to the screen text and illustrations. In the experimental condition, corresponding border colors were shown at posttest.…

Australian Chemistry Test Item Bank: Years 11 and 12. Volume 2.

ERIC Educational Resources Information Center

Commons, C., Ed.; Martin, P., Ed.

The second volume of the Australian Chemistry Test Item Bank, consisting of two volumes, contains nearly 2000 multiple-choice items related to the chemistry taught in Year 11 and Year 12 courses in Australia. Items which were written during 1979 and 1980 were initially published in the "ACER Chemistry Test Item Collection" and in the…

Comparability of Computer- and Paper-Administered Multiple-Choice Tests for K-12 Populations: A Synthesis

ERIC Educational Resources Information Center

Kingston, Neal M.

2009-01-01

There have been many studies of the comparability of computer-administered and paper-administered tests. Not surprisingly (given the variety of measurement and statistical sampling issues that can affect any one study) the results of such studies have not always been consistent. Moreover, the quality of computer-based test administration systems…

The Potential Use of the Discouraging Random Guessing (DRG) Approach in Multiple-Choice Exams in Medical Education.

ERIC Educational Resources Information Center

Friedman, Miriam; And Others

1987-01-01

Test performances of sophomore medical students on a pretest and final exam (under guessing and no-guessing instructions) were compared. Discouraging random guessing produced test information with improved test reliability and less distortion of item difficulty. More able examinees were less compliant than less able examinees. (Author/RH)

Development and Application of a Two-Tier Multiple-Choice Diagnostic Test for High School Students' Understanding of Cell Division and Reproduction

ERIC Educational Resources Information Center

Sesli, Ertugrul; Kara, Yilmaz

2012-01-01

This study involved the development and application of a two-tier diagnostic test for measuring students' understanding of cell division and reproduction. The instrument development procedure had three general steps: defining the content boundaries of the test, collecting information on students' misconceptions, and instrument development.…

A Comparison of Domain-Referenced and Classic Psychometric Test Construction Methods.

ERIC Educational Resources Information Center

Willoughby, Lee; And Others

This study compared a domain referenced approach with a traditional psychometric approach in the construction of a test. Results of the December, 1975 Quarterly Profile Exam (QPE) administered to 400 examinees at a university were the source of data. The 400 item QPE is a five alternative multiple choice test of information a "safe"…

The Disaggregation of Value-Added Test Scores to Assess Learning Outcomes in Economics Courses

ERIC Educational Resources Information Center

Walstad, William B.; Wagner, Jamie

2016-01-01

This study disaggregates posttest, pretest, and value-added or difference scores in economics into four types of economic learning: positive, retained, negative, and zero. The types are derived from patterns of student responses to individual items on a multiple-choice test. The micro and macro data from the "Test of Understanding in College…

Factors influencing tests of auditory processing: a perspective on current issues and relevant concerns.

PubMed

Cacace, Anthony T; McFarland, Dennis J

2013-01-01

Tests of auditory perception, such as those used in the assessment of central auditory processing disorders ([C]APDs), represent a domain in audiological assessment where measurement of this theoretical construct is often confounded by nonauditory abilities due to methodological shortcomings. These confounds include the effects of cognitive variables such as memory and attention and suboptimal testing paradigms, including the use of verbal reproduction as a form of response selection. We argue that these factors need to be controlled more carefully and/or modified so that their impact on tests of auditory and visual perception is only minimal. To advocate for a stronger theoretical framework than currently exists and to suggest better methodological strategies to improve assessment of auditory processing disorders (APDs). Emphasis is placed on adaptive forced-choice psychophysical methods and the use of matched tasks in multiple sensory modalities to achieve these goals. Together, this approach has potential to improve the construct validity of the diagnosis, enhance and develop theory, and evolve into a preferred method of testing. Examination of methods commonly used in studies of APDs. Where possible, currently used methodology is compared to contemporary psychophysical methods that emphasize computer-controlled forced-choice paradigms. In many cases, the procedures used in studies of APD introduce confounding factors that could be minimized if computer-controlled forced-choice psychophysical methods were utilized. Ambiguities of interpretation, indeterminate diagnoses, and unwanted confounds can be avoided by minimizing memory and attentional demands on the input end and precluding the use of response-selection strategies that use complex motor processes on the output end. Advocated are the use of computer-controlled forced-choice psychophysical paradigms in combination with matched tasks in multiple sensory modalities to enhance the prospect of obtaining a valid diagnosis. American Academy of Audiology.

Android worksheet application based on discovery learning on students' achievement for vocational high school: Mechanical behavior of materials topics

NASA Astrophysics Data System (ADS)

Nanto, Dwi; Aini, Anisa Nurul; Mulhayatiah, Diah

2017-05-01

This research reports a study of student worksheet based on discovery learning on Mechanical Behavior of Materials topics under Android application (Android worksheet application) for vocational high school. The samples are Architecture class X students of SMKN 4 (a public vocational high school) in Tangerang Selatan City, province of Banten, Indonesia. We made 3 groups based on Intellectual Quotient (IQ). They are average IQ group, middle IQ group and high IQ group. The method of research is used as a quasi-experimental design with nonequivalent control group design. The technique of sampling is purposive sampling. Instruments used in this research are test instruments and non-test instruments. The test instruments are IQ test and test of student's achievement. For the test of student's achievement (pretest and posttest) we provide 25 multiple choice problems. The non-test instruments are questionnaire responses by the students and the teacher. Without IQ categorized, the result showed that there is an effect of Android worksheet application on student's achievement based on cognitive aspects of Revised Bloom's Taxonomy. However, from the IQ groups point of view, only the middle IQ group and the high IQ group showed a significant effect from the Android worksheet application on student's achievement meanwhile for the average IQ group there was no effect.

The Positive and Negative Effects of Science Concept Tests on Student Conceptual Understanding

NASA Astrophysics Data System (ADS)

Chang, Chun-Yen; Yeh, Ting-Kuang; Barufaldi, James P.

2010-01-01

This study explored the phenomenon of testing effect during science concept assessments, including the mechanism behind it and its impact upon a learner's conceptual understanding. The participants consisted of 208 high school students, in either the 11th or 12th grade. Three types of tests (traditional multiple-choice test, correct concept test, and incorrect concept test) related to the greenhouse effect and global warming were developed to explore the mechanisms underlining the test effect. Interview data analyzed by means of the flow-map method were used to examine the two-week post-test consequences of taking one of these three tests. The results indicated: (1) Traditional tests can affect participants' long-term memory, both positively and negatively; in addition, when students ponder repeatedly and think harder about highly distracting choices during a test, they may gradually develop new conceptions; (2) Students develop more correct conceptions when more true descriptions are provided on the tests; on the other hand, students develop more misconceptions while completing tests in which more false descriptions of choices are provided. Finally, the results of this study revealed a noteworthy phenomenon that tests, if employed appropriately, may be also an effective instrument for assisting students' conceptual understanding.

Using a Systematic Approach and Theoretical Framework to Design a Curriculum for the Shaping Healthy Choices Program.

PubMed

Linnell, Jessica D; Zidenberg-Cherr, Sheri; Briggs, Marilyn; Scherr, Rachel E; Brian, Kelley M; Hillhouse, Carol; Smith, Martin H

2016-01-01

To examine the use of a systematic approach and theoretical framework to develop an inquiry-based, garden-enhanced nutrition curriculum for the Shaping Healthy Choices Program. Curriculum development occurred in 3 steps: identification of learning objectives, determination of evidence of learning, and activity development. Curriculum activities were further refined through pilot-testing, which was conducted in 2 phases. Formative data collected during pilot-testing resulted in improvements to activities. Using a systematic, iterative process resulted in a curriculum called Discovering Healthy Choices, which has a strong foundation in Social Cognitive Theory and constructivist learning theory. Furthermore, the Backward Design method provided the design team with a systematic approach to ensure activities addressed targeted learning objectives and overall Shaping Healthy Choices Program goals. The process by which a nutrition curriculum is developed may have a direct effect on student outcomes. Processes by which nutrition curricula are designed and learning objectives are selected, and how theory and pedagogy are applied should be further investigated so that effective approaches to developing garden-enhanced nutrition interventions can be determined and replicated. Copyright © 2016 Society for Nutrition Education and Behavior. Published by Elsevier Inc. All rights reserved.

Performance of Dental Hygiene Students in Mass Fatality Training and Radiographic Imaging of Dental Remains.

PubMed

Newcomb, Tara L; Bruhn, Ann M; Ulmer, Loreta H; Diawara, Norou

2015-10-01

Mass fatality incidents can overwhelm local, state and national resources quickly. Dental hygienists are widely distributed and have the potential to increase response teams' capacity. However, appropriate training is required. The literature is void of addressing this type of training for dental hygienists and scant in dentistry. Hence, the purpose of this study was to assess one facet of such training: Whether the use of multimedia is likely to enhance educational outcomes related to mass fatality training. A randomized, double-blind, pre- and post-test design was used to evaluate the effectiveness of comparable educational modules for 2 groups: a control group (n=19) that received low media training and a treatment group (n=20) that received multimedia training. Participants were second-year, baccalaureate dental hygiene students. Study instruments included a multiple-choice examination, a clinical competency-based radiology lab scored via a standardized rubric, and an assessment of interest in mass fatality education as a specialty. ANOVA was used to analyze results. Participants' pre- and post-test scores and clinical competency-based radiology lab scores increased following both educational approaches. Interest in mass fatality training also increased significantly for all participants (p=0.45). There was no significant difference in pre- and post-test multiple choice scores (p=0.6455), interest (p=0.9133) or overall competency-based radiology lab scores (p=0.997) between groups. Various educational technique may be effective for mass fatality training. However, mass fatality training that incorporates multimedia is an appropriate avenue for training instruction. Continued research about multimedia's role in this specialty area is encouraged. Copyright © 2015 The American Dental Hygienists’ Association.

A comparative study of students' performance in preclinical physiology assessed by multiple choice and short essay questions.

PubMed

Oyebola, D D; Adewoye, O E; Iyaniwura, J O; Alada, A R; Fasanmade, A A; Raji, Y

2000-01-01

This study was designed to compare the performance of medical students in physiology when assessed by multiple choice questions (MCQs) and short essay questions (SEQs). The study also examined the influence of factors such as age, sex, O/level grades and JAMB scores on performance in the MCQs and SEQs. A structured questionnaire was administered to 264 medical students' four months before the Part I MBBS examination. Apart from personal data of each student, the questionnaire sought information on the JAMB scores and GCE O' Level grades of each student in English Language, Biology, Chemistry, Physics and Mathematics. The physiology syllabus was divided into five parts and the students were administered separate examinations (tests) on each part. Each test consisted of MCQs and SEQs. The performance in MCQs and SEQs were compared. Also, the effects of JAMB scores and GCE O/level grades on the performance in both the MCQs and SEQs were assessed. The results showed that the students performed better in all MCQ tests than in the SEQs. JAMB scores and O' level English Language grade had no significant effect on students' performance in MCQs and SEQs. However O' level grades in Biology, Chemistry, Physics and Mathematics had significant effects on performance in MCQs and SEQs. Inadequate knowledge of physiology and inability to present information in a logical sequence are believed to be major factors contributing to the poorer performance in the SEQs compared with MCQs. In view of the finding of significant association between performance in MCQs and SEQs and GCE O/level grades in science subjects and mathematics, it was recommended that both JAMB results and the GCE results in the four O/level subjects above may be considered when selecting candidates for admission into the medical schools.

Nurse-led immunotreatment DEcision Coaching In people with Multiple Sclerosis (DECIMS) - Feasibility testing, pilot randomised controlled trial and mixed methods process evaluation.

PubMed

Rahn, A C; Köpke, S; Backhus, I; Kasper, J; Anger, K; Untiedt, B; Alegiani, A; Kleiter, I; Mühlhauser, I; Heesen, C

2018-02-01

Treatment decision-making is complex for people with multiple sclerosis. Profound information on available options is virtually not possible in regular neurologist encounters. The "nurse decision coach model" was developed to redistribute health professionals' tasks in supporting immunotreatment decision-making following the principles of informed shared decision-making. To test the feasibility of a decision coaching programme and recruitment strategies to inform the main trial. Feasibility testing and parallel pilot randomised controlled trial, accompanied by a mixed methods process evaluation. Two German multiple sclerosis university centres. People with suspected or relapsing-remitting multiple sclerosis facing immunotreatment decisions on first line drugs were recruited. Randomisation to the intervention (n = 38) or control group (n = 35) was performed on a daily basis. Quantitative and qualitative process data were collected from people with multiple sclerosis, nurses and physicians. We report on the development and piloting of the decision coaching programme. It comprises a training course for multiple sclerosis nurses and the coaching intervention. The intervention consists of up to three structured nurse-led decision coaching sessions, access to an evidence-based online information platform (DECIMS-Wiki) and a final physician consultation. After feasibility testing, a pilot randomised controlled trial was performed. People with multiple sclerosis were randomised to the intervention or control group. The latter had also access to the DECIMS-Wiki, but received otherwise care as usual. Nurses were not blinded to group assignment, while people with multiple sclerosis and physicians were. The primary outcome was 'informed choice' after six months including the sub-dimensions' risk knowledge (after 14 days), attitude concerning immunotreatment (after physician consultation), and treatment uptake (after six months). Quantitative process evaluation data were collected via questionnaires. Qualitative interviews were performed with all nurses and a convenience sample of nine people with multiple sclerosis. 116 people with multiple sclerosis fulfilled the inclusion criteria and 73 (63%) were included. Groups were comparable at baseline. Data of 51 people with multiple sclerosis (70%) were available for the primary endpoint. In the intervention group 15 of 31 (48%) people with multiple sclerosis achieved an informed choice after six months and 6 of 20 (30%) in the control group. Process evaluation data illustrated a positive response towards the coaching programme as well as good acceptance. The pilot-phase showed promising results concerning acceptability and feasibility of the intervention, which was well perceived by people with multiple sclerosis, most nurses and physicians. Delegating parts of the immunotreatment decision-making process to trained nurses has the potential to increase informed choice and participation as well as effectiveness of patient-physician consultations. Copyright © 2017 Elsevier Ltd. All rights reserved.

Spacecraft Antennas

NASA Technical Reports Server (NTRS)

Jamnejad, Vahraz; Manshadi, Farzin; Rahmat-Samii, Yahya; Cramer, Paul

1990-01-01

Some of the various categories of issues that must be considered in the selection and design of spacecraft antennas for a Personal Access Satellite System (PASS) are addressed, and parametric studies for some of the antenna concepts to help the system designer in making the most appropriate antenna choice with regards to weight, size, and complexity, etc. are provided. The question of appropriate polarization for the spacecraft as well as for the User Terminal Antenna required particular attention and was studied in some depth. Circular polarization seems to be the favored outcome of this study. Another problem that has generally been a complicating factor in designing the multiple beam reflector antennas, is the type of feeds (single vs. multiple element and overlapping vs. non-overlapping clusters) needed for generating the beams. This choice is dependent on certain system design factors, such as the required frequency reuse, acceptable interbeam isolation, antenna efficiency, number of beams scanned, and beam-forming network (BFN) complexity. This issue is partially addressed, but is not completely resolved. Indications are that it may be possible to use relatively simple non-overlapping clusters of only a few elements, unless a large frequency reuse and very stringent isolation levels are required.

Development of a special-purpose test surface guided by uncertainty analysis - Introduction of a new uncertainty analysis step

NASA Technical Reports Server (NTRS)

Wang, T.; Simon, T. W.

1988-01-01

Development of a recent experimental program to investigate the effects of streamwise curvature on boundary layer transition required making a bendable, heated and instrumented test wall, a rather nonconventional surface. The present paper describes this surface, the design choices made in its development and how uncertainty analysis was used, beginning early in the test program, to make such design choices. Published uncertainty analysis techniques were found to be of great value; but, it became clear that another step, one herein called the pre-test analysis, would aid the program development. Finally, it is shown how the uncertainty analysis was used to determine whether the test surface was qualified for service.

Factorial versus multi-arm multi-stage designs for clinical trials with multiple treatments.

PubMed

Jaki, Thomas; Vasileiou, Despina

2017-02-20

When several treatments are available for evaluation in a clinical trial, different design options are available. We compare multi-arm multi-stage with factorial designs, and in particular, we will consider a 2 × 2 factorial design, where groups of patients will either take treatments A, B, both or neither. We investigate the performance and characteristics of both types of designs under different scenarios and compare them using both theory and simulations. For the factorial designs, we construct appropriate test statistics to test the hypothesis of no treatment effect against the control group with overall control of the type I error. We study the effect of the choice of the allocation ratios on the critical value and sample size requirements for a target power. We also study how the possibility of an interaction between the two treatments A and B affects type I and type II errors when testing for significance of each of the treatment effects. We present both simulation results and a case study on an osteoarthritis clinical trial. We discover that in an optimal factorial design in terms of minimising the associated critical value, the corresponding allocation ratios differ substantially to those of a balanced design. We also find evidence of potentially big losses in power in factorial designs for moderate deviations from the study design assumptions and little gain compared with multi-arm multi-stage designs when the assumptions hold. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

Students’ Conception on Heat and Temperature toward Science Process Skill

NASA Astrophysics Data System (ADS)

Ratnasari, D.; Sukarmin, S.; Suparmi, S.; Aminah, N. S.

2017-09-01

This research is aimed to analyze the effect of students’ conception toward science process skill. This is a descriptive research with subjects of the research were 10th-grade students in Surakarta from high, medium and low categorized school. The sample selection uses purposive sampling technique based on physics score in national examination four latest years. Data in this research collecting from essay test, two-tier multiple choice test, and interview. Two-tier multiple choice test consists of 30 question that contains an indicator of science process skill. Based on the result of the research and analysis, it shows that students’ conception of heat and temperature affect science process skill of students. The students’ conception that still contains the wrong concept can emerge misconception. For the future research, it is suggested to improve students’ conceptual understanding and students’ science process skill with appropriate learning method and assessment instrument because heat and temperature is one of physics material that closely related with students’ daily life.

The Influence of Using Momentum and Impulse Computer Simulation to Senior High School Students’ Concept Mastery

NASA Astrophysics Data System (ADS)

Kaniawati, I.; Samsudin, A.; Hasopa, Y.; Sutrisno, A. D.; Suhendi, E.

2016-08-01

This research is based on students’ lack of mastery of physics abstract concepts. Thus, this study aims to improve senior high school students’ mastery of momentum and impulse concepts with the use of computer simulation. To achieve these objectives, the research method employed was pre experimental design with one group pre-test post-test. A total of 36 science students of grade 11 in one of public senior high school in Bandung became the sample in this study. The instruments utilized to determine the increase of students’ concept mastery were pretest and posttest in the form of multiple choices. After using computer simulations in physics learning, students’ mastery of momentum and impulse concept has increased as indicated by the normalized gain of 0.64 with the medium category.

Constructing a Criterion Reference Test to Measure the Research and Statistical Competencies of Graduate Students at the Jordanian Governmental Universities

ERIC Educational Resources Information Center

Al-Habashneh, Maher Hussein; Najjar, Nabil Juma

2017-01-01

This study aimed at constructing a criterion-reference test to measure the research and statistical competencies of graduate students at the Jordanian governmental universities, the test has to be in its first form of (50) multiple choice items, then the test was introduced to (5) arbitrators with competence in measurement and evaluation to…

How Well Do Engineering Students Retain Core Mathematical Knowledge after a Series of High Threshold Online Mathematics Tests?

ERIC Educational Resources Information Center

Carr, Michael; Prendergast, Mark; Breen, Cormac; Faulkner, Fiona

2017-01-01

In the Dublin Institute of Technology, high threshold core skills assessments are run in mathematics for third-year engineering students. Such tests require students to reach a threshold of 90% on a multiple choice test based on a randomized question bank. The material covered by the test consists of the more important aspects of undergraduate…

Gradated assembly of multiple proteins into supramolecular nanomaterials

NASA Astrophysics Data System (ADS)

Hudalla, Gregory A.; Sun, Tao; Gasiorowski, Joshua Z.; Han, Huifang; Tian, Ye F.; Chong, Anita S.; Collier, Joel H.

2014-08-01

Biomaterials exhibiting precise ratios of different bioactive protein components are critical for applications ranging from vaccines to regenerative medicine, but their design is often hindered by limited choices and cross-reactivity of protein conjugation chemistries. Here, we describe a strategy for inducing multiple different expressed proteins of choice to assemble into nanofibres and gels with exceptional compositional control. The strategy employs ‘βTail’ tags, which allow for good protein expression in bacteriological cultures, yet can be induced to co-assemble into nanomaterials when mixed with additional β-sheet fibrillizing peptides. Multiple different βTail fusion proteins could be inserted into peptide nanofibres alone or in combination at predictable, smoothly gradated concentrations, providing a simple yet versatile route to install precise combinations of proteins into nanomaterials. The technology is illustrated by achieving precisely targeted hues using mixtures of fluorescent proteins, by creating nanofibres bearing enzymatic activity, and by adjusting antigenic dominance in vaccines.

Detection of Undocumented Changepoints Using Multiple Test Statistics and Composite Reference Series.

NASA Astrophysics Data System (ADS)

Menne, Matthew J.; Williams, Claude N., Jr.

2005-10-01

An evaluation of three hypothesis test statistics that are commonly used in the detection of undocumented changepoints is described. The goal of the evaluation was to determine whether the use of multiple tests could improve undocumented, artificial changepoint detection skill in climate series. The use of successive hypothesis testing is compared to optimal approaches, both of which are designed for situations in which multiple undocumented changepoints may be present. In addition, the importance of the form of the composite climate reference series is evaluated, particularly with regard to the impact of undocumented changepoints in the various component series that are used to calculate the composite.In a comparison of single test changepoint detection skill, the composite reference series formulation is shown to be less important than the choice of the hypothesis test statistic, provided that the composite is calculated from the serially complete and homogeneous component series. However, each of the evaluated composite series is not equally susceptible to the presence of changepoints in its components, which may be erroneously attributed to the target series. Moreover, a reference formulation that is based on the averaging of the first-difference component series is susceptible to random walks when the composition of the component series changes through time (e.g., values are missing), and its use is, therefore, not recommended. When more than one test is required to reject the null hypothesis of no changepoint, the number of detected changepoints is reduced proportionately less than the number of false alarms in a wide variety of Monte Carlo simulations. Consequently, a consensus of hypothesis tests appears to improve undocumented changepoint detection skill, especially when reference series homogeneity is violated. A consensus of successive hypothesis tests using a semihierarchic splitting algorithm also compares favorably to optimal solutions, even when changepoints are not hierarchic.

Science Library of Test Items. Volume Twenty. A Collection of Multiple Choice Test Items Relating Mainly to Physics, 1.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Science Library of Test Items. Volume Twenty-One. A Collection of Multiple Choice Test Items Relating Mainly to Physics, 2.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Science Library of Test Items. Volume Twenty-Two. A Collection of Multiple Choice Test Items Relating Mainly to Skills.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Sex Differences in Dating Choices: The Role of Physical and Personality Attractiveness.

ERIC Educational Resources Information Center

Caputo, G. Craig; Gronwaldt, Virginia H.

There is a persistent belief that, in making dating choices, men are more influenced by physical attractiveness and less influenced by personality characteristics of potential dates than are women. This study was designed to test whether this is true of college students. On the basis of pre-test ratings, a set of nine facial photographs and nine…

After-School Spaces: Looking for Learning in All the Right Places

NASA Astrophysics Data System (ADS)

Schnittka, Christine G.; Evans, Michael A.; Won, Samantha G. L.; Drape, Tiffany A.

2016-06-01

After-school settings provide youth with homework support, social outlets and fun activities, and help build self-confidence. They are safe places for forming relationships with caring adults. More after-school settings are starting to integrate Science, Technology, Engineering, and Mathematics (STEM) topics. What science skills and concepts might youth learn in engineering design-based after-school settings? Traditional assessments often fail to capture the ways youth learn in informal settings, and deep science understandings are notoriously difficult to measure. In this study, we examined three after-school settings where 65 youth were learning science through engineering design challenges. In this informal setting, we examined storyboards, social networking forum (SNF) chat logs, videos of whole-class interactions, interviews with groups and single participants, and traditional multiple-choice pre- and posttest results. As we looked for evidence of learning, we found that the social networking forum was rich with data. Interviews were even more informative, much more so than traditional pencil and paper multiple-choice tests. We found that different kinds of elicitation strategies adopted by site leaders and facilitators played an important role in the ways youth constructed knowledge. These elicitation strategies also helped us find evidence of learning. Based on findings, future iterations of the curricula will involve tighter integration of social networking forums, continued use of videotaped interviews for data collection, an increased focus on training site leaders and facilitators in elicitation strategies, and more open-ended pencil and paper assessments in order to facilitate the process of looking for learning.

Item Order, Response Format, and Examinee Sex and Handedness and Performance on a Multiple-Choice Test.

ERIC Educational Resources Information Center

Kleinke, David J.

Four forms of a 36-item adaptation of the Stanford Achievement Test were administered to 484 fourth graders. External factors potentially influencing test performance were examined, namely: (1) item order (easy-to-difficult vs. uniform); (2) response location (left column vs. right column); (3) handedness which may interact with response location;…

Dividing the Force Concept Inventory into Two Equivalent Half-Length Tests

ERIC Educational Resources Information Center

Han, Jing; Bao, Lei; Chen, Li; Cai, Tianfang; Pi, Yuan; Zhou, Shaona; Tu, Yan; Koenig, Kathleen

2015-01-01

The Force Concept Inventory (FCI) is a 30-question multiple-choice assessment that has been a building block for much of the physics education research done today. In practice, there are often concerns regarding the length of the test and possible test-retest effects. Since many studies in the literature use the mean score of the FCI as the…

A Model-Based Method for Content Validation of Automatically Generated Test Items

ERIC Educational Resources Information Center

Zhang, Xinxin; Gierl, Mark

2016-01-01

The purpose of this study is to describe a methodology to recover the item model used to generate multiple-choice test items with a novel graph theory approach. Beginning with the generated test items and working backward to recover the original item model provides a model-based method for validating the content used to automatically generate test…

Development and Application of a Four-Tier Test to Assess Pre-Service Physics Teachers' Misconceptions about Geometrical Optics

ERIC Educational Resources Information Center

Kaltakci-Gurel, Derya; Eryilmaz, Ali; McDermott, Lillian Christie

2017-01-01

Background: Correct identification of misconceptions is an important first step in order to gain an understanding of student learning. More recently, four-tier multiple choice tests have been found to be effective in assessing misconceptions. Purpose: The purposes of this study are (1) to develop and validate a four-tier misconception test to…

[Development of critical thinking skill evaluation scale for nursing students].

PubMed

You, So Young; Kim, Nam Cho

2014-04-01

To develop a Critical Thinking Skill Test for Nursing Students. The construct concepts were drawn from a literature review and in-depth interviews with hospital nurses and surveys were conducted among students (n=607) from nursing colleges. The data were collected from September 13 to November 23, 2012 and analyzed using the SAS program, 9.2 version. The KR 20 coefficient for reliability, difficulty index, discrimination index, item-total correlation and known group technique for validity were performed. Four domains and 27 skills were identified and 35 multiple choice items were developed. Thirty multiple choice items which had scores higher than .80 on the content validity index were selected for the pre test. From the analysis of the pre test data, a modified 30 items were selected for the main test. In the main test, the KR 20 coefficient was .70 and Corrected Item-Total Correlations range was .11-.38. There was a statistically significant difference between two academic systems (p=.001). The developed instrument is the first critical thinking skill test reflecting nursing perspectives in hospital settings and is expected to be utilized as a tool which contributes to improvement of the critical thinking ability of nursing students.

The Australian Science Item Bank Project

ERIC Educational Resources Information Center

Kings, Clive B.; Cropley, Murray C.

1974-01-01

Describes the development of multiple-choice test item bank for grade ten science by the Australian Council for Educational Research. Other item banks are also being developed at the grade ten level in mathematics and social science. (RH)

Experimental design matters for statistical analysis: how to handle blocking.

PubMed

Jensen, Signe M; Schaarschmidt, Frank; Onofri, Andrea; Ritz, Christian

2018-03-01

Nowadays, evaluation of the effects of pesticides often relies on experimental designs that involve multiple concentrations of the pesticide of interest or multiple pesticides at specific comparable concentrations and, possibly, secondary factors of interest. Unfortunately, the experimental design is often more or less neglected when analysing data. Two data examples were analysed using different modelling strategies. First, in a randomized complete block design, mean heights of maize treated with a herbicide and one of several adjuvants were compared. Second, translocation of an insecticide applied to maize as a seed treatment was evaluated using incomplete data from an unbalanced design with several layers of hierarchical sampling. Extensive simulations were carried out to further substantiate the effects of different modelling strategies. It was shown that results from suboptimal approaches (two-sample t-tests and ordinary ANOVA assuming independent observations) may be both quantitatively and qualitatively different from the results obtained using an appropriate linear mixed model. The simulations demonstrated that the different approaches may lead to differences in coverage percentages of confidence intervals and type 1 error rates, confirming that misleading conclusions can easily happen when an inappropriate statistical approach is chosen. To ensure that experimental data are summarized appropriately, avoiding misleading conclusions, the experimental design should duly be reflected in the choice of statistical approaches and models. We recommend that author guidelines should explicitly point out that authors need to indicate how the statistical analysis reflects the experimental design. © 2017 Society of Chemical Industry. © 2017 Society of Chemical Industry.

THE DEVELOPMENT OF A STANDARDIZED ACHIEVEMENT TEST FOR SMALL GASOLINE ENGINE INSTRUCTION.

ERIC Educational Resources Information Center

EBBERT, J. MARVIN

THE PURPOSE OF THE STUDY WAS TO DEVELOP A STANDARDIZED, MULTIPLE-CHOICE ACHIEVEMENT TEST ON THE OPERATION, CARE, AND MAINTENANCE OF SMALL GASOLINE ENGINES. OBJECTIVES AND A UNIT OUTLINE WERE DEVELOPED WITH THE COOPERATION OF 75 INDIANA VOCATIONAL AGRICULTURE TEACHERS. A PANEL SUGGESTED MODIFICATIONS, AND THE REFINED OBJECTIVES AND OUTLINE WERE…

Regulating Accuracy on University Tests with the Plurality Option

ERIC Educational Resources Information Center

Higham, Philip A.

2013-01-01

A single experiment is reported in which introductory psychology students were administered a multiple-choice test on psychology with either 4 (n = 78) or 5 alternatives (n = 92) prior to any lectures being delivered. Two answers were generated for each question: a small answer consisting of their favorite alternative, and a large answer…

"Making the Difficult Choice": Understanding Georgia's Test-Based Grade Retention Policy in Reading

ERIC Educational Resources Information Center

Huddleston, Andrew P.

2015-01-01

The author uses Bourdieu's concepts of field, capital, and habitus to analyze how students, parents, teachers, and administrators are responding to Georgia's test-based grade retention policy in reading at one Georgia elementary school. In this multiple case study, the author interviewed, observed, and collected documents regarding ten fifth…

Components of Spatial Thinking: Evidence from a Spatial Thinking Ability Test

ERIC Educational Resources Information Center

Lee, Jongwon; Bednarz, Robert

2012-01-01

This article introduces the development and validation of the spatial thinking ability test (STAT). The STAT consists of sixteen multiple-choice questions of eight types. The STAT was validated by administering it to a sample of 532 junior high, high school, and university students. Factor analysis using principal components extraction was applied…

Observed Hierarchy of Student Proficiency with Period, Frequency, and Angular Frequency

ERIC Educational Resources Information Center

Young, Nicholas T.; Heckler, Andrew F.

2018-01-01

In the context of a generic harmonic oscillator, we investigated students' accuracy in determining the period, frequency, and angular frequency from mathematical and graphical representations. In a series of studies including interviews, free response tests, and multiple-choice tests developed in an iterative process, we assessed students in both…

Components of Answers to Multiple-Choice Questions on a Published Reading Comprehension Test: An Application of the Hanna-Oaster Approach.

ERIC Educational Resources Information Center

Entin, Eileen B.; Klare, George B.

1980-01-01

An approach to assessing context dependence was applied to data from the Nelson-Denny Reading Test. The results suggest that scores on the difficult passages are inflated because the examinees can answer the questions without having to comprehend the passage. (MKM)

Comment on 3PL IRT Adjustment for Guessing

ERIC Educational Resources Information Center

Chiu, Ting-Wei; Camilli, Gregory

2013-01-01

Guessing behavior is an issue discussed widely with regard to multiple choice tests. Its primary effect is on number-correct scores for examinees at lower levels of proficiency. This is a systematic error or bias, which increases observed test scores. Guessing also can inflate random error variance. Correction or adjustment for guessing formulas…

Accountability Is More than a Test Score

ERIC Educational Resources Information Center

Turnipseed, Stephan; Darling-Hammond, Linda

2015-01-01

The number one quality business leaders look for in employees is creativity and yet the U.S. education system undermines the development of the higher-order skills that promote creativity by its dogged focus on multiple-choice tests. Stephan Turnipseed and Linda DarlingHammond discuss the kind of rich accountability system that will help students…

Language Learner Strategy by Chinese-Speaking EFL Readers When Comprehending Familiar and Unfamiliar Texts

ERIC Educational Resources Information Center

Lee, Jia-Ying

2015-01-01

This study compares the language learner and test-taking strategies used by Chinese-speaking graduate students when confronted with familiar versus unfamiliar topics in an English multiple-choice format reading comprehension test. Thirty-six participants at a large mid-western university performed three tasks: A content knowledge vocabulary…

Missouri Assessment Program (MAP), Spring 2000: Elementary Health/Physical Education, Released Items, Grade 5.

ERIC Educational Resources Information Center

Missouri State Dept. of Elementary and Secondary Education, Jefferson City.

This document presents 10 released items from the Health/Physical Education Missouri Assessment Program (MAP) test given in the spring of 2000 to fifth graders. Items from the test sessions include: selected-response (multiple choice), constructed-response, and a performance event. The selected-response items consist of individual questions…

Single-Word Intelligibility in Speakers with Repaired Cleft Palate

ERIC Educational Resources Information Center

Whitehill, Tara; Chau, Cynthia

2004-01-01

Many speakers with repaired cleft palate have reduced intelligibility, but there are limitations with current procedures for assessing intelligibility. The aim of this study was to construct a single-word intelligibility test for speakers with cleft palate. The test used a multiple-choice identification format, and was based on phonetic contrasts…

Assessment of Foundation Knowledge: Are Students Confident in Their Ability?

ERIC Educational Resources Information Center

Fenna, Doug S.

2004-01-01

Multiple-choice testing (MCT) has several advantages which are becoming more relevant in the current financial climate. In particular, they can be machine marked. As an objective testing method it is particularly relevant to engineering and other factual courses, but MCTs are not widely used in engineering because students can benefit from…

Students' Achievement in Relation to Reasoning Ability, Prior Knowledge and Gender

ERIC Educational Resources Information Center

Yenilmez, Ayse; Sungur, Semra; Tekkaya, Ceren

2006-01-01

This study investigated students' achievement regarding photosynthesis and respiration in plants in relation to reasoning ability, prior knowledge and gender. A total of 117 eighth-grade students participated in the study. Test of logical thinking and the two-tier multiple choice tests were administered to determine students' reasoning ability and…

Advanced Marketing Core Curriculum. Test Items and Assessment Techniques.

ERIC Educational Resources Information Center

Smith, Clifton L.; And Others

This document contains duties and tasks, multiple-choice test items, and other assessment techniques for Missouri's advanced marketing core curriculum. The core curriculum begins with a list of 13 suggested textbook resources. Next, nine duties with their associated tasks are given. Under each task appears one or more citations to appropriate…

The None-of-the-Above Option: An Empirical Study.

ERIC Educational Resources Information Center

Frary, Robert B.

1991-01-01

The use of the "none-of-the-above" option (NOTA) in 20 college-level multiple-choice tests was evaluated for classes with 100 or more students. Eight academic disciplines were represented, and 295 NOTA and 724 regular test items were used. It appears that the NOTA can be compatible with good classroom measurement. (TJH)

Project Physics Tests 2, Motion in the Heavens.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 2 are presented in this booklet. Included are 70 multiple-choice and 22 problem-and-essay questions. Concepts of motion in the heavens are examined for planetary motions, heliocentric theory, forces exerted on the planets, Kepler's laws, gravitational force, Galileo's work, satellite orbits, Jupiter's…

Potential Values of Incorporating a Multiple-Choice Question Construction in Physics Experimentation Instruction

NASA Astrophysics Data System (ADS)

Yu, Fu-Yun; Liu, Yu-Hsin

2005-09-01

The potential value of a multiple-choice question-construction instructional strategy for the support of students’ learning of physics experiments was examined in the study. Forty-two university freshmen participated in the study for a whole semester. A constant comparison method adopted to categorize students’ qualitative data indicated that the influences of multiple-choice question construction were evident in several significant ways (promoting constructive and productive studying habits; reflecting and previewing course-related materials; increasing in-group communication and interaction; breaking passive learning style and habits, etc.), which, worked together, not only enhanced students’ comprehension and retention of the obtained knowledge, but also helped distil a sense of empowerment and learning community within the participants. Analysis with one-group t-tests, using 3 as the expected mean, on quantitative data further found that students’ satisfaction toward past learning experience, and perceptions toward this strategy’s potentials for promoting learning were statistically significant at the 0.0005 level, while learning anxiety was not statistically significant. Suggestions for incorporating question-generation activities within classroom and topics for future studies were rendered.

A Multiple-Choice Task with Changes of Mind

PubMed Central

Albantakis, Larissa; Branzi, Francesca M.; Costa, Albert; Deco, Gustavo

2012-01-01

The role of changes of mind and multiple choices has recently received increased attention in the study of perceptual decision-making. Previously, these extensions to standard two-alternative tasks have been studied separately. Here we explored how changes of mind depend on the number of choice-alternatives. To this end, we tested 14 human subjects on a 2- and 4-alternative direction-discrimination task. Changes of mind in the participants' movement trajectories could be observed for two and for four choice alternatives. With fewer alternatives, participants responded faster and more accurately. The frequency of changes of mind, however, did not significantly differ for the different numbers of choice alternatives. Nevertheless, mind-changing improved the participants' final performance, particularly for intermediate difficulty levels, in both experimental conditions. Moreover, the mean reaction times of individual participants were negatively correlated with their overall tendency to make changes of mind. We further reproduced these findings with a multi-alternative attractor model for decision-making, while a simple race model could not account for the experimental data. Our experiment, combined with the theoretical models allowed us to shed light on: (1) the differences in choice behavior between two and four alternatives, (2) the differences between the data of our human subjects and previous monkey data, (3) individual differences between participants, and (4) the inhibitory interaction between neural representations of choice alternatives. PMID:22916216

A phantom design and assessment of lesion detectability in PET imaging

NASA Astrophysics Data System (ADS)

Wollenweber, Scott D.; Kinahan, Paul E.; Alessio, Adam M.

2017-03-01

The early detection of abnormal regions with increased tracer uptake in positron emission tomography (PET) is a key driver of imaging system design and optimization as well as choice of imaging protocols. Detectability, however, remains difficult to assess due to the need for realistic objects mimicking the clinical scene, multiple lesion-present and lesion-absent images and multiple observers. Fillable phantoms, with tradeoffs between complexity and utility, provide a means to quantitatively test and compare imaging systems under truth-known conditions. These phantoms, however, often focus on quantification rather than detectability. This work presents extensions to a novel phantom design and analysis techniques to evaluate detectability in the context of realistic, non-piecewise constant backgrounds. The design consists of a phantom filled with small solid plastic balls and a radionuclide solution to mimic heterogeneous background uptake. A set of 3D-printed regular dodecahedral `features' were included at user-defined locations within the phantom to create `holes' within the matrix of chaotically-packed balls. These features fill at approximately 3:1 contrast to the lumpy background. A series of signal-known-present (SP) and signal-known-absent (SA) sub-images were generated and used as input for observer studies. This design was imaged in a head-like 20 cm diameter, 20 cm long cylinder and in a body-like 36 cm wide by 21 cm tall by 40 cm long tank. A series of model observer detectability indices were compared across scan conditions (count levels, number of scan replicates), PET image reconstruction methods (with/without TOF and PSF) and between PET/CT scanner system designs using the same phantom imaged on multiple systems. The detectability index was further compared to the noise-equivalent count (NEC) level to characterize the relationship between NEC and observer SNR.

Participatory Design of Mass Health Communication in Three Languages for Seniors and People With Disabilities on Medicaid

PubMed Central

Rothschild, Beccah; Graham, Carrie; Ivey, Susan L.; Konishi, Susana

2009-01-01

Objectives. We used participatory design methods to develop and test guidebooks about health care choices intended for 600 000 English-, Spanish-, and Chinese-speaking seniors and people with disabilities receiving Medicaid in California. Methods. Design and testing processes were conducted with consumers and professionals; they included 24 advisory group interviews, 36 usability tests, 18 focus groups (105 participants), 51 key informant interviews, guidebook readability and suitability testing, linguistic adaptation, and iterative revisions of 4 prototypes. Results. Participatory design processes identified preferences of intended audiences for guidebook content, linguistic adaptation, and format; guidebook readability was scored at the sixth- to eighth-grade level and suitability at 95%. These findings informed the design of a separate efficacy study that showed high guidebook usage and satisfaction, and better gains in knowledge, confidence, and intended behaviors among intervention participants than among control participants. Conclusions. Participatory design can be used effectively in mass communication to inform vulnerable audiences of health care choices. The techniques described can be adapted for a broad range of health communication interventions. PMID:19833990

Participatory design of mass health communication in three languages for seniors and people with disabilities on Medicaid.

PubMed

Neuhauser, Linda; Rothschild, Beccah; Graham, Carrie; Ivey, Susan L; Konishi, Susana

2009-12-01

We used participatory design methods to develop and test guidebooks about health care choices intended for 600 000 English-, Spanish-, and Chinese-speaking seniors and people with disabilities receiving Medicaid in California. Design and testing processes were conducted with consumers and professionals; they included 24 advisory group interviews, 36 usability tests, 18 focus groups (105 participants), 51 key informant interviews, guidebook readability and suitability testing, linguistic adaptation, and iterative revisions of 4 prototypes. Participatory design processes identified preferences of intended audiences for guidebook content, linguistic adaptation, and format; guidebook readability was scored at the sixth- to eighth-grade level and suitability at 95%. These findings informed the design of a separate efficacy study that showed high guidebook usage and satisfaction, and better gains in knowledge, confidence, and intended behaviors among intervention participants than among control participants. Participatory design can be used effectively in mass communication to inform vulnerable audiences of health care choices. The techniques described can be adapted for a broad range of health communication interventions.

Visual associations to retrieve episodic memory across healthy elderly, mild cognitive impairment, and patients with Alzheimer's disease.

PubMed

Meyer, Sascha R A; De Jonghe, Jos F M; Schmand, Ben; Ponds, Rudolf W H M

2018-05-16

Episodic memory tests need to determine the degree to which patients with moderate to severe memory deficits can still benefit from retrieval support. Especially in the case of Alzheimer's disease (AD), this may support health care to be more closely aligned with patients' memory capacities. We investigated whether the different measures of episodic memory of the Visual Association Test-Extended (VAT-E) can provide a more detailed and informative assessment on memory disturbances across a broad range of cognitive decline, from normal to severe impairment as seen in AD, by examining differences in floor effects. The VAT-E consists of 24 pairs of black-and-white line drawings. In a within-group design, we compared score distributions of VAT-E subtests in healthy elderly controls, mild cognitive impairment (MCI), and AD (n = 144), as well as in relation to global cognitive impairment. Paired associate recall showed a floor effect in 41% of MCI patients and 62% of AD patients. Free recall showed a floor effect in 73% of MCI patients and 84% of AD patients. Multiple-choice cued recognition did not show a floor effect in either of the patient groups. We conclude that the VAT-E covers a broad range of episodic memory decline in patients. As expected, paired associate recall was of intermediate difficulty, free recall was most difficult, and multiple-choice cued recognition was least difficult for patients. These varying levels of difficulty enable a more accurate determination of the level of retrieval support that can still benefit patients across a broad range of cognitive decline.

Nurses' knowledge of evidence-based guidelines for prevention of ventilator-associated pneumonia in critical care areas: a pre and post test design.

PubMed

Meherali, Salima Moez; Parpio, Yasmin; Ali, Tazeen S; Javed, Fawad

2011-01-01

Ventilator associated pneumonia (VAP) is a common hospital acquired pneumonia in ventilated patients. VAP is associated with increased morbidity, mortality duration of hospitalization and cost of treatment. Critical care nurses are usually unaware of evidence based preventive guidelines for VAP, resulting in negative impact on all aspects of patient care. This study investigated the impact of a 5-hour teaching module on nurses' knowledge to practice evidence based guidelines for the prevention of VAP. This study was conducted at a private tertiary care teaching hospital in Karachi, Pakistan. Single group pre-test design was used. Forty nurses were included in the study. The knowledge of nurses was assessed before, immediately after and 4 weeks after the intervention. The final sample (n=40) was selected on the basis of the set inclusion criteria. The demographic data sheet was used to collect relevant information about the participants. Knowledge was assessed through a self-developed validated tool, consisting of multiple choice questions. The difference in knowledge was analysed through repeated measures of analysis of variance. The mean scores at 3 time points were compared using the Tukey's multiple comparison procedure. Knowledge scores of participants increased significantly after the educational intervention in the first post-test; however, there was a decline in the score in post-test 2. the 5-hour teaching module significantly enhanced nurses' knowledge towards evidence based guidelines for the prevention of VAP. Further research is needed to assess the impact of training on nursing practice and to explore factors affecting attitudinal change.

The Test of Basic Mechanics Conceptual Understanding (bMCU): Using Rasch Analysis to Develop and Evaluate an Efficient Multiple Choice Test on Newton's Mechanics

ERIC Educational Resources Information Center

Hofer, Sarah I.; Schumacher, Ralph; Rubin, Herbert

2017-01-01

Background: Valid assessment of the understanding of Newton's mechanics is highly relevant to both physics classrooms and research. Several tests have been developed. What remains missing, however, is an efficient and fair test of conceptual understanding that is adapted to the content taught to secondary school students and that can be validly…

Proceedings of the Mobile Satellite System Architectures and Multiple Access Techniques Workshop

NASA Technical Reports Server (NTRS)

Dessouky, Khaled

1989-01-01

The Mobile Satellite System Architectures and Multiple Access Techniques Workshop served as a forum for the debate of system and network architecture issues. Particular emphasis was on those issues relating to the choice of multiple access technique(s) for the Mobile Satellite Service (MSS). These proceedings contain articles that expand upon the 12 presentations given in the workshop. Contrasting views on Frequency Division Multiple Access (FDMA), Code Division Multiple Access (CDMA), and Time Division Multiple Access (TDMA)-based architectures are presented, and system issues relating to signaling, spacecraft design, and network management constraints are addressed. An overview article that summarizes the issues raised in the numerous discussion periods of the workshop is also included.

Advanced Placement Economics. Macroeconomics: Student Activities.

ERIC Educational Resources Information Center

Morton, John S.

This book is designed to help advanced placement students better understand macroeconomic concepts through various activities. The book contains 6 units with 64 activities, sample multiple-choice questions, sample short essay questions, and sample long essay questions. The units are entitled: (1) "Basic Economic Concepts"; (2) "Measuring Economic…

Student Questionnaire. [Harvard Project Physics

ERIC Educational Resources Information Center

Welch, Wayne W.; Ahlgren, Andrew

This 60-item questionnaire was designed to gather general background information from students who had used the Harvard Project Physics curriculum. The instrument includes three 20-item subscales: (1) attitude toward physics, (2) career interest, and (3) student characteristics. Items are multiple choice (5 options), and the introductory material…

QTest: Quantitative Testing of Theories of Binary Choice.

PubMed

Regenwetter, Michel; Davis-Stober, Clintin P; Lim, Shiau Hong; Guo, Ying; Popova, Anna; Zwilling, Chris; Cha, Yun-Shil; Messner, William

2014-01-01

The goal of this paper is to make modeling and quantitative testing accessible to behavioral decision researchers interested in substantive questions. We provide a novel, rigorous, yet very general, quantitative diagnostic framework for testing theories of binary choice. This permits the nontechnical scholar to proceed far beyond traditionally rather superficial methods of analysis, and it permits the quantitatively savvy scholar to triage theoretical proposals before investing effort into complex and specialized quantitative analyses. Our theoretical framework links static algebraic decision theory with observed variability in behavioral binary choice data. The paper is supplemented with a custom-designed public-domain statistical analysis package, the QTest software. We illustrate our approach with a quantitative analysis using published laboratory data, including tests of novel versions of "Random Cumulative Prospect Theory." A major asset of the approach is the potential to distinguish decision makers who have a fixed preference and commit errors in observed choices from decision makers who waver in their preferences.

A labelled discrete choice experiment adds realism to the choices presented: preferences for surveillance tests for Barrett esophagus

PubMed Central

2009-01-01

Background Discrete choice experiments (DCEs) allow systematic assessment of preferences by asking respondents to choose between scenarios. We conducted a labelled discrete choice experiment with realistic choices to investigate patients' trade-offs between the expected health gains and the burden of testing in surveillance of Barrett esophagus (BE). Methods Fifteen choice scenarios were selected based on 2 attributes: 1) type of test (endoscopy and two less burdensome fictitious tests), 2) frequency of surveillance. Each test-frequency combination was associated with its own realistic decrease in risk of dying from esophageal adenocarcinoma. A conditional logit model was fitted. Results Of 297 eligible patients (155 BE and 142 with non-specific upper GI symptoms), 247 completed the questionnaire (84%). Patients preferred surveillance to no surveillance. Current surveillance schemes of once every 1–2 years were amongst the most preferred alternatives. Higher health gains were preferred over those with lower health gains, except when test frequencies exceeded once a year. For similar health gains, patients preferred video-capsule over saliva swab and least preferred endoscopy. Conclusion This first example of a labelled DCE using realistic scenarios in a healthcare context shows that such experiments are feasible. A comparison of labelled and unlabelled designs taking into account setting and research question is recommended. PMID:19454022

User-Centered Design for Developing Interventions to Improve Clinician Recommendation of Human Papillomavirus Vaccination

PubMed Central

Henninger, Michelle L; McMullen, Carmit K; Firemark, Alison J; Naleway, Allison L; Henrikson, Nora B; Turcotte, Joseph A

2017-01-01

Introduction Human papillomavirus (HPV) is the most common sexually transmitted infection in the US and is associated with multiple types of cancer. Although effective HPV vaccines have been available since 2006, coverage rates in the US remain much lower than with other adolescent vaccinations. Prior research has shown that a strong recommendation from a clinician is a critical determinant in HPV vaccine uptake and coverage. However, few published studies to date have specifically addressed the issue of helping clinicians communicate more effectively with their patients about the HPV vaccine. Objective To develop one or more novel interventions for helping clinicians make strong and effective recommendations for HPV vaccination. Methods Using principles of user-centered design, we conducted qualitative interviews, interviews with persons from analogous industries, and a data synthesis workshop with multiple stakeholders. Results Five potential intervention strategies targeted at health care clinicians, youth, and their parents were developed. The two most popular choices to pursue were a values-based communication strategy and a puberty education workbook. Conclusion User-centered design is a useful strategy for developing potential interventions to improve the rate and success of clinicians recommending the HPV vaccine. Further research is needed to test the effectiveness and acceptability of these interventions in clinical settings. PMID:28898195

User-Centered Design for Developing Interventions to Improve Clinician Recommendation of Human Papillomavirus Vaccination.

PubMed

Henninger, Michelle L; Mcmullen, Carmit K; Firemark, Alison J; Naleway, Allison L; Henrikson, Nora B; Turcotte, Joseph A

2017-01-01

Human papillomavirus (HPV) is the most common sexually transmitted infection in the US and is associated with multiple types of cancer. Although effective HPV vaccines have been available since 2006, coverage rates in the US remain much lower than with other adolescent vaccinations. Prior research has shown that a strong recommendation from a clinician is a critical determinant in HPV vaccine uptake and coverage. However, few published studies to date have specifically addressed the issue of helping clinicians communicate more effectively with their patients about the HPV vaccine. To develop one or more novel interventions for helping clinicians make strong and effective recommendations for HPV vaccination. Using principles of user-centered design, we conducted qualitative interviews, interviews with persons from analogous industries, and a data synthesis workshop with multiple stakeholders. Five potential intervention strategies targeted at health care clinicians, youth, and their parents were developed. The two most popular choices to pursue were a values-based communication strategy and a puberty education workbook. User-centered design is a useful strategy for developing potential interventions to improve the rate and success of clinicians recommending the HPV vaccine. Further research is needed to test the effectiveness and acceptability of these interventions in clinical settings.

FAA Pilot Knowledge Tests: Learning or Rote Memorization?

NASA Technical Reports Server (NTRS)

Casner, Stephen M.; Jones, Karen M.; Puentes, Antonio; Irani, Homi

2004-01-01

The FAA pilot knowledge test is a multiple-choice assessment tool designed to measure the extent to which applicants for FAA pilot certificates and ratings have mastered a corpus of required aeronautical knowledge. All questions that appear on the test are drawn from a database of questions that is made available to the public. The FAA and others are concerned that releasing test questions may encourage students to focus their study on memorizing test questions. To investigate this concern, we created our own database of questions that differed from FAA questions in four different ways. Our first three question types were derived by modifying existing FAA questions: (1) rewording questions and answers; (2) shuffling answers; and (3) substituting different figures for problems that used figures. Our last question type posed a question about required knowledge for which no FAA question currently exists. Forty-eight student pilots completed one of two paper-and-pencil knowledge tests that contained a mix of these experimental questions. The results indicate significantly lower scores for some question types when compared to unaltered FAA questions to which participants had prior access.

A Three-Tier Diagnostic Test to Assess Pre-Service Teachers' Misconceptions about Global Warming, Greenhouse Effect, Ozone Layer Depletion, and Acid Rain

ERIC Educational Resources Information Center

Arslan, Harika Ozge; Cigdemoglu, Ceyhan; Moseley, Christine

2012-01-01

This study describes the development and validation of a three-tier multiple-choice diagnostic test, the atmosphere-related environmental problems diagnostic test (AREPDiT), to reveal common misconceptions of global warming (GW), greenhouse effect (GE), ozone layer depletion (OLD), and acid rain (AR). The development of a two-tier diagnostic test…

End-of-Course Multiple-Choice Test Results, 2008-09. Measuring Up. E&R Report No. 10.04

ERIC Educational Resources Information Center

McMillen, Brad

2010-01-01

End-of-Course (EOC) tests are given statewide in 10 courses typically taken in high school. Results for 2008-09 (and prior years, where available) are reported in terms of both average scale scores and the percentage of students scoring proficient. After the recent introduction of new EOC tests with higher standards, scores in WCPSS have begun to…