Can Multiple-Choice Testing Induce Desirable Difficulties? Evidence from the Laboratory and the Classroom.

PubMed

Bjork, Elizabeth Ligon; Soderstrom, Nicholas C; Little, Jeri L

2015-01-01

The term desirable difficulties (Bjork, 1994) refers to conditions of learning that, though often appearing to cause difficulties for the learner and to slow down the process of acquisition, actually improve long-term retention and transfer. One known desirable difficulty is testing (as compared with restudy), although typically it is tests that clearly involve retrieval--such as free and cued recall tests--that are thought to induce these learning benefits and not multiple-choice tests. Nonetheless, multiple-choice testing is ubiquitous in educational settings and many other high-stakes situations. In this article, we discuss research, in both the laboratory and the classroom, exploring whether multiple-choice testing can also be fashioned to promote the type of retrieval processes known to improve learning, and we speculate about the necessary properties that multiple-choice questions must possess, as well as the metacognitive strategy students need to use in answering such questions, to achieve this goal.

Assessing Multiple Choice Question (MCQ) Tests--A Mathematical Perspective

ERIC Educational Resources Information Center

Scharf, Eric M.; Baldwin, Lynne P.

2007-01-01

The reasoning behind popular methods for analysing the raw data generated by multiple choice question (MCQ) tests is not always appreciated, occasionally with disastrous results. This article discusses and analyses three options for processing the raw data produced by MCQ tests. The article shows that one extreme option is not to penalize a…

Violating Conventional Wisdom in Multiple Choice Test Construction

ERIC Educational Resources Information Center

Taylor, Annette Kujawski

2005-01-01

This research examined 2 elements of multiple-choice test construction, balancing the key and optimal number of options. In Experiment 1 the 3 conditions included a balanced key, overrepresentation of a and b responses, and overrepresentation of c and d responses. The results showed that error-patterns were independent of the key, reflecting…

Multiple-choice tests stabilize access to marginal knowledge.

PubMed

Cantor, Allison D; Eslick, Andrea N; Marsh, Elizabeth J; Bjork, Robert A; Bjork, Elizabeth Ligon

2015-02-01

Marginal knowledge refers to knowledge that is stored in memory, but is not accessible at a given moment. For example, one might struggle to remember who wrote The Call of the Wild, even if that knowledge is stored in memory. Knowing how best to stabilize access to marginal knowledge is important, given that new learning often requires accessing and building on prior knowledge. While even a single opportunity to restudy marginal knowledge boosts its later accessibility (Berger, Hall, & Bahrick, 1999), in many situations explicit relearning opportunities are not available. Our question is whether multiple-choice tests (which by definition expose the learner to the correct answers) can also serve this function and, if so, how testing compares to restudying given that tests can be particularly powerful learning devices (Roediger & Karpicke, 2006). In four experiments, we found that multiple-choice testing had the power to stabilize access to marginal knowledge, and to do so for at least up to a week. Importantly, such tests did not need to be paired with feedback, although testing was no more powerful than studying. Overall, the results support the idea that one's knowledge base is unstable, with individual pieces of information coming in and out of reach. The present findings have implications for a key educational challenge: ensuring that students have continuing access to information they have learned.

Format of Options in Multiple Choice Test vis-a-vis Test Performance

ERIC Educational Resources Information Center

Bendulo, Hermabeth O.; Tibus, Erlinda D.; Bande, Rhodora A.; Oyzon, Voltaire Q.; Milla, Norberto E.; Macalinao, Myrna L.

2017-01-01

Testing or evaluation in an educational context is primarily used to measure or evaluate and authenticate the academic readiness, learning advancement, acquisition of skills, or instructional needs of learners. This study tried to determine whether the varied combinations of arrangements of options and letter cases in a Multiple-Choice Test (MCT)…

Students' Geographic Knowledge and Skills in Different Kinds of Tests: Multiple-Choice versus Performance Assessment.

ERIC Educational Resources Information Center

Kon, Jane Heckley; Martin-Kniep, Giselle O.

1992-01-01

Describes a case study to determine whether performance tests are a feasible alternative to multiple-choice tests. Examines the difficulties of administering and scoring performance assessments. Explains that the study employed three performance tests and one multiple-choice test. Concludes that performance test administration and scoring was no…

"None of the above" as a correct and incorrect alternative on a multiple-choice test: implications for the testing effect.

PubMed

Odegard, Timothy N; Koen, Joshua D

2007-11-01

Both positive and negative testing effects have been demonstrated with a variety of materials and paradigms (Roediger & Karpicke, 2006b). The present series of experiments replicate and extend the research of Roediger and Marsh (2005) with the addition of a "none-of-the-above" response option. Participants (n=32 in both experiments) read a set of passages, took an initial multiple-choice test, completed a filler task, and then completed a final cued-recall test (Experiment 1) or multiple-choice test (Experiment 2). Questions were manipulated on the initial multiple-choice test by adding a "none-of-the-above" response alternative (choice "E") that was incorrect ("E" Incorrect) or correct ("E" Correct). The results from both experiments demonstrated that the positive testing effect was negated when the "none-of-the-above" alternative was the correct response on the initial multiple-choice test, but was still present when the "none-of-the-above" alternative was an incorrect response.

Developing, Analyzing, and Using Distractors for Multiple-Choice Tests in Education: A Comprehensive Review

ERIC Educational Resources Information Center

Gierl, Mark J.; Bulut, Okan; Guo, Qi; Zhang, Xinxin

2017-01-01

Multiple-choice testing is considered one of the most effective and enduring forms of educational assessment that remains in practice today. This study presents a comprehensive review of the literature on multiple-choice testing in education focused, specifically, on the development, analysis, and use of the incorrect options, which are also…

Accommodations for Multiple Choice Tests

ERIC Educational Resources Information Center

Trammell, Jack

2011-01-01

Students with learning or learning-related disabilities frequently struggle with multiple choice assessments due to difficulty discriminating between items, filtering out distracters, and framing a mental best answer. This Practice Brief suggests accommodations and strategies that disability service providers can utilize in conjunction with…

Valuing Assessment in Teacher Education - Multiple-Choice Competency Testing

ERIC Educational Resources Information Center

Martin, Dona L.; Itter, Diane

2014-01-01

When our focus is on assessment educators should work to value the nature of assessment. This paper presents a new approach to multiple-choice competency testing in mathematics education. The instrument discussed here reflects student competence, encourages self-regulatory learning behaviours and links content with current curriculum documents and…

Development of multiple choice pictorial test for measuring the dimensions of knowledge

NASA Astrophysics Data System (ADS)

Nahadi, Siswaningsih, Wiwi; Erna

2017-05-01

This study aims to develop a multiple choice pictorial test as a tool to measure dimension of knowledge in chemical equilibrium subject. The method used is Research and Development and validation that was conducted in the preliminary studies and model development. The product is multiple choice pictorial test. The test was developed by 22 items and tested to 64 high school students in XII grade. The quality of test was determined by value of validity, reliability, difficulty index, discrimination power, and distractor effectiveness. The validity of test was determined by CVR calculation using 8 validators (4 university teachers and 4 high school teachers) with average CVR value 0,89. The reliability of test has very high category with value 0,87. Discrimination power of items with a very good category is 32%, 59% as good category, and 20% as sufficient category. This test has a varying level of difficulty, item with difficult category is 23%, the medium category is 50%, and the easy category is 27%. The distractor effectiveness of items with a very poor category is 1%, poor category is 1%, medium category is 4%, good category is 39%, and very good category is 55%. The dimension of knowledge that was measured consist of factual knowledge, conceptual knowledge, and procedural knowledge. Based on the questionnaire, students responded quite well to the developed test and most of the students like this kind of multiple choice pictorial test that include picture as evaluation tool compared to the naration tests was dominated by text.

A Case Study on Multiple-Choice Testing in Anatomical Sciences

ERIC Educational Resources Information Center

Golda, Stephanie DuPont

2011-01-01

Objective testing techniques, such as multiple-choice examinations, are a widely accepted method of assessment in gross anatomy. In order to deter cheating on these types of examinations, instructors often design several versions of an examination to distribute. These versions usually involve the rearrangement of questions and their corresponding…

Does the Position of Response Options in Multiple-Choice Tests Matter?

ERIC Educational Resources Information Center

Hohensinn, Christine; Baghaei, Purya

2017-01-01

In large scale multiple-choice (MC) tests alternate forms of a test may be developed to prevent cheating by changing the order of items or by changing the position of the response options. The assumption is that since the content of the test forms are the same the order of items or the positions of the response options do not have any effect on…

Cognitive Diagnostic Models for Tests with Multiple-Choice and Constructed-Response Items

ERIC Educational Resources Information Center

Kuo, Bor-Chen; Chen, Chun-Hua; Yang, Chih-Wei; Mok, Magdalena Mo Ching

2016-01-01

Traditionally, teachers evaluate students' abilities via their total test scores. Recently, cognitive diagnostic models (CDMs) have begun to provide information about the presence or absence of students' skills or misconceptions. Nevertheless, CDMs are typically applied to tests with multiple-choice (MC) items, which provide less diagnostic…

Piloting a Polychotomous Partial-Credit Scoring Procedure in a Multiple-Choice Test

ERIC Educational Resources Information Center

Tsopanoglou, Antonios; Ypsilandis, George S.; Mouti, Anna

2014-01-01

Multiple-choice (MC) tests are frequently used to measure language competence because they are quick, economical and straightforward to score. While degrees of correctness have been investigated for partially correct responses in combined-response MC tests, degrees of incorrectness in distractors and the role they play in determining the…

Developing multiple-choices test items as tools for measuring the scientific-generic skills on solar system

NASA Astrophysics Data System (ADS)

Bhakti, Satria Seto; Samsudin, Achmad; Chandra, Didi Teguh; Siahaan, Parsaoran

2017-05-01

The aim of research is developing multiple-choices test items as tools for measuring the scientific of generic skills on solar system. To achieve the aim that the researchers used the ADDIE model consisting Of: Analyzing, Design, Development, Implementation, dan Evaluation, all of this as a method research. While The scientific of generic skills limited research to five indicator including: (1) indirect observation, (2) awareness of the scale, (3) inference logic, (4) a causal relation, and (5) mathematical modeling. The participants are 32 students at one of junior high schools in Bandung. The result shown that multiple-choices that are constructed test items have been declared valid by the expert validator, and after the tests show that the matter of developing multiple-choices test items be able to measuring the scientific of generic skills on solar system.

The Impact of Escape Alternative Position Change in Multiple-Choice Test on the Psychometric Properties of a Test and Its Items Parameters

ERIC Educational Resources Information Center

Hamadneh, Iyad Mohammed

2015-01-01

This study aimed at investigating the impact changing of escape alternative position in multiple-choice test on the psychometric properties of a test and it's items parameters (difficulty, discrimination & guessing), and estimation of examinee ability. To achieve the study objectives, a 4-alternative multiple choice type achievement test…

A Practical Methodology for the Systematic Development of Multiple Choice Tests.

ERIC Educational Resources Information Center

Blumberg, Phyllis; Felner, Joel

Using Guttman's facet design analysis, four parallel forms of a multiple-choice test were developed. A mapping sentence, logically representing the universe of content of a basic cardiology course, specified the facets of the course and the semantic structural units linking them. The facets were: cognitive processes, disease priority, specific…

Multiple-Choice Tests with Correction Allowed in Autism: An Excel Applet

ERIC Educational Resources Information Center

Martinez, Elisabetta Monari

2010-01-01

The valuation of academic achievements in students with severe language impairment is problematic if they also have difficulties in sustaining attention and in praxic skills. In severe autism all of these difficulties may occur together. Multiple-choice tests offer the advantage that simple praxic skills are required, allowing the tasks to be…

[Continuing medical education: how to write multiple choice questions].

PubMed

Soler Fernández, R; Méndez Díaz, C; Rodríguez García, E

2013-06-01

Evaluating professional competence in medicine is a difficult but indispensable task because it makes it possible to evaluate, at different times and from different perspectives, the extent to which the knowledge, skills, and values required for exercising the profession have been acquired. Tests based on multiple choice questions have been and continue to be among the most useful tools for objectively evaluating learning in medicine. When these tests are well designed and correctly used, they can stimulate learning and even measure higher cognitive skills. Designing a multiple choice test is a difficult task that requires knowledge of the material to be tested and of the methodology of test preparation as well as time to prepare the test. The aim of this article is to review what can be evaluated through multiple choice tests, the rules and guidelines that should be taken into account when writing multiple choice questions, the different formats that can be used, the most common errors in elaborating multiple choice tests, and how to analyze the results of the test to verify its quality. Copyright © 2012 SERAM. Published by Elsevier Espana. All rights reserved.

Multiple-choice pretesting potentiates learning of related information.

PubMed

Little, Jeri L; Bjork, Elizabeth Ligon

2016-10-01

Although the testing effect has received a substantial amount of empirical attention, such research has largely focused on the effects of tests given after study. The present research examines the effect of using tests prior to study (i.e., as pretests), focusing particularly on how pretesting influences the subsequent learning of information that is not itself pretested but that is related to the pretested information. In Experiment 1, we found that multiple-choice pretesting was better for the learning of such related information than was cued-recall pretesting or a pre-fact-study control condition. In Experiment 2, we found that the increased learning of non-pretested related information following multiple-choice testing could not be attributed to increased time allocated to that information during subsequent study. Last, in Experiment 3, we showed that the benefits of multiple-choice pretesting over cued-recall pretesting for the learning of related information persist over 48 hours, thus demonstrating the promise of multiple-choice pretesting to potentiate learning in educational contexts. A possible explanation for the observed benefits of multiple-choice pretesting for enhancing the effectiveness with which related nontested information is learned during subsequent study is discussed.

Making the Most of Multiple Choice

ERIC Educational Resources Information Center

Brookhart, Susan M.

2015-01-01

Multiple-choice questions draw criticism because many people perceive they test only recall or atomistic, surface-level objectives and do not require students to think. Although this can be the case, it does not have to be that way. Susan M. Brookhart suggests that multiple-choice questions are a useful part of any teacher's questioning repertoire…

Second Language Reading Topic Familiarity and Test Score: Test-Taking Strategies for Multiple-Choice Comprehension Questions

ERIC Educational Resources Information Center

Lee, Jia-Ying

2011-01-01

The main purpose of this study was to compare the strategies used by Chinese-speaking students when confronted with familiar versus unfamiliar topics in a multiple-choice format reading comprehension test. The focus was on describing what students do when they are taking reading comprehension tests by asking students to verbalize their thoughts.…

Effects of Mayfield's Four Questions (M4Q) on Nursing Students' Self-Efficacy and Multiple-Choice Test Scores

ERIC Educational Resources Information Center

Mayfield, Linda Riggs

2010-01-01

This study examined the effects of being taught the Mayfield's Four Questions multiple-choice test-taking strategy on the perceived self-efficacy and multiple-choice test scores of nursing students in a two-year associate degree program. Experimental and control groups were chosen by stratified random sampling. Subjects completed the 10-statement…

Comparing Assessments of Students' Knowledge by Computerized Open-Ended and Multiple-Choice Tests.

ERIC Educational Resources Information Center

Anbar, Michael

1991-01-01

Interactive computerized tests accepting unrestricted natural-language input were used to assess knowledge of clinical biophysics at the State University of New York at Buffalo. Comparison of responses to open-ended sequential questions and multiple-choice questions on the same material found the two formats test different aspects of competence.…

Multiple-Choice versus Constructed-Response Tests in the Assessment of Mathematics Computation Skills.

ERIC Educational Resources Information Center

Gadalla, Tahany M.

The equivalence of multiple-choice (MC) and constructed response (discrete) (CR-D) response formats as applied to mathematics computation at grade levels two to six was tested. The difference between total scores from the two response formats was tested for statistical significance, and the factor structure of items in both response formats was…

Application of a Multidimensional Nested Logit Model to Multiple-Choice Test Items

ERIC Educational Resources Information Center

Bolt, Daniel M.; Wollack, James A.; Suh, Youngsuk

2012-01-01

Nested logit models have been presented as an alternative to multinomial logistic models for multiple-choice test items (Suh and Bolt in "Psychometrika" 75:454-473, 2010) and possess a mathematical structure that naturally lends itself to evaluating the incremental information provided by attending to distractor selection in scoring. One potential…

"I Don't Know" and Multiple Choice Analysis of Pre- and Post-Tests

ERIC Educational Resources Information Center

Spears, Karen; Wilson, Mary

2010-01-01

Evaluation is an essential component of any Extension education program. One tool, the pre- and post-test, provides measurable evaluation data. Yet often the answer "I don't know" or all possible answers to a multiple choice question are not included in the repeated measure analysis. Because more than two answers are offered, the test of marginal…

Predictive Validity of a Multiple-Choice Test for Placement in a Community College

ERIC Educational Resources Information Center

Verbout, Mary F.

2013-01-01

Multiple-choice tests of punctuation and usage are used throughout the United States to assess the writing skills of new community college students in order to place them in either a basic writing course or first-year composition. To determine whether using the COMPASS Writing Test (CWT) is a valid placement at a community college, student test…

Set of Criteria for Efficiency of the Process Forming the Answers to Multiple-Choice Test Items

ERIC Educational Resources Information Center

Rybanov, Alexander Aleksandrovich

2013-01-01

Is offered the set of criteria for assessing efficiency of the process forming the answers to multiple-choice test items. To increase accuracy of computer-assisted testing results, it is suggested to assess dynamics of the process of forming the final answer using the following factors: loss of time factor and correct choice factor. The model…

Retrieval practice with short-answer, multiple-choice, and hybrid tests.

PubMed

Smith, Megan A; Karpicke, Jeffrey D

2014-01-01

Retrieval practice improves meaningful learning, and the most frequent way of implementing retrieval practice in classrooms is to have students answer questions. In four experiments (N=372) we investigated the effects of different question formats on learning. Students read educational texts and practised retrieval by answering short-answer, multiple-choice, or hybrid questions. In hybrid conditions students first attempted to recall answers in short-answer format, then identified answers in multiple-choice format. We measured learning 1 week later using a final assessment with two types of questions: those that could be answered by recalling information verbatim from the texts and those that required inferences. Practising retrieval in all format conditions enhanced retention, relative to a study-only control condition, on both verbatim and inference questions. However, there were little or no advantages of answering short-answer or hybrid format questions over multiple-choice questions in three experiments. In Experiment 4, when retrieval success was improved under initial short-answer conditions, there was an advantage of answering short-answer or hybrid questions over multiple-choice questions. The results challenge the simple conclusion that short-answer questions always produce the best learning, due to increased retrieval effort or difficulty, and demonstrate the importance of retrieval success for retrieval-based learning activities.

Sex Differences in the Tendency to Omit Items on Multiple-Choice Tests: 1980-2000

ERIC Educational Resources Information Center

von Schrader, Sarah; Ansley, Timothy

2006-01-01

Much has been written concerning the potential group differences in responding to multiple-choice achievement test items. This discussion has included references to possible disparities in tendency to omit such test items. When test scores are used for high-stakes decision making, even small differences in scores and rankings that arise from male…

The Relationship of Deep and Surface Study Approaches on Factual and Applied Test-Bank Multiple-Choice Question Performance

ERIC Educational Resources Information Center

Yonker, Julie E.

2011-01-01

With the advent of online test banks and large introductory classes, instructors have often turned to textbook publisher-generated multiple-choice question (MCQ) exams in their courses. Multiple-choice questions are often divided into categories of factual or applied, thereby implicating levels of cognitive processing. This investigation examined…

Quality Multiple-Choice Test Questions: Item-Writing Guidelines and an Analysis of Auditing Testbanks.

ERIC Educational Resources Information Center

Hansen, James D.; Dexter, Lee

1997-01-01

Analysis of test item banks in 10 auditing textbooks found that 75% of questions violated one or more guidelines for multiple-choice items. In comparison, 70% of a certified public accounting exam bank had no violations. (SK)

Test of understanding of vectors: A reliable multiple-choice vector concept test

NASA Astrophysics Data System (ADS)

Barniol, Pablo; Zavala, Genaro

2014-06-01

In this article we discuss the findings of our research on students' understanding of vector concepts in problems without physical context. First, we develop a complete taxonomy of the most frequent errors made by university students when learning vector concepts. This study is based on the results of several test administrations of open-ended problems in which a total of 2067 students participated. Using this taxonomy, we then designed a 20-item multiple-choice test [Test of understanding of vectors (TUV)] and administered it in English to 423 students who were completing the required sequence of introductory physics courses at a large private Mexican university. We evaluated the test's content validity, reliability, and discriminatory power. The results indicate that the TUV is a reliable assessment tool. We also conducted a detailed analysis of the students' understanding of the vector concepts evaluated in the test. The TUV is included in the Supplemental Material as a resource for other researchers studying vector learning, as well as instructors teaching the material.

Feedback-related brain activity predicts learning from feedback in multiple-choice testing.

PubMed

Ernst, Benjamin; Steinhauser, Marco

2012-06-01

Different event-related potentials (ERPs) have been shown to correlate with learning from feedback in decision-making tasks and with learning in explicit memory tasks. In the present study, we investigated which ERPs predict learning from corrective feedback in a multiple-choice test, which combines elements from both paradigms. Participants worked through sets of multiple-choice items of a Swahili-German vocabulary task. Whereas the initial presentation of an item required the participants to guess the answer, corrective feedback could be used to learn the correct response. Initial analyses revealed that corrective feedback elicited components related to reinforcement learning (FRN), as well as to explicit memory processing (P300) and attention (early frontal positivity). However, only the P300 and early frontal positivity were positively correlated with successful learning from corrective feedback, whereas the FRN was even larger when learning failed. These results suggest that learning from corrective feedback crucially relies on explicit memory processing and attentional orienting to corrective feedback, rather than on reinforcement learning.

The "None of the Above" Option in Multiple-Choice Testing: An Experimental Study

ERIC Educational Resources Information Center

DiBattista, David; Sinnige-Egger, Jo-Anne; Fortuna, Glenda

2014-01-01

The authors assessed the effects of using "none of the above" as an option in a 40-item, general-knowledge multiple-choice test administered to undergraduate students. Examinees who selected "none of the above" were given an incentive to write the correct answer to the question posed. Using "none of the above" as the…

How to Assess Student Performance in Science: Going beyond Multiple-Choice Tests. Third Edition

ERIC Educational Resources Information Center

Butler, Susan M.; McColskey, Wendy; O'Sullivan, Rita

2005-01-01

Educational systems promote student growth in a variety of dimensions. Basic content knowledge can be effectively assessed with multiple-choice and completion tests. However educational reforms have become more concerned with higher-order cognitive dimensions (problem-solving, creativity), social dimensions (communication skills, ability to work…

Teaching Composition Skills with Weekly Multiple Choice Tests in Lieu of Theme Writing. Final Report.

ERIC Educational Resources Information Center

Scannell, Dale P.; Haugh, Oscar M.

The purpose of the study was to compare the effectiveness with which composition skills could be taught by the traditional theme-assignment approach and by an experimental method using weekly multiple-choice composition tests in lieu of theme writing. The weekly tests were based on original but typical first-draft compositions and covered problems…

Post-Graduate Student Performance in "Supervised In-Class" vs. "Unsupervised Online" Multiple Choice Tests: Implications for Cheating and Test Security

ERIC Educational Resources Information Center

Ladyshewsky, Richard K.

2015-01-01

This research explores differences in multiple choice test (MCT) scores in a cohort of post-graduate students enrolled in a management and leadership course. A total of 250 students completed the MCT in either a supervised in-class paper and pencil test or an unsupervised online test. The only statistically significant difference between the nine…

Multiple-Choice Question Tests: A Convenient, Flexible and Effective Learning Tool? A Case Study

ERIC Educational Resources Information Center

Douglas, Mercedes; Wilson, Juliette; Ennis, Sean

2012-01-01

The research presented in this paper is part of a project investigating assessment practices, funded by the Scottish Funding Council. Using established principles of good assessment and feedback, the use of online formative and summative multiple choice tests (MCT's) was piloted to support independent and self-directed learning and improve…

Some Effects of Changes in Question Structure and Sequence on Performance in a Multiple Choice Chemistry Test.

ERIC Educational Resources Information Center

Hodson, D.

1984-01-01

Investigated the effect on student performance of changes in question structure and sequence on a GCE 0-level multiple-choice chemistry test. One finding noted is that there was virtually no change in test reliability on reducing the number of options (from five to per test item). (JN)

High time for a change: psychometric analysis of multiple-choice questions in nursing.

PubMed

Redmond, Sandra P; Hartigan-Rogers, Jackie A; Cobbett, Shelley

2012-11-26

Nurse educators teach students to develop an informed nursing practice but can educators claim the same grounding in the available evidence when formulating multiple-choice assessment tools to evaluate student learning? Multiple-choice questions are a popular assessment format within nursing education. While widely accepted as a credible format to assess student knowledge across disciplines, debate exists among educators regarding the number of options necessary to adequately test cognitive reasoning and optimal discrimination between student abilities. The purpose of this quasi-experimental between groups study was to examine the psychometric properties of three option multiple-choice questions when compared to the more traditional four option questions. Data analysis revealed that there were no statistically significant differences in the item discrimination, difficulty or the mean examination scores when multiple-choice test questions were administered with three versus four option answer choices. This study provides additional guidance for nurse educators to assist in improving multiple-choice question writing and test design.

Comparison of paragraph comprehension test scores with reading versus listening-reading and multiple-choice versus nominal recall administration techniques: justification for the bypass approach.

PubMed

Weinberg, W A; McLean, A; Snider, R L; Rintelmann, J W; Brumback, R A

1989-12-01

Eight groups of learning disabled children (N = 100), categorized by the clinical Lexical Paradigm as good readers or poor readers, were individually administered the Gilmore Oral Reading Test, Form D, by one of four input/retrieval methods: (1) the standardized method of administration in which the child reads each paragraph aloud and then answers five questions relating to the paragraph [read/recall method]; (2) the child reads each paragraph aloud and then for each question selects the correct answer from among three choices read by the examiner [read/choice method]; (3) the examiner reads each paragraph aloud and reads each of the five questions to the child to answer [listen/recall method]; and (4) the examiner reads each paragraph aloud and then for each question reads three multiple-choice answers from which the child selects the correct answer [listen/choice method]. The major difference in scores was between the groups tested by the recall versus the orally read multiple-choice methods. This study indicated that poor readers who listened to the material and were tested by orally read multiple-choice format could perform as well as good readers. The performance of good readers was not affected by listening or by the method of testing. The multiple-choice testing improved the performance of poor readers independent of the input method. This supports the arguments made previously that a "bypass approach" to education of poor readers in which testing is accomplished using an orally read multiple-choice format can enhance the child's school performance on reading-related tasks. Using a listening while reading input method may further enhance performance.

Score Increase and Partial-Credit Validity When Administering Multiple-Choice Tests Using an Answer-Until-Correct Format

ERIC Educational Resources Information Center

Slepkov, Aaron D.; Vreugdenhil, Andrew J.; Shiell, Ralph C.

2016-01-01

There are numerous benefits to answer-until-correct (AUC) approaches to multiple-choice testing, not the least of which is the straightforward allotment of partial credit. However, the benefits of granting partial credit can be tempered by the inevitable increase in test scores and by fears that such increases are further contaminated by a large…

Using Multiple-Choice Questions to Evaluate In-Depth Learning of Economics

ERIC Educational Resources Information Center

Buckles, Stephen; Siegfried, John J.

2006-01-01

Multiple-choice questions are the basis of a significant portion of assessment in introductory economics courses. However, these questions, as found in course assessments, test banks, and textbooks, often fail to evaluate students' abilities to use and apply economic analysis. The authors conclude that multiple-choice questions can be used to…

To Show or Not to Show: The Effects of Item Stems and Answer Options on Performance on a Multiple-Choice Listening Comprehension Test

ERIC Educational Resources Information Center

Yanagawa, Kozo; Green, Anthony

2008-01-01

The purpose of this study is to examine whether the choice between three multiple-choice listening comprehension test formats results in any difference in listening comprehension test performance. The three formats entail (a) allowing test takers to preview both the question stem and answer options prior to listening; (b) allowing test takers to…

Effect of differing PowerPoint slide design on multiple-choice test scores for assessment of knowledge and retention in a theriogenology course.

PubMed

Root Kustritz, Margaret V

2014-01-01

Third-year veterinary students in a required theriogenology diagnostics course were allowed to self-select attendance at a lecture in either the evening or the next morning. One group was presented with PowerPoint slides in a traditional format (T group), and the other group was presented with PowerPoint slides in the assertion-evidence format (A-E group), which uses a single sentence and a highly relevant graphic on each slide to ensure attention is drawn to the most important points in the presentation. Students took a multiple-choice pre-test, attended lecture, and then completed a take-home assignment. All students then completed an online multiple-choice post-test and, one month later, a different online multiple-choice test to evaluate retention. Groups did not differ on pre-test, assignment, or post-test scores, and both groups showed significant gains from pre-test to post-test and from pre-test to retention test. However, the T group showed significant decline from post-test to retention test, while the A-E group did not. Short-term differences between slide designs were most likely unaffected due to required coursework immediately after lecture, but retention of material was superior with the assertion-evidence slide design.

Demand Characteristics of Multiple-Choice Items.

ERIC Educational Resources Information Center

Diamond, James J.; Williams, David V.

Thirteen graduate students were asked to indicate for each of 24 multiple-choice items whether the item tested "recall of specific information," a "higher order skill," or "don't know." The students were also asked to state their general basis for judging the items. The 24 items had been previously classified according to Bloom's cognitive-skills…

Examining Two Strategies to Link Mixed-Format Tests Using Multiple-Choice Anchors. Research Report. ETS RR-10-18

ERIC Educational Resources Information Center

Walker, Michael E.; Kim, Sooyeon

2010-01-01

This study examined the use of an all multiple-choice (MC) anchor for linking mixed format tests containing both MC and constructed-response (CR) items, in a nonequivalent groups design. An MC-only anchor could effectively link two such test forms if either (a) the MC and CR portions of the test measured the same construct, so that the MC anchor…

Measuring the Consistency in Change in Hepatitis B Knowledge among Three Different Types of Tests: True/False, Multiple Choice, and Fill in the Blanks Tests.

ERIC Educational Resources Information Center

Sahai, Vic; Demeyere, Petra; Poirier, Sheila; Piro, Felice

1998-01-01

The recall of information about Hepatitis B demonstrated by 180 seventh graders was tested with three test types: (1) short-answer; (2) true/false; and (3) multiple-choice. Short answer testing was the most reliable. Suggestions are made for the use of short-answer tests in evaluating student knowledge. (SLD)

A Diagnostic Study of Pre-Service Teachers' Competency in Multiple-Choice Item Development

ERIC Educational Resources Information Center

Asim, Alice E.; Ekuri, Emmanuel E.; Eni, Eni I.

2013-01-01

Large class size is an issue in testing at all levels of Education. As a panacea to this, multiple choice test formats has become very popular. This case study was designed to diagnose pre-service teachers' competency in constructing questions (IQT); direct questions (DQT); and best answer (BAT) varieties of multiple choice items. Subjects were 88…

Gender and Ethnicity Differences in Multiple-Choice Testing. Effects of Self-Assessment and Risk-Taking Propensity

DTIC Science & Technology

1993-05-01

correctness of the response provides I some advantages. They are: i 1. Increased reliability of the test; 2. Examinees pay more attention to the multiple...their choice 3 of test date. Each sign up sheet was divided into four cells: Non-Hispanic males and females and Hispanic males and females. 3 I I I...certain prestige and financial rewards; or entering a conservatory of music for advanced training with a well-known pianist . Mr. H realizes that even

Further Support for Changing Multiple-Choice Answers.

ERIC Educational Resources Information Center

Fabrey, Lawrence J.; Case, Susan M.

1985-01-01

The effect on test scores of changing answers to multiple-choice questions was studied and compared to earlier research. The current setting was a nationally administered, in-training, specialty examination for medical residents in obstetrics and gynecology. Both low and high scorers improved their scores when they changed answers. (SW)

A Multiple Choice Version of the Sentence Completion Method

ERIC Educational Resources Information Center

Shouval, Ron; And Others

1975-01-01

It was concluded that a multiple choice form corresponding to a sentence completion measure, test clearly defined personality areas (such as autonomy) could be a reasonable alternative for many purposes. (Author/DEP)

Examining the Prediction of Reading Comprehension on Different Multiple-Choice Tests

ERIC Educational Resources Information Center

Andreassen, Rune; Braten, Ivar

2010-01-01

In this study, 180 Norwegian fifth-grade students with a mean age of 10.5 years were administered measures of word recognition skills, strategic text processing, reading motivation and working memory. Six months later, the same students were given three different multiple-choice reading comprehension measures. Based on three forced-order…

The Effects of Item Preview on Video-Based Multiple-Choice Listening Assessments

ERIC Educational Resources Information Center

Koyama, Dennis; Sun, Angela; Ockey, Gary J.

2016-01-01

Multiple-choice formats remain a popular design for assessing listening comprehension, yet no consensus has been reached on how multiple-choice formats should be employed. Some researchers argue that test takers must be provided with a preview of the items prior to the input (Buck, 1995; Sherman, 1997); others argue that a preview may decrease the…

Of Small Beauties and Large Beasts: The Quality of Distractors on Multiple-Choice Tests Is More Important than Their Quantity

ERIC Educational Resources Information Center

Papenberg, Martin; Musch, Jochen

2017-01-01

In multiple-choice tests, the quality of distractors may be more important than their number. We therefore examined the joint influence of distractor quality and quantity on test functioning by providing a sample of 5,793 participants with five parallel test sets consisting of items that differed in the number and quality of distractors.…

Decision making under internal uncertainty: the case of multiple-choice tests with different scoring rules.

PubMed

Bereby-Meyer, Yoella; Meyer, Joachim; Budescu, David V

2003-02-01

This paper assesses framing effects on decision making with internal uncertainty, i.e., partial knowledge, by focusing on examinees' behavior in multiple-choice (MC) tests with different scoring rules. In two experiments participants answered a general-knowledge MC test that consisted of 34 solvable and 6 unsolvable items. Experiment 1 studied two scoring rules involving Positive (only gains) and Negative (only losses) scores. Although answering all items was the dominating strategy for both rules, the results revealed a greater tendency to answer under the Negative scoring rule. These results are in line with the predictions derived from Prospect Theory (PT) [Econometrica 47 (1979) 263]. The second experiment studied two scoring rules, which allowed respondents to exhibit partial knowledge. Under the Inclusion-scoring rule the respondents mark all answers that could be correct, and under the Exclusion-scoring rule they exclude all answers that might be incorrect. As predicted by PT, respondents took more risks under the Inclusion rule than under the Exclusion rule. The results illustrate that the basic process that underlies choice behavior under internal uncertainty and especially the effect of framing is similar to the process of choice under external uncertainty and can be described quite accurately by PT. Copyright 2002 Elsevier Science B.V.

Multiple-Choice Testing Using Immediate Feedback--Assessment Technique (IF AT®) Forms: Second-Chance Guessing vs. Second-Chance Learning?

ERIC Educational Resources Information Center

Merrel, Jeremy D.; Cirillo, Pier F.; Schwartz, Pauline M.; Webb, Jeffrey A.

2015-01-01

Multiple choice testing is a common but often ineffective method for evaluating learning. A newer approach, however, using Immediate Feedback Assessment Technique (IF AT®, Epstein Educational Enterprise, Inc.) forms, offers several advantages. In particular, a student learns immediately if his or her answer is correct and, in the case of an…

The Effects of Clinically Relevant Multiple-Choice Items on the Statistical Discrimination of Physician Clinical Competence.

ERIC Educational Resources Information Center

Downing, Steven M.; Maatsch, Jack L.

To test the effect of clinically relevant multiple-choice item content on the validity of statistical discriminations of physicians' clinical competence, data were collected from a field test of the Emergency Medicine Examination, test items for the certification of specialists in emergency medicine. Two 91-item multiple-choice subscales were…

Effectiveness of Guided Multiple Choice Objective Questions Test on Students' Academic Achievement in Senior School Mathematics by School Location

ERIC Educational Resources Information Center

Igbojinwaekwu, Patrick Chukwuemeka

2015-01-01

This study investigated, using pretest-posttest quasi-experimental research design, the effectiveness of guided multiple choice objective questions test on students' academic achievement in Senior School Mathematics, by school location, in Delta State Capital Territory, Nigeria. The sample comprised 640 Students from four coeducation secondary…

A simple test of choice stepping reaction time for assessing fall risk in people with multiple sclerosis.

PubMed

Tijsma, Mylou; Vister, Eva; Hoang, Phu; Lord, Stephen R

2017-03-01

Purpose To determine (a) the discriminant validity for established fall risk factors and (b) the predictive validity for falls of a simple test of choice stepping reaction time (CSRT) in people with multiple sclerosis (MS). Method People with MS (n = 210, 21-74y) performed the CSRT, sensorimotor, balance and neuropsychological tests in a single session. They were then followed up for falls using monthly fall diaries for 6 months. Results The CSRT test had excellent discriminant validity with respect to established fall risk factors. Frequent fallers (≥3 falls) performed significantly worse in the CSRT test than non-frequent fallers (0-2 falls). With the odds of suffering frequent falls increasing 69% with each SD increase in CSRT (OR = 1.69, 95% CI: 1.27-2.26, p = <0.001). In regression analysis, CSRT was best explained by sway, time to complete the 9-Hole Peg test, knee extension strength of the weaker leg, proprioception and the time to complete the Trails B test (multiple R 2   =   0.449, p < 0.001). Conclusions A simple low tech CSRT test has excellent discriminative and predictive validity in relation to falls in people with MS. This test may prove useful in documenting longitudinal changes in fall risk in relation to MS disease progression and effects of interventions. Implications for rehabilitation Good choice stepping reaction time (CSRT) is required for maintaining balance. A simple low-tech CSRT test has excellent discriminative and predictive validity in relation to falls in people with MS. This test may prove useful documenting longitudinal changes in fall risk in relation to MS disease progression and effects of interventions.

Performance of Men and Women on Multiple-Choice and Constructed-Response Tests for Beginning Teachers. Research Report. ETS RR-04-48

ERIC Educational Resources Information Center

Livingston, Samuel A.; Rupp, Stacie L.

2004-01-01

Some previous research results imply that women tend to perform better, relative to men, on constructed-response (CR) tests than on multiple-choice (MC) tests in the same subjects. An analysis of data from several tests used in the licensing of beginning teachers supported this hypothesis, to varying degrees, in most of the tests investigated. The…

Dynamic Testing of Analogical Reasoning in 5- to 6-Year-Olds: Multiple-Choice versus Constructed-Response Training Items

ERIC Educational Resources Information Center

Stevenson, Claire E.; Heiser, Willem J.; Resing, Wilma C. M.

2016-01-01

Multiple-choice (MC) analogy items are often used in cognitive assessment. However, in dynamic testing, where the aim is to provide insight into potential for learning and the learning process, constructed-response (CR) items may be of benefit. This study investigated whether training with CR or MC items leads to differences in the strategy…

Student certainty answering misconception question: study of Three-Tier Multiple-Choice Diagnostic Test in Acid-Base and Solubility Equilibrium

NASA Astrophysics Data System (ADS)

Ardiansah; Masykuri, M.; Rahardjo, S. B.

2018-04-01

Students’ concept comprehension in three-tier multiple-choice diagnostic test related to student confidence level. The confidence level related to certainty and student’s self-efficacy. The purpose of this research was to find out students’ certainty in misconception test. This research was quantitative-qualitative research method counting students’ confidence level. The research participants were 484 students that were studying acid-base and equilibrium solubility subject. Data was collected using three-tier multiple-choice (3TMC) with thirty questions and students’ questionnaire. The findings showed that #6 item gives the highest misconception percentage and high student confidence about the counting of ultra-dilute solution’s pH. Other findings were that 1) the student tendency chosen the misconception answer is to increase over item number, 2) student certainty decreased in terms of answering the 3TMC, and 3) student self-efficacy and achievement were related each other in the research. The findings suggest some implications and limitations for further research.

Preliminary Findings on the Computer-Administered Multiple-Choice Online Causal Comprehension Assessment, a Diagnostic Reading Comprehension Test

ERIC Educational Resources Information Center

Davison, Mark L.; Biancarosa, Gina; Carlson, Sarah E.; Seipel, Ben; Liu, Bowen

2018-01-01

The computer-administered Multiple-Choice Online Causal Comprehension Assessment (MOCCA) for Grades 3 to 5 has an innovative, 40-item multiple-choice structure in which each distractor corresponds to a comprehension process upon which poor comprehenders have been shown to rely. This structure requires revised thinking about measurement issues…

Pick-N Multiple Choice-Exams: A Comparison of Scoring Algorithms

ERIC Educational Resources Information Center

Bauer, Daniel; Holzer, Matthias; Kopp, Veronika; Fischer, Martin R.

2011-01-01

To compare different scoring algorithms for Pick-N multiple correct answer multiple-choice (MC) exams regarding test reliability, student performance, total item discrimination and item difficulty. Data from six 3rd year medical students' end of term exams in internal medicine from 2005 to 2008 at Munich University were analysed (1,255 students,…

Force Concept Inventory-based multiple-choice test for investigating students' representational consistency

NASA Astrophysics Data System (ADS)

Nieminen, Pasi; Savinainen, Antti; Viiri, Jouni

2010-07-01

This study investigates students’ ability to interpret multiple representations consistently (i.e., representational consistency) in the context of the force concept. For this purpose we developed the Representational Variant of the Force Concept Inventory (R-FCI), which makes use of nine items from the 1995 version of the Force Concept Inventory (FCI). These original FCI items were redesigned using various representations (such as motion map, vectorial and graphical), yielding 27 multiple-choice items concerning four central concepts underpinning the force concept: Newton’s first, second, and third laws, and gravitation. We provide some evidence for the validity and reliability of the R-FCI; this analysis is limited to the student population of one Finnish high school. The students took the R-FCI at the beginning and at the end of their first high school physics course. We found that students’ (n=168) representational consistency (whether scientifically correct or not) varied considerably depending on the concept. On average, representational consistency and scientifically correct understanding increased during the instruction, although in the post-test only a few students performed consistently both in terms of representations and scientifically correct understanding. We also compared students’ (n=87) results of the R-FCI and the FCI, and found that they correlated quite well.

Effect of response format on cognitive reflection: Validating a two- and four-option multiple choice question version of the Cognitive Reflection Test.

PubMed

Sirota, Miroslav; Juanchich, Marie

2018-03-27

The Cognitive Reflection Test, measuring intuition inhibition and cognitive reflection, has become extremely popular because it reliably predicts reasoning performance, decision-making, and beliefs. Across studies, the response format of CRT items sometimes differs, based on the assumed construct equivalence of tests with open-ended versus multiple-choice items (the equivalence hypothesis). Evidence and theoretical reasons, however, suggest that the cognitive processes measured by these response formats and their associated performances might differ (the nonequivalence hypothesis). We tested the two hypotheses experimentally by assessing the performance in tests with different response formats and by comparing their predictive and construct validity. In a between-subjects experiment (n = 452), participants answered stem-equivalent CRT items in an open-ended, a two-option, or a four-option response format and then completed tasks on belief bias, denominator neglect, and paranormal beliefs (benchmark indicators of predictive validity), as well as on actively open-minded thinking and numeracy (benchmark indicators of construct validity). We found no significant differences between the three response formats in the numbers of correct responses, the numbers of intuitive responses (with the exception of the two-option version, which had a higher number than the other tests), and the correlational patterns of the indicators of predictive and construct validity. All three test versions were similarly reliable, but the multiple-choice formats were completed more quickly. We speculate that the specific nature of the CRT items helps build construct equivalence among the different response formats. We recommend using the validated multiple-choice version of the CRT presented here, particularly the four-option CRT, for practical and methodological reasons. Supplementary materials and data are available at https://osf.io/mzhyc/ .

All of the above: When multiple correct response options enhance the testing effect.

PubMed

Bishara, Anthony J; Lanzo, Lauren A

2015-01-01

Previous research has shown that multiple choice tests often improve memory retention. However, the presence of incorrect lures often attenuates this memory benefit. The current research examined the effects of "all of the above" (AOTA) options. When such options are correct, no incorrect lures are present. In the first three experiments, a correct AOTA option on an initial test led to a larger memory benefit than no test and standard multiple choice test conditions. The benefits of a correct AOTA option occurred even without feedback on the initial test; for both 5-minute and 48-hour retention delays; and for both cued recall and multiple choice final test formats. In the final experiment, an AOTA question led to better memory retention than did a control condition that had identical timing and exposure to response options. However, the benefits relative to this control condition were similar regardless of the type of multiple choice test (AOTA or not). Results suggest that retrieval contributes to multiple choice testing effects. However, the extra testing effect from a correct AOTA option, rather than being due to more retrieval, might be due simply to more exposure to correct information.

Using a Classroom Response System to Improve Multiple-Choice Performance in AP® Physics

NASA Astrophysics Data System (ADS)

Bertrand, Peggy

2009-04-01

Participation in rigorous high school courses such as Advanced Placement (AP®) Physics increases the likelihood of college success, especially for students who are traditionally underserved. Tackling difficult multiple-choice exams should be part of any AP program because well-constructed multiple-choice questions, such as those on AP exams and on the Force Concept Inventory,2 are particularly good at rooting out common and persisting student misconceptions. Additionally, there are barriers to multiple-choice performance that have little to do with content mastery. For example, a student might fail to read the question thoroughly, forget to apply a reasonableness test to the answer, or simply work too slowly.

A Better Benchmark Assessment: Multiple-Choice versus Project-Based

ERIC Educational Resources Information Center

Peariso, Jamon F.

2006-01-01

The purpose of this literature review and Ex Post Facto descriptive study was to determine which type of benchmark assessment, multiple-choice or project-based, provides the best indication of general success on the history portion of the CST (California Standards Tests). The result of the study indicates that although the project-based benchmark…

Fast Assessments with Digital Tools Using Multiple-Choice Questions

ERIC Educational Resources Information Center

Howell, Dusti D.; Tseng, Daphne ChingYu; Colorado-Resa, Jozenia T.

2017-01-01

Multiple Choice Questions (MCQs) have come a long way since they were used in "The Kansas Silent Reading Test" in 1915. After over 100 years of MCQs, new innovative digital tools using this form of assessment can help foster interactivity in today's classrooms. This article describes three free online MCQ tools that are relatively quick…

Delayed, but not immediate, feedback after multiple-choice questions increases performance on a subsequent short-answer, but not multiple-choice, exam: evidence for the dual-process theory of memory.

PubMed

Sinha, Neha; Glass, Arnold Lewis

2015-01-01

Three experiments, two performed in the laboratory and one embedded in a college psychology lecture course, investigated the effects of immediate versus delayed feedback following a multiple-choice exam on subsequent short answer and multiple-choice exams. Performance on the subsequent multiple-choice exam was not affected by the timing of the feedback on the prior exam; however, performance on the subsequent short answer exam was better following delayed than following immediate feedback. This was true regardless of the order in which immediate versus delayed feedback was given. Furthermore, delayed feedback only had a greater effect than immediate feedback on subsequent short answer performance following correct, confident responses on the prior exam. These results indicate that delayed feedback cues a student's prior response and increases subsequent recollection of that response. The practical implication is that delayed feedback is better than immediate feedback during academic testing.

Fixed or mixed: a comparison of three, four and mixed-option multiple-choice tests in a Fetal Surveillance Education Program.

PubMed

Zoanetti, Nathan; Beaves, Mark; Griffin, Patrick; Wallace, Euan M

2013-03-04

Despite the widespread use of multiple-choice assessments in medical education assessment, current practice and published advice concerning the number of response options remains equivocal. This article describes an empirical study contrasting the quality of three 60 item multiple-choice test forms within the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG) Fetal Surveillance Education Program (FSEP). The three forms are described below. The first form featured four response options per item. The second form featured three response options, having removed the least functioning option from each item in the four-option counterpart. The third test form was constructed by retaining the best performing version of each item from the first two test forms. It contained both three and four option items. Psychometric and educational factors were taken into account in formulating an approach to test construction for the FSEP. The four-option test performed better than the three-option test overall, but some items were improved by the removal of options. The mixed-option test demonstrated better measurement properties than the fixed-option tests, and has become the preferred test format in the FSEP program. The criteria used were reliability, errors of measurement and fit to the item response model. The position taken is that decisions about the number of response options be made at the item level, with plausible options being added to complete each item on both psychometric and educational grounds rather than complying with a uniform policy. The point is to construct the better performing item in providing the best psychometric and educational information.

Cognitive Validity: Can Multiple-Choice Items Tap Historical Thinking Processes?

ERIC Educational Resources Information Center

Smith, Mark D.

2017-01-01

Cognitive validity examines the relationship between what an assessment aims to measure and what it actually elicits from test takers. The present study examined whether multiple-choice items from the National Assessment of Educational Progress (NAEP) grade 12 U.S. history exam elicited the historical thinking processes they were designed to…

Multiple-Choice Exams and Guessing: Results from a One-Year Study of General Chemistry Tests Designed to Discourage Guessing

ERIC Educational Resources Information Center

Campbell, Mark L.

2015-01-01

Multiple-choice exams, while widely used, are necessarily imprecise due to the contribution of the final student score due to guessing. This past year at the United States Naval Academy the construction and grading scheme for the department-wide general chemistry multiple-choice exams were revised with the goal of decreasing the contribution of…

A Multiple-Choice Task with Changes of Mind

PubMed Central

Albantakis, Larissa; Branzi, Francesca M.; Costa, Albert; Deco, Gustavo

2012-01-01

The role of changes of mind and multiple choices has recently received increased attention in the study of perceptual decision-making. Previously, these extensions to standard two-alternative tasks have been studied separately. Here we explored how changes of mind depend on the number of choice-alternatives. To this end, we tested 14 human subjects on a 2- and 4-alternative direction-discrimination task. Changes of mind in the participants' movement trajectories could be observed for two and for four choice alternatives. With fewer alternatives, participants responded faster and more accurately. The frequency of changes of mind, however, did not significantly differ for the different numbers of choice alternatives. Nevertheless, mind-changing improved the participants' final performance, particularly for intermediate difficulty levels, in both experimental conditions. Moreover, the mean reaction times of individual participants were negatively correlated with their overall tendency to make changes of mind. We further reproduced these findings with a multi-alternative attractor model for decision-making, while a simple race model could not account for the experimental data. Our experiment, combined with the theoretical models allowed us to shed light on: (1) the differences in choice behavior between two and four alternatives, (2) the differences between the data of our human subjects and previous monkey data, (3) individual differences between participants, and (4) the inhibitory interaction between neural representations of choice alternatives. PMID:22916216

Mechanical waves conceptual survey: Its modification and conversion to a standard multiple-choice test

NASA Astrophysics Data System (ADS)

Barniol, Pablo; Zavala, Genaro

2016-06-01

In this article we present several modifications of the mechanical waves conceptual survey, the most important test to date that has been designed to evaluate university students' understanding of four main topics in mechanical waves: propagation, superposition, reflection, and standing waves. The most significant changes are (i) modification of several test questions that had some problems in their original design, (ii) standardization of the number of options for each question to five, (iii) conversion of the two-tier questions to multiple-choice questions, and (iv) modification of some questions to make them independent of others. To obtain a final version of the test, we administered both the original and modified versions several times to students at a large private university in Mexico. These students were completing a course that covers the topics tested by the survey. The final modified version of the test was administered to 234 students. In this study we present the modifications for each question, and discuss the reasons behind them. We also analyze the results obtained by the final modified version and offer a comparison between the original and modified versions. In the Supplemental Material we present the final modified version of the test. It can be used by teachers and researchers to assess students' understanding of, and learning about, mechanical waves.

Comparing comprehension measured by multiple-choice and open-ended questions.

PubMed

Ozuru, Yasuhiro; Briner, Stephen; Kurby, Christopher A; McNamara, Danielle S

2013-09-01

This study compared the nature of text comprehension as measured by multiple-choice format and open-ended format questions. Participants read a short text while explaining preselected sentences. After reading the text, participants answered open-ended and multiple-choice versions of the same questions based on their memory of the text content. The results indicated that performance on open-ended questions was correlated with the quality of self-explanations, but performance on multiple-choice questions was correlated with the level of prior knowledge related to the text. These results suggest that open-ended and multiple-choice format questions measure different aspects of comprehension processes. The results are discussed in terms of dual process theories of text comprehension. PsycINFO Database Record (c) 2013 APA, all rights reserved

Fixed or mixed: a comparison of three, four and mixed-option multiple-choice tests in a Fetal Surveillance Education Program

PubMed Central

2013-01-01

Background Despite the widespread use of multiple-choice assessments in medical education assessment, current practice and published advice concerning the number of response options remains equivocal. This article describes an empirical study contrasting the quality of three 60 item multiple-choice test forms within the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG) Fetal Surveillance Education Program (FSEP). The three forms are described below. Methods The first form featured four response options per item. The second form featured three response options, having removed the least functioning option from each item in the four-option counterpart. The third test form was constructed by retaining the best performing version of each item from the first two test forms. It contained both three and four option items. Results Psychometric and educational factors were taken into account in formulating an approach to test construction for the FSEP. The four-option test performed better than the three-option test overall, but some items were improved by the removal of options. The mixed-option test demonstrated better measurement properties than the fixed-option tests, and has become the preferred test format in the FSEP program. The criteria used were reliability, errors of measurement and fit to the item response model. Conclusions The position taken is that decisions about the number of response options be made at the item level, with plausible options being added to complete each item on both psychometric and educational grounds rather than complying with a uniform policy. The point is to construct the better performing item in providing the best psychometric and educational information. PMID:23453056

Investigating the potential influence of established multiple-choice test-taking cues on item response in a pharmacotherapy board certification examination preparatory manual: a pilot study.

PubMed

Gettig, Jacob P

2006-04-01

To determine the prevalence of established multiple-choice test-taking correct and incorrect answer cues in the American College of Clinical Pharmacy's Updates in Therapeutics: The Pharmacotherapy Preparatory Course, 2005 Edition, as an equal or lesser surrogate indication of the prevalence of such cues in the Pharmacotherapy board certification examination. All self-assessment and patient case question-and-answer sets were assessed individually to determine if they were subject to selected correct and incorrect answer cues commonly seen in multiple-choice question writing. If the question was considered evaluable, correct answer cues-longest answer, mid-range number, one of two similar choices, and one of two opposite choices-were tallied. In addition, incorrect answer cues- inclusionary language and grammatical mismatch-were also tallied. Each cue was counted if it did what was expected or did the opposite of what was expected. Multiple cues could be identified in each question. A total of 237 (47.7%) of 497 questions in the manual were deemed evaluable. A total of 325 correct answer cues and 35 incorrect answer cues were identified in the 237 evaluable questions. Most evaluable questions contained one to two correct and/or incorrect answer cue(s). Longest answer was the most frequently identified correct answer cue; however, it was the least likely to identify the correct answer. Inclusionary language was the most frequently identified incorrect answer cue. Incorrect answer cues were considerably more likely to identify incorrect answer choices than correct answer cues were able to identify correct answer choices. The use of established multiple-choice test-taking cues is unlikely to be of significant help when taking the Pharmacotherapy board certification examination, primarily because of the lack of questions subject to such cues and the inability of correct answer cues to accurately identify correct answers. Incorrect answer cues, especially the use of

Multiple Choice Test Bias Uncovered by Use of an "I Don't Know" Alternative.

ERIC Educational Resources Information Center

Sherman, Susan W.

The multiple-choice science exercises used by the National Assessment of Educational Progress include an "I Don't Know" (IDK) alternative to estimate more accurately knowledge of groups of respondents. Group percentages of IDK responses were examined and compared with correct responses to see if the IDK introduces bias. Variance common…

Nested Logit Models for Multiple-Choice Item Response Data

ERIC Educational Resources Information Center

Suh, Youngsuk; Bolt, Daniel M.

2010-01-01

Nested logit item response models for multiple-choice data are presented. Relative to previous models, the new models are suggested to provide a better approximation to multiple-choice items where the application of a solution strategy precedes consideration of response options. In practice, the models also accommodate collapsibility across all…

Assessment of item-writing flaws in multiple-choice questions.

PubMed

Nedeau-Cayo, Rosemarie; Laughlin, Deborah; Rus, Linda; Hall, John

2013-01-01

This study evaluated the quality of multiple-choice questions used in a hospital's e-learning system. Constructing well-written questions is fraught with difficulty, and item-writing flaws are common. Study results revealed that most items contained flaws and were written at the knowledge/comprehension level. Few items had linked objectives, and no association was found between the presence of objectives and flaws. Recommendations include education for writing test questions.

Psychometrics of Multiple Choice Questions with Non-Functioning Distracters: Implications to Medical Education.

PubMed

Deepak, Kishore K; Al-Umran, Khalid Umran; AI-Sheikh, Mona H; Dkoli, B V; Al-Rubaish, Abdullah

2015-01-01

The functionality of distracters in a multiple choice question plays a very important role. We examined the frequency and impact of functioning and non-functioning distracters on psychometric properties of 5-option items in clinical disciplines. We analyzed item statistics of 1115 multiple choice questions from 15 summative assessments of undergraduate medical students and classified the items into five groups by their number of non-functioning distracters. We analyzed the effect of varying degree of non-functionality ranging from 0 to 4, on test reliability, difficulty index, discrimination index and point biserial correlation. The non-functionality of distracters inversely affected the test reliability and quality of items in a predictable manner. The non-functioning distracters made the items easier and lowered the discrimination index significantly. Three non-functional distracters in a 5-option MCQ significantly affected all psychometric properties (p < 0.5). The corrected point biserial correlation revealed that the items with 3 functional options were psychometrically as effective as 5-option items. Our study reveals that a multiple choice question with 3 functional options provides lower most limit of item format that has adequate psychometric property. The test containing items with less number of functioning options have significantly lower reliability. The distracter function analysis and revision of nonfunctioning distracters can serve as important methods to improve the psychometrics and reliability of assessment.

Does Linking Mixed-Format Tests Using a Multiple-Choice Anchor Produce Comparable Results for Male and Female Subgroups? Research Report. ETS RR-11-44

ERIC Educational Resources Information Center

Kim, Sooyeon; Walker, Michael E.

2011-01-01

This study examines the use of subpopulation invariance indices to evaluate the appropriateness of using a multiple-choice (MC) item anchor in mixed-format tests, which include both MC and constructed-response (CR) items. Linking functions were derived in the nonequivalent groups with anchor test (NEAT) design using an MC-only anchor set for 4…

Comparison between three option, four option and five option multiple choice question tests for quality parameters: A randomized study.

PubMed

Vegada, Bhavisha; Shukla, Apexa; Khilnani, Ajeetkumar; Charan, Jaykaran; Desai, Chetna

2016-01-01

Most of the academic teachers use four or five options per item of multiple choice question (MCQ) test as formative and summative assessment. Optimal number of options in MCQ item is a matter of considerable debate among academic teachers of various educational fields. There is a scarcity of the published literature regarding the optimum number of option in each item of MCQ in the field of medical education. To compare three options, four options, and five options MCQs test for the quality parameters - reliability, validity, item analysis, distracter analysis, and time analysis. Participants were 3 rd semester M.B.B.S. students. Students were divided randomly into three groups. Each group was given one set of MCQ test out of three options, four options, and five option randomly. Following the marking of the multiple choice tests, the participants' option selections were analyzed and comparisons were conducted of the mean marks, mean time, validity, reliability and facility value, discrimination index, point biserial value, distracter analysis of three different option formats. Students score more ( P = 0.000) and took less time ( P = 0.009) for the completion of three options as compared to four options and five options groups. Facility value was more ( P = 0.004) in three options group as compared to four and five options groups. There was no significant difference between three groups for the validity, reliability, and item discrimination. Nonfunctioning distracters were more in the four and five options group as compared to three option group. Assessment based on three option MCQs is can be preferred over four option and five option MCQs.

A Two-Tier Multiple Choice Questions to Diagnose Thermodynamic Misconception of Thai and Laos Students

NASA Astrophysics Data System (ADS)

Kamcharean, Chanwit; Wattanakasiwich, Pornrat

The objective of this study was to diagnose misconceptions of Thai and Lao students in thermodynamics by using a two-tier multiple-choice test. Two-tier multiple choice questions consist of the first tier, a content-based question and the second tier, a reasoning-based question. Data of student understanding was collected by using 10 two-tier multiple-choice questions. Thai participants were the first-year students (N = 57) taking a fundamental physics course at Chiang Mai University in 2012. Lao participants were high school students in Grade 11 (N = 57) and Grade 12 (N = 83) at Muengnern high school in Xayaboury province, Lao PDR. As results, most students answered content-tier questions correctly but chose incorrect answers for reason-tier questions. When further investigating their incorrect reasons, we found similar misconceptions as reported in previous studies such as incorrectly relating pressure with temperature when presenting with multiple variables.

Multiple-Choice Exams: An Obstacle for Higher-Level Thinking in Introductory Science Classes

ERIC Educational Resources Information Center

Stanger-Hall, Kathrin F.

2012-01-01

Learning science requires higher-level (critical) thinking skills that need to be practiced in science classes. This study tested the effect of exam format on critical-thinking skills. Multiple-choice (MC) testing is common in introductory science courses, and students in these classes tend to associate memorization with MC questions and may not…

Validity and Realibility of Chemistry Systemic Multiple Choices Questions (CSMCQs)

ERIC Educational Resources Information Center

Priyambodo, Erfan; Marfuatun

2016-01-01

Nowadays, Rasch model analysis is used widely in social research, moreover in educational research. In this research, Rasch model is used to determine the validation and the reliability of systemic multiple choices question in chemistry teaching and learning. There were 30 multiple choices question with systemic approach for high school student…

COMPUTER TECHNIQUES FOR WEEKLY MULTIPLE-CHOICE TESTING.

ERIC Educational Resources Information Center

BROYLES, DAVID

TO ENCOURAGE POLITICAL SCIENCE STUDENTS TO READ PROPERLY AND CONTINUOUSLY, THE AUTHOR GIVES FREQUENT SHORT QUIZZES BASED ON THE ASSIGNED READINGS. FOR EASE IN ADMINISTRATION AND SCORING, HE USES MARK-SENSE CARDS, ON WHICH THE STUDENT MARKS DESIGNATED AREAS TO INDICATE HIS NUMBER AND HIS CHOICE OF ANSWERS. TO EMPHASIZE THE VALUE OF CONTINUED HIGH…

Improving Measures via Examining the Behavior of Distractors in Multiple-Choice Tests: Assessment and Remediation

ERIC Educational Resources Information Center

Sideridis, Georgios; Tsaousis, Ioannis; Al Harbi, Khaleel

2017-01-01

The purpose of the present article was to illustrate, using an example from a national assessment, the value from analyzing the behavior of distractors in measures that engage the multiple-choice format. A secondary purpose of the present article was to illustrate four remedial actions that can potentially improve the measurement of the…

Development and Preliminary Testing of the Food Choice Priorities Survey (FCPS): Assessing the Importance of Multiple Factors on College Students' Food Choices.

PubMed

Vilaro, Melissa J; Zhou, Wenjun; Colby, Sarah E; Byrd-Bredbenner, Carol; Riggsbee, Kristin; Olfert, Melissa D; Barnett, Tracey E; Mathews, Anne E

2017-12-01

Understanding factors that influence food choice may help improve diet quality. Factors that commonly affect adults' food choices have been described, but measures that identify and assess food choice factors specific to college students are lacking. This study developed and tested the Food Choice Priorities Survey (FCPS) among college students. Thirty-seven undergraduates participated in two focus groups ( n = 19; 11 in the male-only group, 8 in the female-only group) and interviews ( n = 18) regarding typical influences on food choice. Qualitative data informed the development of survey items with a 5-point Likert-type scale (1 = not important, 5 = extremely important). An expert panel rated FCPS items for clarity, relevance, representativeness, and coverage using a content validity form. To establish test-retest reliability, 109 first-year college students completed the 14-item FCPS at two time points, 0-48 days apart ( M = 13.99, SD = 7.44). Using Cohen's weighted κ for responses within 20 days, 11 items demonstrated moderate agreement and 3 items had substantial agreement. Factor analysis revealed a three-factor structure (9 items). The FCPS is designed for college students and provides a way to determine the factors of greatest importance regarding food choices among this population. From a public health perspective, practical applications include using the FCPS to tailor health communications and behavior change interventions to factors most salient for food choices of college students.

Teaching Critical Thinking without (Much) Writing: Multiple-Choice and Metacognition

ERIC Educational Resources Information Center

Bassett, Molly H.

2016-01-01

In this essay, I explore an exam format that pairs multiple-choice questions with required rationales. In a space adjacent to each multiple-choice question, students explain why or how they arrived at the answer they selected. This exercise builds the critical thinking skill known as metacognition, thinking about thinking, into an exam that also…

Assessing the Validity of Multiple-Choice Questions in Measuring Fourth Graders' Ability to Interpret Graphs about Motion and Temperature

ERIC Educational Resources Information Center

Dulger, Mehmet; Deniz, Hasan

2017-01-01

The purpose of this paper is to assess the validity of multiple-choice questions in measuring fourth grade students' ability to interpret graphs related to physical science topics such as motion and temperature. We administered a test including 6 multiple-choice questions to 28 fourth grade students. Students were asked to explain their thinking…

The Effect of Images on Item Statistics in Multiple Choice Anatomy Examinations

ERIC Educational Resources Information Center

Notebaert, Andrew J.

2017-01-01

Although multiple choice examinations are often used to test anatomical knowledge, these often forgo the use of images in favor of text-based questions and answers. Because anatomy is reliant on visual resources, examinations using images should be used when appropriate. This study was a retrospective analysis of examination items that were text…

A multiple choice testing program coupled with a year-long elective experience is associated with improved performance on the internal medicine in-training examination.

PubMed

Mathis, Bradley R; Warm, Eric J; Schauer, Daniel P; Holmboe, Eric; Rouan, Gregory W

2011-11-01

The Internal Medicine In-Training Exam (IM-ITE) assesses the content knowledge of internal medicine trainees. Many programs use the IM-ITE to counsel residents, to create individual remediation plans, and to make fundamental programmatic and curricular modifications. To assess the association between a multiple-choice testing program administered during 12 consecutive months of ambulatory and inpatient elective experience and IM-ITE percentile scores in third post-graduate year (PGY-3) categorical residents. Retrospective cohort study. One hundred and four categorical internal medicine residents. Forty-five residents in the 2008 and 2009 classes participated in the study group, and the 59 residents in the three classes that preceded the use of the testing program, 2005-2007, served as controls. A comprehensive, elective rotation specific, multiple-choice testing program and a separate board review program, both administered during a continuous long-block elective experience during the twelve months between the second post-graduate year (PGY-2) and PGY-3 in-training examinations. We analyzed the change in median individual percent correct and percentile scores between the PGY-1 and PGY-2 IM-ITE and between the PGY-2 and PGY-3 IM-ITE in both control and study cohorts. For our main outcome measure, we compared the change in median individual percentile rank between the control and study cohorts between the PGY-2 and the PGY-3 IM-ITE testing opportunities. After experiencing the educational intervention, the study group demonstrated a significant increase in median individual IM-ITE percentile score between PGY-2 and PGY-3 examinations of 8.5 percentile points (p < 0.01). This is significantly better than the increase of 1.0 percentile point seen in the control group between its PGY-2 and PGY-3 examination (p < 0.01). A comprehensive multiple-choice testing program aimed at PGY-2 residents during a 12-month continuous long-block elective experience is associated with

What influences participation in genetic carrier testing? Results from a discrete choice experiment.

PubMed

Hall, Jane; Fiebig, Denzil G; King, Madeleine T; Hossain, Ishrat; Louviere, Jordan J

2006-05-01

This study explores factors that influence participation in genetic testing programs and the acceptance of multiple tests. Tay Sachs and cystic fibrosis are both genetically determined recessive disorders with differing severity, treatment availability, and prevalence in different population groups. We used a discrete choice experiment with a general community and an Ashkenazi Jewish sample; data were analysed using multinomial logit with random coefficients. Although Jewish respondents were more likely to be tested, both groups seem to be making very similar tradeoffs across attributes when they make genetic testing choices.

The Prevalence of Multiple-Choice Testing in Registered Nurse Licensure-Qualifying Nursing Education Programs in New York State.

PubMed

Birkhead, Susan; Kelman, Glenda; Zittel, Barbara; Jatulis, Linnea

The aim of this study was to describe nurse educators' use of multiple-choice questions (MCQs) in testing in registered nurse licensure-qualifying nursing education programs in New York State. This study was a descriptive correlational analysis of data obtained from surveying 1,559 nurse educators; 297 educators from 61 institutions responded (response rate [RR] = 19 percent), yielding a final cohort of 200. MCQs were reported to comprise a mean of 81 percent of questions on a typical test. Baccalaureate program respondents were equally likely to use MCQs as associate degree program respondents (p > .05) but were more likely to report using other methods of assessing student achievement to construct course grades (p < .01). Both groups reported little use of alternate format-type questions. Respondent educators reported substantial reliance upon the use of MCQs, corroborating the limited data quantifying the prevalence of use of MCQ tests in licensure-qualifying nursing education programs.

Comedy workshop: an enjoyable way to develop multiple-choice questions.

PubMed

Droegemueller, William; Gant, Norman; Brekken, Alvin; Webb, Lynn

2005-01-01

To describe an innovative method of developing multiple-choice items for a board certification examination. The development of appropriate multiple-choice items is definitely more of an art, rather than a science. The comedy workshop format for developing questions for a certification examination is similar to the process used by comedy writers composing scripts for television shows. This group format dramatically diminishes the frustrations faced by an individual question writer attempting to create items. The vast majority of our comedy workshop participants enjoy and prefer the comedy workshop format. It provides an ideal environment in which to teach and blend the talents of inexperienced and experienced question writers. This is a descriptive article, in which we suggest an innovative process in the art of creating multiple-choice items for a high-stakes examination.

Multiple Choice Items: How to Gain the Most out of Them.

ERIC Educational Resources Information Center

Talmir, Pinchas

1991-01-01

Describes how multiple-choice items can be designed and used as an effective diagnostic tool by avoiding their pitfalls and by taking advantage of their potential benefits. The following issues are discussed: correct' versus best answers; construction of diagnostic multiple-choice items; the problem of guessing; the use of justifications of…

Multiple-Choice and Short-Answer Exam Performance in a College Classroom

ERIC Educational Resources Information Center

Funk, Steven C.; Dickson, K. Laurie

2011-01-01

The authors experimentally investigated the effects of multiple-choice and short-answer format exam items on exam performance in a college classroom. They randomly assigned 50 students to take a 10-item short-answer pretest or posttest on two 50-item multiple-choice exams in an introduction to personality course. Students performed significantly…

Tailoring auditory training to patient needs with single and multiple talkers: transfer-appropriate gains on a four-choice discrimination test.

PubMed

Barcroft, Joe; Sommers, Mitchell S; Tye-Murray, Nancy; Mauzé, Elizabeth; Schroy, Catherine; Spehar, Brent

2011-11-01

Our long-term objective is to develop an auditory training program that will enhance speech recognition in those situations where patients most want improvement. As a first step, the current investigation trained participants using either a single talker or multiple talkers to determine if auditory training leads to transfer-appropriate gains. The experiment implemented a 2 × 2 × 2 mixed design, with training condition as a between-participants variable and testing interval and test version as repeated-measures variables. Participants completed a computerized six-week auditory training program wherein they heard either the speech of a single talker or the speech of six talkers. Training gains were assessed with single-talker and multi-talker versions of the Four-choice discrimination test. Participants in both groups were tested on both versions. Sixty-nine adult hearing-aid users were randomly assigned to either single-talker or multi-talker auditory training. Both groups showed significant gains on both test versions. Participants who trained with multiple talkers showed greater improvement on the multi-talker version whereas participants who trained with a single talker showed greater improvement on the single-talker version. Transfer-appropriate gains occurred following auditory training, suggesting that auditory training can be designed to target specific patient needs.

The Effect of the Multiple-Choice Item Format on the Measurement of Knowledge of Language Structure

ERIC Educational Resources Information Center

Currie, Michael; Chiramanee, Thanyapa

2010-01-01

Noting the widespread use of multiple-choice items in tests in English language education in Thailand, this study compared their effect against that of constructed-response items. One hundred and fifty-two university undergraduates took a test of English structure first in constructed-response format, and later in three, stem-equivalent…

Multiple Choice Questions Can Be Designed or Revised to Challenge Learners' Critical Thinking

ERIC Educational Resources Information Center

Tractenberg, Rochelle E.; Gushta, Matthew M.; Mulroney, Susan E.; Weissinger, Peggy A.

2013-01-01

Multiple choice (MC) questions from a graduate physiology course were evaluated by cognitive-psychology (but not physiology) experts, and analyzed statistically, in order to test the independence of content expertise and cognitive complexity ratings of MC items. Integration of higher order thinking into MC exams is important, but widely known to…

Does Correct Answer Distribution Influence Student Choices When Writing Multiple Choice Examinations?

ERIC Educational Resources Information Center

Carnegie, Jacqueline A.

2017-01-01

Summative evaluation for large classes of first- and second-year undergraduate courses often involves the use of multiple choice question (MCQ) exams in order to provide timely feedback. Several versions of those exams are often prepared via computer-based question scrambling in an effort to deter cheating. An important parameter to consider when…

Step by Step: Biology Undergraduates' Problem-Solving Procedures during Multiple-Choice Assessment

ERIC Educational Resources Information Center

Prevost, Luanna B.; Lemons, Paula P.

2016-01-01

This study uses the theoretical framework of domain-specific problem solving to explore the procedures students use to solve multiple-choice problems about biology concepts. We designed several multiple-choice problems and administered them on four exams. We trained students to produce written descriptions of how they solved the problem, and this…

Solomon design analysis of multiple-choice Rorschach animal content.

PubMed

Feigenbaum, D; Costello, R M

1975-10-01

The Solomon four-group design was used to study the effects of a persuasive message on a selected multiple-choice Rorschach index--animal content. The independent variable elicited behavior in a predictable manner. Pretesting as a main effect was not significant, but as an interactional effect obviated the effect of the persuasive message. Although knowledge of test rationale can elicit behavior that conforms to experimental demand characteristics, some subjects nonetheless acted in defiance of such information. A condition for defiance in this experimental arrangement, however, was pretesting. Other possibilities regarding the study of compliance behavior and the use of pathognomonic indicators were suggested. Ethical issues were raised.

Models for Scoring Missing Responses to Multiple-Choice Items. Program Statistics Research Technical Report No. 94-1.

ERIC Educational Resources Information Center

Longford, Nicholas T.

This study is a critical evaluation of the roles for coding and scoring of missing responses to multiple-choice items in educational tests. The focus is on tests in which the test-takers have little or no motivation; in such tests omitting and not reaching (as classified by the currently adopted operational rules) is quite frequent. Data from the…

Genetic Algorithms for Multiple-Choice Problems

NASA Astrophysics Data System (ADS)

Aickelin, Uwe

2010-04-01

This thesis investigates the use of problem-specific knowledge to enhance a genetic algorithm approach to multiple-choice optimisation problems.It shows that such information can significantly enhance performance, but that the choice of information and the way it is included are important factors for success.Two multiple-choice problems are considered.The first is constructing a feasible nurse roster that considers as many requests as possible.In the second problem, shops are allocated to locations in a mall subject to constraints and maximising the overall income.Genetic algorithms are chosen for their well-known robustness and ability to solve large and complex discrete optimisation problems.However, a survey of the literature reveals room for further research into generic ways to include constraints into a genetic algorithm framework.Hence, the main theme of this work is to balance feasibility and cost of solutions.In particular, co-operative co-evolution with hierarchical sub-populations, problem structure exploiting repair schemes and indirect genetic algorithms with self-adjusting decoder functions are identified as promising approaches.The research starts by applying standard genetic algorithms to the problems and explaining the failure of such approaches due to epistasis.To overcome this, problem-specific information is added in a variety of ways, some of which are designed to increase the number of feasible solutions found whilst others are intended to improve the quality of such solutions.As well as a theoretical discussion as to the underlying reasons for using each operator,extensive computational experiments are carried out on a variety of data.These show that the indirect approach relies less on problem structure and hence is easier to implement and superior in solution quality.

The Relationship of Expert-System Scored Constrained Free-Response Items to Multiple-Choice and Open-Ended Items.

ERIC Educational Resources Information Center

Bennett, Randy Elliot; And Others

1990-01-01

The relationship of an expert-system-scored constrained free-response item type to multiple-choice and free-response items was studied using data for 614 students on the College Board's Advanced Placement Computer Science (APCS) Examination. Implications for testing and the APCS test are discussed. (SLD)

Performance of Certification and Recertification Examinees on Multiple Choice Test Items: Does Physician Age Have an Impact?

PubMed

Shen, Linjun; Juul, Dorthea; Faulkner, Larry R

2016-01-01

The development of recertification programs (now referred to as Maintenance of Certification or MOC) by the members of the American Board of Medical Specialties provides the opportunity to study knowledge base across the professional lifespan of physicians. Research results to date are mixed with some studies finding negative associations between age and various measures of competency and others finding no or minimal relationships. Four groups of multiple choice test items that were independently developed for certification and MOC examinations in psychiatry and neurology were administered to certification and MOC examinees within each specialty. Percent correct scores were calculated for each examinee. Differences between certification and MOC examinees were compared using unpaired t tests, and logistic regression was used to compare MOC and certification examinee performance on the common test items. Except for the neurology certification test items that addressed basic neurology concepts, the performance of the certification and MOC examinees was similar. The differences in performance on individual test items did not consistently favor one group or the other and could not be attributed to any distinguishable content or format characteristics of those items. The findings of this study are encouraging in that physicians who had recently completed residency training possessed clinical knowledge that was comparable to that of experienced physicians, and the experienced physicians' clinical knowledge was equivalent to that of recent residency graduates. The role testing can play in enhancing expertise is described.

Validating Measurement of Knowledge Integration in Science Using Multiple-Choice and Explanation Items

ERIC Educational Resources Information Center

Lee, Hee-Sun; Liu, Ou Lydia; Linn, Marcia C.

2011-01-01

This study explores measurement of a construct called knowledge integration in science using multiple-choice and explanation items. We use construct and instructional validity evidence to examine the role multiple-choice and explanation items plays in measuring students' knowledge integration ability. For construct validity, we analyze item…

The Effect of English Language on Multiple Choice Question Scores of Thai Medical Students.

PubMed

Phisalprapa, Pochamana; Muangkaew, Wayuda; Assanasen, Jintana; Kunavisarut, Tada; Thongngarm, Torpong; Ruchutrakool, Theera; Kobwanthanakun, Surapon; Dejsomritrutai, Wanchai

2016-04-01

Universities in Thailand are preparing for Thailand's integration into the ASEAN Economic Community (AEC) by increasing the number of tests in English language. English language is not the native language of Thailand Differences in English language proficiency may affect scores among test-takers, even when subject knowledge among test-takers is comparable and may falsely represent the knowledge level of the test-taker. To study the impact of English language multiple choice test questions on test scores of medical students. The final examination of fourth-year medical students completing internal medicine rotation contains 120 multiple choice questions (MCQ). The languages used on the test are Thai and English at a ratio of 3:1. Individual scores of tests taken in both languages were collected and the effect of English language on MCQ was analyzed Individual MCQ scores were then compared with individual student English language proficiency and student grade point average (GPA). Two hundred ninety five fourth-year medical students were enrolled. The mean percentage of MCQ scores in Thai and English were significantly different (65.0 ± 8.4 and 56.5 ± 12.4, respectively, p < 0.001). The correlation between MCQ scores in Thai and English was fair (Spearman's correlation coefficient = 0.41, p < 0.001). Of 295 students, only 73 (24.7%) students scored higher when being tested in English than in Thai language. Students were classified into six grade categories (A, B+, B, C+, C, and D+), which cumulatively measured total internal medicine rotation performance score plus final examination score. MCQ scores from Thai language examination were more closely correlated with total course grades than were the scores from English language examination (Spearman's correlation coefficient = 0.73 (p < 0.001) and 0.53 (p < 0.001), respectively). The gap difference between MCQ scores in both languages was higher in borderline students than in the excellent student group (11.2 ± 11

Comparison of Difficulties and Reliabilities of Math-Completion and Multiple-Choice Item Formats.

ERIC Educational Resources Information Center

Oosterhof, Albert C.; Coats, Pamela K.

Instructors who develop classroom examinations that require students to provide a numerical response to a mathematical problem are often very concerned about the appropriateness of the multiple-choice format. The present study augments previous research relevant to this concern by comparing the difficulty and reliability of multiple-choice and…

Analyzing Student Confidence in Classroom Voting with Multiple Choice Questions

ERIC Educational Resources Information Center

Stewart, Ann; Storm, Christopher; VonEpps, Lahna

2013-01-01

The purpose of this paper is to present results of a recent study in which students voted on multiple choice questions in mathematics courses of varying levels. Students used clickers to select the best answer among the choices given; in addition, they were also asked whether they were confident in their answer. In this paper we analyze data…

Identifying student mental models from their response pattern to a physics multiple-choice test

NASA Astrophysics Data System (ADS)

Montenegro Maggio, Maximiliano Jose

Previous work has shown that students present different misconceptions across different but similar physical situations, but the cause of these differences is still not clear. In this study, a novel analysis method was introduced to help to gain a better understanding of how different physical situations affect students' responses and learning. This novel analysis groups students into mental model groups (MMG) by similarities in their responses to multiple-choice test items, under the assumption that they have similar mental models. The Mass and Energy Conservation test was developed to probe the common misconception that objects with greater mass fall faster than objects with lesser mass across four physical situations and four knowledge sub-domains: information, dynamics, work, and energy. The test was applied before and after energy instruction to 144 college students in a large Midwestern university attending a calculus-based introductory physics course. Test time along with instruction and physical situation were the two factors. It was found that physical situation did not have a significant effect on mental models: The number of MMGs identified and the fraction of students belonging to the same MMG were not significantly different (p > .05) across physical situations. However, there was a significant effect of test time on mental models (p < .05): the fraction of students belonging to the same MMG changed from the pretest to the posttest, in that the MMG representing higher performance became predominant than the MMG with lower performance for the posttest results. A MANOVA for the average scores for each sub-domain and physical situation combination was applied to validate the previous results. It was found that a significant effect (p < .01) by physical situation resulted due to a lower average dynamics sub-domain score for the friction physical-situation attribute when compared to the no-friction physical-situation attribute. A significant effect (p < .01

The impact of two multiple-choice question formats on the problem-solving strategies used by novices and experts.

PubMed

Coderre, Sylvain P; Harasym, Peter; Mandin, Henry; Fick, Gordon

2004-11-05

Pencil-and-paper examination formats, and specifically the standard, five-option multiple-choice question, have often been questioned as a means for assessing higher-order clinical reasoning or problem solving. This study firstly investigated whether two paper formats with differing number of alternatives (standard five-option and extended-matching questions) can test problem-solving abilities. Secondly, the impact of the alternatives number on psychometrics and problem-solving strategies was examined. Think-aloud protocols were collected to determine the problem-solving strategy used by experts and non-experts in answering Gastroenterology questions, across the two pencil-and-paper formats. The two formats demonstrated equal ability in testing problem-solving abilities, while the number of alternatives did not significantly impact psychometrics or problem-solving strategies utilized. These results support the notion that well-constructed multiple-choice questions can in fact test higher order clinical reasoning. Furthermore, it can be concluded that in testing clinical reasoning, the question stem, or content, remains more important than the number of alternatives.

Multiple objective optimization in reliability demonstration test

DOE PAGES

Lu, Lu; Anderson-Cook, Christine Michaela; Li, Mingyang

2016-10-01

Reliability demonstration tests are usually performed in product design or validation processes to demonstrate whether a product meets specified requirements on reliability. For binomial demonstration tests, the zero-failure test has been most commonly used due to its simplicity and use of minimum sample size to achieve an acceptable consumer’s risk level. However, this test can often result in unacceptably high risk for producers as well as a low probability of passing the test even when the product has good reliability. This paper explicitly explores the interrelationship between multiple objectives that are commonly of interest when planning a demonstration test andmore » proposes structured decision-making procedures using a Pareto front approach for selecting an optimal test plan based on simultaneously balancing multiple criteria. Different strategies are suggested for scenarios with different user priorities and graphical tools are developed to help quantify the trade-offs between choices and to facilitate informed decision making. As a result, potential impacts of some subjective user inputs on the final decision are studied to offer insights and useful guidance for general applications.« less

A Systematic Assessment of "None of the Above" on Multiple Choice Tests in a First Year Psychology Classroom

ERIC Educational Resources Information Center

Pachai, Matthew V.; DiBattista, David; Kim, Joseph A.

2015-01-01

Multiple choice writing guidelines are decidedly split on the use of "none of the above" (NOTA), with some authors discouraging and others advocating its use. Moreover, empirical studies of NOTA have produced mixed results. Generally, these studies have utilized NOTA as either the correct response or a distractor and assessed its effect…

A Cognitive Diagnosis Model for Cognitively Based Multiple-Choice Options

ERIC Educational Resources Information Center

de la Torre, Jimmy

2009-01-01

Cognitive or skills diagnosis models are discrete latent variable models developed specifically for the purpose of identifying the presence or absence of multiple fine-grained skills. However, applications of these models typically involve dichotomous or dichotomized data, including data from multiple-choice (MC) assessments that are scored as…

Graded Multiple Choice Questions: Rewarding Understanding and Preventing Plagiarism

NASA Astrophysics Data System (ADS)

Denyer, G. S.; Hancock, D.

2002-08-01

This paper describes an easily implemented method that allows the generation and analysis of graded multiple-choice examinations. The technique, which uses standard functions in user-end software (Microsoft Excel 5+), can also produce several different versions of an examination that can be employed to prevent the reward of plagarism. The manuscript also discusses the advantages of having a graded marking system for the elimination of ambiguities, use in multi-step calculation questions, and questions that require extrapolation or reasoning. The advantages of the scrambling strategy, which maintains the same question order, is discussed with reference to student equity. The system provides a non-confrontational mechanism for dealing with cheating in large-class multiple-choice examinations, as well as providing a reward for problem solving over surface learning.

A hybrid heuristic for the multiple choice multidimensional knapsack problem

NASA Astrophysics Data System (ADS)

Mansi, Raïd; Alves, Cláudio; Valério de Carvalho, J. M.; Hanafi, Saïd

2013-08-01

In this article, a new solution approach for the multiple choice multidimensional knapsack problem is described. The problem is a variant of the multidimensional knapsack problem where items are divided into classes, and exactly one item per class has to be chosen. Both problems are NP-hard. However, the multiple choice multidimensional knapsack problem appears to be more difficult to solve in part because of its choice constraints. Many real applications lead to very large scale multiple choice multidimensional knapsack problems that can hardly be addressed using exact algorithms. A new hybrid heuristic is proposed that embeds several new procedures for this problem. The approach is based on the resolution of linear programming relaxations of the problem and reduced problems that are obtained by fixing some variables of the problem. The solutions of these problems are used to update the global lower and upper bounds for the optimal solution value. A new strategy for defining the reduced problems is explored, together with a new family of cuts and a reformulation procedure that is used at each iteration to improve the performance of the heuristic. An extensive set of computational experiments is reported for benchmark instances from the literature and for a large set of hard instances generated randomly. The results show that the approach outperforms other state-of-the-art methods described so far, providing the best known solution for a significant number of benchmark instances.

Asymmetry in Student Achievement on Multiple-Choice and Constructed-Response Items in Reversible Mathematics Processes

ERIC Educational Resources Information Center

Sangwin, Christopher J.; Jones, Ian

2017-01-01

In this paper we report the results of an experiment designed to test the hypothesis that when faced with a question involving the inverse direction of a reversible mathematical process, students solve a multiple-choice version by verifying the answers presented to them by the direct method, not by undertaking the actual inverse calculation.…

Delay and probability discounting of multiple commodities in smokers and never-smokers using multiple-choice tasks.

PubMed

Poltavski, Dmitri V; Weatherly, Jeffrey N

2013-12-01

The purpose of the present study was to investigate temporal and probabilistic discounting in smokers and never-smokers, across a number of commodities, using a multiple-choice method. One hundred and eighty-two undergraduate university students, of whom 90 had never smoked, 73 were self-reported light smokers (<10 cigarettes/day), and 17 were heavy smokers (10+cigarettes/day), completed computerized batteries of delay and probability discounting questions pertaining to a total of eight commodities and administered in a multiple-choice format. In addition to cigarettes, monetary rewards, and health outcomes, the tasks included novel commodities such as ideal dating partner and retirement income. The results showed that heavy smokers probability discounted commodities at a significantly shallower rate than never-smokers, suggesting greater risk-taking. No effect of smoking status was observed for delay discounting questions. The only commodity that was probability discounted significantly less than others was 'finding an ideal dating partner'. The results suggest that probability discounting tasks using the multiple-choice format can discriminate between non-abstaining smokers and never-smokers and could be further explored in the context of behavioral and drug addictions.

Potential Values of Incorporating a Multiple-Choice Question Construction in Physics Experimentation Instruction

NASA Astrophysics Data System (ADS)

Yu, Fu-Yun; Liu, Yu-Hsin

2005-09-01

The potential value of a multiple-choice question-construction instructional strategy for the support of students’ learning of physics experiments was examined in the study. Forty-two university freshmen participated in the study for a whole semester. A constant comparison method adopted to categorize students’ qualitative data indicated that the influences of multiple-choice question construction were evident in several significant ways (promoting constructive and productive studying habits; reflecting and previewing course-related materials; increasing in-group communication and interaction; breaking passive learning style and habits, etc.), which, worked together, not only enhanced students’ comprehension and retention of the obtained knowledge, but also helped distil a sense of empowerment and learning community within the participants. Analysis with one-group t-tests, using 3 as the expected mean, on quantitative data further found that students’ satisfaction toward past learning experience, and perceptions toward this strategy’s potentials for promoting learning were statistically significant at the 0.0005 level, while learning anxiety was not statistically significant. Suggestions for incorporating question-generation activities within classroom and topics for future studies were rendered.

Comparing narrative and multiple-choice formats in online communication skill assessment.

PubMed

Kim, Sara; Spielberg, Freya; Mauksch, Larry; Farber, Stu; Duong, Cuong; Fitch, Wes; Greer, Tom

2009-06-01

We compared multiple-choice and open-ended responses collected from a web-based tool designated 'Case for Change', which had been developed for assessing and teaching medical students in the skills involved in integrating sexual risk assessment and behaviour change discussions into patient-centred primary care visits. A total of 111 Year 3 students completed the web-based tool. A series of videos from one patient encounter illustrated how a clinician uses patient-centred communication and health behaviour change skills while caring for a patient presenting with a urinary tract infection. Each video clip was followed by a request for students to respond in two ways to the question: 'What would you do next?' Firstly, students typed their statements of what they would say to the patient. Secondly, students selected from a multiple-choice list the statements that most closely resembled their free text entries. These two modes of students' answers were analysed and compared. When articulating what they would say to the patient in a narrative format, students frequently used doctor-centred approaches that focused on premature diagnostic questioning or neglected to elicit patient perspectives. Despite the instruction to select a matching statement from the multiple-choice list, students tended to choose the most exemplary patient-centred statement, which was contrary to the doctor-centred approaches reflected in their narrative responses. Open-ended questions facilitate in-depth understanding of students' educational needs, although the scoring of narrative responses is time-consuming. Multiple-choice questions allow efficient scoring and individualised feedback associated with question items but do not fully elicit students' thought processes.

Using a Classroom Response System to Improve Multiple-Choice Performance in AP[R] Physics

ERIC Educational Resources Information Center

Bertrand, Peggy

2009-01-01

Participation in rigorous high school courses such as Advanced Placement (AP[R]) Physics increases the likelihood of college success, especially for students who are traditionally underserved. Tackling difficult multiple-choice exams should be part of any AP program because well-constructed multiple-choice questions, such as those on AP exams and…

Benford's Law: textbook exercises and multiple-choice testbanks.

PubMed

Slepkov, Aaron D; Ironside, Kevin B; DiBattista, David

2015-01-01

Benford's Law describes the finding that the distribution of leading (or leftmost) digits of innumerable datasets follows a well-defined logarithmic trend, rather than an intuitive uniformity. In practice this means that the most common leading digit is 1, with an expected frequency of 30.1%, and the least common is 9, with an expected frequency of 4.6%. Currently, the most common application of Benford's Law is in detecting number invention and tampering such as found in accounting-, tax-, and voter-fraud. We demonstrate that answers to end-of-chapter exercises in physics and chemistry textbooks conform to Benford's Law. Subsequently, we investigate whether this fact can be used to gain advantage over random guessing in multiple-choice tests, and find that while testbank answers in introductory physics closely conform to Benford's Law, the testbank is nonetheless secure against such a Benford's attack for banal reasons.

Evaluation of five guidelines for option development in multiple-choice item-writing.

PubMed

Martínez, Rafael J; Moreno, Rafael; Martín, Irene; Trigo, M Eva

2009-05-01

This paper evaluates certain guidelines for writing multiple-choice test items. The analysis of the responses of 5013 subjects to 630 items from 21 university classroom achievement tests suggests that an option should not differ in terms of heterogeneous content because such error has a slight but harmful effect on item discrimination. This also occurs with the "None of the above" option when it is the correct one. In contrast, results do not show the supposedly negative effects of a different-length option, the use of specific determiners, or the use of the "All of the above" option, which not only decreases difficulty but also improves discrimination when it is the correct option.

Polytomous versus Dichotomous Scoring on Multiple-Choice Examinations: Development of a Rubric for Rating Partial Credit

ERIC Educational Resources Information Center

Grunert, Megan L.; Raker, Jeffrey R.; Murphy, Kristen L.; Holme, Thomas A.

2013-01-01

The concept of assigning partial credit on multiple-choice test items is considered for items from ACS Exams. Because the items on these exams, particularly the quantitative items, use common student errors to define incorrect answers, it is possible to assign partial credits to some of these incorrect responses. To do so, however, it becomes…

Role of the plurality rule in multiple choices

NASA Astrophysics Data System (ADS)

Calvão, A. M.; Ramos, M.; Anteneodo, C.

2016-02-01

People are often challenged to select one among several alternatives. This situation is present not only in decisions about complex issues, e.g. political or academic choices, but also about trivial ones, such as in daily purchases at a supermarket. We tackle this scenario by means of the tools of statistical mechanics. Following this approach, we introduce and analyse a model of opinion dynamics, using a Potts-like state variable to represent the multiple choices, including the ‘undecided state’, which represents the individuals who do not make a choice. We investigate the dynamics over Erdös-Rényi and Barabási-Albert networks, two paradigmatic classes with the small-world property, and we show the impact of the type of network on the opinion dynamics. Depending on the number of available options q and on the degree distribution of the network of contacts, different final steady states are accessible: from a wide distribution of choices to a state where a given option largely dominates. The abrupt transition between them is consistent with the sudden viral dominance of a given option over many similar ones. Moreover, the probability distributions produced by the model are validated by real data. Finally, we show that the model also contemplates the real situation of overchoice, where a large number of similar alternatives makes the choice process harder and indecision prevail.

Format Effects of Empirically Derived Multiple-Choice versus Free-Response Instruments When Assessing Graphing Abilities

ERIC Educational Resources Information Center

Berg, Craig; Boote, Stacy

2017-01-01

Prior graphing research has demonstrated that clinical interviews and free-response instruments produce very different results than multiple-choice instruments, indicating potential validity problems when using multiple-choice instruments to assess graphing skills (Berg & Smith in "Science Education," 78(6), 527-554, 1994). Extending…

An Empirical Comparison of DDF Detection Methods for Understanding the Causes of DIF in Multiple-Choice Items

ERIC Educational Resources Information Center

Suh, Youngsuk; Talley, Anna E.

2015-01-01

This study compared and illustrated four differential distractor functioning (DDF) detection methods for analyzing multiple-choice items. The log-linear approach, two item response theory-model-based approaches with likelihood ratio tests, and the odds ratio approach were compared to examine the congruence among the four DDF detection methods.…

The Answering Process for Multiple-Choice Questions in Collaborative Learning: A Mathematical Learning Model Analysis

ERIC Educational Resources Information Center

Nakamura, Yasuyuki; Nishi, Shinnosuke; Muramatsu, Yuta; Yasutake, Koichi; Yamakawa, Osamu; Tagawa, Takahiro

2014-01-01

In this paper, we introduce a mathematical model for collaborative learning and the answering process for multiple-choice questions. The collaborative learning model is inspired by the Ising spin model and the model for answering multiple-choice questions is based on their difficulty level. An intensive simulation study predicts the possibility of…

Initial Correction versus Negative Marking in Multiple Choice Examinations

ERIC Educational Resources Information Center

Van Hecke, Tanja

2015-01-01

Optimal assessment tools should measure in a limited time the knowledge of students in a correct and unbiased way. A method for automating the scoring is multiple choice scoring. This article compares scoring methods from a probabilistic point of view by modelling the probability to pass: the number right scoring, the initial correction (IC) and…

The Display of Multiple Choice Question Bank on Microfilm

ERIC Educational Resources Information Center

Stevens, J. M.; Harris, F. T. C.

1977-01-01

An automated question bank maintained by the Department of Research and Services in Education at the Middlesex Hospital Medical School provides a printed copy of each of 25,000 multiple choice questions (95 percent relating to the whole spectrum of the medical curriculum). Problems with this procedure led to experimental work storing the data on…

Improving Measures via Examining the Behavior of Distractors in Multiple-Choice Tests

PubMed Central

Sideridis, Georgios; Tsaousis, Ioannis; Al Harbi, Khaleel

2017-01-01

The purpose of the present article was to illustrate, using an example from a national assessment, the value from analyzing the behavior of distractors in measures that engage the multiple-choice format. A secondary purpose of the present article was to illustrate four remedial actions that can potentially improve the measurement of the construct(s) under study. Participants were 2,248 individuals who took a national examination of chemistry. The behavior of the distractors was analyzed by modeling their behavior within the Rasch model. Potentially informative distractors were (a) further modeled using the partial credit model, (b) split onto separate items and retested for model fit and parsimony, (c) combined to form a “super” item or testlet, and (d) reexamined after deleting low-ability individuals who likely guessed on those informative, albeit erroneous, distractors. Results indicated that all but the item split strategies were associated with better model fit compared with the original model. The best fitted model, however, involved modeling and crediting informative distractors via the partial credit model or eliminating the responses of low-ability individuals who likely guessed on informative distractors. The implications, advantages, and disadvantages of modeling informative distractors for measurement purposes are discussed. PMID:29795904

Application of Item Analysis to Assess Multiple-Choice Examinations in the Mississippi Master Cattle Producer Program

ERIC Educational Resources Information Center

Parish, Jane A.; Karisch, Brandi B.

2013-01-01

Item analysis can serve as a useful tool in improving multiple-choice questions used in Extension programming. It can identify gaps between instruction and assessment. An item analysis of Mississippi Master Cattle Producer program multiple-choice examination responses was performed to determine the difficulty of individual examinations, assess the…

Using Module Analysis for Multiple Choice Responses: A New Method Applied to Force Concept Inventory Data

ERIC Educational Resources Information Center

Brewe, Eric; Bruun, Jesper; Bearden, Ian G.

2016-01-01

We describe "Module Analysis for Multiple Choice Responses" (MAMCR), a new methodology for carrying out network analysis on responses to multiple choice assessments. This method is used to identify modules of non-normative responses which can then be interpreted as an alternative to factor analysis. MAMCR allows us to identify conceptual…

The Use of a Comprehensive Multiple Choice Final Exam in the Macroeconomics Principles Course: An Assessment.

ERIC Educational Resources Information Center

Petrowsky, Michael C.

This paper analyzes the results of a pilot study at Glendale Community College (Arizona) to assess the effectiveness of a comprehensive multiple choice final exam in the macroeconomic principles course. The "pilot project" involved the administration of a 50-question multiple choice exam to 71 students in three macroeconomics sections.…

Sustainable Assessment for Large Science Classes: Non-Multiple Choice, Randomised Assignments through a Learning Management System

ERIC Educational Resources Information Center

Schultz, Madeleine

2011-01-01

This paper reports on the development of a tool that generates randomised, non-multiple choice assessment within the BlackBoard Learning Management System interface. An accepted weakness of multiple-choice assessment is that it cannot elicit learning outcomes from upper levels of Biggs' SOLO taxonomy. However, written assessment items require…

Multiple choice questions can be designed or revised to challenge learners' critical thinking.

PubMed

Tractenberg, Rochelle E; Gushta, Matthew M; Mulroney, Susan E; Weissinger, Peggy A

2013-12-01

Multiple choice (MC) questions from a graduate physiology course were evaluated by cognitive-psychology (but not physiology) experts, and analyzed statistically, in order to test the independence of content expertise and cognitive complexity ratings of MC items. Integration of higher order thinking into MC exams is important, but widely known to be challenging-perhaps especially when content experts must think like novices. Expertise in the domain (content) may actually impede the creation of higher-complexity items. Three cognitive psychology experts independently rated cognitive complexity for 252 multiple-choice physiology items using a six-level cognitive complexity matrix that was synthesized from the literature. Rasch modeling estimated item difficulties. The complexity ratings and difficulty estimates were then analyzed together to determine the relative contributions (and independence) of complexity and difficulty to the likelihood of correct answers on each item. Cognitive complexity was found to be statistically independent of difficulty estimates for 88 % of items. Using the complexity matrix, modifications were identified to increase some item complexities by one level, without affecting the item's difficulty. Cognitive complexity can effectively be rated by non-content experts. The six-level complexity matrix, if applied by faculty peer groups trained in cognitive complexity and without domain-specific expertise, could lead to improvements in the complexity targeted with item writing and revision. Targeting higher order thinking with MC questions can be achieved without changing item difficulties or other test characteristics, but this may be less likely if the content expert is left to assess items within their domain of expertise.

Benford’s Law: Textbook Exercises and Multiple-Choice Testbanks

PubMed Central

Slepkov, Aaron D.; Ironside, Kevin B.; DiBattista, David

2015-01-01

Benford’s Law describes the finding that the distribution of leading (or leftmost) digits of innumerable datasets follows a well-defined logarithmic trend, rather than an intuitive uniformity. In practice this means that the most common leading digit is 1, with an expected frequency of 30.1%, and the least common is 9, with an expected frequency of 4.6%. Currently, the most common application of Benford’s Law is in detecting number invention and tampering such as found in accounting-, tax-, and voter-fraud. We demonstrate that answers to end-of-chapter exercises in physics and chemistry textbooks conform to Benford’s Law. Subsequently, we investigate whether this fact can be used to gain advantage over random guessing in multiple-choice tests, and find that while testbank answers in introductory physics closely conform to Benford’s Law, the testbank is nonetheless secure against such a Benford’s attack for banal reasons. PMID:25689468

How did you guess? Or, what do multiple-choice questions measure?

PubMed

Cox, K R

1976-06-05

Multiple-choice questions classified as requiring problem-solving skills have been interpreted as measuring problem-solving skills within students, with the implicit hypothesis that questions needing an increasingly complex intellectual process should present increasing difficulty to the student. This hypothesis was tested in a 150-question paper taken by 721 students in seven Australian medical schools. No correlation was observed between difficulty and assigned process. Consequently, the question-answering process was explored with a group of final-year students. Anecdotal recall by students gave heavy weight to knowledge rather than problem solving in answering these questions. Assignment of the 150 questions to the classification by three teachers and six students showed their congruence to be a little above random probability.

Undergraduate Students' Preferences for Constructed versus Multiple-Choice Assessment of Learning

ERIC Educational Resources Information Center

Mingo, Maya A.; Chang, Hsin-Hui; Williams, Robert L.

2018-01-01

Students (N = 161) in seven sections of an undergraduate educational psychology course rated ten performance-assessment options in collegiate courses. They rated in-class essay exams as their most preferred assessment and multiple-choice exams (in-class and out-of-class) as their least preferred. Also, student ratings of multiple papers and a term…

On School Choice and Test-Based Accountability

ERIC Educational Resources Information Center

Betebenner, Damian W.; Howe, Kenneth R.; Foster, Samara S.

2005-01-01

Among the two most prominent school reform measures currently being implemented in The United States are school choice and test-based accountability. Until recently, the two policy initiatives remained relatively distinct from one another. With the passage of the No Child Left Behind Act of 2001 (NCLB), a mutualism between choice and…

A multiple indicator solution approach to endogeneity in discrete-choice models for environmental valuation.

PubMed

Mariel, Petr; Hoyos, David; Artabe, Alaitz; Guevara, C Angelo

2018-08-15

Endogeneity is an often neglected issue in empirical applications of discrete choice modelling despite its severe consequences in terms of inconsistent parameter estimation and biased welfare measures. This article analyses the performance of the multiple indicator solution method to deal with endogeneity arising from omitted explanatory variables in discrete choice models for environmental valuation. We also propose and illustrate a factor analysis procedure for the selection of the indicators in practice. Additionally, the performance of this method is compared with the recently proposed hybrid choice modelling framework. In an empirical application we find that the multiple indicator solution method and the hybrid model approach provide similar results in terms of welfare estimates, although the multiple indicator solution method is more parsimonious and notably easier to implement. The empirical results open a path to explore the performance of this method when endogeneity is thought to have a different cause or under a different set of indicators. Copyright © 2018 Elsevier B.V. All rights reserved.

Using the Multiple Choice Procedure to Measure College Student Gambling

ERIC Educational Resources Information Center

Butler, Leon Harvey

2010-01-01

Research suggests that gambling is similar to addictive behaviors such as substance use. In the current study, gambling was investigated from a behavioral economics perspective. The Multiple Choice Procedure (MCP) with gambling as the target behavior was used to assess for relative reinforcing value, the effect of alternative reinforcers, and…

Negatively-Worded Multiple Choice Questions: An Avoidable Threat to Validity

ERIC Educational Resources Information Center

Chiavaroli, Neville

2017-01-01

Despite the majority of MCQ writing guides discouraging the use of negatively-worded multiple choice questions (NWQs), they continue to be regularly used both in locally produced examinations and commercially available questions. There are several reasons why the use of NWQs may prove resistant to sound pedagogical advice. Nevertheless, systematic…

FormScanner: Open-Source Solution for Grading Multiple-Choice Exams

ERIC Educational Resources Information Center

Young, Chadwick; Lo, Glenn; Young, Kaisa; Borsetta, Alberto

2016-01-01

The multiple-choice exam remains a staple for many introductory physics courses. In the past, people have graded these by hand or even flaming needles. Today, one usually grades the exams with a form scanner that utilizes optical mark recognition (OMR). Several companies provide these scanners and particular forms, such as the eponymous…

The Development of Multiple-Choice Items Consistent with the AP Chemistry Curriculum Framework to More Accurately Assess Deeper Understanding

ERIC Educational Resources Information Center

Domyancich, John M.

2014-01-01

Multiple-choice questions are an important part of large-scale summative assessments, such as the advanced placement (AP) chemistry exam. However, past AP chemistry exam items often lacked the ability to test conceptual understanding and higher-order cognitive skills. The redesigned AP chemistry exam shows a distinctive shift in item types toward…

The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments.

PubMed

Tarrant, Marie; Knierim, Aimee; Hayes, Sasha K; Ware, James

2006-12-01

Multiple-choice questions are a common assessment method in nursing examinations. Few nurse educators, however, have formal preparation in constructing multiple-choice questions. Consequently, questions used in baccalaureate nursing assessments often contain item-writing flaws, or violations to accepted item-writing guidelines. In one nursing department, 2770 MCQs were collected from tests and examinations administered over a five-year period from 2001 to 2005. Questions were evaluated for 19 frequently occurring item-writing flaws, for cognitive level, for question source, and for the distribution of correct answers. Results show that almost half (46.2%) of the questions contained violations of item-writing guidelines and over 90% were written at low cognitive levels. Only a small proportion of questions were teacher generated (14.1%), while 36.2% were taken from testbanks and almost half (49.4%) had no source identified. MCQs written at a lower cognitive level were significantly more likely to contain item-writing flaws. While there was no relationship between the source of the question and item-writing flaws, teacher-generated questions were more likely to be written at higher cognitive levels (p<0.001). Correct answers were evenly distributed across all four options and no bias was noted in the placement of correct options. Further training in item-writing is recommended for all faculty members who are responsible for developing tests. Pre-test review and quality assessment is also recommended to reduce the occurrence of item-writing flaws and to improve the quality of test questions.

Guide to Developing High-Quality, Reliable, and Valid Multiple-Choice Assessments

ERIC Educational Resources Information Center

Towns, Marcy H.

2014-01-01

Chemistry faculty members are highly skilled in obtaining, analyzing, and interpreting physical measurements, but often they are less skilled in measuring student learning. This work provides guidance for chemistry faculty from the research literature on multiple-choice item development in chemistry. Areas covered include content, stem, and…

Cheating on Multiple-Choice Exams: Monitoring, Assessment, and an Optional Assignment

ERIC Educational Resources Information Center

Nath, Leda; Lovaglia, Michael

2009-01-01

Academic dishonesty is unethical. Exam cheating is viewed as more serious than most other forms (Pincus and Schmelkin 2003). The authors review the general cheating problem, introduce a program to conservatively identify likely cheaters on multiple-choice exams, and offer a procedure for handling likely cheaters. Feedback from students who confess…

QTest: Quantitative Testing of Theories of Binary Choice.

PubMed

Regenwetter, Michel; Davis-Stober, Clintin P; Lim, Shiau Hong; Guo, Ying; Popova, Anna; Zwilling, Chris; Cha, Yun-Shil; Messner, William

2014-01-01

The goal of this paper is to make modeling and quantitative testing accessible to behavioral decision researchers interested in substantive questions. We provide a novel, rigorous, yet very general, quantitative diagnostic framework for testing theories of binary choice. This permits the nontechnical scholar to proceed far beyond traditionally rather superficial methods of analysis, and it permits the quantitatively savvy scholar to triage theoretical proposals before investing effort into complex and specialized quantitative analyses. Our theoretical framework links static algebraic decision theory with observed variability in behavioral binary choice data. The paper is supplemented with a custom-designed public-domain statistical analysis package, the QTest software. We illustrate our approach with a quantitative analysis using published laboratory data, including tests of novel versions of "Random Cumulative Prospect Theory." A major asset of the approach is the potential to distinguish decision makers who have a fixed preference and commit errors in observed choices from decision makers who waver in their preferences.

QTest: Quantitative Testing of Theories of Binary Choice

PubMed Central

Regenwetter, Michel; Davis-Stober, Clintin P.; Lim, Shiau Hong; Guo, Ying; Popova, Anna; Zwilling, Chris; Cha, Yun-Shil; Messner, William

2014-01-01

The goal of this paper is to make modeling and quantitative testing accessible to behavioral decision researchers interested in substantive questions. We provide a novel, rigorous, yet very general, quantitative diagnostic framework for testing theories of binary choice. This permits the nontechnical scholar to proceed far beyond traditionally rather superficial methods of analysis, and it permits the quantitatively savvy scholar to triage theoretical proposals before investing effort into complex and specialized quantitative analyses. Our theoretical framework links static algebraic decision theory with observed variability in behavioral binary choice data. The paper is supplemented with a custom-designed public-domain statistical analysis package, the QTest software. We illustrate our approach with a quantitative analysis using published laboratory data, including tests of novel versions of “Random Cumulative Prospect Theory.” A major asset of the approach is the potential to distinguish decision makers who have a fixed preference and commit errors in observed choices from decision makers who waver in their preferences. PMID:24999495

NHA choice test : feeding dilemma for Reticulitermes flavipes (Kollar)

Treesearch

Frederick Green; S. Nami Kartal; Patricia K. Lebow; Rachel A. Arango

2011-01-01

An AWPA laboratory choice test was performed with Reticulitermes flavipes (Kollar) to determine the role of feeding stimulation or aggregation by N’N napthaloylhydroxylamine (NHA). Pine blocks were vacuum treated with aqueous concentrations of NHA ranging from 250-10,000ppm and exposed to one gram R. flavipes workers in the dual block choice test which included one...

Kernel Machine SNP-set Testing under Multiple Candidate Kernels

PubMed Central

Wu, Michael C.; Maity, Arnab; Lee, Seunggeun; Simmons, Elizabeth M.; Harmon, Quaker E.; Lin, Xinyi; Engel, Stephanie M.; Molldrem, Jeffrey J.; Armistead, Paul M.

2013-01-01

Joint testing for the cumulative effect of multiple single nucleotide polymorphisms grouped on the basis of prior biological knowledge has become a popular and powerful strategy for the analysis of large scale genetic association studies. The kernel machine (KM) testing framework is a useful approach that has been proposed for testing associations between multiple genetic variants and many different types of complex traits by comparing pairwise similarity in phenotype between subjects to pairwise similarity in genotype, with similarity in genotype defined via a kernel function. An advantage of the KM framework is its flexibility: choosing different kernel functions allows for different assumptions concerning the underlying model and can allow for improved power. In practice, it is difficult to know which kernel to use a priori since this depends on the unknown underlying trait architecture and selecting the kernel which gives the lowest p-value can lead to inflated type I error. Therefore, we propose practical strategies for KM testing when multiple candidate kernels are present based on constructing composite kernels and based on efficient perturbation procedures. We demonstrate through simulations and real data applications that the procedures protect the type I error rate and can lead to substantially improved power over poor choices of kernels and only modest differences in power versus using the best candidate kernel. PMID:23471868

Free-Response and Multiple-Choice Items: Measures of the Same Ability?

ERIC Educational Resources Information Center

Bennett, Randy Elliot; And Others

This study examined the relationship of multiple-choice and free-response items contained on the College Board's Advanced Placement Computer Science (APCS) examination. Subjects were two samples of 1,000 randomly drawn from the population of 7,372 high school students taking the 1988 examination of the APCS "AB" form. Most were high…

How do STEM-interested students pursue multiple interests in their higher educational choice?

NASA Astrophysics Data System (ADS)

Vulperhorst, Jonne Pieter; Wessels, Koen Rens; Bakker, Arthur; Akkerman, Sanne Floor

2018-05-01

Interest in science, technology, engineering and mathematics (STEM) has lately received attention in research due to a gap between the number of STEM students and the needs of the labour market. As interest seems to be one of the most important factors in deciding what to study, we focus in the present study on how STEM-interested students weigh multiple interests in making educational choices. A questionnaire with both open-ended and closed-ended items was administered to 91 STEM-interested students enrolled in a STEM programme of a Dutch University for secondary school students. Results indicate that students find it important that a study programme allows them to pursue multiple interests. Some students pursued multiple interests by choosing to enrol in two programmes at the same time. Most students chose one programme that enabled them to combine multiple interests. Combinations of pursued interests were dependent on the disciplinary range of interests of students. Students who were interested in diverse domains combined interests in an educational programme across academic and non-academic domains, whilst students who were mainly interested in STEM combined only STEM-focused interests. Together these findings stress the importance of taking a multiple interest perspective on interest development and educational choice.

Multiple Choice Knapsack Problem: example of planning choice in transportation.

PubMed

Zhong, Tao; Young, Rhonda

2010-05-01

Transportation programming, a process of selecting projects for funding given budget and other constraints, is becoming more complex as a result of new federal laws, local planning regulations, and increased public involvement. This article describes the use of an integer programming tool, Multiple Choice Knapsack Problem (MCKP), to provide optimal solutions to transportation programming problems in cases where alternative versions of projects are under consideration. In this paper, optimization methods for use in the transportation programming process are compared and then the process of building and solving the optimization problems is discussed. The concepts about the use of MCKP are presented and a real-world transportation programming example at various budget levels is provided. This article illustrates how the use of MCKP addresses the modern complexities and provides timely solutions in transportation programming practice. While the article uses transportation programming as a case study, MCKP can be useful in other fields where a similar decision among a subset of the alternatives is required. Copyright 2009 Elsevier Ltd. All rights reserved.

Test-Taking Strategies of Arab EFL Learners on Multiple Choice Tests

ERIC Educational Resources Information Center

Al Fraidan, Abdullah; Al-Khalaf, Khadija

2012-01-01

Many studies have focused on the function of learners' strategies in a variety of EFL domains. However, research on test-taking strategies (TTSs) has been limited, even though such strategies might influence test scores and, as a result, test validity. Motivated by this fact and in light of our own experience as EFL test-makers, this article will…

An intuitive graphical webserver for multiple-choice protein sequence search.

PubMed

Banky, Daniel; Szalkai, Balazs; Grolmusz, Vince

2014-04-10

Every day tens of thousands of sequence searches and sequence alignment queries are submitted to webservers. The capitalized word "BLAST" becomes a verb, describing the act of performing sequence search and alignment. However, if one needs to search for sequences that contain, for example, two hydrophobic and three polar residues at five given positions, the query formation on the most frequently used webservers will be difficult. Some servers support the formation of queries with regular expressions, but most of the users are unfamiliar with their syntax. Here we present an intuitive, easily applicable webserver, the Protein Sequence Analysis server, that allows the formation of multiple choice queries by simply drawing the residues to their positions; if more than one residue are drawn to the same position, then they will be nicely stacked on the user interface, indicating the multiple choice at the given position. This computer-game-like interface is natural and intuitive, and the coloring of the residues makes possible to form queries requiring not just certain amino acids in the given positions, but also small nonpolar, negatively charged, hydrophobic, positively charged, or polar ones. The webserver is available at http://psa.pitgroup.org. Copyright © 2014 Elsevier B.V. All rights reserved.

Modeling Incorrect Responses to Multiple-Choice Items with Multilinear Formula Score Theory.

ERIC Educational Resources Information Center

Drasgow, Fritz; And Others

This paper addresses the information revealed in incorrect option selection on multiple choice items. Multilinear Formula Scoring (MFS), a theory providing methods for solving psychological measurement problems of long standing, is first used to estimate option characteristic curves for the Armed Services Vocational Aptitude Battery Arithmetic…

FormScanner: Open-Source Solution for Grading Multiple-Choice Exams

NASA Astrophysics Data System (ADS)

Young, Chadwick; Lo, Glenn; Young, Kaisa; Borsetta, Alberto

2016-01-01

The multiple-choice exam remains a staple for many introductory physics courses. In the past, people have graded these by hand or even flaming needles. Today, one usually grades the exams with a form scanner that utilizes optical mark recognition (OMR). Several companies provide these scanners and particular forms, such as the eponymous "Scantron." OMR scanners combine hardware and software—a scanner and OMR program—to read and grade student-filled forms.

Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper

PubMed Central

Palmer, Edward J; Devitt, Peter G

2007-01-01

Background Reliable and valid written tests of higher cognitive function are difficult to produce, particularly for the assessment of clinical problem solving. Modified Essay Questions (MEQs) are often used to assess these higher order abilities in preference to other forms of assessment, including multiple-choice questions (MCQs). MEQs often form a vital component of end-of-course assessments in higher education. It is not clear how effectively these questions assess higher order cognitive skills. This study was designed to assess the effectiveness of the MEQ to measure higher-order cognitive skills in an undergraduate institution. Methods An analysis of multiple-choice questions and modified essay questions (MEQs) used for summative assessment in a clinical undergraduate curriculum was undertaken. A total of 50 MCQs and 139 stages of MEQs were examined, which came from three exams run over two years. The effectiveness of the questions was determined by two assessors and was defined by the questions ability to measure higher cognitive skills, as determined by a modification of Bloom's taxonomy, and its quality as determined by the presence of item writing flaws. Results Over 50% of all of the MEQs tested factual recall. This was similar to the percentage of MCQs testing factual recall. The modified essay question failed in its role of consistently assessing higher cognitive skills whereas the MCQ frequently tested more than mere recall of knowledge. Conclusion Construction of MEQs, which will assess higher order cognitive skills cannot be assumed to be a simple task. Well-constructed MCQs should be considered a satisfactory replacement for MEQs if the MEQs cannot be designed to adequately test higher order skills. Such MCQs are capable of withstanding the intellectual and statistical scrutiny imposed by a high stakes exit examination. PMID:18045500

Visual Attention for Solving Multiple-Choice Science Problem: An Eye-Tracking Analysis

ERIC Educational Resources Information Center

Tsai, Meng-Jung; Hou, Huei-Tse; Lai, Meng-Lung; Liu, Wan-Yi; Yang, Fang-Ying

2012-01-01

This study employed an eye-tracking technique to examine students' visual attention when solving a multiple-choice science problem. Six university students participated in a problem-solving task to predict occurrences of landslide hazards from four images representing four combinations of four factors. Participants' responses and visual attention…

Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables

PubMed Central

Kennerley, Steven W.; Wallis, Jonathan D.

2009-01-01

Damage to the frontal lobe can cause severe decision-making impairments. A mechanism that may underlie this is that neurons in the frontal cortex encode many variables that contribute to the valuation of a choice, such as its costs, benefits and probability of success. However, optimal decision-making requires that one considers these variables, not only when faced with the choice, but also when evaluating the outcome of the choice, in order to adapt future behaviour appropriately. To examine the role of the frontal cortex in encoding the value of different choice outcomes, we simultaneously recorded the activity of multiple single neurons in the anterior cingulate cortex (ACC), orbitofrontal cortex (OFC) and lateral prefrontal cortex (LPFC) while subjects evaluated the outcome of choices involving manipulations of probability, payoff and cost. Frontal neurons encoded many of the parameters that enabled the calculation of the value of these variables, including the onset and offset of reward and the amount of work performed, and often encoded the value of outcomes across multiple decision variables. In addition, many neurons encoded both the predicted outcome during the choice phase of the task as well as the experienced outcome in the outcome phase of the task. These patterns of selectivity were more prevalent in ACC relative to OFC and LPFC. These results support a role for the frontal cortex, principally ACC, in selecting between choice alternatives and evaluating the outcome of that selection thereby ensuring that choices are optimal and adaptive. PMID:19453638

Facilitating informed choice in prenatal testing: how well are we doing?

PubMed

Marteau, T M; Dormandy, E

2001-01-01

There is a consensus that prenatal testing services need to provide the information and support necessary for women to make informed choices about prenatal testing. Informed choices are those based on relevant information that reflect the decision-maker's values. To date, most research has focused on the information provided to women deciding whether to undergo tests. This has highlighted the poor quality of information provided to many women. There is agreement on the need to provide information on three key aspects of any test: the condition for which testing is being offered, characteristics of the test, and the implications of testing. Very little research has been conducted on decisions after the diagnosis of a fetal abnormality and how information and emotional and decisional support are and should be provided. Research is now needed in four key areas: first, on the optimal ways of organizing services to facilitate choices that are not only based on relevant information, but also reflect the decision-maker's values; second, on the most effective ways of framing information needed for the different decisions involved in prenatal testing; third, on the most effective media in which to deliver information; and, fourth, to identify aspects of counseling that facilitate informed choices following diagnoses of fetal abnormality. If we value women's ability to make informed choices about prenatal tests as highly as we value reliable laboratory tests, evidence-based quality standards need to be developed for the information and support women are given at all stages of the process of prenatal testing.

Measuring University students' understanding of the greenhouse effect - a comparison of multiple-choice, short answer and concept sketch assessment tools with respect to students' mental models

NASA Astrophysics Data System (ADS)

Gold, A. U.; Harris, S. E.

2013-12-01

The greenhouse effect comes up in most discussions about climate and is a key concept related to climate change. Existing studies have shown that students and adults alike lack a detailed understanding of this important concept or might hold misconceptions. We studied the effectiveness of different interventions on University-level students' understanding of the greenhouse effect. Introductory level science students were tested for their pre-knowledge of the greenhouse effect using validated multiple-choice questions, short answers and concept sketches. All students participated in a common lesson about the greenhouse effect and were then randomly assigned to one of two lab groups. One group explored an existing simulation about the greenhouse effect (PhET-lesson) and the other group worked with absorption spectra of different greenhouse gases (Data-lesson) to deepen the understanding of the greenhouse effect. All students completed the same assessment including multiple choice, short answers and concept sketches after participation in their lab lesson. 164 students completed all the assessments, 76 completed the PhET lesson and 77 completed the data lesson. 11 students missed the contrasting lesson. In this presentation we show the comparison between the multiple-choice questions, short answer questions and the concept sketches of students. We explore how well each of these assessment types represents student's knowledge. We also identify items that are indicators of the level of understanding of the greenhouse effect as measured in correspondence of student answers to an expert mental model and expert responses. Preliminary data analysis shows that student who produce concept sketch drawings that come close to expert drawings also choose correct multiple-choice answers. However, correct multiple-choice answers are not necessarily an indicator that a student produces an expert-like correlating concept sketch items. Multiple-choice questions that require detailed

Written Justifications to Multiple-Choice Concept Questions during Active Learning in Class

ERIC Educational Resources Information Center

Koretsky, Milo D.; Brooks, Bill J.; Higgins, Adam Z.

2016-01-01

Increasingly, instructors of large, introductory STEM courses are having students actively engage during class by answering multiple-choice concept questions individually and in groups. This study investigates the use of a technology-based tool that allows students to answer such questions during class. The tool also allows the instructor to…

Writing Multiple Choice Outcome Questions to Assess Knowledge and Competence.

PubMed

Brady, Erik D

2015-11-01

Few articles contemplate the need for good guidance in question item-writing in the continuing education (CE) space. Although many of the core principles of sound item design translate to the CE health education team, the need exists for specific examples for nurse educators that clearly describe how to measure changes in competence and knowledge using multiple choice items. In this article, some keys points and specific examples for nursing CE providers are shared. Copyright 2015, SLACK Incorporated.

Behavioral economic analysis of drug preference using multiple choice procedure data.

PubMed

Greenwald, Mark K

2008-01-11

The multiple choice procedure has been used to evaluate preference for psychoactive drugs, relative to money amounts (price), in human subjects. The present re-analysis shows that MCP data are compatible with behavioral economic analysis of drug choices. Demand curves were constructed from studies with intravenous fentanyl, intramuscular hydromorphone and oral methadone in opioid-dependent individuals; oral d-amphetamine, oral MDMA alone and during fluoxetine treatment, and smoked marijuana alone or following naltrexone pretreatment in recreational drug users. For each participant and dose, the MCP crossover point was converted into unit price (UP) by dividing the money value ($) by the drug dose (mg/70kg). At the crossover value, the dose ceases to function as a reinforcer, so "0" was entered for this and higher UPs to reflect lack of drug choice. At lower UPs, the dose functions as a reinforcer and "1" was entered to reflect drug choice. Data for UP vs. average percent choice were plotted in log-log space to generate demand functions. Rank of order of opioid inelasticity (slope of non-linear regression) was: fentanyl>hydromorphone (continuing heroin users)>methadone>hydromorphone (heroin abstainers). Rank order of psychostimulant inelasticity was d-amphetamine>MDMA>MDMA+fluoxetine. Smoked marijuana was more inelastic with high-dose naltrexone. These findings show this method translates individuals' drug preferences into estimates of population demand, which has the potential to yield insights into pharmacotherapy efficacy, abuse liability assessment, and individual differences in susceptibility to drug abuse.

Multiple data sets and modelling choices in a comparative LCA of disposable beverage cups.

PubMed

van der Harst, Eugenie; Potting, José; Kroeze, Carolien

2014-10-01

This study used multiple data sets and modelling choices in an environmental life cycle assessment (LCA) to compare typical disposable beverage cups made from polystyrene (PS), polylactic acid (PLA; bioplastic) and paper lined with bioplastic (biopaper). Incineration and recycling were considered as waste processing options, and for the PLA and biopaper cup also composting and anaerobic digestion. Multiple data sets and modelling choices were systematically used to calculate average results and the spread in results for each disposable cup in eleven impact categories. The LCA results of all combinations of data sets and modelling choices consistently identify three processes that dominate the environmental impact: (1) production of the cup's basic material (PS, PLA, biopaper), (2) cup manufacturing, and (3) waste processing. The large spread in results for impact categories strongly overlaps among the cups, however, and therefore does not allow a preference for one type of cup material. Comparison of the individual waste treatment options suggests some cautious preferences. The average waste treatment results indicate that recycling is the preferred option for PLA cups, followed by anaerobic digestion and incineration. Recycling is slightly preferred over incineration for the biopaper cups. There is no preferred waste treatment option for the PS cups. Taking into account the spread in waste treatment results for all cups, however, none of these preferences for waste processing options can be justified. The only exception is composting, which is least preferred for both PLA and biopaper cups. Our study illustrates that using multiple data sets and modelling choices can lead to considerable spread in LCA results. This makes comparing products more complex, but the outcomes more robust. Copyright © 2014 Elsevier B.V. All rights reserved.

Modeling Confidence Judgments, Response Times, and Multiple Choices in Decision Making: Recognition Memory and Motion Discrimination

PubMed Central

Ratcliff, Roger; Starns, Jeffrey J.

2014-01-01

Confidence in judgments is a fundamental aspect of decision making, and tasks that collect confidence judgments are an instantiation of multiple-choice decision making. We present a model for confidence judgments in recognition memory tasks that uses a multiple-choice diffusion decision process with separate accumulators of evidence for the different confidence choices. The accumulator that first reaches its decision boundary determines which choice is made. Five algorithms for accumulating evidence were compared, and one of them produced proportions of responses for each of the choices and full response time distributions for each choice that closely matched empirical data. With this algorithm, an increase in the evidence in one accumulator is accompanied by a decrease in the others so that the total amount of evidence in the system is constant. Application of the model to the data from an earlier experiment (Ratcliff, McKoon, & Tindall, 1994) uncovered a relationship between the shapes of z-transformed receiver operating characteristics and the behavior of response time distributions. Both are explained in the model by the behavior of the decision boundaries. For generality, we also applied the decision model to a 3-choice motion discrimination task and found it accounted for data better than a competing class of models. The confidence model presents a coherent account of confidence judgments and response time that cannot be explained with currently popular signal detection theory analyses or dual-process models of recognition. PMID:23915088

Grading Multiple Choice Exams with Low-Cost and Portable Computer-Vision Techniques

ERIC Educational Resources Information Center

Fisteus, Jesus Arias; Pardo, Abelardo; García, Norberto Fernández

2013-01-01

Although technology for automatic grading of multiple choice exams has existed for several decades, it is not yet as widely available or affordable as it should be. The main reasons preventing this adoption are the cost and the complexity of the setup procedures. In this paper, "Eyegrade," a system for automatic grading of multiple…

Developing an Array Binary Code Assessment Rubric for Multiple- Choice Questions Using Item Arrays and Binary-Coded Responses

ERIC Educational Resources Information Center

Haro, Elizabeth K.; Haro, Luis S.

2014-01-01

The multiple-choice question (MCQ) is the foundation of knowledge assessment in K-12, higher education, and standardized entrance exams (including the GRE, MCAT, and DAT). However, standard MCQ exams are limited with respect to the types of questions that can be asked when there are only five choices. MCQs offering additional choices more…

Grading Multiple Choice Exams with Low-Cost and Portable Computer-Vision Techniques

NASA Astrophysics Data System (ADS)

Fisteus, Jesus Arias; Pardo, Abelardo; García, Norberto Fernández

2013-08-01

Although technology for automatic grading of multiple choice exams has existed for several decades, it is not yet as widely available or affordable as it should be. The main reasons preventing this adoption are the cost and the complexity of the setup procedures. In this paper, Eyegrade, a system for automatic grading of multiple choice exams is presented. While most current solutions are based on expensive scanners, Eyegrade offers a truly low-cost solution requiring only a regular off-the-shelf webcam. Additionally, Eyegrade performs both mark recognition as well as optical character recognition of handwritten student identification numbers, which avoids the use of bubbles in the answer sheet. When compared with similar webcam-based systems, the user interface in Eyegrade has been designed to provide a more efficient and error-free data collection procedure. The tool has been validated with a set of experiments that show the ease of use (both setup and operation), the reduction in grading time, and an increase in the reliability of the results when compared with conventional, more expensive systems.

Responses of horses offered a choice between stables containing single or multiple forages.

PubMed

Goodwin, D; Davidson, H P B; Harris, P

2007-04-21

To investigate the choices of foraging location of horses, 10 to 12 horses were introduced for five minutes into each of two similar stables containing a single forage or six forages, in four replicated trials. The horses were then removed and released into the gangway between the stables, and allowed five minutes to choose between the stables. Their initial and final choices, mean duration in each stable and proportional frequency of change of location were compared. Most of the horses initially entered the closest stable on release (P<0.05); if the closest stable contained a single hay, most horses transferred to the stable containing multiple forages (P<0.001). The length of time spent by the horses in the two stables suggested that they preferred multiple forages in multiple locations (P<0.001). Eleven horses moved from one stable to the other on one or more occasions during trials when hay or a preferred forage was available in both stables, possibly indicating a motivation to move between foraging locations regardless of the palatability of the forages offered or the horses' preference for a forage.

A labelled discrete choice experiment adds realism to the choices presented: preferences for surveillance tests for Barrett esophagus

PubMed Central

2009-01-01

Background Discrete choice experiments (DCEs) allow systematic assessment of preferences by asking respondents to choose between scenarios. We conducted a labelled discrete choice experiment with realistic choices to investigate patients' trade-offs between the expected health gains and the burden of testing in surveillance of Barrett esophagus (BE). Methods Fifteen choice scenarios were selected based on 2 attributes: 1) type of test (endoscopy and two less burdensome fictitious tests), 2) frequency of surveillance. Each test-frequency combination was associated with its own realistic decrease in risk of dying from esophageal adenocarcinoma. A conditional logit model was fitted. Results Of 297 eligible patients (155 BE and 142 with non-specific upper GI symptoms), 247 completed the questionnaire (84%). Patients preferred surveillance to no surveillance. Current surveillance schemes of once every 1–2 years were amongst the most preferred alternatives. Higher health gains were preferred over those with lower health gains, except when test frequencies exceeded once a year. For similar health gains, patients preferred video-capsule over saliva swab and least preferred endoscopy. Conclusion This first example of a labelled DCE using realistic scenarios in a healthcare context shows that such experiments are feasible. A comparison of labelled and unlabelled designs taking into account setting and research question is recommended. PMID:19454022

Exploring problem solving strategies on multiple-choice science items: Comparing native Spanish-speaking English Language Learners and mainstream monolinguals

NASA Astrophysics Data System (ADS)

Kachchaf, Rachel Rae

The purpose of this study was to compare how English language learners (ELLs) and monolingual English speakers solved multiple-choice items administered with and without a new form of testing accommodation---vignette illustration (VI). By incorporating theories from second language acquisition, bilingualism, and sociolinguistics, this study was able to gain more accurate and comprehensive input into the ways students interacted with items. This mixed methods study used verbal protocols to elicit the thinking processes of thirty-six native Spanish-speaking English language learners (ELLs), and 36 native-English speaking non-ELLs when solving multiple-choice science items. Results from both qualitative and quantitative analyses show that ELLs used a wider variety of actions oriented to making sense of the items than non-ELLs. In contrast, non-ELLs used more problem solving strategies than ELLs. There were no statistically significant differences in student performance based on the interaction of presence of illustration and linguistic status or the main effect of presence of illustration. However, there were significant differences based on the main effect of linguistic status. An interaction between the characteristics of the students, the items, and the illustrations indicates considerable heterogeneity in the ways in which students from both linguistic groups think about and respond to science test items. The results of this study speak to the need for more research involving ELLs in the process of test development to create test items that do not require ELLs to carry out significantly more actions to make sense of the item than monolingual students.

Predicting social and communicative ability in school-age children with autism spectrum disorder: A pilot study of the Social Attribution Task, Multiple Choice.

PubMed

Burger-Caplan, Rebecca; Saulnier, Celine; Jones, Warren; Klin, Ami

2016-11-01

The Social Attribution Task, Multiple Choice is introduced as a measure of implicit social cognitive ability in children, addressing a key challenge in quantification of social cognitive function in autism spectrum disorder, whereby individuals can often be successful in explicit social scenarios, despite marked social adaptive deficits. The 19-question Social Attribution Task, Multiple Choice, which presents ambiguous stimuli meant to elicit social attribution, was administered to children with autism spectrum disorder (N = 23) and to age-matched and verbal IQ-matched typically developing children (N = 57). The Social Attribution Task, Multiple Choice performance differed between autism spectrum disorder and typically developing groups, with typically developing children performing significantly better than children with autism spectrum disorder. The Social Attribution Task, Multiple Choice scores were positively correlated with age (r = 0.474) while being independent from verbal IQ (r = 0.236). The Social Attribution Task, Multiple Choice was strongly correlated with Vineland Adaptive Behavior Scales Communication (r = 0.464) and Socialization (r = 0.482) scores, but not with Daily Living Skills scores (r = 0.116), suggesting that the implicit social cognitive ability underlying performance on the Social Attribution Task, Multiple Choice is associated with real-life social adaptive function. © The Author(s) 2016.

Behavioral Economic Analysis of Drug Preference Using Multiple Choice Procedure Data

PubMed Central

Greenwald, Mark K.

2008-01-01

The Multiple Choice Procedure has been used to evaluate preference for psychoactive drugs, relative to money amounts (price), in human subjects. The present re-analysis shows that MCP data are compatible with behavioral economic analysis of drug choices. Demand curves were constructed from studies with intravenous fentanyl, intramuscular hydromorphone and oral methadone in opioid-dependent individuals; oral d-amphetamine, oral MDMA alone and during fluoxetine treatment, and smoked marijuana alone or following naltrexone pretreatment in recreational drug users. For each participant and dose, the MCP crossover point was converted into unit price (UP) by dividing the money value ($) by the drug dose (mg/70 kg). At the crossover value, the dose ceases to function as a reinforcer, so “0” was entered for this and higher UPs to reflect lack of drug choice. At lower UPs, the dose functions as a reinforcer and “1” was entered to reflect drug choice. Data for UP vs. average percent choice were plotted in log-log space to generate demand functions. Rank of order of opioid inelasticity (slope of non-linear regression) was: fentanyl > hydromorphone (continuing heroin users) > methadone > hydromorphone (heroin abstainers). Rank order of psychostimulant inelasticity was d-amphetamine > MDMA > MDMA + fluoxetine. Smoked marijuana was more inelastic with high-dose naltrexone. These findings show this method translates individuals’ drug preferences into estimates of population demand, which has the potential to yield insights into pharmacotherapy efficacy, abuse liability assessment, and individual differences in susceptibility to drug abuse. PMID:17949924

An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis.

PubMed

Tarrant, Marie; Ware, James; Mohammed, Ahmed M

2009-07-07

Four- or five-option multiple choice questions (MCQs) are the standard in health-science disciplines, both on certification-level examinations and on in-house developed tests. Previous research has shown, however, that few MCQs have three or four functioning distractors. The purpose of this study was to investigate non-functioning distractors in teacher-developed tests in one nursing program in an English-language university in Hong Kong. Using item-analysis data, we assessed the proportion of non-functioning distractors on a sample of seven test papers administered to undergraduate nursing students. A total of 514 items were reviewed, including 2056 options (1542 distractors and 514 correct responses). Non-functioning options were defined as ones that were chosen by fewer than 5% of examinees and those with a positive option discrimination statistic. The proportion of items containing 0, 1, 2, and 3 functioning distractors was 12.3%, 34.8%, 39.1%, and 13.8% respectively. Overall, items contained an average of 1.54 (SD = 0.88) functioning distractors. Only 52.2% (n = 805) of all distractors were functioning effectively and 10.2% (n = 158) had a choice frequency of 0. Items with more functioning distractors were more difficult and more discriminating. The low frequency of items with three functioning distractors in the four-option items in this study suggests that teachers have difficulty developing plausible distractors for most MCQs. Test items should consist of as many options as is feasible given the item content and the number of plausible distractors; in most cases this would be three. Item analysis results can be used to identify and remove non-functioning distractors from MCQs that have been used in previous tests.

The Use of Management and Marketing Textbook Multiple-Choice Questions: A Case Study.

ERIC Educational Resources Information Center

Hampton, David R.; And Others

1993-01-01

Four management and four marketing professors classified multiple-choice questions in four widely adopted introductory textbooks according to the two levels of Bloom's taxonomy of educational objectives: knowledge and intellectual ability and skill. Inaccuracies may cause instructors to select questions that require less thinking than they intend.…

Assessing understanding of relative clauses: a comparison of multiple-choice comprehension versus sentence repetition.

PubMed

Frizelle, Pauline; O'Neill, Clodagh; Bishop, Dorothy V M

2017-11-01

Although sentence repetition is considered a reliable measure of children's grammatical knowledge, few studies have directly compared children's sentence repetition performance with their understanding of grammatical structures. The current study aimed to compare children's performance on these two assessment measures, using a multiple-choice picture-matching sentence comprehension task and a sentence repetition task. Thirty-three typically developing children completed both assessments, which included relative clauses representing a range of syntactic roles. Results revealed a similar order of difficulty of constructions on both measures but little agreement between them when evaluating individual differences. Interestingly, repetition was the easier of the two measures, with children showing the ability to repeat sentences they did not understand. This discrepancy is primarily attributed to the additional processing load resulting from the design of multiple-choice comprehension tasks, and highlights the fact that these assessments are invoking skills beyond those of linguistic competence.

Cost analysis for computer supported multiple-choice paper examinations

PubMed Central

Mandel, Alexander; Hörnlein, Alexander; Ifland, Marianus; Lüneburg, Edeltraud; Deckert, Jürgen; Puppe, Frank

2011-01-01

Introduction: Multiple-choice-examinations are still fundamental for assessment in medical degree programs. In addition to content related research, the optimization of the technical procedure is an important question. Medical examiners face three options: paper-based examinations with or without computer support or completely electronic examinations. Critical aspects are the effort for formatting, the logistic effort during the actual examination, quality, promptness and effort of the correction, the time for making the documents available for inspection by the students, and the statistical analysis of the examination results. Methods: Since three semesters a computer program for input and formatting of MC-questions in medical and other paper-based examinations is used and continuously improved at Wuerzburg University. In the winter semester (WS) 2009/10 eleven, in the summer semester (SS) 2010 twelve and in WS 2010/11 thirteen medical examinations were accomplished with the program and automatically evaluated. For the last two semesters the remaining manual workload was recorded. Results: The cost of the formatting and the subsequent analysis including adjustments of the analysis of an average examination with about 140 participants and about 35 questions was 5-7 hours for exams without complications in the winter semester 2009/2010, about 2 hours in SS 2010 and about 1.5 hours in the winter semester 2010/11. Including exams with complications, the average time was about 3 hours per exam in SS 2010 and 2.67 hours for the WS 10/11. Discussion: For conventional multiple-choice exams the computer-based formatting and evaluation of paper-based exams offers a significant time reduction for lecturers in comparison with the manual correction of paper-based exams and compared to purely electronically conducted exams it needs a much simpler technological infrastructure and fewer staff during the exam. PMID:22205913

Cost analysis for computer supported multiple-choice paper examinations.

PubMed

Mandel, Alexander; Hörnlein, Alexander; Ifland, Marianus; Lüneburg, Edeltraud; Deckert, Jürgen; Puppe, Frank

2011-01-01

Multiple-choice-examinations are still fundamental for assessment in medical degree programs. In addition to content related research, the optimization of the technical procedure is an important question. Medical examiners face three options: paper-based examinations with or without computer support or completely electronic examinations. Critical aspects are the effort for formatting, the logistic effort during the actual examination, quality, promptness and effort of the correction, the time for making the documents available for inspection by the students, and the statistical analysis of the examination results. Since three semesters a computer program for input and formatting of MC-questions in medical and other paper-based examinations is used and continuously improved at Wuerzburg University. In the winter semester (WS) 2009/10 eleven, in the summer semester (SS) 2010 twelve and in WS 2010/11 thirteen medical examinations were accomplished with the program and automatically evaluated. For the last two semesters the remaining manual workload was recorded. The cost of the formatting and the subsequent analysis including adjustments of the analysis of an average examination with about 140 participants and about 35 questions was 5-7 hours for exams without complications in the winter semester 2009/2010, about 2 hours in SS 2010 and about 1.5 hours in the winter semester 2010/11. Including exams with complications, the average time was about 3 hours per exam in SS 2010 and 2.67 hours for the WS 10/11. For conventional multiple-choice exams the computer-based formatting and evaluation of paper-based exams offers a significant time reduction for lecturers in comparison with the manual correction of paper-based exams and compared to purely electronically conducted exams it needs a much simpler technological infrastructure and fewer staff during the exam.

Neural bases of human mate choice: multiple value dimensions, sex difference, and self-assessment system.

PubMed

Funayama, Risa; Sugiura, Motoaki; Sassa, Yuko; Jeong, Hyeonjeong; Wakusawa, Keisuke; Horie, Kaoru; Sato, Shigeru; Kawashima, Ryuta

2012-01-01

Mate choice is an example of sophisticated daily decision making supported by multiple componential processes. In mate-choice literature, different characteristics of the value dimensions, including the sex difference in the value dimensions, and the involvement of self-assessment due to the mutual nature of the choice, have been suggested. We examined whether the brain-activation pattern during virtual mate choice would be congruent with these characteristics in terms of stimulus selectivity and activated brain regions. In measuring brain activity, young men and women were shown two pictures of either faces or behaviors, and they indicated which person they would choose either as a spouse or as a friend. Activation selective to spouse choice was observed face-selectively in men's amygdala and behavior-selectively in women's motor system. During both partner-choice conditions, behavior-selective activation was observed in the temporoparietal regions. Taking the available knowledge of these regions into account, these results are congruent with the suggested characteristics of value dimensions for physical attractiveness, parenting resources, and beneficial personality traits for a long-lasting relationship, respectively. The medial prefrontal and posterior cingulate cortices were nonselectively activated during the partner choices, suggesting the involvement of a self-assessment process. The results thus provide neuroscientific support for the multi-component mate-choice mechanism.

A Method for Imputing Response Options for Missing Data on Multiple-Choice Assessments

ERIC Educational Resources Information Center

Wolkowitz, Amanda A.; Skorupski, William P.

2013-01-01

When missing values are present in item response data, there are a number of ways one might impute a correct or incorrect response to a multiple-choice item. There are significantly fewer methods for imputing the actual response option an examinee may have provided if he or she had not omitted the item either purposely or accidentally. This…

Multiple-Choice Exams: An Obstacle for Higher-Level Thinking in Introductory Science Classes

PubMed Central

Stanger-Hall, Kathrin F.

2012-01-01

Learning science requires higher-level (critical) thinking skills that need to be practiced in science classes. This study tested the effect of exam format on critical-thinking skills. Multiple-choice (MC) testing is common in introductory science courses, and students in these classes tend to associate memorization with MC questions and may not see the need to modify their study strategies for critical thinking, because the MC exam format has not changed. To test the effect of exam format, I used two sections of an introductory biology class. One section was assessed with exams in the traditional MC format, the other section was assessed with both MC and constructed-response (CR) questions. The mixed exam format was correlated with significantly more cognitively active study behaviors and a significantly better performance on the cumulative final exam (after accounting for grade point average and gender). There was also less gender-bias in the CR answers. This suggests that the MC-only exam format indeed hinders critical thinking in introductory science classes. Introducing CR questions encouraged students to learn more and to be better critical thinkers and reduced gender bias. However, student resistance increased as students adjusted their perceptions of their own critical-thinking abilities. PMID:22949426

More Than the Verbal Stimulus Matters: Visual Attention in Language Assessment for People With Aphasia Using Multiple-Choice Image Displays

PubMed Central

Ivanova, Maria V.; Hallowell, Brooke

2017-01-01

Purpose Language comprehension in people with aphasia (PWA) is frequently evaluated using multiple-choice displays: PWA are asked to choose the image that best corresponds to the verbal stimulus in a display. When a nontarget image is selected, comprehension failure is assumed. However, stimulus-driven factors unrelated to linguistic comprehension may influence performance. In this study we explore the influence of physical image characteristics of multiple-choice image displays on visual attention allocation by PWA. Method Eye fixations of 41 PWA were recorded while they viewed 40 multiple-choice image sets presented with and without verbal stimuli. Within each display, 3 images (majority images) were the same and 1 (singleton image) differed in terms of 1 image characteristic. The mean proportion of fixation duration (PFD) allocated across majority images was compared against the PFD allocated to singleton images. Results PWA allocated significantly greater PFD to the singleton than to the majority images in both nonverbal and verbal conditions. Those with greater severity of comprehension deficits allocated greater PFD to nontarget singleton images in the verbal condition. Conclusion When using tasks that rely on multiple-choice displays and verbal stimuli, one cannot assume that verbal stimuli will override the effect of visual-stimulus characteristics. PMID:28520866

Step by Step: Biology Undergraduates’ Problem-Solving Procedures during Multiple-Choice Assessment

PubMed Central

Prevost, Luanna B.; Lemons, Paula P.

2016-01-01

This study uses the theoretical framework of domain-specific problem solving to explore the procedures students use to solve multiple-choice problems about biology concepts. We designed several multiple-choice problems and administered them on four exams. We trained students to produce written descriptions of how they solved the problem, and this allowed us to systematically investigate their problem-solving procedures. We identified a range of procedures and organized them as domain general, domain specific, or hybrid. We also identified domain-general and domain-specific errors made by students during problem solving. We found that students use domain-general and hybrid procedures more frequently when solving lower-order problems than higher-order problems, while they use domain-specific procedures more frequently when solving higher-order problems. Additionally, the more domain-specific procedures students used, the higher the likelihood that they would answer the problem correctly, up to five procedures. However, if students used just one domain-general procedure, they were as likely to answer the problem correctly as if they had used two to five domain-general procedures. Our findings provide a categorization scheme and framework for additional research on biology problem solving and suggest several important implications for researchers and instructors. PMID:27909021

Examen en Vue du Diplome Douzieme Annee. Langue et Litterature 30. Partie B: Lecture (Choix Multiples). Livret de Questions (Examination for the Twelfth Grade Diploma, Language and Literature 30. Part B: Reading--Multiple Choice. Questions Booklet).

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

As part of an examination required by the Alberta (Canada) Department of Education in order for 12th grade students to receive a diploma in French, this booklet contains the 80 multiple choice questions portion of Part B, the language and literature component of the January 1987 tests. Representing the genres of poetry, short story, the novel, and…

Examen en Vue du Diplome Douzieme Annee. Langue et Litterature 30. Partie B: Lecture (Choix Multiples). Livret de Questions (Examination for the Twelfth Grade Diploma, Language and Literature 30. Part B: Reading--Multiple Choice. Questions Booklet.)

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

As part of an examination required by the Alberta (Canada) Department of Education in order for 12th grade students to receive a diploma in French, this booklet contains the 80 multiple choice questions portion of Part B, the language and literature component of the January 1988 tests. Representing the genres of poetry, short story, novel, and…

A Participatory Learning Approach to Biochemistry Using Student Authored and Evaluated Multiple-Choice Questions

ERIC Educational Resources Information Center

Bottomley, Steven; Denny, Paul

2011-01-01

A participatory learning approach, combined with both a traditional and a competitive assessment, was used to motivate students and promote a deep approach to learning biochemistry. Students were challenged to research, author, and explain their own multiple-choice questions (MCQs). They were also required to answer, evaluate, and discuss MCQs…

Validity and Reliability of Scores Obtained on Multiple-Choice Questions: Why Functioning Distractors Matter

ERIC Educational Resources Information Center

Ali, Syed Haris; Carr, Patrick A.; Ruit, Kenneth G.

2016-01-01

Plausible distractors are important for accurate measurement of knowledge via multiple-choice questions (MCQs). This study demonstrates the impact of higher distractor functioning on validity and reliability of scores obtained on MCQs. Freeresponse (FR) and MCQ versions of a neurohistology practice exam were given to four cohorts of Year 1 medical…

Online evaluation of novel choices by simultaneous representation of multiple memories

PubMed Central

Barron, Helen C; Dolan, Raymond J; Behrens, Timothy E J

2014-01-01

Prior experience plays a critical role in decision making. It enables explicit representation of potential outcomes and provides training to valuation mechanisms. However, we can also make choices in the absence of prior experience, by merely imagining the consequences of a new experience. Here, using fMRI repetition suppression in humans, we show how neuronal representations of novel rewards can be constructed and evaluated. A likely novel experience is constructed by invoking multiple independent memories within hippocampus and medial prefrontal cortex. This construction persists for only a short time period, during which new associations are observed between the memories for component items. Together these findings suggest that in the absence of direct experience, co-activation of multiple relevant memories can provide a training signal to the valuation system which allows the consequences of new experiences to be imagined and acted upon. PMID:24013592

Influence of test duration on the sensitivity of the two-bottle choice test.

PubMed

Tordoff, Michael G; Bachmanov, Alexander A

2002-11-01

The long-term two-bottle choice test is commonly used as a simple screen to examine the acceptance of taste solutions by rodents. As part of an investigation of factors influencing the sensitivity of the two-bottle choice test, we determined the extent to which test duration influenced test sensitivity. C57BL6/J and 129X1/SvJ mice received four series of eight two-bottle tests, with each test lasting 1, 2, 4 or 6 days. Each series involved sequential tests with water, 2 mM saccharin, 5 and 50 mM citric acid, 30 and 300 micro M quinine hydrochloride, 75 mM NaCl and 10% ethanol. There were significant differences between the strains in intake of saccharin, 5 and 50 mM citric acid, NaCl and ethanol in 4 and 6 day tests, but only saccharin and ethanol in 2 day tests, and 5 mM citric acid and ethanol in 1 day tests. To compare the sensitivity of the tests, we developed an analytical approach based on the comparison of deviations of individual 129X1/SvJ mice from the C57BL6/J strain mean. Our results suggest that to discriminate between strains or treatments when using 'standard' laboratory conditions and methods, 1 day tests are generally inadequate and 2 day tests are useful only if large effects are anticipated. Tests lasting 4 or 6 days are more sensitive, but conducting 6 day tests provides little additional benefit and sometimes is detrimental relative to conducting 4 day tests.

Difficulty and Discriminability of Introductory Psychology Test Items.

ERIC Educational Resources Information Center

Scialfa, Charles; Legare, Connie; Wenger, Larry; Dingley, Louis

2001-01-01

Analyzes multiple-choice questions provided in test banks for introductory psychology textbooks. Study 1 offered a consistent picture of the objective difficulty of multiple-choice tests for introductory psychology students, while both studies 1 and 2 indicated that test items taken from commercial test banks have poor psychometric properties.…

Representing Heterogeneity in Structural Relationships Among Multiple Choice Variables Using a Latent Segmentation Approach

DOE Office of Scientific and Technical Information (OSTI.GOV)

Garikapati, Venu; Astroza, Sebastian; Pendyala, Ram M.

Travel model systems often adopt a single decision structure that links several activity-travel choices together. The single decision structure is then used to predict activity-travel choices, with those downstream in the decision-making chain influenced by those upstream in the sequence. The adoption of a singular sequential causal structure to depict relationships among activity-travel choices in travel demand model systems ignores the possibility that some choices are made jointly as a bundle as well as the possible presence of structural heterogeneity in the population with respect to decision-making processes. As different segments in the population may adopt and follow different causalmore » decision-making mechanisms when making selected choices jointly, it would be of value to develop simultaneous equations model systems relating multiple endogenous choice variables that are able to identify population subgroups following alternative causal decision structures. Because the segments are not known a priori, they are considered latent and determined endogenously within a joint modeling framework proposed in this paper. The methodology is applied to a national mobility survey data set to identify population segments that follow different causal structures relating residential location choice, vehicle ownership, and car-share and mobility service usage. It is found that the model revealing three distinct latent segments best describes the data, confirming the efficacy of the modeling approach and the existence of structural heterogeneity in decision-making in the population. Future versions of activity-travel model systems should strive to incorporate such structural heterogeneity to better reflect varying decision processes across population subgroups.« less

Make Better Choices (MBC): Study design of a randomized controlled trial testing optimal technology-supported change in multiple diet and physical activity risk behaviors

PubMed Central

2010-01-01

Background Suboptimal diet and physical inactivity are prevalent, co-occurring chronic disease risk factors, yet little is known about how to maximize multiple risk behavior change. Make Better Choices, a randomized controlled trial, tests competing hypotheses about the optimal way to promote healthy change in four bundled risk behaviors: high saturated fat intake, low fruit and vegetable intake, low physical activity, and high sedentary leisure screen time. The study aim is to determine which combination of two behavior change goals - one dietary, one activity - yields greatest overall healthy lifestyle change. Methods/Design Adults (n = 200) with poor quality diet and sedentary lifestyle will be recruited and screened for study eligibility. Participants will be trained to record their diet and activities onto a personal data assistant, and use it to complete two weeks of baseline. Those who continue to show all four risk behaviors after baseline recording will be randomized to one of four behavior change prescriptions: 1) increase fruits and vegetables and increase physical activity, 2) decrease saturated fat and increase physical activity, 3) increase fruits and vegetable and decrease saturated fat, or 4) decrease saturated fat and decrease sedentary activity. They will use decision support feedback on the personal digital assistant and receive counseling from a coach to alter their diet and activity during a 3-week prescription period when payment is contingent upon meeting behavior change goals. They will continue recording on an intermittent schedule during a 4.5-month maintenance period when payment is not contingent upon goal attainment. The primary outcome is overall healthy lifestyle change, aggregated across all four risk behaviors. Discussion The Make Better Choices trial tests a disseminable lifestyle intervention supported by handheld technology. Findings will fill a gap in knowledge about optimal goal prescription to facilitate simultaneous diet and

Adaptive graph-based multiple testing procedures

PubMed Central

Klinglmueller, Florian; Posch, Martin; Koenig, Franz

2016-01-01

Multiple testing procedures defined by directed, weighted graphs have recently been proposed as an intuitive visual tool for constructing multiple testing strategies that reflect the often complex contextual relations between hypotheses in clinical trials. Many well-known sequentially rejective tests, such as (parallel) gatekeeping tests or hierarchical testing procedures are special cases of the graph based tests. We generalize these graph-based multiple testing procedures to adaptive trial designs with an interim analysis. These designs permit mid-trial design modifications based on unblinded interim data as well as external information, while providing strong family wise error rate control. To maintain the familywise error rate, it is not required to prespecify the adaption rule in detail. Because the adaptive test does not require knowledge of the multivariate distribution of test statistics, it is applicable in a wide range of scenarios including trials with multiple treatment comparisons, endpoints or subgroups, or combinations thereof. Examples of adaptations are dropping of treatment arms, selection of subpopulations, and sample size reassessment. If, in the interim analysis, it is decided to continue the trial as planned, the adaptive test reduces to the originally planned multiple testing procedure. Only if adaptations are actually implemented, an adjusted test needs to be applied. The procedure is illustrated with a case study and its operating characteristics are investigated by simulations. PMID:25319733

Sequential and simultaneous choices: testing the diet selection and sequential choice models.

PubMed

Freidin, Esteban; Aw, Justine; Kacelnik, Alex

2009-03-01

We investigate simultaneous and sequential choices in starlings, using Charnov's Diet Choice Model (DCM) and Shapiro, Siller and Kacelnik's Sequential Choice Model (SCM) to integrate function and mechanism. During a training phase, starlings encountered one food-related option per trial (A, B or R) in random sequence and with equal probability. A and B delivered food rewards after programmed delays (shorter for A), while R ('rejection') moved directly to the next trial without reward. In this phase we measured latencies to respond. In a later, choice, phase, birds encountered the pairs A-B, A-R and B-R, the first implementing a simultaneous choice and the second and third sequential choices. The DCM predicts when R should be chosen to maximize intake rate, and SCM uses latencies of the training phase to predict choices between any pair of options in the choice phase. The predictions of both models coincided, and both successfully predicted the birds' preferences. The DCM does not deal with partial preferences, while the SCM does, and experimental results were strongly correlated to this model's predictions. We believe that the SCM may expose a very general mechanism of animal choice, and that its wider domain of success reflects the greater ecological significance of sequential over simultaneous choices.

A discrete-choice experiment to determine patient preferences for injectable multiple sclerosis treatments in Germany.

PubMed

Poulos, Christine; Kinter, Elizabeth; Yang, Jui-Chen; Bridges, John F P; Posner, Joshua; Gleißner, Erika; Mühlbacher, Axel; Kieseier, Bernd

2016-03-01

The aim of this study was to assess the relative importance of features of a hypothetical injectable disease-modifying treatment for patients with multiple sclerosis using a discrete-choice experiment. German residents at least 18 years of age with a self-reported physician diagnosis of multiple sclerosis completed a 25-30 minute online discrete-choice experiment. Patients were asked to choose one of two hypothetical injectable treatments for multiple sclerosis, defined by different levels of six attributes (disability progression, the number of relapses in the next 4 years, injection time, frequency of injections, presence of flu-like symptoms, and presence of injection-site reactions). The data were analyzed using a random-parameters logit model. Of 202 adults who completed the survey, results from 189 were used in the analysis. Approximately 50% of all patients reported a diagnosis of relapsing-remitting multiple sclerosis, and 31% reported secondary progressive multiple sclerosis. Approximately 71% of patients had current or prior experience with injectable multiple sclerosis medication. Approximately 53% had experienced flu-like symptoms caused by their medication, and 47% had experienced mild injection-site reactions. At least one significant difference was seen between levels in all attributes, except injection time. The greatest change in relative importance between levels of an attribute was years until symptoms get worse from 1 to 4 years. The magnitude of this difference was about twice that of relapses in the next 4 years, frequency of injections, and flu-like symptoms. Most attributes examined in this experiment had an influence on patient preference. Patients placed a significant value on improvements in the frequency of dosing and disability progression. Results suggest that changes in injection frequency can be as important as changes in efficacy and safety attributes. Understanding which attributes of injectable therapies influence patient

Multiple Hypothesis Testing for Experimental Gingivitis Based on Wilcoxon Signed Rank Statistics

PubMed Central

Preisser, John S.; Sen, Pranab K.; Offenbacher, Steven

2011-01-01

Dental research often involves repeated multivariate outcomes on a small number of subjects for which there is interest in identifying outcomes that exhibit change in their levels over time as well as to characterize the nature of that change. In particular, periodontal research often involves the analysis of molecular mediators of inflammation for which multivariate parametric methods are highly sensitive to outliers and deviations from Gaussian assumptions. In such settings, nonparametric methods may be favored over parametric ones. Additionally, there is a need for statistical methods that control an overall error rate for multiple hypothesis testing. We review univariate and multivariate nonparametric hypothesis tests and apply them to longitudinal data to assess changes over time in 31 biomarkers measured from the gingival crevicular fluid in 22 subjects whereby gingivitis was induced by temporarily withholding tooth brushing. To identify biomarkers that can be induced to change, multivariate Wilcoxon signed rank tests for a set of four summary measures based upon area under the curve are applied for each biomarker and compared to their univariate counterparts. Multiple hypothesis testing methods with choice of control of the false discovery rate or strong control of the family-wise error rate are examined. PMID:21984957

40 CFR 791.52 - Multiple tests.

Code of Federal Regulations, 2014 CFR

2014-07-01

... 40 Protection of Environment 32 2014-07-01 2014-07-01 false Multiple tests. 791.52 Section 791.52...) DATA REIMBURSEMENT Basis for Proposed Order § 791.52 Multiple tests. When more than one of a particular kind of test required by the test rule is performed, the additional costs will be shared among all...

40 CFR 791.52 - Multiple tests.

Code of Federal Regulations, 2011 CFR

2011-07-01

... 40 Protection of Environment 32 2011-07-01 2011-07-01 false Multiple tests. 791.52 Section 791.52...) DATA REIMBURSEMENT Basis for Proposed Order § 791.52 Multiple tests. When more than one of a particular kind of test required by the test rule is performed, the additional costs will be shared among all...

A practical method to test the validity of the standard Gumbel distribution in logit-based multinomial choice models of travel behavior

DOE PAGES

Ye, Xin; Garikapati, Venu M.; You, Daehyun; ...

2017-11-08

Most multinomial choice models (e.g., the multinomial logit model) adopted in practice assume an extreme-value Gumbel distribution for the random components (error terms) of utility functions. This distributional assumption offers a closed-form likelihood expression when the utility maximization principle is applied to model choice behaviors. As a result, model coefficients can be easily estimated using the standard maximum likelihood estimation method. However, maximum likelihood estimators are consistent and efficient only if distributional assumptions on the random error terms are valid. It is therefore critical to test the validity of underlying distributional assumptions on the error terms that form the basismore » of parameter estimation and policy evaluation. In this paper, a practical yet statistically rigorous method is proposed to test the validity of the distributional assumption on the random components of utility functions in both the multinomial logit (MNL) model and multiple discrete-continuous extreme value (MDCEV) model. Based on a semi-nonparametric approach, a closed-form likelihood function that nests the MNL or MDCEV model being tested is derived. The proposed method allows traditional likelihood ratio tests to be used to test violations of the standard Gumbel distribution assumption. Simulation experiments are conducted to demonstrate that the proposed test yields acceptable Type-I and Type-II error probabilities at commonly available sample sizes. The test is then applied to three real-world discrete and discrete-continuous choice models. For all three models, the proposed test rejects the validity of the standard Gumbel distribution in most utility functions, calling for the development of robust choice models that overcome adverse effects of violations of distributional assumptions on the error terms in random utility functions.« less

A practical method to test the validity of the standard Gumbel distribution in logit-based multinomial choice models of travel behavior

DOE Office of Scientific and Technical Information (OSTI.GOV)

Ye, Xin; Garikapati, Venu M.; You, Daehyun

Most multinomial choice models (e.g., the multinomial logit model) adopted in practice assume an extreme-value Gumbel distribution for the random components (error terms) of utility functions. This distributional assumption offers a closed-form likelihood expression when the utility maximization principle is applied to model choice behaviors. As a result, model coefficients can be easily estimated using the standard maximum likelihood estimation method. However, maximum likelihood estimators are consistent and efficient only if distributional assumptions on the random error terms are valid. It is therefore critical to test the validity of underlying distributional assumptions on the error terms that form the basismore » of parameter estimation and policy evaluation. In this paper, a practical yet statistically rigorous method is proposed to test the validity of the distributional assumption on the random components of utility functions in both the multinomial logit (MNL) model and multiple discrete-continuous extreme value (MDCEV) model. Based on a semi-nonparametric approach, a closed-form likelihood function that nests the MNL or MDCEV model being tested is derived. The proposed method allows traditional likelihood ratio tests to be used to test violations of the standard Gumbel distribution assumption. Simulation experiments are conducted to demonstrate that the proposed test yields acceptable Type-I and Type-II error probabilities at commonly available sample sizes. The test is then applied to three real-world discrete and discrete-continuous choice models. For all three models, the proposed test rejects the validity of the standard Gumbel distribution in most utility functions, calling for the development of robust choice models that overcome adverse effects of violations of distributional assumptions on the error terms in random utility functions.« less

Are Faculty Predictions or Item Taxonomies Useful for Estimating the Outcome of Multiple-Choice Examinations?

ERIC Educational Resources Information Center

Kibble, Jonathan D.; Johnson, Teresa

2011-01-01

The purpose of this study was to evaluate whether multiple-choice item difficulty could be predicted either by a subjective judgment by the question author or by applying a learning taxonomy to the items. Eight physiology faculty members teaching an upper-level undergraduate human physiology course consented to participate in the study. The…

Test analysis and research on static choice reaction ability of commercial vehicle drivers

NASA Astrophysics Data System (ADS)

Zhang, Lingchao; Wei, Lang; Qiao, Jie; Tian, Shun; Wang, Shengchang

2017-03-01

Drivers' choice reaction ability has a certain relation with safe driving. It has important significance to research its influence on traffic safety. Firstly, the paper uses a choice reaction detector developed by research group to detect drivers' choice reaction ability of commercial vehicles, and gets 2641 effective samples. Then by using mathematical statistics method, the paper founds that average reaction time from accident group has no difference with non-accident group, and then introduces a variance rate of reaction time as a new index to replace it. The result shows that the test index choice reaction errors and variance rate of reaction time have positive correlations with accidents. Finally, according to testing results of the detector, the paper formulates a detection threshold with four levels for helping transportation companies to assess commercial vehicles drivers.

Teaching Choice Making to Children with Visual Impairments and Multiple Disabilities in Preschool and Kindergarten Classrooms

ERIC Educational Resources Information Center

Clark, Christine; McDonnell, Andrea P.

2008-01-01

This study examined the effectiveness of an intervention package that included visual accommodations, daily preference assessments, and naturalistic instructional strategies on the accuracy of choice-making responses for three participants with visual impairments and multiple disabilities. It also examined the participants' ability to maintain and…

Learners' choices and beliefs about self-testing.

PubMed

Kornell, Nate; Son, Lisa K

2009-07-01

Students have to make scores of practical decisions when they study. We investigated the effectiveness of, and beliefs underlying, one such practical decision: the decision to test oneself while studying. Using a flashcards-like procedure, participants studied lists of word pairs. On the second of two study trials, participants either saw the entire pair again (pair mode) or saw the cue and attempted to generate the target (test mode). Participants were asked either to rate the effectiveness of each study mode (Experiment 1) or to choose between the two modes (Experiment 2). The results demonstrated a mismatch between metacognitive beliefs and study choices: Participants (incorrectly) judged that the pair mode resulted in the most learning, but chose the test mode most frequently. A post-experimental questionnaire suggested that self-testing was motivated by a desire to diagnose learning rather than a desire to improve learning.

The Incidence of Clueing in Multiple Choice Testbank Questions in Accounting: Some Evidence from Australia

ERIC Educational Resources Information Center

Ibbett, Nicole L.; Wheldon, Brett J.

2016-01-01

In 2014 Central Queensland University (CQU) in Australia banned the use of multiple choice questions (MCQs) as an assessment tool. One of the reasons given for this decision was that MCQs provide an opportunity for students to "pass" by merely guessing their answers. The mathematical likelihood of a student passing by guessing alone can…

Using a Fine-Grained Multiple-Choice Response Format in Educational Drill-and-Practice Video Games

ERIC Educational Resources Information Center

Beserra, Vagner; Nussbaum, Miguel; Grass, Antonio

2017-01-01

When using educational video games, particularly drill-and-practice video games, there are several ways of providing an answer to a quiz. The majority of paper-based options can be classified as being either multiple-choice or constructed-response. Therefore, in the process of creating an educational drill-and-practice video game, one fundamental…

Multiple-choice examinations: adopting an evidence-based approach to exam technique.

PubMed

Hammond, E J; McIndoe, A K; Sansome, A J; Spargo, P M

1998-11-01

Negatively marked multiple-choice questions (MCQs) are part of the assessment process in both the Primary and Final examinations for the fellowship of the Royal College of Anaesthetists. It is said that candidates who guess will lose marks in the MCQ paper. We studied candidates attending a pre-examination revision course and have shown that an evaluation of examination technique is an important part of an individual's preparation. All candidates benefited substantially from backing their educated guesses while only 3 out of 27 lost marks from backing their wild guesses. Failure to appreciate the relationship between knowledge and technique may significantly affect a candidate's performance in the examination.

The Social Attribution Task - Multiple Choice (SAT-MC): Psychometric comparison with social cognitive measures for schizophrenia research.

PubMed

Johannesen, Jason K; Fiszdon, Joanna M; Weinstein, Andrea; Ciosek, David; Bell, Morris D

2018-04-01

The Social Attribution Task-Multiple Choice (SAT-MC) tests the ability to extract social themes from viewed object motion. This form of animacy perception is thought to aid the development of social inference, but appears impaired in schizophrenia. The current study was undertaken to examine psychometric equivalence of two forms of the SAT-MC and to compare their performance against social cognitive tests recommended for schizophrenia research. Thirty-two schizophrenia (SZ) and 30 substance use disorder (SUD) participants completed both SAT-MC forms, the Bell-Lysaker Emotion Recognition Task (BLERT), Hinting Task, The Awareness of Social Inference Test (TASIT), Ambiguous Intentions and Hostility Questionnaire (AIHQ) and questionnaire measures of interpersonal function. Test sensitivity, construct and external validity, test-retest reliability, and internal consistency were evaluated. SZ scored significantly lower than SUD on both SAT-MC forms, each classifying ~60% of SZ as impaired, compared with ~30% of SUD. SAT-MC forms demonstrated good test-retest and parallel form reliability, minimal practice effect, high internal consistency, and similar patterns of correlation with social cognitive and external validity measures. The SAT-MC compared favorably to recommended social cognitive tests across psychometric features and, with exception of TASIT, was most sensitive to impairment in schizophrenia when compared to a chronic substance use sample. Published by Elsevier B.V.

Evaluating the Psychometric Characteristics of Generated Multiple-Choice Test Items

ERIC Educational Resources Information Center

Gierl, Mark J.; Lai, Hollis; Pugh, Debra; Touchie, Claire; Boulais, André-Philippe; De Champlain, André

2016-01-01

Item development is a time- and resource-intensive process. Automatic item generation integrates cognitive modeling with computer technology to systematically generate test items. To date, however, items generated using cognitive modeling procedures have received limited use in operational testing situations. As a result, the psychometric…

Analyzing Multiple-Choice Questions by Model Analysis and Item Response Curves

NASA Astrophysics Data System (ADS)

Wattanakasiwich, P.; Ananta, S.

2010-07-01

In physics education research, the main goal is to improve physics teaching so that most students understand physics conceptually and be able to apply concepts in solving problems. Therefore many multiple-choice instruments were developed to probe students' conceptual understanding in various topics. Two techniques including model analysis and item response curves were used to analyze students' responses from Force and Motion Conceptual Evaluation (FMCE). For this study FMCE data from more than 1000 students at Chiang Mai University were collected over the past three years. With model analysis, we can obtain students' alternative knowledge and the probabilities for students to use such knowledge in a range of equivalent contexts. The model analysis consists of two algorithms—concentration factor and model estimation. This paper only presents results from using the model estimation algorithm to obtain a model plot. The plot helps to identify a class model state whether it is in the misconception region or not. Item response curve (IRC) derived from item response theory is a plot between percentages of students selecting a particular choice versus their total score. Pros and cons of both techniques are compared and discussed.

The Social Attribution Task-Multiple Choice (SAT-MC): A Psychometric and Equivalence Study of an Alternate Form.

PubMed

Johannesen, Jason K; Lurie, Jessica B; Fiszdon, Joanna M; Bell, Morris D

2013-01-01

The Social Attribution Task-Multiple Choice (SAT-MC) uses a 64-second video of geometric shapes set in motion to portray themes of social relatedness and intentions. Considered a test of "Theory of Mind," the SAT-MC assesses implicit social attribution formation while reducing verbal and basic cognitive demands required of other common measures. We present a comparability analysis of the SAT-MC and the new SAT-MC-II, an alternate form created for repeat testing, in a university sample (n = 92). Score distributions and patterns of association with external validation measures were nearly identical between the two forms, with convergent and discriminant validity supported by association with affect recognition ability and lack of association with basic visual reasoning. Internal consistency of the SAT-MC-II was superior (alpha = .81) to the SAT-MC (alpha = .56). Results support the use of SAT-MC and new SAT-MC-II as equivalent test forms. Demonstrating relatively higher association to social cognitive than basic cognitive abilities, the SAT-MC may provide enhanced sensitivity as an outcome measure of social cognitive intervention trials.

The Social Attribution Task-Multiple Choice (SAT-MC): A Psychometric and Equivalence Study of an Alternate Form

PubMed Central

Johannesen, Jason K.; Lurie, Jessica B.; Fiszdon, Joanna M.; Bell, Morris D.

2013-01-01

The Social Attribution Task-Multiple Choice (SAT-MC) uses a 64-second video of geometric shapes set in motion to portray themes of social relatedness and intentions. Considered a test of “Theory of Mind,” the SAT-MC assesses implicit social attribution formation while reducing verbal and basic cognitive demands required of other common measures. We present a comparability analysis of the SAT-MC and the new SAT-MC-II, an alternate form created for repeat testing, in a university sample (n = 92). Score distributions and patterns of association with external validation measures were nearly identical between the two forms, with convergent and discriminant validity supported by association with affect recognition ability and lack of association with basic visual reasoning. Internal consistency of the SAT-MC-II was superior (alpha = .81) to the SAT-MC (alpha = .56). Results support the use of SAT-MC and new SAT-MC-II as equivalent test forms. Demonstrating relatively higher association to social cognitive than basic cognitive abilities, the SAT-MC may provide enhanced sensitivity as an outcome measure of social cognitive intervention trials. PMID:23864984

Validation and Structural Analysis of the Kinematics Concept Test

ERIC Educational Resources Information Center

Lichtenberger, A.; Wagner, C.; Hofer, S. I.; Stem, E.; Vaterlaus, A.

2017-01-01

The kinematics concept test (KCT) is a multiple-choice test designed to evaluate students' conceptual understanding of kinematics at the high school level. The test comprises 49 multiple-choice items about velocity and acceleration, which are based on seven kinematic concepts and which make use of three different representations. In the first part…

Exploring informed choice in the context of prenatal testing: findings from a qualitative study

PubMed Central

Potter, Beth K.; O’Reilly, Natasha; Etchegary, Holly; Howley, Heather; Graham, Ian D.; Walker, Mark; Coyle, Doug; Chorny, Yelena; Cappelli, Mario; Boland, Isabelle; Wilson, Brenda J.

2008-01-01

Abstract Purpose  This study explored whether and how a sample of women made informed choices about prenatal testing for foetal anomalies; its aim was to provide insights for future health policy and service provision. Methods  We conducted semi‐structured interviews with 38 mothers in Ottawa, Ontario, all of whom had been offered prenatal tests in at least one pregnancy. Using the Multi‐dimensional Measure of Informed Choice as a general guide to analysis, we explored themes relevant to informed choice, including values and knowledge, and interactions with health professionals. Results  Many, but not all, participants seemed to have made informed decisions about prenatal testing. Values and knowledge were interrelated and important components of informed choice, but the way they were discussed differed from the way they have been presented in scientific literature. In particular, ‘values’ related to expressions of women’s moral views or ideas about ‘how life should be lived’ and ‘knowledge’ related to the ways in which women prioritized and interpreted factual information, through their own and others’ experiences and in ‘thinking through’ the personal implications of testing. While some women described non‐directive discussions with health professionals, others perceived testing as routine or felt pressured to accept it. Conclusions  Our findings suggest a need for maternity care providers to be vigilant in promoting active decision making about prenatal testing, particularly around the consideration of personal implications. Further development of measures of informed choice may be necessary to fully evaluate decision support tools and to determine whether prenatal testing programmes are meeting their objectives. PMID:18798759

Multiple comparison analysis testing in ANOVA.

PubMed

McHugh, Mary L

2011-01-01

The Analysis of Variance (ANOVA) test has long been an important tool for researchers conducting studies on multiple experimental groups and one or more control groups. However, ANOVA cannot provide detailed information on differences among the various study groups, or on complex combinations of study groups. To fully understand group differences in an ANOVA, researchers must conduct tests of the differences between particular pairs of experimental and control groups. Tests conducted on subsets of data tested previously in another analysis are called post hoc tests. A class of post hoc tests that provide this type of detailed information for ANOVA results are called "multiple comparison analysis" tests. The most commonly used multiple comparison analysis statistics include the following tests: Tukey, Newman-Keuls, Scheffee, Bonferroni and Dunnett. These statistical tools each have specific uses, advantages and disadvantages. Some are best used for testing theory while others are useful in generating new theory. Selection of the appropriate post hoc test will provide researchers with the most detailed information while limiting Type 1 errors due to alpha inflation.

Diagnosing Secondary Students' Misconceptions of Photosynthesis and Respiration in Plants Using a Two-Tier Multiple Choice Instrument.

ERIC Educational Resources Information Center

Haslam, Filocha; Treagust, David F.

1987-01-01

Describes a multiple-choice instrument that reliably and validly diagnoses secondary students' understanding of photosynthesis and respiration in plants. Highlights the consistency of students' misconceptions across secondary levels and indicates a high percentage of students have misconceptions regarding plant physiology. (CW)

Multiple choices of time in quantum cosmology

NASA Astrophysics Data System (ADS)

Małkiewicz, Przemysław

2015-07-01

It is often conjectured that a choice of time function merely sets up a frame for the quantum evolution of the gravitational field, meaning that all choices should be in some sense compatible. In order to explore this conjecture (and the meaning of compatibility), we develop suitable tools for determining the relation between quantum theories based on different time functions. First, we discuss how a time function fixes a canonical structure on the constraint surface. The presentation includes both the kinematical and the reduced perspective, and the relation between them. Second, we formulate twin theorems about the existence of two inequivalent maps between any two deparameterizations, a formal canonical and a coordinate one. They are used to separate the effects induced by choice of clock and other factors. We show, in an example, how the spectra of quantum observables are transformed under the change of clock and prove, via a general argument, the existence of choice-of-time-induced semiclassical effects. Finally, we study an example, in which we find that the semiclassical discrepancies can in fact be arbitrarily large for dynamical observables. We conclude that the values of critical energy density or critical volume in the bouncing scenarios of quantum cosmology cannot in general be at the Planck scale, and always need to be given with reference to a specific time function.

The Multiple-Choice Model: Some Solutions for Estimation of Parameters in the Presence of Omitted Responses

ERIC Educational Resources Information Center

Abad, Francisco J.; Olea, Julio; Ponsoda, Vicente

2009-01-01

This article deals with some of the problems that have hindered the application of Samejima's and Thissen and Steinberg's multiple-choice models: (a) parameter estimation difficulties owing to the large number of parameters involved, (b) parameter identifiability problems in the Thissen and Steinberg model, and (c) their treatment of omitted…

Measurement of impulsive choice in rats: Same and alternate form test-retest reliability and temporal tracking

PubMed Central

Peterson, Jennifer R.; Hill, Catherine C.; Kirkpatrick, Kimberly

2016-01-01

Impulsive choice is typically measured by presenting smaller-sooner (SS) versus larger-later (LL) rewards, with biases towards the SS indicating impulsivity. The current study tested rats on different impulsive choice procedures with LL delay manipulations to assess same-form and alternate-form test-retest reliability. In the systematic-GE procedure (Green & Estle, 2003), the LL delay increased after several sessions of training; in the systematic-ER procedure (Evenden & Ryan, 1996), the delay increased within each session; and in the adjusting-M procedure (Mazur, 1987), the delay changed after each block of trials within a session based on each rat’s choices in the previous block. In addition to measuring choice behavior, we also assessed temporal tracking of the LL delays using the median times of responding during LL trials. The two systematic procedures yielded similar results in both choice and temporal tracking measures following extensive training, whereas the adjusting procedure resulted in relatively more impulsive choices and poorer temporal tracking. Overall, the three procedures produced acceptable same form test-retest reliability over time, but the adjusting procedure did not show significant alternate form test-retest reliability with the other two procedures. The results suggest that systematic procedures may supply better measurements of impulsive choice in rats. PMID:25490901

Test of Understanding of Vectors: A Reliable Multiple-Choice Vector Concept Test

ERIC Educational Resources Information Center

Barniol, Pablo; Zavala, Genaro

2014-01-01

In this article we discuss the findings of our research on students' understanding of vector concepts in problems without physical context. First, we develop a complete taxonomy of the most frequent errors made by university students when learning vector concepts. This study is based on the results of several test administrations of open-ended…

Nurse-led immunotreatment DEcision Coaching In people with Multiple Sclerosis (DECIMS) - Feasibility testing, pilot randomised controlled trial and mixed methods process evaluation.

PubMed

Rahn, A C; Köpke, S; Backhus, I; Kasper, J; Anger, K; Untiedt, B; Alegiani, A; Kleiter, I; Mühlhauser, I; Heesen, C

2018-02-01

Treatment decision-making is complex for people with multiple sclerosis. Profound information on available options is virtually not possible in regular neurologist encounters. The "nurse decision coach model" was developed to redistribute health professionals' tasks in supporting immunotreatment decision-making following the principles of informed shared decision-making. To test the feasibility of a decision coaching programme and recruitment strategies to inform the main trial. Feasibility testing and parallel pilot randomised controlled trial, accompanied by a mixed methods process evaluation. Two German multiple sclerosis university centres. People with suspected or relapsing-remitting multiple sclerosis facing immunotreatment decisions on first line drugs were recruited. Randomisation to the intervention (n = 38) or control group (n = 35) was performed on a daily basis. Quantitative and qualitative process data were collected from people with multiple sclerosis, nurses and physicians. We report on the development and piloting of the decision coaching programme. It comprises a training course for multiple sclerosis nurses and the coaching intervention. The intervention consists of up to three structured nurse-led decision coaching sessions, access to an evidence-based online information platform (DECIMS-Wiki) and a final physician consultation. After feasibility testing, a pilot randomised controlled trial was performed. People with multiple sclerosis were randomised to the intervention or control group. The latter had also access to the DECIMS-Wiki, but received otherwise care as usual. Nurses were not blinded to group assignment, while people with multiple sclerosis and physicians were. The primary outcome was 'informed choice' after six months including the sub-dimensions' risk knowledge (after 14 days), attitude concerning immunotreatment (after physician consultation), and treatment uptake (after six months). Quantitative process evaluation data

Backwash Effects of Language-Testing in Primary and Secondary Education.

ERIC Educational Resources Information Center

Wesdorp, H.

A debate has been carried on in Dutch educational circles about the widespread use of multiple-choice tests, and a number of objections have been raised against the use of such tests. This paper reports on research into the validity of the objections, in particular with respect to the possible effect of multiple-choice tests on the teaching of…

Considering medical risk information and communicating values: A mixed-method study of women’s choice in prenatal testing

PubMed Central

Tenhunen, Henni; Torkki, Paulus; Heinonen, Seppo; Lillrank, Paul; Stefanovic, Vedran

2017-01-01

Introduction Nowadays, an important decision for pregnant women is whether to undergo prenatal testing for aneuploidies and which tests to uptake. We investigate the factors influencing women’s choices between non-invasive prenatal testing (NIPT) and invasive prenatal tests in pregnancies with elevated a priori risk of fetal aneuploidies. Methodology This is a mixed-method study. We used medical data (1st Jan 2015-31st Dec 2015) about women participating in further testing at Fetomaternal Medical Center at Helsinki University Hospital and employed Chi-square tests and ANOVA to compare the groups of women choosing different methods. Multinomial logistic regressions revealed the significant clinical factors influencing women’s choice. We explored the underlying values, beliefs, attitudes and other psychosocial factors that affect women’s choice by interviewing women with the Theory of Planned Behavior framework. The semi-structured interview data were processed by thematic analysis. Results Statistical data indicated that gestational age and counseling day were strong factors influencing women’s choice. Interview data revealed that women’s values and moral principles on pregnancy and childbirth chiefly determined the choices. Behavioral beliefs (e.g. safety and accuracy) and perceived choice control (e.g. easiness, rapidness and convenience) were also important and the major trade-offs happened between these constructs. Discussion Values are the determinants of women’s choice. Service availability and convenience are strong factors. Medical risk status in this choice context is not highly influential. Choice aids can be developed by helping women to identify their leading values in prenatal testing and by providing lists of value-matching test options and attributes. PMID:28355226

Force Concept Inventory-Based Multiple-Choice Test for Investigating Students' Representational Consistency

ERIC Educational Resources Information Center

Nieminen, Pasi; Savinainen, Antti; Viiri, Jouni

2010-01-01

This study investigates students' ability to interpret multiple representations consistently (i.e., representational consistency) in the context of the force concept. For this purpose we developed the Representational Variant of the Force Concept Inventory (R-FCI), which makes use of nine items from the 1995 version of the Force Concept Inventory…

Step by Step: Biology Undergraduates' Problem-Solving Procedures during Multiple-Choice Assessment.

PubMed

Prevost, Luanna B; Lemons, Paula P

2016-01-01

This study uses the theoretical framework of domain-specific problem solving to explore the procedures students use to solve multiple-choice problems about biology concepts. We designed several multiple-choice problems and administered them on four exams. We trained students to produce written descriptions of how they solved the problem, and this allowed us to systematically investigate their problem-solving procedures. We identified a range of procedures and organized them as domain general, domain specific, or hybrid. We also identified domain-general and domain-specific errors made by students during problem solving. We found that students use domain-general and hybrid procedures more frequently when solving lower-order problems than higher-order problems, while they use domain-specific procedures more frequently when solving higher-order problems. Additionally, the more domain-specific procedures students used, the higher the likelihood that they would answer the problem correctly, up to five procedures. However, if students used just one domain-general procedure, they were as likely to answer the problem correctly as if they had used two to five domain-general procedures. Our findings provide a categorization scheme and framework for additional research on biology problem solving and suggest several important implications for researchers and instructors. © 2016 L. B. Prevost and P. P. Lemons. CBE—Life Sciences Education © 2016 The American Society for Cell Biology. This article is distributed by The American Society for Cell Biology under license from the author(s). It is available to the public under an Attribution–Noncommercial–Share Alike 3.0 Unported Creative Commons License (http://creativecommons.org/licenses/by-nc-sa/3.0).

Dairy cow feeding space requirements assessed in a Y-maze choice test.

PubMed

Rioja-Lang, F C; Roberts, D J; Healy, S D; Lawrence, A B; Haskell, M J

2012-07-01

The effect of proximity to a dominant cow on a low-ranking cow's willingness to feed was assessed using choice tests. The main aim of the experiment was to determine the feeding space allowance at which the majority of subordinate cows would choose to feed on high-palatability food (HPF) next to a dominant cow rather than feeding alone on low-palatability food (LPF). Thirty Holstein-Friesian cows were used in the study. Half of the cows were trained to make an association between a black bin and HPF and a white bin and LPF, and the other half were trained with the opposite combination. Observations of pair-wise aggressive interactions were observed during feeding to determine the relative social status of each cow. From this, dominant and subordinate cows were allocated to experimental pairs. When cows had achieved an HPF preference with an 80% success rate in training, they were presented with choices using a Y-maze test apparatus, in which cows were offered choices between feeding on HPF with a dominant cow and feeding on LPF alone. Four different space allowances were tested at the HPF feeder: 0.3, 0.45, 0.6, and 0.75 m. At the 2 smaller space allowances, cows preferred to feed alone (choices between feeding alone or not for 0.3- and 0.45-m tests were significantly different). For the 2 larger space allowances, cows had no significant preferences (number of choices for feeding alone or with a dominant). Given that low-status cows are willing to sacrifice food quality to avoid close contact with a dominant animal, we suggest that the feeding space allowance should be at least 0.6m per cow whenever possible. However, even when space allowances are large, it is clear that some subordinate cows will still prefer to avoid proximity to dominant individuals. Copyright © 2012 American Dairy Science Association. Published by Elsevier Inc. All rights reserved.

Determinants of choice of delivery place: Testing rational choice theory and habitus theory.

PubMed

Broda, Anja; Krüger, Juliane; Schinke, Stephanie; Weber, Andreas

2018-05-07

The current study uses two antipodal social science theories, the rational choice theory and the habitus theory, and applies these to describe how women choose between intraclinical (i.e., hospital-run birth clinics) and extraclinical (i.e., midwife-led birth centres or home births) delivery places. Data were collected in a cross-sectional questionnaire-based survey among 189 women. A list of 22 determinants, conceptualized to capture the two theoretical concepts, were rated on a 7-point Likert scale with 1 = unimportant to 7 = very important. The analytic method was structural equation modelling. A model was built, in which the rational choice theory and the habitus theory as latent variables predicted the choice of delivery place. With regards to the choice of delivery place, 89.3% of the women wanted an intraclinical and 10.7% an extraclinical delivery place at the time of their last child's birth. Significant differences between women with a choice of an intraclinical or extraclinical delivery place were found for 14 of the 22 determinants. In the structural equation model, rational choice theory determinants predicted a choice of intraclinical delivery and habitus theory determinants predicted a choice of extraclinical delivery. The two theories had diametrically opposed effects on the choice of delivery place. Women are more likely to decide on intraclinical delivery when arguments such as high medical standards, positive evaluations, or good advanced information are rated important. In contrast, women are more likely to decide on extraclinical delivery when factors such as family atmosphere during birth, friendliness of health care professionals, or consideration of the woman's interests are deemed important. A practical implication of our study is that intraclinical deliveries may be promoted by providing comprehensive information, data and facts on various delivery-related issues, while extraclinical deliveries may be fostered by healthcare

Exploring examinee behaviours as validity evidence for multiple-choice question examinations.

PubMed

Surry, Luke T; Torre, Dario; Durning, Steven J

2017-10-01

Clinical-vignette multiple choice question (MCQ) examinations are used widely in medical education. Standardised MCQ examinations are used by licensure and certification bodies to award credentials that are meant to assure stakeholders as to the quality of physicians. Such uses are based on the interpretation of MCQ examination performance as giving meaningful information about the quality of clinical reasoning. There are several assumptions foundational to these interpretations and uses of standardised MCQ examinations. This study explores the implicit assumption that cognitive processes elicited by clinical-vignette MCQ items are like the processes thought to occur with 'real-world' clinical reasoning as theorised by dual-process theory. Fourteen participants (three medical students, five residents and six staff physicians) completed three sets of five timed MCQ items (total 15) from the Medical Knowledge Self-Assessment Program (MKSAP). Upon answering a set of MCQs, each participant completed a retrospective think aloud (TA) protocol. Using constant comparative analysis (CCA) methods sensitised by dual-process theory, we performed a qualitative thematic analysis. Examinee behaviours fell into three categories: clinical reasoning behaviours, test-taking behaviours and reactions to the MCQ. Consistent with dual-process theory, statements about clinical reasoning behaviours were divided into two sub-categories: analytical reasoning and non-analytical reasoning. Each of these categories included several themes. Our study provides some validity evidence that test-takers' descriptions of their cognitive processes during completion of high-quality clinical-vignette MCQs align with processes expected in real-world clinical reasoning. This supports one of the assumptions important for interpretations of MCQ examination scores as meaningful measures of clinical reasoning. Our observations also suggest that MCQs elicit other cognitive processes, including certain test

Testing Transitivity of Preferences on Two-Alternative Forced Choice Data

PubMed Central

Regenwetter, Michel; Dana, Jason; Davis-Stober, Clintin P.

2010-01-01

As Duncan Luce and other prominent scholars have pointed out on several occasions, testing algebraic models against empirical data raises difficult conceptual, mathematical, and statistical challenges. Empirical data often result from statistical sampling processes, whereas algebraic theories are nonprobabilistic. Many probabilistic specifications lead to statistical boundary problems and are subject to nontrivial order constrained statistical inference. The present paper discusses Luce's challenge for a particularly prominent axiom: Transitivity. The axiom of transitivity is a central component in many algebraic theories of preference and choice. We offer the currently most complete solution to the challenge in the case of transitivity of binary preference on the theory side and two-alternative forced choice on the empirical side, explicitly for up to five, and implicitly for up to seven, choice alternatives. We also discuss the relationship between our proposed solution and weak stochastic transitivity. We recommend to abandon the latter as a model of transitive individual preferences. PMID:21833217

The development and validation of a two-tiered multiple-choice instrument to identify alternative conceptions in earth science

NASA Astrophysics Data System (ADS)

Mangione, Katherine Anna

This study was to determine reliability and validity for a two-tiered, multiple- choice instrument designed to identify alternative conceptions in earth science. Additionally, this study sought to identify alternative conceptions in earth science held by preservice teachers, to investigate relationships between self-reported confidence scores and understanding of earth science concepts, and to describe relationships between content knowledge and alternative conceptions and planning instruction in the science classroom. Eighty-seven preservice teachers enrolled in the MAT program participated in this study. Sixty-eight participants were female, twelve were male, and seven chose not to answer. Forty-seven participants were in the elementary certification program, five were in the middle school certification program, and twenty-nine were pursuing secondary certification. Results indicate that the two-tiered, multiple-choice format can be a reliable and valid method for identifying alternative conceptions. Preservice teachers in all certification areas who participated in this study may possess common alternative conceptions previously identified in the literature. Alternative conceptions included: all rivers flow north to south, the shadow of the Earth covers the Moon causing lunar phases, the Sun is always directly overhead at noon, weather can be predicted by animal coverings, and seasons are caused by the Earth's proximity to the Sun. Statistical analyses indicated differences, however not all of them significant, among all subgroups according to gender and certification area. Generally males outperformed females and preservice teachers pursuing middle school certification had higher scores on the questionnaire followed by those obtaining secondary certification. Elementary preservice teachers scored the lowest. Additionally, self-reported scores of confidence in one's answers and understanding of the earth science concept in question were analyzed. There was a

Reproducibility of an objective four-choice canine vision testing technique that assesses vision at differing light intensities.

PubMed

Annear, Matthew J; Gornik, Kara R; Venturi, Francesca L; Hauptman, Joe G; Bartoe, Joshua T; Petersen-Jones, Simon M

2013-09-01

The increasing importance of canine retinal dystrophy models means accurate vision testing is needed. This study was performed to evaluate a four-choice vision testing technique for any difference in outcome measures with repeated evaluations of the same dogs.   Four 11-month-old RPE65-deficient dogs. Vision was evaluated using a previously described four-choice vision testing device. Four evaluations were performed at 2-week intervals. Vision was assessed at six different white light intensities (bright through dim), and each eye was evaluated separately. The ability to select the one of the four exit tunnels that was open at the far end was assessed ('choice of exit') and recorded as correct or incorrect first tunnel choice. 'Time to exit' the device was also recorded. Both outcomes were analyzed for significance using anova. We hypothesized that performance would improve with repeated testing (more correct choices and more rapid time to exit). 'Choice of exit' did not vary significantly between each evaluation (P = 0.12), in contrast 'time to exit' increased significantly (P = 0.012), and showed greater variability in dim light conditions. We found no evidence to support the hypothesis that either measure of outcome worsened with repeated testing; in fact, the 'time to exit' outcome worsened rather than improved. The 'choice of exit' gave consistent results between trials. These outcome data indicate the importance of including a choice-based assessment of vision in addition to measurement of device transit time. © 2012 American College of Veterinary Ophthalmologists.

Understanding Rasch Measurement: Distractors with Information in Multiple Choice Items: A Rationale Based on the Rasch Model

ERIC Educational Resources Information Center

Andrich, David; Styles, Irene

2011-01-01

There is a substantial literature on attempts to obtain information on the proficiency of respondents from distractors in multiple choice items. Information in a distractor implies that a person who chooses that distractor has greater proficiency than if the person chose another distractor with no information. A further implication is that the…

Equal Opportunity in the Classroom: Test Construction in a Diversity-Sensitive Environment.

ERIC Educational Resources Information Center

Ghorpade, Jai; Lackritz, James R.

1998-01-01

Two multiple-choice tests and one essay test were taken by 231 students (50/50 male/female, 192 White, 39 East Asian, Black, Mexican American, or Middle Eastern). Multiple-choice tests showed no significant differences in equal employment opportunity terms; women and men scored about the same on essays, but minority students had significantly…

Detection of Undocumented Changepoints Using Multiple Test Statistics and Composite Reference Series.

NASA Astrophysics Data System (ADS)

Menne, Matthew J.; Williams, Claude N., Jr.

2005-10-01

An evaluation of three hypothesis test statistics that are commonly used in the detection of undocumented changepoints is described. The goal of the evaluation was to determine whether the use of multiple tests could improve undocumented, artificial changepoint detection skill in climate series. The use of successive hypothesis testing is compared to optimal approaches, both of which are designed for situations in which multiple undocumented changepoints may be present. In addition, the importance of the form of the composite climate reference series is evaluated, particularly with regard to the impact of undocumented changepoints in the various component series that are used to calculate the composite.In a comparison of single test changepoint detection skill, the composite reference series formulation is shown to be less important than the choice of the hypothesis test statistic, provided that the composite is calculated from the serially complete and homogeneous component series. However, each of the evaluated composite series is not equally susceptible to the presence of changepoints in its components, which may be erroneously attributed to the target series. Moreover, a reference formulation that is based on the averaging of the first-difference component series is susceptible to random walks when the composition of the component series changes through time (e.g., values are missing), and its use is, therefore, not recommended. When more than one test is required to reject the null hypothesis of no changepoint, the number of detected changepoints is reduced proportionately less than the number of false alarms in a wide variety of Monte Carlo simulations. Consequently, a consensus of hypothesis tests appears to improve undocumented changepoint detection skill, especially when reference series homogeneity is violated. A consensus of successive hypothesis tests using a semihierarchic splitting algorithm also compares favorably to optimal solutions, even when

Pushing Critical Thinking Skills With Multiple-Choice Questions: Does Bloom's Taxonomy Work?

PubMed

Zaidi, Nikki L Bibler; Grob, Karri L; Monrad, Seetha M; Kurtz, Joshua B; Tai, Andrew; Ahmed, Asra Z; Gruppen, Larry D; Santen, Sally A

2018-06-01

Medical school assessments should foster the development of higher-order thinking skills to support clinical reasoning and a solid foundation of knowledge. Multiple-choice questions (MCQs) are commonly used to assess student learning, and well-written MCQs can support learner engagement in higher levels of cognitive reasoning such as application or synthesis of knowledge. Bloom's taxonomy has been used to identify MCQs that assess students' critical thinking skills, with evidence suggesting that higher-order MCQs support a deeper conceptual understanding of scientific process skills. Similarly, clinical practice also requires learners to develop higher-order thinking skills that include all of Bloom's levels. Faculty question writers and examinees may approach the same material differently based on varying levels of knowledge and expertise, and these differences can influence the cognitive levels being measured by MCQs. Consequently, faculty question writers may perceive that certain MCQs require higher-order thinking skills to process the question, whereas examinees may only need to employ lower-order thinking skills to render a correct response. Likewise, seemingly lower-order questions may actually require higher-order thinking skills to respond correctly. In this Perspective, the authors describe some of the cognitive processes examinees use to respond to MCQs. The authors propose that various factors affect both the question writer and examinee's interaction with test material and subsequent cognitive processes necessary to answer a question.

Does Educator Training or Experience Affect the Quality of Multiple-Choice Questions?

PubMed

Webb, Emily M; Phuong, Jonathan S; Naeger, David M

2015-10-01

Physicians receive little training on proper multiple-choice question (MCQ) writing methods. Well-constructed MCQs follow rules, which ensure that a question tests what it is intended to test. Questions that break these are described as "flawed." We examined whether the prevalence of flawed questions differed significantly between those with or without prior training in question writing and between those with different levels of educator experience. We assessed 200 unedited MCQs from a question bank for our senior medical student radiology elective: an equal number of questions (50) were written by faculty with previous training in MCQ writing, other faculty, residents, and medical students. Questions were scored independently by two readers for the presence of 11 distinct flaws described in the literature. Questions written by faculty with MCQ writing training had significantly fewer errors: mean 0.4 errors per question compared to a mean of 1.5-1.7 errors per question for the other groups (P < .001). There were no significant differences in the total number of errors between the untrained faculty, residents, and students (P values .35-.91). Among trained faculty 17/50 questions (34%) were flawed, whereas other faculty wrote 38/50 (76%) flawed questions, residents 37/50 (74%), and students 44/50 (88%). Trained question writers' higher performance was mainly manifest in the reduced frequency of five specific errors. Faculty with training in effective MCQ writing made fewer errors in MCQ construction. Educator experience alone had no effect on the frequency of flaws; faculty without dedicated training, residents, and students performed similarly. Copyright © 2015 AUR. Published by Elsevier Inc. All rights reserved.

Estimating the Effect on Grades of Using Multiple-Choice versus Constructive-Response Questions: Data from the Classroom

ERIC Educational Resources Information Center

Hickson, Stephen; Reed, W. Robert; Sander, Nicholas

2012-01-01

This study investigates the degree to which grades based solely on constructed-response (CR) questions differ from grades based solely on multiple-choice (MC) questions. If CR questions are to justify their higher costs, they should produce different grade outcomes than MC questions. We use a data set composed of thousands of observations on…

Pattern recognition as a concept for multiple-choice questions in a national licensing exam.

PubMed

Freiwald, Tilo; Salimi, Madjid; Khaljani, Ehsan; Harendza, Sigrid

2014-11-14

Multiple-choice questions (MCQ) are still widely used in high stakes medical exams. We wanted to examine whether and to what extent a national licensing exam uses the concept of pattern recognition to test applied clinical knowledge. We categorized all 4,134 German National medical licensing exam questions between October 2006 and October 2012 by discipline, year, and type. We analyzed questions from the four largest disciplines: internal medicine (n = 931), neurology (n = 305), pediatrics (n = 281), and surgery (n = 233), with respect to the following question types: knowledge questions (KQ), pattern recognition questions (PRQ), inverse PRQ (IPRQ), and pseudo PRQ (PPRQ). A total 51.1% of all questions were of a higher taxonomical order (PRQ and IPRQ) with a significant decrease in the percentage of these questions (p <0.001) from 2006 (61.5%) to 2012 (41.6%). The proportion of PRQs and IPRQs was significantly lower (p <0.001) in internal medicine and surgery, compared to neurology and pediatrics. PRQs were mostly used in questions about diagnoses (71.7%). A significantly higher (p <0.05) percentage of PR/therapy questions was found for internal medicine compared with neurology and pediatrics. The concept of pattern recognition is used with different priorities and to various extents by the different disciplines in a high stakes exam to test applied clinical knowledge. Being aware of this concept may aid in the design and balance of MCQs in an exam with respect to testing clinical reasoning as a desired skill at the threshold of postgraduate medical education.

The Development and Validation of a Two-Tiered Multiple-Choice Instrument to Identify Alternative Conceptions in Earth Science

ERIC Educational Resources Information Center

Mangione, Katherine Anna

2010-01-01

This study was to determine reliability and validity for a two-tiered, multiple- choice instrument designed to identify alternative conceptions in earth science. Additionally, this study sought to identify alternative conceptions in earth science held by preservice teachers, to investigate relationships between self-reported confidence scores and…

21 CFR 866.5660 - Multiple autoantibodies immunological test system.

Code of Federal Regulations, 2010 CFR

2010-04-01

... SERVICES (CONTINUED) MEDICAL DEVICES IMMUNOLOGY AND MICROBIOLOGY DEVICES Immunological Test Systems § 866.5660 Multiple autoantibodies immunological test system. (a) Identification. A multiple autoantibodies... 21 Food and Drugs 8 2010-04-01 2010-04-01 false Multiple autoantibodies immunological test system...

Augmenting Fellow Education Through Spaced Multiple-Choice Questions.

PubMed

Barsoumian, Alice E; Yun, Heather C

2018-01-01

The San Antonio Uniformed Services Health Education Consortium Infectious Disease Fellowship program historically included a monthly short-answer and multiple-choice quiz. The intent was to ensure medical knowledge in relevant content areas that may not be addressed through clinical rotations, such as operationally relevant infectious disease. After completion, it was discussed in a small group with faculty. Over time, faculty noted increasing dissatisfaction with the activity. Spaced interval education is useful in retention of medical knowledge and skills by medical students and residents. Its use in infectious disease fellow education has not been described. To improve the quiz experience, we assessed the introduction of spaced education curriculum in our program. A pre-intervention survey was distributed to assess the monthly quiz with Likert scale and open-ended questions. A multiple-choice question spaced education curriculum was created using the Qstream(R) platform in 2011. Faculty development on question writing was conducted. Two questions were delivered every 2 d. Incorrectly and correctly answered questions were repeated after 7 and 13 d, respectively. Questions needed to be answered correctly twice to be retired. Fellow satisfaction was assessed at semi-annual fellowship reviews over 5 yr and by a one-time repeat survey. Pre-intervention survey of six fellows indicated dissatisfaction with the time commitment of the monthly quiz (median Likert score of 2, mean 6.5 h to complete), neutral in perceived utility, but satisfaction with knowledge retention (Likert score 4). Eighteen fellows over 5 yr participated in the spaced education curriculum. Three quizzes with 20, 39, and 48 questions were designed. Seventeen percentage of questions addressed operationally relevant topics. Fifty-nine percentage of questions were answered correctly on first attempt, improving to 93% correct answer rate at the end of the analysis. Questions were attempted 2,999 times

Estimating Statistical Power When Making Adjustments for Multiple Tests

ERIC Educational Resources Information Center

Porter, Kristin E.

2016-01-01

In recent years, there has been increasing focus on the issue of multiple hypotheses testing in education evaluation studies. In these studies, researchers are typically interested in testing the effectiveness of an intervention on multiple outcomes, for multiple subgroups, at multiple points in time or across multiple treatment groups. When…

School Choice in Suburbia: Test Scores, Race, and Housing Markets

ERIC Educational Resources Information Center

Dougherty, Jack; Harelson, Jeffrey; Maloney, Laura; Murphy, Drew; Smith, Russell; Snow, Michael; Zannoni, Diane

2009-01-01

Home buyers exercise school choice when shopping for a private residence due to its location in a public school district or attendance area. In this quantitative study of one Connecticut suburban district, we measure the effect of elementary school test scores and racial composition on home buyers' willingness to purchase single-family homes over…

Common pitfalls in statistical analysis: The perils of multiple testing

PubMed Central

Ranganathan, Priya; Pramesh, C. S.; Buyse, Marc

2016-01-01

Multiple testing refers to situations where a dataset is subjected to statistical testing multiple times - either at multiple time-points or through multiple subgroups or for multiple end-points. This amplifies the probability of a false-positive finding. In this article, we look at the consequences of multiple testing and explore various methods to deal with this issue. PMID:27141478

Developing Achievement Test: A Research for Assessment of 5th Grade Biology Subject

ERIC Educational Resources Information Center

Sener, Nilay; Tas, Erol

2017-01-01

The purpose of this study is to prepare a multiple-choice achievement test with high reliability and validity for the "Let's Solve the Puzzle of Our Body" unit. For this purpose, a multiple choice achievement test consisting of 46 items was applied to 178 fifth grade students in total. As a result of the test and material analysis…

Collaborative Testing: Cognitive and Interpersonal Processes Related to Enhanced Test Performance

ERIC Educational Resources Information Center

Kapitanoff, Susan H.

2009-01-01

Research has demonstrated that collaborative testing, working on tests in groups, leads to improved test scores but the mechanism by which this occurs has not been specified. Three factors were proposed as mediators: cognitive processes, interpersonal interactions and reduced test-anxiety. Thirty-three students completed a multiple-choice exam…

Women's and healthcare professionals' preferences for prenatal testing: a discrete choice experiment.

PubMed

Beulen, Lean; Grutters, Janneke P C; Faas, Brigitte H W; Feenstra, Ilse; Groenewoud, Hans; van Vugt, John M G; Bekker, Mireille N

2015-06-01

This study evaluates pregnant women's and healthcare professionals' preferences regarding specific prenatal screening and diagnostic test characteristics. A discrete choice experiment was developed to assess preferences for prenatal tests that differed in seven attributes: minimal gestational age, time to test results, level of information, detection rate, false positive rate, miscarriage risk and costs. The questionnaire was completed by 596 (70.2%) pregnant women and 297 (51.7%) healthcare professionals, of whom 507 (85.1%) and 283 (95.3%), respectively, were included in further analyses as their choice behavior indicated prenatal testing was an option to them. Comparison of results showed differences in relative importance attached to attributes, further reflected by differences in willingness to trade between attributes. Pregnant women are willing to accept a less accurate test to obtain more information on fetal chromosomal status or to exclude the risk of procedure-related miscarriage. Healthcare professionals consider level of information and miscarriage risk to be most important as well but put more emphasis on timing and accuracy. Pregnant women and healthcare professionals differ significantly in their preferences regarding prenatal test characteristics. Healthcare professionals should take these differences into consideration when counseling pregnant women on prenatal testing. © 2015 John Wiley & Sons, Ltd.

An omnibus test for family-based association studies with multiple SNPs and multiple phenotypes.

PubMed

Lasky-Su, Jessica; Murphy, Amy; McQueen, Matthew B; Weiss, Scott; Lange, Christoph

2010-06-01

We propose an omnibus family-based association test (MFBAT) that can be applied to multiple markers and multiple phenotypes and that has only one degree of freedom. The proposed test statistic extends current FBAT methodology to incorporate multiple markers as well as multiple phenotypes. Using simulation studies, power estimates for the proposed methodology are compared with the standard methodologies. On the basis of these simulations, we find that MFBAT substantially outperforms other methods, including haplotypic approaches and doing multiple tests with single single-nucleotide polymorphisms (SNPs) and single phenotypes. The practical relevance of the approach is illustrated by an application to asthma in which SNP/phenotype combinations are identified and reach overall significance that would not have been identified using other approaches. This methodology is directly applicable to cases in which there are multiple SNPs, such as candidate gene studies, cases in which there are multiple phenotypes, such as expression data, and cases in which there are multiple phenotypes and genotypes, such as genome-wide association studies that incorporate expression profiles as phenotypes. This program is available in the PBAT analysis package.

Retention of Prose Following Testing with Different Types of Tests.

ERIC Educational Resources Information Center

Duchastel, Philippe C.

1981-01-01

Taking a test on a passage one has just studied is known to enhance later retention. This effect was influenced by the type of initial test used. It was evident in the case of the initial short-answer test, but not in the case of multiple choice and free recall tests. (Author/RD)

Multiple composite grafts (k, π or double-Y) in coronary artery surgery: a choice or a necessity?

PubMed

Mannacio, Vito; Cirillo, Plinio; Mannacio, Luigi; Antignano, Anita; Mottola, Michele; Vosa, Carlo

2015-01-01

Composite grafts allow complete arterial revascularization with minimal aortic manipulations. The Y-T configuration supplies all distal branches adequately, whereas it is unclear whether complex composite configurations (K, Π or double-Y) are equally at rest or when challenged by maximal requirements. Forty-seven patients who underwent off-pump coronary artery revascularization by multiple arterial composite grafts (K, Π or double-Y) were retrospectively evaluated. Indication for this surgical option was porcelain aorta or conduit unavailability. Composite systems were evaluated by intraoperative flow measurements and perioperative transthoracic Doppler ultrasonography, 12 months later also by exercise test, sestamibi scintigraphy at rest and during induced hyperaemia and by 64-slice multidetector CT angiography. A total of 141 distal anastomoses were implanted as composite grafts. Perioperative flow measurements and 12-month Doppler ultrasonography were adequate at rest. At stress test, chest pain and/or induced ECG evidence of ischaemia are found in 16 patients (39%). During dipyridamole-induced hyperaemia, single-photon emission computed tomography image revealed that mean summed stress score was 7.2 ± 5.7, summed difference score 5.3 ± 4.2 and coronary flow reserve 1.7 ± 0.2. Multiple composite grafts, albeit adequate at rest, were unable to meet flow requirements during maximal hyperaemia. In daily practice, their use must be not a choice but rather a necessity in those patients without alternative options. © The Author 2014. Published by Oxford University Press on behalf of the European Association for Cardio-Thoracic Surgery. All rights reserved.

Pressure scanning choices - Rotary vs electronic

NASA Astrophysics Data System (ADS)

Pemberton, Addison

The choices available for present-day pressure scanning applications are described. Typical pressure scanning applications include wind tunnels, flight testing, turbine engine testing, process control, and laboratory/bench testing. The Scanivalve concept is discussed and it is noted that their use eliminates the cost of multiple individual pressure transducers and their signal conditioners as well as associated wiring for each pressure to be measured. However, they are limited to a maximum acquisition speed of 20 ports/sec/scanner. The advantages of electronic pressure scanners include in-situ calibration on demand, fast data acquisition speed, and high reliability. On the other hand, they are three times more expensive than rotary Scanivalves.

Multiple testing and power calculations in genetic association studies.

PubMed

So, Hon-Cheong; Sham, Pak C

2011-01-01

Modern genetic association studies typically involve multiple single-nucleotide polymorphisms (SNPs) and/or multiple genes. With the development of high-throughput genotyping technologies and the reduction in genotyping cost, investigators can now assay up to a million SNPs for direct or indirect association with disease phenotypes. In addition, some studies involve multiple disease or related phenotypes and use multiple methods of statistical analysis. The combination of multiple genetic loci, multiple phenotypes, and multiple methods of evaluating associations between genotype and phenotype means that modern genetic studies often involve the testing of an enormous number of hypotheses. When multiple hypothesis tests are performed in a study, there is a risk of inflation of the type I error rate (i.e., the chance of falsely claiming an association when there is none). Several methods for multiple-testing correction are in popular use, and they all have strengths and weaknesses. Because no single method is universally adopted or always appropriate, it is important to understand the principles, strengths, and weaknesses of the methods so that they can be applied appropriately in practice. In this article, we review the three principle methods for multiple-testing correction and provide guidance for calculating statistical power.

PERMANOVA-S: association test for microbial community composition that accommodates confounders and multiple distances.

PubMed

Tang, Zheng-Zheng; Chen, Guanhua; Alekseyenko, Alexander V

2016-09-01

Recent advances in sequencing technology have made it possible to obtain high-throughput data on the composition of microbial communities and to study the effects of dysbiosis on the human host. Analysis of pairwise intersample distances quantifies the association between the microbiome diversity and covariates of interest (e.g. environmental factors, clinical outcomes, treatment groups). In the design of these analyses, multiple choices for distance metrics are available. Most distance-based methods, however, use a single distance and are underpowered if the distance is poorly chosen. In addition, distance-based tests cannot flexibly handle confounding variables, which can result in excessive false-positive findings. We derive presence-weighted UniFrac to complement the existing UniFrac distances for more powerful detection of the variation in species richness. We develop PERMANOVA-S, a new distance-based method that tests the association of microbiome composition with any covariates of interest. PERMANOVA-S improves the commonly-used Permutation Multivariate Analysis of Variance (PERMANOVA) test by allowing flexible confounder adjustments and ensembling multiple distances. We conducted extensive simulation studies to evaluate the performance of different distances under various patterns of association. Our simulation studies demonstrate that the power of the test relies on how well the selected distance captures the nature of the association. The PERMANOVA-S unified test combines multiple distances and achieves good power regardless of the patterns of the underlying association. We demonstrate the usefulness of our approach by reanalyzing several real microbiome datasets. miProfile software is freely available at https://medschool.vanderbilt.edu/tang-lab/software/miProfile z.tang@vanderbilt.edu or g.chen@vanderbilt.edu Supplementary data are available at Bioinformatics online. © The Author 2016. Published by Oxford University Press.

PERMANOVA-S: association test for microbial community composition that accommodates confounders and multiple distances

PubMed Central

Tang, Zheng-Zheng; Chen, Guanhua; Alekseyenko, Alexander V.

2016-01-01

Motivation: Recent advances in sequencing technology have made it possible to obtain high-throughput data on the composition of microbial communities and to study the effects of dysbiosis on the human host. Analysis of pairwise intersample distances quantifies the association between the microbiome diversity and covariates of interest (e.g. environmental factors, clinical outcomes, treatment groups). In the design of these analyses, multiple choices for distance metrics are available. Most distance-based methods, however, use a single distance and are underpowered if the distance is poorly chosen. In addition, distance-based tests cannot flexibly handle confounding variables, which can result in excessive false-positive findings. Results: We derive presence-weighted UniFrac to complement the existing UniFrac distances for more powerful detection of the variation in species richness. We develop PERMANOVA-S, a new distance-based method that tests the association of microbiome composition with any covariates of interest. PERMANOVA-S improves the commonly-used Permutation Multivariate Analysis of Variance (PERMANOVA) test by allowing flexible confounder adjustments and ensembling multiple distances. We conducted extensive simulation studies to evaluate the performance of different distances under various patterns of association. Our simulation studies demonstrate that the power of the test relies on how well the selected distance captures the nature of the association. The PERMANOVA-S unified test combines multiple distances and achieves good power regardless of the patterns of the underlying association. We demonstrate the usefulness of our approach by reanalyzing several real microbiome datasets. Availability and Implementation: miProfile software is freely available at https://medschool.vanderbilt.edu/tang-lab/software/miProfile. Contact: z.tang@vanderbilt.edu or g.chen@vanderbilt.edu Supplementary information: Supplementary data are available at Bioinformatics

Relationships among Testing Medium, Test Performance, and Testing Time of High School Students Who Are Visually Impaired

ERIC Educational Resources Information Center

Erin, Jane N.; Hong, Sunggye; Schoch, Christina; Kuo, YaJu

2006-01-01

This study compared the test scores and time required by high school students who are blind, sighted, or have low vision to complete tests administered in written and oral formats. The quantitative results showed that the blind students performed better on multiple-choice tests in braille and needed more time while taking tests in braille. The…

Pathfinding in a large vertebrate axon tract: isotypic interactions guide retinotectal axons at multiple choice points

PubMed Central

Pittman, Andrew J.; Law, Mei-Yee; Chien, Chi-Bin

2008-01-01

Summary Navigating axons respond to environmental guidance signals, but can also follow axons that have gone before—pioneer axons. Pioneers have been studied extensively in simple systems, but the role of axon-axon interactions remains largely unexplored in large vertebrate axon tracts, where cohorts of identical axons could potentially use isotypic interactions to guide each other through multiple choice points. Furthermore, the relative importance of axon-axon interactions compared to axon-autonomous receptor function has not been assessed. Here we test the role of axon-axon interactions in retinotectal development, by devising a technique to selectively remove or replace early-born retinal ganglion cells (RGCs). We find that early RGCs are both necessary and sufficient for later axons to exit the eye. Furthermore, introducing misrouted axons by transplantation reveals that guidance from eye to tectum relies heavily on interactions between axons, including both pioneer-follower and community effects. We conclude that axon-axon interactions and ligand-receptor signaling have coequal roles, cooperating to ensure the fidelity of axon guidance in developing vertebrate tracts. PMID:18653554

An Australian Study Comparing the Use of Multiple-Choice Questionnaires with Assignments as Interim, Summative Law School Assessment

ERIC Educational Resources Information Center

Huang, Vicki

2017-01-01

To the author's knowledge, this is the first Australian study to empirically compare the use of a multiple-choice questionnaire (MCQ) with the use of a written assignment for interim, summative law school assessment. This study also surveyed the same student sample as to what types of assessments are preferred and why. In total, 182 undergraduate…

Test Design Project: Studies in Test Bias. Annual Report.

ERIC Educational Resources Information Center

McArthur, David

Item bias in a multiple-choice test can be detected by appropriate analyses of the persons x items scoring matrix. This permits comparison of groups of examinees tested with the same instrument. The test may be biased if it is not measuring the same thing in comparable groups, if groups are responding to different aspects of the test items, or if…

Decision making and preferences for acoustic signals in choice situations by female crickets.

PubMed

Gabel, Eileen; Kuntze, Janine; Hennig, R Matthias

2015-08-01

Multiple attributes usually have to be assessed when choosing a mate. Efficient choice of the best mate is complicated if the available cues are not positively correlated, as is often the case during acoustic communication. Because of varying distances of signalers, a female may be confronted with signals of diverse quality at different intensities. Here, we examined how available cues are weighted for a decision by female crickets. Two songs with different temporal patterns and/or sound intensities were presented in a choice paradigm and compared with female responses from a no-choice test. When both patterns were presented at equal intensity, preference functions became wider in choice situations compared with a no-choice paradigm. When the stimuli in two-choice tests were presented at different intensities, this effect was counteracted as preference functions became narrower compared with choice tests using stimuli of equal intensity. The weighting of intensity differences depended on pattern quality and was therefore non-linear. A simple computational model based on pattern and intensity cues reliably predicted female decisions. A comparison of processing schemes suggested that the computations for pattern recognition and directionality are performed in a network with parallel topology. However, the computational flow of information corresponded to serial processing. © 2015. Published by The Company of Biologists Ltd.

Development and validation of a measure of informed choice for women undergoing non-invasive prenatal testing for aneuploidy.

PubMed

Lewis, Celine; Hill, Melissa; Skirton, Heather; Chitty, Lyn S

2016-06-01

Non-invasive prenatal testing (NIPT) using cell-free DNA for aneuploidy is a highly accurate screening test; however, concerns exist around the potential for routinisation of testing. The multidimensional measure of informed choice (MMIC) is a quantitative instrument developed to assess informed choice for Down syndrome screening (DSS). We have validated a modified MMIC for NIPT and measured informed choice among women offered NIPT in a public health service. The measure was distributed to women recruited across eight maternity units in the United Kingdom who had accepted DSS. Construct validity was assessed by simultaneously conducting qualitative interviews. Five hundred and eighty-five questionnaires were completed and 45 interviews conducted after blood-draw (or equivalent for those that declined NIPT). The measure demonstrated good internal consistency and internal validity. Results indicate the vast majority of women (89%) made an informed choice; 95% were judged to have good knowledge, 88% had a positive attitude and 92% had deliberated. Of the 11% judged to have made an uninformed choice, 55% had not deliberated, 41% had insufficient knowledge, and 19% had a negative attitude. Ethnicity (OR=2.78, P=0.003) and accepting NIPT (OR=16.05, P=0.021) were found to be significant predictors of informed choice. The high rate of informed choice is likely to reflect the importance placed on the provision of pre-test counselling in this study. It will be vital to ensure that this is maintained once NIPT is offered in routine clinical practice.

Do Sequentially-Presented Answer Options Prevent the Use of Testwiseness Cues on Continuing Medical Education Tests?

ERIC Educational Resources Information Center

Willing, Sonja; Ostapczuk, Martin; Musch, Jochen

2015-01-01

Testwiseness--that is, the ability to find subtle cues towards the solution by the simultaneous comparison of the available answer options--threatens the validity of multiple-choice (MC) tests. Discrete-option multiple-choice (DOMC) has recently been proposed as a computerized alternative testing format for MC tests, and presumably allows for a…

Multiple Testing with Modified Bonferroni Methods.

ERIC Educational Resources Information Center

Li, Jianmin; And Others

This paper discusses the issue of multiple testing and overall Type I error rates in contexts other than multiple comparisons of means. It demonstrates, using a 5 x 5 correlation matrix, the application of 5 recently developed modified Bonferroni procedures developed by the following authors: (1) Y. Hochberg (1988); (2) B. S. Holland and M. D.…

The detection of cheating in multiple choice examinations

NASA Astrophysics Data System (ADS)

Richmond, Peter; Roehner, Bertrand M.

2015-10-01

Cheating in examinations is acknowledged by an increasing number of organizations to be widespread. We examine two different approaches to assess their effectiveness at detecting anomalous results, suggestive of collusion, using data taken from a number of multiple-choice examinations organized by the UK Radio Communication Foundation. Analysis of student pair overlaps of correct answers is shown to give results consistent with more orthodox statistical correlations for which confidence limits as opposed to the less familiar "Bonferroni method" can be used. A simulation approach is also developed which confirms the interpretation of the empirical approach. Then the variables Xi =(1 -Ui) Yi +Ui Z are a system of symmetric dependent binary variables (0 , 1 ; p) whose correlation matrix is ρij = r. The proof is easy and given in the paper. Let us add two remarks. • We used the expression "symmetric variables" to reflect the fact that all Xi play the same role. The expression "exchangeable variables" is often used with the same meaning. • The correlation matrix has only positive elements. This is of course imposed by the symmetry condition. ρ12 < 0 and ρ23 < 0 would imply ρ13 > 0, thus violating the symmetry requirement. In the following subsections we will be concerned with the question of uniqueness of the set of Xi generated above. Needless to say, it is useful to know whether the proposition gives the answer or only one among many. More precisely, the problem can be stated as follows.

Making School Choice Work

ERIC Educational Resources Information Center

DeArmond, Michael; Jochim, Ashley; Lake, Robin

2014-01-01

School choice is increasingly the new normal in urban education. But in cities with multiple public school options, how can civic leaders create a choice system that works for all families, whether they choose a charter or district public school? To answer this question, the Center on Reinventing Public Education (CRPE) researchers surveyed 4,000…

A set partitioning reformulation for the multiple-choice multidimensional knapsack problem

NASA Astrophysics Data System (ADS)

Voß, Stefan; Lalla-Ruiz, Eduardo

2016-05-01

The Multiple-choice Multidimensional Knapsack Problem (MMKP) is a well-known ?-hard combinatorial optimization problem that has received a lot of attention from the research community as it can be easily translated to several real-world problems arising in areas such as allocating resources, reliability engineering, cognitive radio networks, cloud computing, etc. In this regard, an exact model that is able to provide high-quality feasible solutions for solving it or being partially included in algorithmic schemes is desirable. The MMKP basically consists of finding a subset of objects that maximizes the total profit while observing some capacity restrictions. In this article a reformulation of the MMKP as a set partitioning problem is proposed to allow for new insights into modelling the MMKP. The computational experimentation provides new insights into the problem itself and shows that the new model is able to improve on the best of the known results for some of the most common benchmark instances.

Model Choice and Sample Size in Item Response Theory Analysis of Aphasia Tests

ERIC Educational Resources Information Center

Hula, William D.; Fergadiotis, Gerasimos; Martin, Nadine

2012-01-01

Purpose: The purpose of this study was to identify the most appropriate item response theory (IRT) measurement model for aphasia tests requiring 2-choice responses and to determine whether small samples are adequate for estimating such models. Method: Pyramids and Palm Trees (Howard & Patterson, 1992) test data that had been collected from…

Acoustic Features Influence Musical Choices Across Multiple Genres.

PubMed

Barone, Michael D; Bansal, Jotthi; Woolhouse, Matthew H

2017-01-01

Based on a large behavioral dataset of music downloads, two analyses investigate whether the acoustic features of listeners' preferred musical genres influence their choice of tracks within non-preferred, secondary musical styles. Analysis 1 identifies feature distributions for pairs of genre-defined subgroups that are distinct. Using correlation analysis, these distributions are used to test the degree of similarity between subgroups' main genres and the other music within their download collections. Analysis 2 explores the issue of main-to-secondary genre influence through the production of 10 feature-influence matrices, one per acoustic feature, in which cell values indicate the percentage change in features for genres and subgroups compared to overall population averages. In total, 10 acoustic features and 10 genre-defined subgroups are explored within the two analyses. Results strongly indicate that the acoustic features of people's main genres influence the tracks they download within non-preferred, secondary musical styles. The nature of this influence and its possible actuating mechanisms are discussed with respect to research on musical preference, personality, and statistical learning.

Acoustic Features Influence Musical Choices Across Multiple Genres

PubMed Central

Barone, Michael D.; Bansal, Jotthi; Woolhouse, Matthew H.

2017-01-01

Based on a large behavioral dataset of music downloads, two analyses investigate whether the acoustic features of listeners' preferred musical genres influence their choice of tracks within non-preferred, secondary musical styles. Analysis 1 identifies feature distributions for pairs of genre-defined subgroups that are distinct. Using correlation analysis, these distributions are used to test the degree of similarity between subgroups' main genres and the other music within their download collections. Analysis 2 explores the issue of main-to-secondary genre influence through the production of 10 feature-influence matrices, one per acoustic feature, in which cell values indicate the percentage change in features for genres and subgroups compared to overall population averages. In total, 10 acoustic features and 10 genre-defined subgroups are explored within the two analyses. Results strongly indicate that the acoustic features of people's main genres influence the tracks they download within non-preferred, secondary musical styles. The nature of this influence and its possible actuating mechanisms are discussed with respect to research on musical preference, personality, and statistical learning. PMID:28725200

Development and validation of a measure of informed choice for women undergoing non-invasive prenatal testing for aneuploidy

PubMed Central

Lewis, Celine; Hill, Melissa; Skirton, Heather; Chitty, Lyn S

2016-01-01

Non-invasive prenatal testing (NIPT) using cell-free DNA for aneuploidy is a highly accurate screening test; however, concerns exist around the potential for routinisation of testing. The multidimensional measure of informed choice (MMIC) is a quantitative instrument developed to assess informed choice for Down syndrome screening (DSS). We have validated a modified MMIC for NIPT and measured informed choice among women offered NIPT in a public health service. The measure was distributed to women recruited across eight maternity units in the United Kingdom who had accepted DSS. Construct validity was assessed by simultaneously conducting qualitative interviews. Five hundred and eighty-five questionnaires were completed and 45 interviews conducted after blood-draw (or equivalent for those that declined NIPT). The measure demonstrated good internal consistency and internal validity. Results indicate the vast majority of women (89%) made an informed choice; 95% were judged to have good knowledge, 88% had a positive attitude and 92% had deliberated. Of the 11% judged to have made an uninformed choice, 55% had not deliberated, 41% had insufficient knowledge, and 19% had a negative attitude. Ethnicity (OR=2.78, P=0.003) and accepting NIPT (OR=16.05, P=0.021) were found to be significant predictors of informed choice. The high rate of informed choice is likely to reflect the importance placed on the provision of pre-test counselling in this study. It will be vital to ensure that this is maintained once NIPT is offered in routine clinical practice. PMID:26508572

Sequential Test Strategies for Multiple Fault Isolation

NASA Technical Reports Server (NTRS)

Shakeri, M.; Pattipati, Krishna R.; Raghavan, V.; Patterson-Hine, Ann; Kell, T.

1997-01-01

In this paper, we consider the problem of constructing near optimal test sequencing algorithms for diagnosing multiple faults in redundant (fault-tolerant) systems. The computational complexity of solving the optimal multiple-fault isolation problem is super-exponential, that is, it is much more difficult than the single-fault isolation problem, which, by itself, is NP-hard. By employing concepts from information theory and Lagrangian relaxation, we present several static and dynamic (on-line or interactive) test sequencing algorithms for the multiple fault isolation problem that provide a trade-off between the degree of suboptimality and computational complexity. Furthermore, we present novel diagnostic strategies that generate a static diagnostic directed graph (digraph), instead of a static diagnostic tree, for multiple fault diagnosis. Using this approach, the storage complexity of the overall diagnostic strategy reduces substantially. Computational results based on real-world systems indicate that the size of a static multiple fault strategy is strictly related to the structure of the system, and that the use of an on-line multiple fault strategy can diagnose faults in systems with as many as 10,000 failure sources.

Milwaukee Longitudinal School Choice Evaluation: Annual School Testing Summary Report. SCDP Milwaukee Evaluation Report #4

ERIC Educational Resources Information Center

Gray, Nathan L.; Wolf, Patrick J.; Jensen, Laura I.

2008-01-01

With the passage of 2005 Wisconsin Act 125, private schools participating in the Milwaukee Parental Choice Program (MPCP) are now required to administer a nationally normed standardized test annually in reading, mathematics, and science to their MPCP (a.k.a. "Choice") students enrolled in the 4th, 8th, and 10th grades. The law further…

Developing and pilot testing a shared decision-making intervention for dialysis choice.

PubMed

Finderup, Jeanette; Jensen, Jens K D; Lomborg, Kirsten

2018-04-17

Evidence is inconclusive on how best to guide the patient in decision-making around haemodialysis and peritoneal dialysis choice. International guidelines recommend involvement of the patient in the decision to choose the dialysis modality most suitable for the individual patient. Nevertheless, studies have shown lack of involvement of the patient in decision-making. To develop and pilot test an intervention for shared decision-making targeting the choice of dialysis modality. This study reflects the first two phases of a complex intervention design: phase 1, the development process and phase 2, feasibility and piloting. Because decision aids were a part of the intervention, the International Patient Decision Aid Standards were considered. The pilot test included both the intervention and the feasibility of the validated shared decision-making questionnaire (SDM Q9) and the Decision Quality Measure (DQM) applied to evaluate the intervention. A total of 137 patients tested the intervention. After the intervention, 80% of the patients chose dialysis at home reflecting an increase of 23% in starting dialysis at home prior to the study. The SDM Q9 showed the majority of the patients experienced this intervention as shared decision-making. An intervention based on shared decision-making supported by decision aids seemed to increase the number of patients choosing home dialysis. The SDM Q9 and DQM were feasible evaluation tools. Further research is needed to gain insight into the patients' experiences of involvement and the implications for their choice of dialysis modality. © 2018 European Dialysis and Transplant Nurses Association/European Renal Care Association.

A Stratified Study of Students' Understanding of Basic Optics Concepts in Different Contexts Using Two-Tier Multiple-Choice Items

ERIC Educational Resources Information Center

Chu, Hye-Eun; Treagust, David F.; Chandrasegaran, A. L.

2009-01-01

A large scale study involving 1786 year 7-10 Korean students from three school districts in Seoul was undertaken to evaluate their understanding of basic optics concepts using a two-tier multiple-choice diagnostic instrument consisting of four pairs of items, each of which evaluated the same concept in two different contexts. The instrument, which…

E1-09 Termite Choice Testing: What is it good for?

Treesearch

Frederick III Green; Patti Lebow; S. Nami Kartal

2014-01-01

The prime objective of this investigation was to compare mass loss from termite attack on NHA treated and untreated pine blocks simultaneously in a two-block choice test using Reticulitermes flavipes (Kollar). Southern pine blocks were vacuum treated with aqueous solutions of NHA ranging from 250-10,000 ppm and exposed to one gram of R....

Teaching and Evaluation Materials Utilizing Multiple Representations in Mechanics

ERIC Educational Resources Information Center

Savinainen, A.; Nieminen, P.; Makynen, A.; Viiri, J.

2013-01-01

In this paper, we present materials and teaching ideas utilizing multiple representations in the contexts of kinematics and the force concept. These ideas and materials are substantiated by evidence and can be readily used in teaching with no special training. In addition, we briefly discuss two multiple-choice tests based on physics education…

Reducing the number of options on multiple-choice questions: response time, psychometrics and standard setting.

PubMed

Schneid, Stephen D; Armour, Chris; Park, Yoon Soo; Yudkowsky, Rachel; Bordage, Georges

2014-10-01

Despite significant evidence supporting the use of three-option multiple-choice questions (MCQs), these are rarely used in written examinations for health professions students. The purpose of this study was to examine the effects of reducing four- and five-option MCQs to three-option MCQs on response times, psychometric characteristics, and absolute standard setting judgements in a pharmacology examination administered to health professions students. We administered two versions of a computerised examination containing 98 MCQs to 38 Year 2 medical students and 39 Year 3 pharmacy students. Four- and five-option MCQs were converted into three-option MCQs to create two versions of the examination. Differences in response time, item difficulty and discrimination, and reliability were evaluated. Medical and pharmacy faculty judges provided three-level Angoff (TLA) ratings for all MCQs for both versions of the examination to allow the assessment of differences in cut scores. Students answered three-option MCQs an average of 5 seconds faster than they answered four- and five-option MCQs (36 seconds versus 41 seconds; p = 0.008). There were no significant differences in item difficulty and discrimination, or test reliability. Overall, the cut scores generated for three-option MCQs using the TLA ratings were 8 percentage points higher (p = 0.04). The use of three-option MCQs in a health professions examination resulted in a time saving equivalent to the completion of 16% more MCQs per 1-hour testing period, which may increase content validity and test score reliability, and minimise construct under-representation. The higher cut scores may result in higher failure rates if an absolute standard setting method, such as the TLA method, is used. The results from this study provide a cautious indication to health professions educators that using three-option MCQs does not threaten validity and may strengthen it by allowing additional MCQs to be tested in a fixed amount

Some controversial multiple testing problems in regulatory applications.

PubMed

Hung, H M James; Wang, Sue-Jane

2009-01-01

Multiple testing problems in regulatory applications are often more challenging than the problems of handling a set of mathematical symbols representing multiple null hypotheses under testing. In the union-intersection setting, it is important to define a family of null hypotheses relevant to the clinical questions at issue. The distinction between primary endpoint and secondary endpoint needs to be considered properly in different clinical applications. Without proper consideration, the widely used sequential gate keeping strategies often impose too many logical restrictions to make sense, particularly to deal with the problem of testing multiple doses and multiple endpoints, the problem of testing a composite endpoint and its component endpoints, and the problem of testing superiority and noninferiority in the presence of multiple endpoints. Partitioning the null hypotheses involved in closed testing into clinical relevant orderings or sets can be a viable alternative to resolving the illogical problems requiring more attention from clinical trialists in defining the clinical hypotheses or clinical question(s) at the design stage. In the intersection-union setting there is little room for alleviating the stringency of the requirement that each endpoint must meet the same intended alpha level, unless the parameter space under the null hypothesis can be substantially restricted. Such restriction often requires insurmountable justification and usually cannot be supported by the internal data. Thus, a possible remedial approach to alleviate the possible conservatism as a result of this requirement is a group-sequential design strategy that starts with a conservative sample size planning and then utilizes an alpha spending function to possibly reach the conclusion early.

More than the Verbal Stimulus Matters: Visual Attention in Language Assessment for People with Aphasia Using Multiple-Choice Image Displays

ERIC Educational Resources Information Center

Heuer, Sabine; Ivanova, Maria V.; Hallowell, Brooke

2017-01-01

Purpose: Language comprehension in people with aphasia (PWA) is frequently evaluated using multiple-choice displays: PWA are asked to choose the image that best corresponds to the verbal stimulus in a display. When a nontarget image is selected, comprehension failure is assumed. However, stimulus-driven factors unrelated to linguistic…

A Multiple-Choice Mushroom: Schools, Colleges Rely More than Ever on Standardized Tests.

ERIC Educational Resources Information Center

Hawkins, B. Denise

1995-01-01

This discussion of college entrance examinations reviews differences between the Scholastic Assessment Test (SAT) and the American College Test. It then focuses on the SAT, discussing numbers of students taking the tests, changes in test construction to recognize contributions of women and minorities, involvement of African Americans in…

Testing for Nonuniform Differential Item Functioning with Multiple Indicator Multiple Cause Models

ERIC Educational Resources Information Center

Woods, Carol M.; Grimm, Kevin J.

2011-01-01

In extant literature, multiple indicator multiple cause (MIMIC) models have been presented for identifying items that display uniform differential item functioning (DIF) only, not nonuniform DIF. This article addresses, for apparently the first time, the use of MIMIC models for testing both uniform and nonuniform DIF with categorical indicators. A…

Identification of technical item flaws leads to improvement of the quality of single best Multiple Choice Questions.

PubMed

Fayyaz Khan, Humaira; Farooq Danish, Khalid; Saeed Awan, Azra; Anwar, Masood

2013-05-01

The purpose of the study was to identify technical item flaws in the multiple choice questions submitted for the final exams for the years 2009, 2010 and 2011. This descriptive analytical study was carried out in Islamic International Medical College (IIMC). The Data was collected from the MCQ's submitted by the faculty for the final exams for the year 2009, 2010 and 2011. The data was compiled and evaluated by a three member assessment committee. The data was analyzed for frequency and percentages the categorical data was analyzed by chi-square test. Overall percentage of flawed item was 67% for the year 2009 of which 21% were for testwiseness and 40% were for irrelevant difficulty. In year 2010 the total item flaws were 36% and 11% testwiseness and 22% were for irrelevant difficulty. The year 2011 data showed decreased overall flaws of 21%. The flaws of testwisness were 7%, irrelevant difficulty were 11%. Technical item flaws are frequently encountered during MCQ construction, and the identification of flaws leads to improved quality of the single best MCQ's.

Using the Multiple-Choice Procedure to Measure the Relative Reinforcing Efficacy of Gambling: Initial Validity Evidence Among College Students.

PubMed

Butler, Leon H; Irons, Jessica G; Bassett, Drew T; Correia, Christopher J

2018-06-01

The multiple choice procedure (MCP) is used to assess the relative reinforcing value of concurrently available stimuli. The MCP was originally developed to assess the reinforcing value of drugs; the current within-subjects study employed the MCP to assess the reinforcing value of gambling behavior. Participants (N = 323) completed six versions of the MCP that presented hypothetical choices between money to be used while gambling ($10 or $25) versus escalating amounts of guaranteed money available immediately or after delays of either 1 week or 1 month. Results suggest that choices on the MCP are correlated with other measures of gambling behavior, thus providing concurrent validity data for using the MCP to quantify the relative reinforcing value of gambling. The MCP for gambling also displayed sensitivity to reinforcer magnitude and delay effects, which provides evidence of criterion validity. The results are consistent with a behavioral economic model of addiction and suggest that the MCP could be a valid tool for future research on gambling behavior.

Unscaled Bayes factors for multiple hypothesis testing in microarray experiments.

PubMed

Bertolino, Francesco; Cabras, Stefano; Castellanos, Maria Eugenia; Racugno, Walter

2015-12-01

Multiple hypothesis testing collects a series of techniques usually based on p-values as a summary of the available evidence from many statistical tests. In hypothesis testing, under a Bayesian perspective, the evidence for a specified hypothesis against an alternative, conditionally on data, is given by the Bayes factor. In this study, we approach multiple hypothesis testing based on both Bayes factors and p-values, regarding multiple hypothesis testing as a multiple model selection problem. To obtain the Bayes factors we assume default priors that are typically improper. In this case, the Bayes factor is usually undetermined due to the ratio of prior pseudo-constants. We show that ignoring prior pseudo-constants leads to unscaled Bayes factor which do not invalidate the inferential procedure in multiple hypothesis testing, because they are used within a comparative scheme. In fact, using partial information from the p-values, we are able to approximate the sampling null distribution of the unscaled Bayes factor and use it within Efron's multiple testing procedure. The simulation study suggests that under normal sampling model and even with small sample sizes, our approach provides false positive and false negative proportions that are less than other common multiple hypothesis testing approaches based only on p-values. The proposed procedure is illustrated in two simulation studies, and the advantages of its use are showed in the analysis of two microarray experiments. © The Author(s) 2011.

Instructor perspectives of multiple-choice questions in summative assessment for novice programmers

NASA Astrophysics Data System (ADS)

Shuhidan, Shuhaida; Hamilton, Margaret; D'Souza, Daryl

2010-09-01

Learning to program is known to be difficult for novices. High attrition and high failure rates in foundation-level programming courses undertaken at tertiary level in Computer Science programs, are commonly reported. A common approach to evaluating novice programming ability is through a combination of formative and summative assessments, with the latter typically represented by a final examination. Preparation of such assessment is driven by instructor perceptions of student learning of programming concepts. This in turn may yield instructor perspectives of summative assessment that do not necessarily correlate with student expectations or abilities. In this article, we present results of our study around instructor perspectives of summative assessment for novice programmers. Both quantitative and qualitative data have been obtained via survey responses from programming instructors with varying teaching experience, and from novice student responses to targeted examination questions. Our findings highlight that most of the instructors believed that summative assessment is, and is meant to be, a valid measure of a student's ability to program. Most instructors further believed that Multiple-choice Questions (MCQs) provide a means of testing a low level of understanding, and a few added qualitative comments to suggest that MCQs are easy questions, and others refused to use them at all. There was no agreement around the proposition that if a question was designed to test a low level of skill, or a low level in a hierarchy of a body of knowledge, that such a question should or would be found to be easy by the student. To aid our analysis of assessment questions, we introduced four measures: Syntax Knowledge; Semantic Knowledge; Problem Solving Skill and the Level of Difficulty of the Problem. We applied these measures to selected examination questions, and have identified gaps between the instructor perspectives of what is considered to be an easy question and also in

Semantic Similarity Measures for the Generation of Science Tests in Basque

ERIC Educational Resources Information Center

Aldabe, Itziar; Maritxalar, Montse

2014-01-01

The work we present in this paper aims to help teachers create multiple-choice science tests. We focus on a scientific vocabulary-learning scenario taking place in a Basque-language educational environment. In this particular scenario, we explore the option of automatically generating Multiple-Choice Questions (MCQ) by means of Natural Language…

Choice of Reading Comprehension Test Influences the Outcomes of Genetic Analyses

PubMed Central

Betjemann, Rebecca S.; Keenan, Janice M.; Olson, Richard K.; DeFries, John C.

2010-01-01

Does the choice of test for assessing reading comprehension influence the outcome of genetic analyses? A twin design compared two types of reading comprehension tests classified as primarily associated with word decoding (RC-D) or listening comprehension (RC-LC). For both types of tests, the overall genetic influence is high and nearly identical. However, the tests differed significantly in how they covary with the genes associated with decoding and listening comprehension. Although Cholesky decomposition showed that both types of comprehension tests shared significant genetic influence with both decoding and listening comprehension, RC-D tests shared most genetic variance with decoding, and RC-LC tests shared most with listening comprehension. Thus, different tests used to measure the same construct may manifest very different patterns of genetic covariation. These results suggest that the apparent discrepancies among the findings of previous twin studies of reading comprehension could be due at least in part to test differences. PMID:21804757

Further definition on the multiple partner choice arena: a potential animal model for the study of premature ejaculation.

PubMed

Olayo-Lortia, Jesús; Ferreira-Nuño, Armando; Velázquez-Moctezuma, Javier; Morales-Otal, Adriana

2014-10-01

The multiple partner choice arena (MPCA) is an experimental setup in which male rats display a significant shortening of ejaculation latency, which is the main characteristic of premature ejaculation (PE) in men. Thus, the MPCA is a potential animal model for PE. In this study, we further analyze whether the features of the MPCA satisfy the validity criteria for it to be considered an animal model as well as the possible participation of the serotoninergic system in the faster ejaculation exhibited by male rats in the MPCA. In Experiment 1, male rats were tested in a standard arena to assess their sexual behavior, then were assessed 1 week later in the MPCA. Another group was first tested in the MPCA, then in a standard arena. In Experiment 2, male rats divided into two groups were treated daily with WAY-100635 (5-HT(1A) antagonist) or vehicle for 15 days. In each group, half of the subjects were tested in a standard arena and half were tested in the MPCA on days 1, 8, and 15 of treatment. Number of intromissions and intromission and ejaculation latencies were the main outcome measures. In Experiment 1, males tested in the MPCA ejaculated significantly faster, regardless of the order in which they were evaluated in both arenas. In Experiment 2, the administration of WAY-100635 increased intromission and ejaculation latencies, and the number of intromissions in the MPCA. The results obtained in the MPCA support its use as an animal model for PE evaluation. © 2014 International Society for Sexual Medicine.

The CHOICE study: a "taste-test" of utilitarian vs. leisure walking among older adults.

PubMed

Hekler, Eric B; Castro, Cynthia M; Buman, Matthew P; King, Abby C

2012-01-01

Utilitarian walking (e.g., walking for transport) and leisure walking (e.g., walking for health/recreation) are encouraged to promote health, yet few studies have explored specific preferences for these two forms of physical activity or factors that impact such preferences. A quasi-experimental crossover design was used to evaluate how training underactive midlife and older adults in each type of walking impacted total steps taken and how it was linked to their subsequent choice of walking types. Participants (N = 16) were midlife and older adults (M age = 64 ± 8 years) who were mostly women (81%) and white (75%). To control for order effects, participants were randomized to instruction in either utilitarian or leisure walking for 2 weeks and then the other type for 2 weeks. Participants then entered a 2-week "free choice" phase in which they chose any mixture of the walking types. Outcome variables included walking via OMRON pedometer and the ratio of utilitarian versus leisure walking during the free-choice phase. Participants completed surveys about their neighborhood (NEWS) and daily travel to multiple locations. Instruction in leisure-only, utilitarian-only, and a freely chosen mixture of the two each resulted in significant increases in steps taken relative to baseline (ps < 0.05). Having to go to multiple locations daily and traveling greater distances to locations were associated with engagement in more utilitarian walking. In contrast, good walking paths, neighborhood aesthetics, easy access to exercise facilities, and perceiving easier access to neighborhood services were associated with more leisure walking. Results from this pilot study suggest that midlife and older adults may most easily meet guidelines through either leisure only or a mixture of leisure and utilitarian walking, and tailored suggestions based on the person's neighborhood may be useful.

Constructing food choice decisions.

PubMed

Sobal, Jeffery; Bisogni, Carole A

2009-12-01

Food choice decisions are frequent, multifaceted, situational, dynamic, and complex and lead to food behaviors where people acquire, prepare, serve, give away, store, eat, and clean up. Many disciplines and fields examine decision making. Several classes of theories are applicable to food decision making, including social behavior, social facts, and social definition perspectives. Each offers some insights but also makes limiting assumptions that prevent fully explaining food choice decisions. We used constructionist social definition perspectives to inductively develop a food choice process model that organizes a broad scope of factors and dynamics involved in food behaviors. This food choice process model includes (1) life course events and experiences that establish a food choice trajectory through transitions, turning points, timing, and contexts; (2) influences on food choices that include cultural ideals, personal factors, resources, social factors, and present contexts; and (3) a personal system that develops food choice values, negotiates and balances values, classifies foods and situations, and forms/revises food choice strategies, scripts, and routines. The parts of the model dynamically interact to make food choice decisions leading to food behaviors. No single theory can fully explain decision making in food behavior. Multiple perspectives are needed, including constructionist thinking.

Science Library of Test Items. Volume Three. Mastery Testing Programme. Introduction and Manual.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

A set of short tests aimed at measuring student mastery of specific skills in the natural sciences are presented with a description of the mastery program's purposes, development, and methods. Mastery learning, criterion-referenced testing, and the scope of skills to be tested are defined. Each of the multiple choice tests for grades 7 through 10…

Effects of Repeated Testing on Short- and Long-Term Memory Performance across Different Test Formats

ERIC Educational Resources Information Center

Stenlund, Tova; Sundström, Anna; Jonsson, Bert

2016-01-01

This study examined whether practice testing with short-answer (SA) items benefits learning over time compared to practice testing with multiple-choice (MC) items, and rereading the material. More specifically, the aim was to test the hypotheses of "retrieval effort" and "transfer appropriate processing" by comparing retention…

Bees Algorithm for Construction of Multiple Test Forms in E-Testing

ERIC Educational Resources Information Center

Songmuang, Pokpong; Ueno, Maomi

2011-01-01

The purpose of this research is to automatically construct multiple equivalent test forms that have equivalent qualities indicated by test information functions based on item response theory. There has been a trade-off in previous studies between the computational costs and the equivalent qualities of test forms. To alleviate this problem, we…

Constructing objective tests

NASA Astrophysics Data System (ADS)

Aubrecht, Gordon J.; Aubrecht, Judith D.

1983-07-01

True-false or multiple-choice tests can be useful instruments for evaluating student progress. We examine strategies for planning objective tests which serve to test the material covered in science (physics) courses. We also examine strategies for writing questions for tests within a test blueprint. The statistical basis for judging the quality of test items are discussed. Reliability, difficulty, and discrimination indices are defined and examples presented. Our recommendation are rather easily put into practice.

Incorporating Multiple-Choice Questions into an AACSB Assurance of Learning Process: A Course-Embedded Assessment Application to an Introductory Finance Course

ERIC Educational Resources Information Center

Santos, Michael R.; Hu, Aidong; Jordan, Douglas

2014-01-01

The authors offer a classification technique to make a quantitative skills rubric more operational, with the groupings of multiple-choice questions to match the student learning levels in knowledge, calculation, quantitative reasoning, and analysis. The authors applied this classification technique to the mid-term exams of an introductory finance…

Hierarchical group testing for multiple infections.

PubMed

Hou, Peijie; Tebbs, Joshua M; Bilder, Christopher R; McMahan, Christopher S

2017-06-01

Group testing, where individuals are tested initially in pools, is widely used to screen a large number of individuals for rare diseases. Triggered by the recent development of assays that detect multiple infections at once, screening programs now involve testing individuals in pools for multiple infections simultaneously. Tebbs, McMahan, and Bilder (2013, Biometrics) recently evaluated the performance of a two-stage hierarchical algorithm used to screen for chlamydia and gonorrhea as part of the Infertility Prevention Project in the United States. In this article, we generalize this work to accommodate a larger number of stages. To derive the operating characteristics of higher-stage hierarchical algorithms with more than one infection, we view the pool decoding process as a time-inhomogeneous, finite-state Markov chain. Taking this conceptualization enables us to derive closed-form expressions for the expected number of tests and classification accuracy rates in terms of transition probability matrices. When applied to chlamydia and gonorrhea testing data from four states (Region X of the United States Department of Health and Human Services), higher-stage hierarchical algorithms provide, on average, an estimated 11% reduction in the number of tests when compared to two-stage algorithms. For applications with rarer infections, we show theoretically that this percentage reduction can be much larger. © 2016, The International Biometric Society.

Hierarchical group testing for multiple infections

PubMed Central

Hou, Peijie; Tebbs, Joshua M.; Bilder, Christopher R.; McMahan, Christopher S.

2016-01-01

Summary Group testing, where individuals are tested initially in pools, is widely used to screen a large number of individuals for rare diseases. Triggered by the recent development of assays that detect multiple infections at once, screening programs now involve testing individuals in pools for multiple infections simultaneously. Tebbs, McMahan, and Bilder (2013, Biometrics) recently evaluated the performance of a two-stage hierarchical algorithm used to screen for chlamydia and gonorrhea as part of the Infertility Prevention Project in the United States. In this article, we generalize this work to accommodate a larger number of stages. To derive the operating characteristics of higher-stage hierarchical algorithms with more than one infection, we view the pool decoding process as a time-inhomogeneous, finite-state Markov chain. Taking this conceptualization enables us to derive closed-form expressions for the expected number of tests and classification accuracy rates in terms of transition probability matrices. When applied to chlamydia and gonorrhea testing data from four states (Region X of the United States Department of Health and Human Services), higher-stage hierarchical algorithms provide, on average, an estimated 11 percent reduction in the number of tests when compared to two-stage algorithms. For applications with rarer infections, we show theoretically that this percentage reduction can be much larger. PMID:27657666

Adhesive Defect Monitoring of Glass Fiber Epoxy Plate Using an Impedance-Based Non-Destructive Testing Method for Multiple Structures

PubMed Central

Na, Wongi S.; Baek, Jongdae

2017-01-01

The emergence of composite materials has revolutionized the approach to building engineering structures. With the number of applications for composites increasing every day, maintaining structural integrity is of utmost importance. For composites, adhesive bonding is usually the preferred choice over the mechanical fastening method, and monitoring for delamination is an essential factor in the field of composite materials. In this study, a non-destructive method known as the electromechanical impedance method is used with an approach of monitoring multiple areas by specifying certain frequency ranges to correspond to a certain test specimen. Experiments are conducted using various numbers of stacks created by attaching glass fiber epoxy composite plates onto one another, and two different debonding damage types are introduced to evaluate the performance of the multiple monitoring electromechanical impedance method. PMID:28629194

Patient autonomy and choice in healthcare: self-testing devices as a case in point.

PubMed

Greaney, Anna-Marie; O'Mathúna, Dónal P; Scott, P Anne

2012-11-01

This paper aims to critique the phenomenon of advanced patient autonomy and choice in healthcare within the specific context of self-testing devices. A growing number of self-testing medical devices are currently available for home use. The premise underpinning many of these devices is that they assist individuals to be more autonomous in the assessment and management of their health. Increased patient autonomy is assumed to be a good thing. We take issue with this assumption and argue that self-testing provides a specific example how increased patient autonomy and choice within healthcare might not best serve the patient population. We propose that current interpretations of autonomy in healthcare are based on negative accounts of liberty to the detriment of a more relational understanding. We also propose that Kantian philosophy is often applied to the healthcare arena in an inappropriate manner. We draw on the philosophical literature and examples from the self-testing process to support these claims. We conclude by offering an alternative account of autonomy based on the interrelated concepts of relationality, care and responsibility.

Handbook for Driving Knowledge Testing.

ERIC Educational Resources Information Center

Pollock, William T.; McDole, Thomas L.

Materials intended for driving knowledge test development for use by operational licensing and education agencies are presented. A pool of 1,313 multiple choice test items is included, consisting of sets of specially developed and tested items covering principles of safe driving, legal regulations, and traffic control device knowledge pertinent to…

Are Learning Disabled Students "Test-Wise?": An Inquiry into Reading Comprehension Test Items.

ERIC Educational Resources Information Center

Scruggs, Thomas E.; Lifson, Steve

The ability to correctly answer reading comprehension test items, without having read the accompanying reading passage, was compared for third grade learning disabled students and their peers from a regular classroom. In the first experiment, fourteen multiple choice items were selected from the Stanford Achievement Test. No reading passages were…

Choice-Based Conjoint Analysis: Classification vs. Discrete Choice Models

NASA Astrophysics Data System (ADS)

Giesen, Joachim; Mueller, Klaus; Taneva, Bilyana; Zolliker, Peter

Conjoint analysis is a family of techniques that originated in psychology and later became popular in market research. The main objective of conjoint analysis is to measure an individual's or a population's preferences on a class of options that can be described by parameters and their levels. We consider preference data obtained in choice-based conjoint analysis studies, where one observes test persons' choices on small subsets of the options. There are many ways to analyze choice-based conjoint analysis data. Here we discuss the intuition behind a classification based approach, and compare this approach to one based on statistical assumptions (discrete choice models) and to a regression approach. Our comparison on real and synthetic data indicates that the classification approach outperforms the discrete choice models.

Test Pool Questions, Area III.

ERIC Educational Resources Information Center

Sloan, Jamee Reid

This manual contains multiple choice questions to be used in testing students on nurse training objectives. Each test includes several questions covering each concept. The concepts in section A, medical surgical nursing, are diseases of the following systems: musculoskeletal; central nervous; cardiovascular; gastrointestinal; urinary and male…

Development and Application of a Two-Tier Multiple Choice Diagnostic Instrument To Assess High School Students' Understanding of Inorganic Chemistry Qualitative Analysis.

ERIC Educational Resources Information Center

Tan, Kim Chwee Daniel; Goh, Ngoh Khang; Chia, Lian Sai; Treagust, David F.

2002-01-01

Describes the development and application of a two-tier multiple choice diagnostic instrument to assess high school students' understanding of inorganic chemistry qualitative analysis. Shows that the Grade 10 students had difficulty understanding the reactions involved in the identification of cations and anions, for example, double decomposition…

Testing the Multiple in the Multiple Read-Out Model of Visual Word Recognition

ERIC Educational Resources Information Center

De Moor, Wendy; Verguts, Tom; Brysbaert, Marc

2005-01-01

This study provided a test of the multiple criteria concept used for lexical decision, as implemented in J. Grainger and A. M. Jacobs's (1996) multiple read-out model. This account predicts more inhibition (or less facilitation) from a masked neighbor when accuracy is stressed more but more facilitation (or less inhibition) when the speed of…

Testing Multiple Outcomes in Repeated Measures Designs

ERIC Educational Resources Information Center

Lix, Lisa M.; Sajobi, Tolulope

2010-01-01

This study investigates procedures for controlling the familywise error rate (FWR) when testing hypotheses about multiple, correlated outcome variables in repeated measures (RM) designs. A content analysis of RM research articles published in 4 psychology journals revealed that 3 quarters of studies tested hypotheses about 2 or more outcome…

Improvement of individual camouflage through background choice in ground-nesting birds.

PubMed

Stevens, Martin; Troscianko, Jolyon; Wilson-Aggarwal, Jared K; Spottiswoode, Claire N

2017-09-01

Animal camouflage is a longstanding example of adaptation. Much research has tested how camouflage prevents detection and recognition, largely focusing on changes to an animal's own appearance over evolution. However, animals could also substantially alter their camouflage by behaviourally choosing appropriate substrates. Recent studies suggest that individuals from several animal taxa could select backgrounds or positions to improve concealment. Here, we test whether individual wild animals choose backgrounds in complex environments, and whether this improves camouflage against predator vision. We studied nest site selection by nine species of ground-nesting birds (nightjars, plovers and coursers) in Zambia, and used image analysis and vision modeling to quantify egg and plumage camouflage to predator vision. Individual birds chose backgrounds that enhanced their camouflage, being better matched to their chosen backgrounds than to other potential backgrounds with respect to multiple aspects of camouflage. This occurred at all three spatial scales tested (a few cm and five meters from the nest, and compared to other sites chosen by conspecifics), and was the case for the eggs of all bird groups studied, and for adult nightjar plumage. Thus, individual wild animals improve their camouflage through active background choice, with choices highly refined across multiple spatial scales.

Improvement of individual camouflage through background choice in ground-nesting birds

PubMed Central

Stevens, Martin; Troscianko, Jolyon; Wilson-Aggarwal, Jared K.; Spottiswoode, Claire N.

2017-01-01

Animal camouflage is a longstanding example of adaptation. Much research has tested how camouflage prevents detection and recognition, largely focusing on changes to an animal's own appearance over evolution. However, animals could also substantially alter their camouflage by behaviourally choosing appropriate substrates. Recent studies suggest that individuals from several animal taxa could select backgrounds or positions to improve concealment. Here, we test whether individual wild animals choose backgrounds in complex environments, and whether this improves camouflage against predator vision. We studied nest site selection by nine species of ground-nesting birds (nightjars, plovers and coursers) in Zambia, and used image analysis and vision modeling to quantify egg and plumage camouflage to predator vision. Individual birds chose backgrounds that enhanced their camouflage, being better matched to their chosen backgrounds than to other potential backgrounds with respect to multiple aspects of camouflage. This occurred at all three spatial scales tested (a few cm and five meters from the nest, and compared to other sites chosen by conspecifics), and was the case for the eggs of all bird groups studied, and for adult nightjar plumage. Thus, individual wild animals improve their camouflage through active background choice, with choices highly refined across multiple spatial scales. PMID:28890937

Increasing Choice Making in Students with Intellectual Disability

ERIC Educational Resources Information Center

Sparks, Shannon Lynn; Pierce, Tom; Higgins, Kyle; Miller, Susan; Tandy, Richard

2016-01-01

The purpose of this study was to examine the effectiveness of choice-making training with six high school students with intellectual disability. A multiple probe design with one replication was used to evaluate the efficacy of the choice-making training. The results suggest participants increased and maintained their choice-making abilities.…

Development and psychometric testing of an instrument to compare career choice influences and perceptions of nursing among healthcare students.

PubMed

Liaw, Sok Ying; Wu, Ling Ting; Lopez, Violeta; Chow, Yeow Leng; Lim, Siriwan; Holroyd, Eleanor; Tan, Khoon Kiat; Wang, Wenru

2017-04-27

With the availability of more healthcare courses and an increased intake of nursing students, education institutions are facing challenges to attract school leavers to enter nursing courses. The comparison of career choice influences and perception of nursing among healthcare students can provide information for recruitment strategies. An instrument to compare the influences of healthcare career choice is lacking. The purpose of this study is to develop and evaluate the psychometric properties of an instrument to compare the influences of healthcare career choice with perceptions of nursing as a career choice. The study was conducted in two phases. In phase one, two sets of scales with parallel items that measure the influences of healthcare career choice and perceptions of nursing as a career choice were developed through an earlier qualitative study, literature review, and expert validation. Phase two involved testing the construct validity, concurrent validity and reliability with a convenience sample of 283 first year healthcare students who were recruited at two education institutions in Singapore. An exploratory factor analysis revealed 35-parallel items in a six-factor solution (personal interest, prior healthcare exposure, self-efficacy, perceived nature of work, job prospects, and social influences) that explained 59 and 64% of the variance for healthcare career choice and nursing as a career choice respectively. A high correlation (r = 0.76, p < 0.001) was obtained with an existing tool, confirming the concurrent validity. The internal consistency was sufficient with Cronbach's alpha of 0.93 for healthcare career choice and 0.94 for nursing as a career choice. The test-retest reliability was acceptable with an Intraclass Correlation Coefficient of 0.63 for healthcare career choice and 0.60 for nursing as a career choice. The instrument provides opportunities for understanding the differences between influences of healthcare career choice and

Physics Achievement Test.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

This document is an evaluation instrument developed as a part of Harvard Project Physics (HPP). It consists of a 36-item, multiple choice (five options) Physics Achievement Test (PAT) designed to measure general knowledge of physics as well as the material emphasized in HPP. (PEB)

Grade 9 Pilot Test. Mathematics. June 1988 = 9e Annee Test Pilote. Mathematiques. Juin 1988.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

This pilot test for ninth grade mathematics is written in both French and English. The test consists of 75 multiple-choice items. Students are given 90 minutes to complete the examination and the use of a calculator is highly recommended. The test content covers a wide range of mathematical topics including: decimals; exponents; arithmetic word…

Effect of Multiple Testing Adjustment in Differential Item Functioning Detection

ERIC Educational Resources Information Center

Kim, Jihye; Oshima, T. C.

2013-01-01

In a typical differential item functioning (DIF) analysis, a significance test is conducted for each item. As a test consists of multiple items, such multiple testing may increase the possibility of making a Type I error at least once. The goal of this study was to investigate how to control a Type I error rate and power using adjustment…

I can't wait: Methods for measuring and moderating individual differences in impulsive choice.

PubMed

Peterson, Jennifer R; Hill, Catherine C; Marshall, Andrew T; Stuebing, Sarah L; Kirkpatrick, Kimberly

2015-01-01

Impulsive choice behavior occurs when individuals make choices without regard for future consequences. This behavior is often maladaptive and is a common symptom in many disorders, including drug abuse, compulsive gambling, and obesity. Several proposed mechanisms may influence impulsive choice behavior. These mechanisms provide a variety of pathways that may provide the basis for individual differences that are often evident when measuring choice behavior. This review provides an overview of these different pathways to impulsive choice, and the behavioral intervention strategies being developed to moderate impulsive choice. Because of the compelling link between impulsive choice behavior and the near-epidemic pervasiveness of obesity in the United States, we focus on the relationship between impulsive choice behavior and obesity as a test case for application of the multiple pathways approach. Choosing immediate gratification over healthier long term food choices is a contributing factor to the obesity crisis. Behavioral interventions can lead to more self controlled choices in a rat pre-clinical model, suggesting a possible gateway for translation to human populations. Designing and implementing effective impulsive choice interventions is crucial to improving the overall health and well-being of impulsive individuals.

I can't wait: Methods for measuring and moderating individual differences in impulsive choice

PubMed Central

Peterson, Jennifer R.; Hill, Catherine C.; Marshall, Andrew T.; Stuebing, Sarah L.; Kirkpatrick, Kimberly

2016-01-01

Impulsive choice behavior occurs when individuals make choices without regard for future consequences. This behavior is often maladaptive and is a common symptom in many disorders, including drug abuse, compulsive gambling, and obesity. Several proposed mechanisms may influence impulsive choice behavior. These mechanisms provide a variety of pathways that may provide the basis for individual differences that are often evident when measuring choice behavior. This review provides an overview of these different pathways to impulsive choice, and the behavioral intervention strategies being developed to moderate impulsive choice. Because of the compelling link between impulsive choice behavior and the near-epidemic pervasiveness of obesity in the United States, we focus on the relationship between impulsive choice behavior and obesity as a test case for application of the multiple pathways approach. Choosing immediate gratification over healthier long term food choices is a contributing factor to the obesity crisis. Behavioral interventions can lead to more self controlled choices in a rat pre-clinical model, suggesting a possible gateway for translation to human populations. Designing and implementing effective impulsive choice interventions is crucial to improving the overall health and well-being of impulsive individuals. PMID:27695664

Memory-Context Effects of Screen Color in Multiple-Choice and Fill-In Tests

ERIC Educational Resources Information Center

Prestera, Gustavo E.; Clariana, Roy; Peck, Andrew

2005-01-01

In this experimental study, 44 undergraduates completed five computer-based instructional lessons and either two multiplechoice tests or two fill-in-the-blank tests. Color-coded borders were displayed during the lesson, adjacent to the screen text and illustrations. In the experimental condition, corresponding border colors were shown at posttest.…

Using Tests as Learning Opportunities.

ERIC Educational Resources Information Center

Foos, Paul W.; Fisher, Ronald P.

1988-01-01

A study involving 105 undergraduates assessed the value of testing as a means of increasing, rather than simply monitoring, learning. Results indicate that fill-in-the-blank and items requiring student inferences were more effective, respectively, than multiple-choice tests and verbatim items in furthering student learning. (TJH)

Milwaukee Longitudinal School Choice Evaluation: Annual School Testing Summary Report 2009-10. SCDP Milwaukee Evaluation. Report # 26

ERIC Educational Resources Information Center

McShane, Michael Q.; Wolf, Patrick J.

2011-01-01

The purpose of this report is to provide descriptive data regarding the test scores of Milwaukee Parental Choice Program (MPCP) students in grades 4, 8 and 10 in reading, math and science, as reported to the School Choice Demonstration Project 2009-2010. The tables, graphs, and histograms presented in this report provide a snapshot of these…

Milwaukee Longitudinal School Choice Evaluation: Annual School Testing Summary Report 2008-09. SCDP Milwaukee Evaluation Report #18

ERIC Educational Resources Information Center

Dean, Jeffery R.; Wolf, Patrick J.

2010-01-01

The purpose of this report is to provide descriptive data regarding the test scores of Milwaukee Parental Choice Program (MPCP) students in grades 4, 8 and 10 in reading, math, and science, as reported to the School Choice Demonstration Project 2008-2009. The tables, graphs, and histograms presented in this paper provide a snapshot of these…

Milwaukee Longitudinal School Choice Evaluation: Annual School Testing Summary Report 2007-08. SCDP Milwaukee Evaluation Report #9

ERIC Educational Resources Information Center

Gray, Nathan L.; Wolf, Patrick J.; Jensen, Laura I.

2009-01-01

The purpose of this report is to provide descriptive data regarding the test scores of Milwaukee Parental Choice Program (MPCP) students in grades 4, 8 and 10 in reading, math and science, as reported to the School Choice Demonstration Project 2007-2008. The tables, graphs, and histograms presented in this report provide a snapshot of these…

Milwaukee Longitudinal School Choice Evaluation: Annual School Testing Summary Report 2010-11. SCDP Milwaukee Evaluation Report #32

ERIC Educational Resources Information Center

Jacob, Anna M.; Wolf, Patrick J.

2012-01-01

The Milwaukee Parental Choice Program (MPCP) is the nation's oldest and largest urban school choice program, offering private-school scholarships to low-income students in the city of Milwaukee since 1990. In the early years of the program, voucher schools were not required to test their students, though many of them did so using nationally normed…

Analysis of strength-of-preference measures in dichotomous choice models

Treesearch

Donald F. Dennis; Peter Newman; Robert Manning

2008-01-01

Choice models are becoming increasingly useful for soliciting and analyzing multiple objective decisions faced by recreation managers and others interested in decisions involving natural resources. Choice models are used to estimate relative values for multiple aspects of natural resource management, not individually but within the context of other relevant decision...

The decision-making capacity of elderly hospitalized patients: validation of a test on their choice of return home.

PubMed

Romdhani, Mouna; Abbas, Rachid; Peyneau, Cécile; Koskas, Pierre; Houenou Quenum, Nadège; Galleron, Sandrine; Drunat, Olivier

2018-03-01

Elderly hospitalized patients have uncertain or questionable capacity to make decisions about their care. Determining whether an elderly patient possesses decision-making capacity to return at home is a major concern for geriatricians in everyday practice. To construct and internally validate a new tool, the dream of home test (DROM-test), as support for decision making hospitalization discharge destination for the elderly in the acute or sub-acute care setting. The DROM-test consists of 10 questions and 4 vignettes based upon the 4 relevant criteria for decision-making: capacity to understand information, to appreciate and reason about medical risks and to communicate a choice. A prospective observational study was conducted during 6 months in 2 geriatric care units in Bretonneau Hospital (Assistance publique, Hôpitaux de Paris). We compared the patient decision of DROM-test regarding discharge recommendations with those of an Expert committee and of the team in charge of the patient. 102 were included: mean age 83.1 + 6.7 [70; 97], 66.67% females. Principal components analysis revealed four dimensions: choice, understanding, reasoning and understanding. The area under the ROC curve was 0.64 for the choice dimension, 0.59 for the understanding, 0.53 for the reasoning and 0.52 for the apprehension. Only the choice dimension was statistically associated with the decision of the committee of experts (p=0.017). Even though Drom-test has limitations, it provides an objective way to ascertain decision-making capacity for hospitalised elderly patients.

Preference of Students on the Format of Options in a Multiple-Choice Test

ERIC Educational Resources Information Center

Oyzon, Voltaire Q.; Bendulo, Hermabeth O.; Tibus, Erlinda D.; Bande, Rhodora A.; Macalinao, Myrna L.

2016-01-01

Schools in the Philippines, especially those that are offering teacher education programs, are advised to construct examinations that are Licensure Examination for Teachers (LET)-like test items. This is because "if any aspect of a test is unfamiliar to candidates, they are likely to perform less well than they would do otherwise on…

Patient choice in opt-in, active choice, and opt-out HIV screening: randomized clinical trial.

PubMed

Montoy, Juan Carlos C; Dow, William H; Kaplan, Beth C

2016-01-19

What is the effect of default test offers--opt-in, opt-out, and active choice--on the likelihood of acceptance of an HIV test among patients receiving care in an emergency department? This was a randomized clinical trial conducted in the emergency department of an urban teaching hospital and regional trauma center. Patients aged 13-64 years were randomized to opt-in, opt-out, and active choice HIV test offers. The primary outcome was HIV test acceptance percentage. The Denver Risk Score was used to categorize patients as being at low, intermediate, or high risk of HIV infection. 38.0% (611/1607) of patients in the opt-in testing group accepted an HIV test, compared with 51.3% (815/1628) in the active choice arm (difference 13.3%, 95% confidence interval 9.8% to 16.7%) and 65.9% (1031/1565) in the opt-out arm (difference 27.9%, 24.4% to 31.3%). Compared with active choice testing, opt-out testing led to a 14.6 (11.1 to 18.1) percentage point increase in test acceptance. Patients identified as being at intermediate and high risk were more likely to accept testing than were those at low risk in all arms (difference 6.4% (3.4% to 9.3%) for intermediate and 8.3% (3.3% to 13.4%) for high risk). The opt-out effect was significantly smaller among those reporting high risk behaviors, but the active choice effect did not significantly vary by level of reported risk behavior. Patients consented to inclusion in the study after being offered an HIV test, and inclusion varied slightly by treatment assignment. The study took place at a single county hospital in a city that is somewhat unique with respect to HIV testing; although the test acceptance percentages themselves might vary, a different pattern for opt-in versus active choice versus opt-out test schemes would not be expected. Active choice is a distinct test regimen, with test acceptance patterns that may best approximate patients' true preferences. Opt-out regimens can substantially increase HIV testing, and opt

Informed Choice in Direct-to-Consumer Genetic Testing for Alzheimer and Other Diseases: Lessons from Two Cases

PubMed Central

Messner, Donna A.

2011-01-01

Health-related direct-to-consumer (DTC) genetic testing has been a controversial practice. Especially problematic is predictive testing for Alzheimer disease (AD), since the disease is incurable, prevention is inconclusive, and testing does not definitively predict an individual’s future disease status. In this paper, I examine two contrasting cases of subjects who learn through genetic testing that they have an elevated risk of developing AD later in life. In these cases, the subject’s emotional response to the result is related to how well prepared she was for the real-life personal implications of possible test results. Analysis leads to the conclusion that when groups of health-related genetic tests are offered as packages by DTC companies, informed consumer choice is rendered impossible. Moreover, I argue, this marketing approach contravenes U.S. Federal Trade Commission policies for non-deceptive commercial communications. I conclude by suggesting ways to improve the prospects for informed consumer choice in DTC testing. PMID:21603253

Analysis of the Difficulty and Discrimination Indices of Multiple-Choice Questions According to Cognitive Levels in an Open and Distance Learning Context

ERIC Educational Resources Information Center

Koçdar, Serpil; Karadag, Nejdet; Sahin, Murat Dogan

2016-01-01

This is a descriptive study which intends to determine whether the difficulty and discrimination indices of the multiple-choice questions show differences according to cognitive levels of the Bloom's Taxonomy, which are used in the exams of the courses in a business administration bachelor's degree program offered through open and distance…

Multiple sclerosis: Left advantage for auditory laterality in dichotic tests of central auditory processing and relationship of psychoacoustic tests with the Multiple Sclerosis Disability Scale-EDSS.

PubMed

Peñaloza López, Yolanda Rebeca; Orozco Peña, Xóchitl Daisy; Pérez Ruiz, Santiago Jesús

2018-04-03

To evaluate the central auditory processing disorders in patients with multiple sclerosis, emphasizing auditory laterality by applying psychoacoustic tests and to identify their relationship with the Multiple Sclerosis Disability Scale (EDSS) functions. Depression scales (HADS), EDSS, and 9 psychoacoustic tests to study CAPD were applied to 26 individuals with multiple sclerosis and 26 controls. Correlation tests were performed between the EDSS and psychoacoustic tests. Seven out of 9 psychoacoustic tests were significantly different (P<.05); right or left (14/19 explorations) with respect to control. In dichotic digits there was a left-ear advantage compared to the usual predominance of RDD. There was significant correlation in five psychoacoustic tests and the specific functions of EDSS. The left-ear advantage detected and interpreted as an expression of deficient influences of the corpus callosum and attention in multiple sclerosis should be investigated. There was a correlation between psychoacoustic tests and specific EDSS functions. Copyright © 2018 Sociedad Española de Otorrinolaringología y Cirugía de Cabeza y Cuello. Publicado por Elsevier España, S.L.U. All rights reserved.

Screening for Sex Chromosome Aneuploidy by Cell-Free DNA Testing: Patient Choice and Performance.

PubMed

Bevilacqua, Elisa; Ordóñez, Elena; Hurtado, Ivan; Rueda, Laura; Mazzone, Eléonora; Cirigliano, Vincenzo; Jani, Jacques C

2017-08-23

To study patient choice regarding testing for sex chromosome aneuploidy (SCA) and the performance of cell-free DNA (cfDNA) screening for SCA. Patient choice regarding screening for SCA and factors influencing this choice were evaluated in a single center. In a subsequent two-center study, cases that screened positive for SCA were analyzed to determine the positive predictive value (PPV) for each SCA. In all, 1,957 (61.9%) of the 3,162 patients undergoing cfDNA testing opted for SCA screening. Regression analysis demonstrated that independent predictors of a patient's decision for SCA were earlier gestational age, spontaneous conception, and cfDNA chosen as a primary method of screening. A total of 161 cases screened positive for SCA and follow-up data were available for 118 (73.3%). Forty-six of the 61 cases of 45,X were false-positive results and 15 were concordant with the fetal karyotype (PPV = 24.6%). Seventeen of the 22 cases of 47,XXX were false positive and 5 concordant (PPV = 22.7%). Eleven of the 30 cases of 47,XXY were false positive and 19 concordant (PPV = 63.3%). All 5 cases of 47,XYY were correctly identified, thus yielding a PPV of 100%. More than half of the patients undergoing cfDNA aneuploidy screening also opted for SCA testing, but they were less likely to do so in the presence of an increased risk of trisomy. SCAs involving the X chromosome had a lower PPV than those involving the Y chromosome. © 2017 S. Karger AG, Basel.

Simple model for multiple-choice collective decision making

NASA Astrophysics Data System (ADS)

Lee, Ching Hua; Lucas, Andrew

2014-11-01

We describe a simple model of heterogeneous, interacting agents making decisions between n ≥2 discrete choices. For a special class of interactions, our model is the mean field description of random field Potts-like models and is effectively solved by finding the extrema of the average energy E per agent. In these cases, by studying the propagation of decision changes via avalanches, we argue that macroscopic dynamics is well captured by a gradient flow along E . We focus on the permutation symmetric case, where all n choices are (on average) the same, and spontaneous symmetry breaking (SSB) arises purely from cooperative social interactions. As examples, we show that bimodal heterogeneity naturally provides a mechanism for the spontaneous formation of hierarchies between decisions and that SSB is a preferred instability to discontinuous phase transitions between two symmetric points. Beyond the mean field limit, exponentially many stable equilibria emerge when we place this model on a graph of finite mean degree. We conclude with speculation on decision making with persistent collective oscillations. Throughout the paper, we emphasize analogies between methods of solution to our model and common intuition from diverse areas of physics, including statistical physics and electromagnetism.

Sure, or unsure? Measuring students' confidence and the potential impact on patient safety in multiple-choice questions.

PubMed

Rangel, Rafael Henrique; Möller, Leona; Sitter, Helmut; Stibane, Tina; Strzelczyk, Adam

2017-11-01

Multiple-choice questions (MCQs) provide useful information about correct and incorrect answers, but they do not offer information about students' confidence. Ninety and another 81 medical students participated each in a curricular neurology multiple-choice exam and indicated their confidence for every single MCQ. Each MCQ had a defined level of potential clinical impact on patient safety (uncritical, risky, harmful). Our first objective was to detect informed (IF), guessed (GU), misinformed (MI), and uninformed (UI) answers. Further, we evaluated whether there were significant differences for confidence at correct and incorrect answers. Then, we explored if clinical impact had a significant influence on students' confidence. There were 1818 IF, 635 GU, 71 MI, and 176 UI answers in exam I and 1453 IF, 613 GU, 92 MI, and 191 UI answers in exam II. Students' confidence was significantly higher for correct than for incorrect answers at both exams (p < 0.001). For exam I, students' confidence was significantly higher for incorrect harmful than for incorrect risky classified MCQs (p = 0.01). At exam II, students' confidence was significantly higher for incorrect harmful than for incorrect benign (p < 0.01) and significantly higher for correct benign than for correct harmful categorized MCQs (p = 0.01). We were pleased to see that there were more informed than guessed, more uninformed than misinformed answers and higher students' confidence for correct than for incorrect answers. Our expectation that students state higher confidence in correct and harmful and lower confidence in incorrect and harmful MCQs could not be confirmed.

Accelerated spike resampling for accurate multiple testing controls.

PubMed

Harrison, Matthew T

2013-02-01

Controlling for multiple hypothesis tests using standard spike resampling techniques often requires prohibitive amounts of computation. Importance sampling techniques can be used to accelerate the computation. The general theory is presented, along with specific examples for testing differences across conditions using permutation tests and for testing pairwise synchrony and precise lagged-correlation between many simultaneously recorded spike trains using interval jitter.

Testing Ecological Theories of Offender Spatial Decision Making Using a Discrete Choice Model

PubMed Central

Summers, Lucia

2015-01-01

Research demonstrates that crime is spatially concentrated. However, most research relies on information about where crimes occur, without reference to where offenders reside. This study examines how the characteristics of neighborhoods and their proximity to offender home locations affect offender spatial decision making. Using a discrete choice model and data for detected incidents of theft from vehicles (TFV), we test predictions from two theoretical perspectives—crime pattern and social disorganization theories. We demonstrate that offenders favor areas that are low in social cohesion and closer to their home, or other age-related activity nodes. For adult offenders, choices also appear to be influenced by how accessible a neighborhood is via the street network. The implications for criminological theory and crime prevention are discussed. PMID:25866412

Testing Ecological Theories of Offender Spatial Decision Making Using a Discrete Choice Model.

PubMed

Johnson, Shane D; Summers, Lucia

2015-04-01

Research demonstrates that crime is spatially concentrated. However, most research relies on information about where crimes occur, without reference to where offenders reside. This study examines how the characteristics of neighborhoods and their proximity to offender home locations affect offender spatial decision making. Using a discrete choice model and data for detected incidents of theft from vehicles (TFV) , we test predictions from two theoretical perspectives-crime pattern and social disorganization theories. We demonstrate that offenders favor areas that are low in social cohesion and closer to their home, or other age-related activity nodes. For adult offenders, choices also appear to be influenced by how accessible a neighborhood is via the street network. The implications for criminological theory and crime prevention are discussed.

More Than Just Accuracy: A Novel Method to Incorporate Multiple Test Attributes in Evaluating Diagnostic Tests Including Point of Care Tests.

PubMed

Thompson, Matthew; Weigl, Bernhard; Fitzpatrick, Annette; Ide, Nicole

2016-01-01

Current frameworks for evaluating diagnostic tests are constrained by a focus on diagnostic accuracy, and assume that all aspects of the testing process and test attributes are discrete and equally important. Determining the balance between the benefits and harms associated with new or existing tests has been overlooked. Yet, this is critically important information for stakeholders involved in developing, testing, and implementing tests. This is particularly important for point of care tests (POCTs) where tradeoffs exist between numerous aspects of the testing process and test attributes. We developed a new model that multiple stakeholders (e.g., clinicians, patients, researchers, test developers, industry, regulators, and health care funders) can use to visualize the multiple attributes of tests, the interactions that occur between these attributes, and their impacts on health outcomes. We use multiple examples to illustrate interactions between test attributes (test availability, test experience, and test results) and outcomes, including several POCTs. The model could be used to prioritize research and development efforts, and inform regulatory submissions for new diagnostics. It could potentially provide a way to incorporate the relative weights that various subgroups or clinical settings might place on different test attributes. Our model provides a novel way that multiple stakeholders can use to visualize test attributes, their interactions, and impacts on individual and population outcomes. We anticipate that this will facilitate more informed decision making around diagnostic tests.

Association of Reference Pricing for Diagnostic Laboratory Testing With Changes in Patient Choices, Prices, and Total Spending for Diagnostic Tests.

PubMed

Robinson, James C; Whaley, Christopher; Brown, Timothy T

2016-09-01

Prices for laboratory and other clinical services vary widely. Employers and insurers increasingly are adopting "reference pricing" policies to create incentives for patients to select lower-priced facilities. To measure the association between implementation of reference pricing and patient choice of laboratory, test prices, patient out-of-pocket spending, and insurer spending. We conducted an observational study of changes in laboratory pricing and selection by employees of a large national grocery firm (n = 30 415) before and after the firm implemented a reference pricing policy for laboratory services and compared the findings with changes over the same period for policy holders of a large national insurer that did not implement reference pricing (n = 181 831). The grocery firm established a maximum payment limit at the 60th percentile of the distribution of prices for each laboratory test in each region. Employees were provided with data on prices at all laboratories through a mobile digital platform. Patients selecting a laboratory that charged more than the payment limit were required to pay the full difference themselves. A total of 2.13 million claims were analyzed for 285 types of in vitro diagnostic tests between 2010 and 2013. Patient choice of laboratory, price paid per test, patient out-of-pocket costs, and employer spending. Compared with trends in prices paid by insurance policy holders not subject to reference pricing, and after adjusting for characteristics of tests and patients, implementation of reference pricing was associated with a 31.9% reduction (95% CI, 20.6%-41.6%) in average price paid per test by the third year of the program. In these 3 years, total spending on laboratory tests declined by $2.57 million (95% CI, $1.59-$3.35 million). Out-of-pocket costs by patients declined by $1.05 million (95% CI, $0.73-$1.37 million). Spending by the employer declined by $1.70 million (95% CI, $0.92-$2.48 million). When combined with

What counts as a choice? U.S. Americans are more likely than Indians to construe actions as choices.

PubMed

Savani, Krishna; Markus, Hazel Rose; Naidu, N V R; Kumar, Satishchandra; Berlia, Neha

2010-03-01

People everywhere select among multiple alternatives, but are they always making choices? In five studies, we found that people in U.S. American contexts, where the disjoint model of agency is prevalent, are more likely than those in Indian contexts to construe their own and other individuals' behaviors as choices, to construe ongoing behaviors and behaviors recalled from memory as choices, to construe naturally occurring and experimentally controlled behaviors as choices, to construe mundane and important actions as choices, and to construe personal and interpersonal actions as choices. Indians showed a greater tendency to construe actions as choices when these actions involved responding to other people than when they did not. These findings show that whether people construe actions as choices is significantly shaped by sociocultural systems of meanings and practices. Together, they suggest that the positive consequences associated with maximizing the availability of personal choice may not be universal and instead may be limited to North American contexts.

General solutions to multiple testing problems. Translation of "Sonnemann, E. (1982). Allgemeine Lösungen multipler Test probleme. EDV in Medizin und Biologie 13(4), 120-128".

PubMed

Sonnemann, Eckart

2008-10-01

The introduction of sequentially rejective multiple test procedures (Einot and Gabriel, 1975; Naik, 1975; Holm, 1977; Holm, 1979) has caused considerable progress in the theory of multiple comparisons. Emphasizing the closure of multiple tests we give a survey of the general theory and its recent results in applications. Some new applications are given including a discussion of the connection with the theory of confidence regions.

Memory-Based Simple Heuristics as Attribute Substitution: Competitive Tests of Binary Choice Inference Models.

PubMed

Honda, Hidehito; Matsuka, Toshihiko; Ueda, Kazuhiro

2017-05-01

Some researchers on binary choice inference have argued that people make inferences based on simple heuristics, such as recognition, fluency, or familiarity. Others have argued that people make inferences based on available knowledge. To examine the boundary between heuristic and knowledge usage, we examine binary choice inference processes in terms of attribute substitution in heuristic use (Kahneman & Frederick, 2005). In this framework, it is predicted that people will rely on heuristic or knowledge-based inference depending on the subjective difficulty of the inference task. We conducted competitive tests of binary choice inference models representing simple heuristics (fluency and familiarity heuristics) and knowledge-based inference models. We found that a simple heuristic model (especially a familiarity heuristic model) explained inference patterns for subjectively difficult inference tasks, and that a knowledge-based inference model explained subjectively easy inference tasks. These results were consistent with the predictions of the attribute substitution framework. Issues on usage of simple heuristics and psychological processes are discussed. Copyright © 2016 Cognitive Science Society, Inc.

Test Performance Vis a Vis Test-Taking Strategies in Reading Comprehension of English as a Second Language.

ERIC Educational Resources Information Center

Roizen, Merle A.

A study of students' self-expressed test-taking strategies for reading comprehension tests in English as a second language (ESL) had as subjects 86 Hebrew-speaking university students in Israel. The students were given a reading comprehension test, half with open-ended questions and half with multiple choice. Half the responses were anonymous and…

Predicting Social and Communicative Ability in School-Age Children with Autism Spectrum Disorder: A Pilot Study of the Social Attribution Task, Multiple Choice

ERIC Educational Resources Information Center

Burger-Caplan, Rebecca; Saulnier, Celine; Jones, Warren; Klin, Ami

2016-01-01

The Social Attribution Task, Multiple Choice is introduced as a measure of implicit social cognitive ability in children, addressing a key challenge in quantification of social cognitive function in autism spectrum disorder, whereby individuals can often be successful in explicit social scenarios, despite marked social adaptive deficits. The…

Item Analysis in Introductory Economics Testing.

ERIC Educational Resources Information Center

Tinari, Frank D.

1979-01-01

Computerized analysis of multiple choice test items is explained. Examples of item analysis applications in the introductory economics course are discussed with respect to three objectives: to evaluate learning; to improve test items; and to help improve classroom instruction. Problems, costs and benefits of the procedures are identified. (JMD)

The effects of Georgia's Choice curricular reform model on third grade science scores on the Georgia Criterion Referenced Competency Test

NASA Astrophysics Data System (ADS)

Phemister, Art W.

The purpose of this study was to evaluate the effectiveness of the Georgia's Choice reading curriculum on third grade science scores on the Georgia Criterion Referenced Competency Test from 2002 to 2008. In assessing the effectiveness of the Georgia's Choice curriculum model this causal comparative study examined the 105 elementary schools that implemented Georgia's Choice and 105 randomly selected elementary schools that did not elect to use Georgia's Choice. The Georgia's Choice reading program used intensified instruction in an effort to increase reading levels for all students. The study used a non-equivalent control group with a pretest and posttest design to determine the effectiveness of the Georgia's Choice curriculum model. Findings indicated that third grade students in Non-Georgia's Choice schools outscored third grade students in Georgia's Choice schools across the span of the study.

Do the Guideline Violations Influence Test Difficulty of High-Stake Test?: An Investigation on University Entrance Examination in Turkey

ERIC Educational Resources Information Center

Atalmis, Erkan Hasan

2016-01-01

Multiple-choice (MC) items are commonly used in high-stake tests. Thus, each item of such tests should be meticulously constructed to increase the accuracy of decisions based on test results. Haladyna and his colleagues (2002) addressed the valid item-writing guidelines to construct high quality MC items in order to increase test reliability and…

Development of Achievement Test: Validity and Reliability Study for Achievement Test on Matter Changing

ERIC Educational Resources Information Center

Kara, Filiz; Celikler, Dilek

2015-01-01

For "Matter Changing" unit included in the Secondary School 5th Grade Science Program, it is intended to develop a test conforming the gains described in the program, and that can determine students' achievements. For this purpose, a multiple-choice test of 48 questions is arranged, consisting of 8 questions for each gain included in the…

Instructional Innovation, School Choice, and Student Achievement

ERIC Educational Resources Information Center

Berends, Mark; Penaloza, Roberto V.; Cannata, Marisa; Goldring, Ellen

2009-01-01

There is limited empirical research about innovation in various types of schools of choice, although viable choice policies tend to assume clear differentiation amongst schools. Innovation can be conceptualized in many ways and takes place at multiple levels of the school organization. Schools can innovate in terms of the roles and responsibility…

Environmental change mediates mate choice for an extended phenotype, but not for mate quality.

PubMed

Head, Megan L; Fox, Rebecca J; Barber, Iain

2017-01-01

Sexual cues, including extended phenotypes, are expected to be reliable indicators of male genetic quality and/or provide information on parental quality. However, the reliability of these cues may be dependent on stability of the environment, with heterogeneity affecting how selection acts on such traits. Here, we test how environmental change mediates mate choice for multiple sexual traits, including an extended phenotype--the structure of male-built nests - in stickleback fish. First, we manipulated the dissolved oxygen (DO) content of water to create high or low DO environments in which male fish built nests. Then we recorded the mate choice of females encountering these males (and their nests), under either the same or reversed DO conditions. Males in high DO environments built more compact nests than those in low DO conditions and males adjusted their nest structure in response to changing conditions. Female mate choice for extended phenotype (male nests) was environmentally dependent (females chose more compact nests in high DO conditions), while female choice for male phenotype was not (females chose large, vigorous males regardless of DO level). Examining mate choice in this dynamic context suggests that females evaluate the reliability of multiple sexual cues, taking into account environmental heterogeneity. © 2016 The Author(s). Evolution published by Wiley Periodicals, Inc. on behalf of The Society for the Study of Evolution.

Science Library of Test Items. Volume Nineteen. A Collection of Multiple Choice Test Items Relating Mainly to Geology.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Science Library of Test Items. Volume Seventeen. A Collection of Multiple Choice Test Items Relating Mainly to Biology.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Science Library of Test Items. Volume Eighteen. A Collection of Multiple Choice Test Items Relating Mainly to Chemistry.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Patient choice in opt-in, active choice, and opt-out HIV screening: randomized clinical trial

PubMed Central

Dow, William H; Kaplan, Beth C

2016-01-01

Study question What is the effect of default test offers—opt-in, opt-out, and active choice—on the likelihood of acceptance of an HIV test among patients receiving care in an emergency department? Methods This was a randomized clinical trial conducted in the emergency department of an urban teaching hospital and regional trauma center. Patients aged 13-64 years were randomized to opt-in, opt-out, and active choice HIV test offers. The primary outcome was HIV test acceptance percentage. The Denver Risk Score was used to categorize patients as being at low, intermediate, or high risk of HIV infection. Study answer and limitations 38.0% (611/1607) of patients in the opt-in testing group accepted an HIV test, compared with 51.3% (815/1628) in the active choice arm (difference 13.3%, 95% confidence interval 9.8% to 16.7%) and 65.9% (1031/1565) in the opt-out arm (difference 27.9%, 24.4% to 31.3%). Compared with active choice testing, opt-out testing led to a 14.6 (11.1 to 18.1) percentage point increase in test acceptance. Patients identified as being at intermediate and high risk were more likely to accept testing than were those at low risk in all arms (difference 6.4% (3.4% to 9.3%) for intermediate and 8.3% (3.3% to 13.4%) for high risk). The opt-out effect was significantly smaller among those reporting high risk behaviors, but the active choice effect did not significantly vary by level of reported risk behavior. Patients consented to inclusion in the study after being offered an HIV test, and inclusion varied slightly by treatment assignment. The study took place at a single county hospital in a city that is somewhat unique with respect to HIV testing; although the test acceptance percentages themselves might vary, a different pattern for opt-in versus active choice versus opt-out test schemes would not be expected. What this paper adds Active choice is a distinct test regimen, with test acceptance patterns that may best approximate patients’ true

Understanding Test-Takers' Perceptions of Difficulty in EAP Vocabulary Tests: The Role of Experiential Factors

ERIC Educational Resources Information Center

Oruç Ertürk, Nesrin; Mumford, Simon E.

2017-01-01

This study, conducted by two researchers who were also multiple-choice question (MCQ) test item writers at a private English-medium university in an English as a foreign language (EFL) context, was designed to shed light on the factors that influence test-takers' perceptions of difficulty in English for academic purposes (EAP) vocabulary, with the…

The role of attitudes towards the targets of behaviour in predicting and informing prenatal testing choices.

PubMed

Bryant, Louise D; Green, Josephine M; Hewison, Jenny

2010-12-01

Research considering the role of attitudes in prenatal testing choices has commonly focused on the relationship between the attitude towards undergoing testing and actual testing behaviour. In contrast, this study focused on the relationship between testing behaviour and attitudes towards the targets of the behaviour (in this case people with Down syndrome (DS) and having a baby with DS). A cross-sectional, prospective survey of 197 pregnant women measured attitudes towards the targets of prenatal testing along with intentions to use screening and diagnostic testing, and the termination of an affected pregnancy. Screening uptake was established via patient records. Although attitudes towards DS and having a baby with DS were significantly associated with screening uptake and testing and termination intentions, unfavourable attitudes were better than favourable ones at predicting these outcomes. For example, in the quartile of women with the 'most favourable' attitude towards people with DS 67% used screening although only 8% said they would terminate an affected pregnancy. Qualitative data suggested that not all women considered personal attitudes towards DS to be relevant to their screening decisions. This finding has implications for the way in which informed choice is currently understood and measured in the prenatal testing context.

A discrete choice experiment to determine UK patient preference for attributes of disease modifying treatments in Multiple Sclerosis.

PubMed

Bottomley, Catherine; Lloyd, Andrew; Bennett, Gary; Adlard, Nicholas

2017-08-01

The recent licensing of Disease Modifying Treatments (DMTs) for Multiple Sclerosis (MS) has increased available treatment options. The aim of this study was to explore MS patients' preference for the different attributes of DMTs in the UK. Attributes (treatment characteristics) for inclusion in the discrete choice experiment (DCE) were determined through published literature and interviews with 12 people with MS. Seven attributes were selected. Participants were presented with three hypothetical treatment options sampled from included attributes and asked for their most/least preferred options. The influence of patient characteristics and demographics on patient preference was also investigated. The DCE was completed by 350 people with MS (81% female, mean age = 39). Results showed that method of taking medication was the strongest determinant of preference (27%; relative importance out of 100%), followed by relapse free rate (21%) and symptom progression (14%). Risk of fatigue (8%) and type of monitoring (6%) were the weakest determinants of preference. Once-daily oral treatment was preferred over all other methods of administration. Participant characteristics did not influence data on strength of preference. This study assumed adequate participant understanding of the discrete choice experiment task, and recruitment targeted those with access to the internet. These results, derived from people with MS in the UK, should be used to inform individual discussions with patients about DMT choices.

Pursuing the Qualities of a "Good" Test

ERIC Educational Resources Information Center

Coniam, David

2014-01-01

This article examines the issue of the quality of teacher-produced tests, limiting itself in the current context to objective, multiple-choice tests. The article investigates a short, two-part 20-item English language test. After a brief overview of the key test qualities of reliability and validity, the article examines the two subtests in terms…

Testing and analysis of flat and curved panels with multiple cracks

NASA Technical Reports Server (NTRS)

Broek, David; Jeong, David Y.; Thomson, Douglas

1994-01-01

An experimental and analytical investigation of multiple cracking in various types of test specimens is described in this paper. The testing phase is comprised of a flat unstiffened panel series and curved stiffened and unstiffened panel series. The test specimens contained various configurations for initial damage. Static loading was applied to these specimens until ultimate failure, while loads and crack propagation were recorded. This data provides the basis for developing and validating methodologies for predicting linkup of multiple cracks, progression to failure, and overall residual strength. The results from twelve flat coupon and ten full scale curved panel tests are presented. In addition, an engineering analysis procedure was developed to predict multiple crack linkup. Reasonable agreement was found between predictions and actual test results for linkup and residual strength for both flat and curved panels. The results indicate that an engineering analysis approach has the potential to quantitatively assess the effect of multiple cracks in the arrest capability of an aircraft fuselage structure.

Equating in Small-Scale Language Testing Programs

ERIC Educational Resources Information Center

LaFlair, Geoffrey T.; Isbell, Daniel; May, L. D. Nicolas; Gutierrez Arvizu, Maria Nelly; Jamieson, Joan

2017-01-01

Language programs need multiple test forms for secure administrations and effective placement decisions, but can they have confidence that scores on alternate test forms have the same meaning? In large-scale testing programs, various equating methods are available to ensure the comparability of forms. The choice of equating method is informed by…

Determinants of Awareness, Consideration, and Choice Set Size in University Choice.

ERIC Educational Resources Information Center

Dawes, Philip L.; Brown, Jennifer

2002-01-01

Developed and tested a model of students' university "brand" choice using five individual-level variables (ethnic group, age, gender, number of parents going to university, and academic ability) and one situational variable (duration of search) to explain variation in the sizes of awareness, consideration, and choice decision sets. (EV)

Motives underlying food choice: dentists, porters and dietary health promotion.

PubMed

Crossley, M L; Khan, S N

2001-08-25

Differences in dental decay and disease amongst socioeconomic groups are thought to derive, in part, from variations in dietary practices and differences in education. The aim of this exploratory study was to examine whether differences in motivating factors affecting food choice could be found in a comparison of two groups at very different ends of the social spectrum: dentists and porters/cleaners. A convenience sample of 100 people (51 porters/cleaners and 49 dentists) working in the dental school at a university in the North West of England were approached to interview face-to-face and complete the Food Choice Questionnaire (FCQ), a previously validated measure designed to assess nine main factors relevant to peoples' food choices. A sample size of 100 was chosen because it was adequate to test validity (using a two-group Chi-square test with a 0.050 two sided significance). Findings were analysed using independent sample t-test and multiple linear regression. Results indicated significant differences between porters/cleaners and dentists in terms of their motives for food choice on six of the nine FCQ factors. These included convenience (p < 0.001), natural content (p < 0.05), price (p < 0.005), familiarity (p < .0001), mood (p < 0.03) and ethical concern (p < 0.01). Porters/cleaners tended to rate the factors covenience, price, mood and familiarity more highly, whereas dentists did the same for natural content and ethical concern. Awareness of the differences in motivational factors affecting food choice between different social groups is important to dental practitioners who are being taught to play an increasing role in health promotion. If dental practitioners are to partake meaningfully in such a role, it is necessary for them to be aware not only of their own motives in food selection, but also of the way in which those motives may differ from those of their clients.

Combining food type(s) and food quantity choice in a new food choice paradigm based on vice-virtue bundles.

PubMed

Haws, Kelly L; Liu, Peggy J

2016-08-01

Given the prevalence and rising rates of obesity in many countries, including the United States, much food decision-making research ultimately aims at understanding how consumers can make healthier choices. The two predominant choice paradigms used in food decision-making research ask consumers to choose (a) between a "vice" (or unhealthy food) and a "virtue" (or healthy food) or (b) among varying portion sizes of "vice." We propose a new food choice paradigm that encourages consumers to jointly consider both food type(s) choice and food portion size at each decision point. The purpose of this paradigm is two-fold. First, it aims to allow examination of more comprehensive eating behavior (e.g., to examine the overall composition of a plate of food rather than choice of a single food). Second, it aims to shift consumers towards including large proportions of virtues and smaller proportions of vice in their overall consumption portfolios. For this paradigm, we draw upon a recently introduced food product innovation called "vice-virtue bundles" (Liu et al., 2015) that illustrates the basis of this new food choice paradigm, in which food type(s) and portion decisions are made simultaneously. Accordingly, we first discuss relevant findings on vice-virtue bundles as well as the differences between simultaneous and sequential choice of multiple products. Second, we examine the benefits for managing and controlling one's consumption that are provided by vice-virtue bundles and this joint food choice paradigm more generally. Third and finally, we point out opportunities for future research by discussing (a) multiple factors that influence food choices, (b) decision processes affected by food choice paradigms, and (c) issues of generalizability related to the presence of vice-virtue bundles. Copyright © 2015 Elsevier Ltd. All rights reserved.

Practical Implications of Test Dimensionality for Item Response Theory Calibration of the Medical College Admission Test. MCAT Monograph.

ERIC Educational Resources Information Center

Childs, Ruth A.; Oppler, Scott H.

The use of item response theory (IRT) in the Medical College Admission Test (MCAT) testing program has been limited. This study provides a basis for future IRT analyses of the MCAT by exploring the dimensionality of each of the MCAT's three multiple-choice test sections (Verbal Reasoning, Physical Sciences, and Biological Sciences) and the…

Bilingual Test as a Test Accommodation to Determine the Mathematics Achievement of Mainstream Students with Limited English Proficiency

ERIC Educational Resources Information Center

Shanmugam, S. Kanageswari Suppiah; Lan, Ong Saw

2013-01-01

Purpose: This study aims to investigate the validity of using bilingual test to measure the mathematics achievement of students who have limited English proficiency (LEP). The bilingual test and the English-only test consist of 20 computation and 20 word problem multiple-choice questions (from TIMSS 2003 and 2007 released items. The bilingual test…

Testing Signal-Detection Models of Yes/No and Two-Alternative Forced-Choice Recognition Memory

ERIC Educational Resources Information Center

Jang, Yoonhee; Wixted, John T.; Huber, David E.

2009-01-01

The current study compared 3 models of recognition memory in their ability to generalize across yes/no and 2-alternative forced-choice (2AFC) testing. The unequal-variance signal-detection model assumes a continuous memory strength process. The dual-process signal-detection model adds a thresholdlike recollection process to a continuous…

A comparative study of students' performance in preclinical physiology assessed by multiple choice and short essay questions.

PubMed

Oyebola, D D; Adewoye, O E; Iyaniwura, J O; Alada, A R; Fasanmade, A A; Raji, Y

2000-01-01

This study was designed to compare the performance of medical students in physiology when assessed by multiple choice questions (MCQs) and short essay questions (SEQs). The study also examined the influence of factors such as age, sex, O/level grades and JAMB scores on performance in the MCQs and SEQs. A structured questionnaire was administered to 264 medical students' four months before the Part I MBBS examination. Apart from personal data of each student, the questionnaire sought information on the JAMB scores and GCE O' Level grades of each student in English Language, Biology, Chemistry, Physics and Mathematics. The physiology syllabus was divided into five parts and the students were administered separate examinations (tests) on each part. Each test consisted of MCQs and SEQs. The performance in MCQs and SEQs were compared. Also, the effects of JAMB scores and GCE O/level grades on the performance in both the MCQs and SEQs were assessed. The results showed that the students performed better in all MCQ tests than in the SEQs. JAMB scores and O' level English Language grade had no significant effect on students' performance in MCQs and SEQs. However O' level grades in Biology, Chemistry, Physics and Mathematics had significant effects on performance in MCQs and SEQs. Inadequate knowledge of physiology and inability to present information in a logical sequence are believed to be major factors contributing to the poorer performance in the SEQs compared with MCQs. In view of the finding of significant association between performance in MCQs and SEQs and GCE O/level grades in science subjects and mathematics, it was recommended that both JAMB results and the GCE results in the four O/level subjects above may be considered when selecting candidates for admission into the medical schools.

Multiple Hub Network Choice in the Liberalized European Market

NASA Technical Reports Server (NTRS)

Berechman, Joseph; deWit, Jaap

1997-01-01

airport slots, this is not a guarantee for a stable transport volume growth of individual airports. The more volatile the market is, the more vulnerable privatized airports become. Therefore, the main issue of this study is the analysis of the opportunities of major European airports to become a central hub as a result of the network choices made by the new European airlines in a completely liberalized market. In a previous study (Berechman and de Wit, 1996), we already explored the potential of Amsterdam Airport Schiphol of becoming the major West-European hub, once European aviation markets are deregulated. A major hindrance of that study was the use of a single hub-and-spoke network. For example that model could not analyze the viability of different combinations of European hubs within a multiple hub network of alternative airline alliances. In this study, we have formulated the model of a multi-hub network where two West-European airports are used for inter-continental and intra-European travel to enable a more realistic analysis of hub choice. Like the previous one also this multi-hub model is primarily used to assess the potential ability of Amsterdam Airport Schiphol for becoming a major West-European hub. Thus, in particular, the policy tests focus on this airport in a double hub network.

A Rejection Principle for Sequential Tests of Multiple Hypotheses Controlling Familywise Error Rates

PubMed Central

BARTROFF, JAY; SONG, JINLIN

2015-01-01

We present a unifying approach to multiple testing procedures for sequential (or streaming) data by giving sufficient conditions for a sequential multiple testing procedure to control the familywise error rate (FWER). Together we call these conditions a “rejection principle for sequential tests,” which we then apply to some existing sequential multiple testing procedures to give simplified understanding of their FWER control. Next the principle is applied to derive two new sequential multiple testing procedures with provable FWER control, one for testing hypotheses in order and another for closed testing. Examples of these new procedures are given by applying them to a chromosome aberration data set and to finding the maximum safe dose of a treatment. PMID:26985125

Taking the Test Taker's Perspective: Response Process and Test Motivation in Multidimensional Forced-Choice Versus Rating Scale Instruments.

PubMed

Sass, Rachelle; Frick, Susanne; Reips, Ulf-Dietrich; Wetzel, Eunike

2018-03-01

The multidimensional forced-choice (MFC) format has been proposed as an alternative to the rating scale (RS) response format. However, it is unclear how changing the response format may affect the response process and test motivation of participants. In Study 1, we investigated the MFC response process using the think-aloud technique. In Study 2, we compared test motivation between the RS format and different versions of the MFC format (presenting 2, 3, 4, and 5 items simultaneously). The response process to MFC item blocks was similar to the RS response process but involved an additional step of weighing the items within a block against each other. The RS and MFC response format groups did not differ in their test motivation. Thus, from the test taker's perspective, the MFC format is somewhat more demanding to respond to, but this does not appear to decrease test motivation.

Expanding the Reach of Participatory Risk Management: Testing an Online Decision-Aiding Framework for Informing Internally Consistent Choices.

PubMed

Bessette, Douglas L; Campbell-Arvai, Victoria; Arvai, Joseph

2016-05-01

This article presents research aimed at developing and testing an online, multistakeholder decision-aiding framework for informing multiattribute risk management choices associated with energy development and climate change. The framework was designed to provide necessary background information and facilitate internally consistent choices, or choices that are in line with users' prioritized objectives. In order to test different components of the decision-aiding framework, a six-part, 2 × 2 × 2 factorial experiment was conducted, yielding eight treatment scenarios. The three factors included: (1) whether or not users could construct their own alternatives; (2) the level of detail regarding the composition of alternatives users would evaluate; and (3) the way in which a final choice between users' own constructed (or highest-ranked) portfolio and an internally consistent portfolio was presented. Participants' self-reports revealed the framework was easy to use and providing an opportunity to develop one's own risk-management alternatives (Factor 1) led to the highest knowledge gains. Empirical measures showed the internal consistency of users' decisions across all treatments to be lower than expected and confirmed that providing information about alternatives' composition (Factor 2) resulted in the least internally consistent choices. At the same time, those users who did not develop their own alternatives and were not shown detailed information about the composition of alternatives believed their choices to be the most internally consistent. These results raise concerns about how the amount of information provided and the ability to construct alternatives may inversely affect users' real and perceived internal consistency. © 2015 Society for Risk Analysis.

Using multiple group modeling to test moderators in meta-analysis.

PubMed

Schoemann, Alexander M

2016-12-01

Meta-analysis is a popular and flexible analysis that can be fit in many modeling frameworks. Two methods of fitting meta-analyses that are growing in popularity are structural equation modeling (SEM) and multilevel modeling (MLM). By using SEM or MLM to fit a meta-analysis researchers have access to powerful techniques associated with SEM and MLM. This paper details how to use one such technique, multiple group analysis, to test categorical moderators in meta-analysis. In a multiple group meta-analysis a model is fit to each level of the moderator simultaneously. By constraining parameters across groups any model parameter can be tested for equality. Using multiple groups to test for moderators is especially relevant in random-effects meta-analysis where both the mean and the between studies variance of the effect size may be compared across groups. A simulation study and the analysis of a real data set are used to illustrate multiple group modeling with both SEM and MLM. Issues related to multiple group meta-analysis and future directions for research are discussed. Copyright © 2016 John Wiley & Sons, Ltd. Copyright © 2016 John Wiley & Sons, Ltd.

Paraquat detoxication with multiple emulsions.

PubMed

Frasca, S; Couvreur, P; Seiller, M; Pareau, D; Lacour, B; Stambouli, M; Grossiord, J L

2009-10-01

In this study, we show that detoxifying W/O/W multiple emulsions, prepared with an appropriate extractant/trapping couple, represent a promising technology for quick and safe poisoning treatments, with application to the highly toxic herbicide Paraquat, responsible of poisonings from low-dose exposure leading to several deaths every year. In vitro tests led to the choice of an appropriate extractant/trapping couple system with significant detoxication performance. In vivo tests showed (i) that rats receiving high doses of Paraquat, then a detoxifying emulsion, presented an increase from 50% to 100% of the MST (median survival time) and (ii) that no mortality was observed during 30 days with rats dosed with emulsions initially loaded with Paraquat at a concentration much higher than the lethal dose, proving the stability and the inocuity of the detoxifying multiple emulsion in the gastrointestinal tract.

Distinct encoding of risk and value in economic choice between multiple risky options☆

PubMed Central

Wright, Nicholas D.; Symmonds, Mkael; Dolan, Raymond J.

2013-01-01

Neural encoding of value-based stimuli is suggested to involve representations of summary statistics, including risk and expected value (EV). A more complex, but ecologically more common, context is when multiple risky options are evaluated together. However, it is unknown whether encoding related to option evaluation in these situations involves similar principles. Here we employed fMRI during a task that parametrically manipulated EV and risk in two simultaneously presented lotteries, both of which contained either gains or losses. We found representations of EV in medial prefrontal cortex and anterior insula, an encoding that was dependent on which option was chosen (i.e. chosen and unchosen EV) and whether the choice was over gains or losses. Parietal activity reflected whether the riskier or surer option was selected, whilst activity in a network of regions that also included parietal cortex reflected both combined risk and difference in risk for the two options. Our findings provide support for the idea that summary statistics underpin a representation of value-based stimuli, and further that these summary statistics undergo distinct forms of encoding. PMID:23684860

Methods for the Joint Meta-Analysis of Multiple Tests

ERIC Educational Resources Information Center

Trikalinos, Thomas A.; Hoaglin, David C.; Small, Kevin M.; Terrin, Norma; Schmid, Christopher H.

2014-01-01

Existing methods for meta-analysis of diagnostic test accuracy focus primarily on a single index test. We propose models for the joint meta-analysis of studies comparing multiple index tests on the same participants in paired designs. These models respect the grouping of data by studies, account for the within-study correlation between the tests'…

Test-Wiseness Cues in the Options of Mathematics Items.

ERIC Educational Resources Information Center

Kuntz, Patricia

The quality of mathematics multiple choice items and their susceptibility to test wiseness were examined. Test wiseness was defined as "a subject's capacity to utilize the characteristics and formats of the test and/or test taking situation to receive a high score." The study used results of the Graduate Record Examinations Aptitude Test (GRE) and…

Using Distractor-Driven Standards-Based Multiple-Choice Assessments and Rasch Modeling to Investigate Hierarchies of Chemistry Misconceptions and Detect Structural Problems with Individual Items

ERIC Educational Resources Information Center

Herrmann-Abell, Cari F.; DeBoer, George E.

2011-01-01

Distractor-driven multiple-choice assessment items and Rasch modeling were used as diagnostic tools to investigate students' understanding of middle school chemistry ideas. Ninety-one items were developed according to a procedure that ensured content alignment to the targeted standards and construct validity. The items were administered to 13360…

Relationship of awards in multiple choice questions and structured answer questions in the undergraduate years and their effectiveness in evaluation.

PubMed

Khan, Junaid Sarfraz; Mukhtar, Osama; Tabasum, Saima; Shaheen, Naveed; Farooq, M; Irfan, M Abdul; Sattar, Ajmal; Nabeel, M; Imran, M; Rafique, Sadia; Iqbal, Maryam; Afzal, M Sheraz; Hameed, M Shahbaz; Habib, Maryam; Jabeen, Uzma; Mubbashar, Malik Hussain

2010-01-01

A number of evaluation tools for assessing the cognitive and affective domains in accordance with Bloom's taxonomy are available for summative assessment. At the University of Health Sciences, Lahore, Multiple Choice Questions (MCQs) and Structured Answer Questions (SAQs) are used for the evaluation of the cognitive domain at all six hierarch levels of taxonomy using the tables of specifications to ensure content validity. The rationale of having two evaluation tools seemingly similar in their evaluative competency yet differing in feasibility of construction, administration and marking is being challenged in this study. The MCQ and SAQ awards of the ten percent sample population amounting to 985 students in fifteen Medical and Dental Colleges across Punjab were entered into SPSS-15 and correlated according to the cognitive and affective level of assessment in relation to the Bloom's taxonomy and their grouping in the Tables of Specifications, using parametric tests. 3494 anonymously administered questionnaires were analyzed using ethnograph. No statistically significant difference was found in the mean marks obtained by the students when MCQs and SAQs were compared according to their groupings in the Tables of Specifications at all levels of cognitive hierarchical testing. End-of-yearcognitive level testing targets set were not met and more questions were set at the lower cognitive testing levels. Expenses incurred in setting MCQs and SAQs were comparable but conduct and assessment costs for MCQs and SAQs were 6% and 94% of the total respectively. In both MCQs and SAQs students performed better at higher cognitive testing levels whereas the SAQs and MCQs were able to marginally test the lower levels of affective domain only. Student's feedback showed that attempting MCQs required critical thinking, experience and practice. MCQs are more cost effective means at levels of cognitive domain assessment.

Passage Independence within Standardized Reading Comprehension Tests

ERIC Educational Resources Information Center

Roy-Charland, Annie; Colangelo, Gabrielle; Foglia, Victoria; Reguigui, Leïla

2017-01-01

In tests used to measure reading comprehension, validity is important in obtaining accurate results. Unfortunately, studies have shown that people can correctly answer some questions of these tests without reading the related passage. These findings bring forth the need to address whether this phenomenon is observed in multiple-choice only tests…

Y-balance test: a reliability study involving multiple raters.

PubMed

Shaffer, Scott W; Teyhen, Deydre S; Lorenson, Chelsea L; Warren, Rick L; Koreerat, Christina M; Straseske, Crystal A; Childs, John D

2013-11-01

The Y-balance test (YBT) is one of the few field expedient tests that have shown predictive validity for injury risk in an athletic population. However, analysis of the YBT in a heterogeneous population of active adults (e.g., military, specific occupations) involving multiple raters with limited experience in a mass screening setting is lacking. The primary purpose of this study was to determine interrater test-retest reliability of the YBT in a military setting using multiple raters. Sixty-four service members (53 males, 11 females) actively conducting military training volunteered to participate. Interrater test-retest reliability of the maximal reach had intraclass correlation coefficients (2,1) of 0.80 to 0.85 with a standard error of measurement ranging from 3.1 to 4.2 cm for the 3 reach directions (anterior, posteromedial, and posterolateral). Interrater test-retest reliability of the average reach of 3 trails had an intraclass correlation coefficients (2,3) range of 0.85 to 0.93 with an associated standard error of measurement ranging from 2.0 to 3.5cm. The YBT showed good interrater test-retest reliability with an acceptable level of measurement error among multiple raters screening active duty service members. In addition, 31.3% (n = 20 of 64) of participants exhibited an anterior reach asymmetry of >4cm, suggesting impaired balance symmetry and potentially increased risk for injury. Reprint & Copyright © 2013 Association of Military Surgeons of the U.S.

Kinetics of Aggregation with Choice

DOE PAGES

Ben-Naim, Eli; Krapivsky, Paul

2016-12-01

Here we generalize the ordinary aggregation process to allow for choice. In ordinary aggregation, two random clusters merge and form a larger aggregate. In our implementation of choice, a target cluster and two candidate clusters are randomly selected and the target cluster merges with the larger of the two candidate clusters.We study the long-time asymptotic behavior and find that as in ordinary aggregation, the size density adheres to the standard scaling form. However, aggregation with choice exhibits a number of different features. First, the density of the smallest clusters exhibits anomalous scaling. Second, both the small-size and the large-size tailsmore » of the density are overpopulated, at the expense of the density of moderate-size clusters. Finally, we also study the complementary case where the smaller candidate cluster participates in the aggregation process and find an abundance of moderate clusters at the expense of small and large clusters. Additionally, we investigate aggregation processes with choice among multiple candidate clusters and a symmetric implementation where the choice is between two pairs of clusters.« less

The 48-Pictures Test: a two-alternative forced-choice recognition test for the detection of malingering.

PubMed

Chouinard, M J; Rouleau, I

1997-11-01

We tested the validity of the 48-Pictures Test, a 2-alternative forced-choice recognition test, in detecting exaggerated memory impairments. This test maximizes subjective difficulty, through a large number of stimuli and shows minimal objective difficulty. We compared 17 suspected malingerers to 39 patients with memory impairments (6 amnesic, 15 frontal lobe dysfunctions, 18 other etiologies), and 17 normal adults instructed to simulate malingering on three memory tests: the 48-Pictures Test, the Rey Auditory Verbal Learning Test (RAVLT), and the Rey Complex Figure Test (RCFT). On the 48-Pictures Test, the clinical groups showed good recognition performance (amnesics: 85%; frontal dysfunction: 94%; other memory impairments: 97%), whereas the two simulator groups showed a poor performance (suspected malingerers: 62% correct; volunteer simulators 68% correct). The two other tests did not show a high degree of discrimination between the clinical groups and the simulator groups, except in 2 measures: the 2 simulator groups tended to show a performance decrement from the last recall trial to immediate recognition of the RAVLT and also performed better than the clinical groups on the immediate recall of the RCFT. A discriminant analysis with the latter 2 measures and the 48-Pictures Test correctly classified 96% of the participants. These results suggest that the 48-Pictures Test is a useful tool for the detection of possible simulated memory impairment and that when combined to the RAVLT recall-recognition difference score and to the immediate recall score on the RCFT can provide strong evidence of exaggerated memory impairment.

A Multiple Objective Test Assembly Approach for Exposure Control Problems in Computerized Adaptive Testing

ERIC Educational Resources Information Center

Veldkamp, Bernard P.; Verschoor, Angela J.; Eggen, Theo J. H. M.

2010-01-01

Overexposure and underexposure of items in the bank are serious problems in operational computerized adaptive testing (CAT) systems. These exposure problems might result in item compromise, or point at a waste of investments. The exposure control problem can be viewed as a test assembly problem with multiple objectives. Information in the test has…

Ventromedial prefrontal and anterior cingulate cortex adopt choice and default reference frames during sequential multialternative choice

PubMed Central

Boorman, Erie D; Rushworth, Matthew F; Behrens, Tim E

2013-01-01

Although damage to medial frontal cortex causes profound decision-making impairments, it has been difficult to pinpoint the relative contributions of key anatomical subdivisions. Here we use fMRI to examine the contributions of human ventromedial prefrontal cortex (vmPFC) and dorsal anterior cingulate cortex (dACC) during sequential choices between multiple alternatives – two key features of choices made in ecological settings. By carefully constructing options whose current value at any given decision was dissociable from their longer-term value, we were able to examine choices in current and long-term frames of reference. We present evidence showing that activity at choice and feedback in vmPFC and dACC was tied to the current choice and the best long-term option, respectively. vmPFC, mid-cingulate, and PCC encoded the relative value between the chosen and next-best option at each sequential decision, whereas dACC encoded the relative value of adapting choices from the option with the highest value in the longer-term. Furthermore, at feedback we identify temporally dissociable effects that predict repetition of the current choice and adaptation away from the long-term best option in vmPFC and dACC, respectively. These functional dissociations at choice and feedback suggest that sequential choices are subject to competing cortical mechanisms. PMID:23392656

Understanding Adolescent Contraceptive Choice: An Empirical Test.

ERIC Educational Resources Information Center

Adler, Nancy E.; And Others

Research using expectancy models has shown contraceptive choice among adults to be a rational process in that intentions and behaviors reflect an individual's beliefs, values, attitudes, and perceptions of social norms. This study examined whether such an approach could accurately represent adolescents' contraceptive decision-making. It used the…

Test-Retest Reliability of the Multiple Sleep Latency Test in Narcolepsy without Cataplexy and Idiopathic Hypersomnia

PubMed Central

Trotti, Lynn Marie; Staab, Beth A.; Rye, David B.

2013-01-01

Study Objectives: Differentiation of narcolepsy without cataplexy from idiopathic hypersomnia relies entirely upon the multiple sleep latency test (MSLT). However, the test-retest reliability for these central nervous system hypersomnias has never been determined. Methods: Patients with narcolepsy without cataplexy, idiopathic hypersomnia, and physiologic hypersomnia who underwent two diagnostic multiple sleep latency tests were identified retrospectively. Correlations between the mean sleep latencies on the two studies were evaluated, and we probed for demographic and clinical features associated with reproducibility versus change in diagnosis. Results: Thirty-six patients (58% women, mean age 34 years) were included. Inter -test interval was 4.2 ± 3.8 years (range 2.5 months to 16.9 years). Mean sleep latencies on the first and second tests were 5.5 (± 3.7 SD) and 7.3 (± 3.9) minutes, respectively, with no significant correlation (r = 0.17, p = 0.31). A change in diagnosis occurred in 53% of patients, and was accounted for by a difference in the mean sleep latency (N = 15, 42%) or the number of sleep onset REM periods (N = 11, 31%). The only feature predictive of a diagnosis change was a history of hypnagogic or hypnopompic hallucinations. Conclusions: The multiple sleep latency test demonstrates poor test-retest reliability in a clinical population of patients with central nervous system hypersomnia evaluated in a tertiary referral center. Alternative diagnostic tools are needed. Citation: Trotti LM; Staab BA; Rye DB. Test- retest reliability of the multiple sleep latency test in narcolepsy without cataplexy and idiopathic hypersomnia. J Clin Sleep Med 2013;9(8):789-795. PMID:23946709

A Novel Multiple Choice Question Generation Strategy: Alternative Uses for Controlled Vocabulary Thesauri in Biomedical-Sciences Education.

PubMed

Lopetegui, Marcelo A; Lara, Barbara A; Yen, Po-Yin; Çatalyürek, Ümit V; Payne, Philip R O

2015-01-01

Multiple choice questions play an important role in training and evaluating biomedical science students. However, the resource intensive nature of question generation limits their open availability, reducing their contribution to evaluation purposes mainly. Although applied-knowledge questions require a complex formulation process, the creation of concrete-knowledge questions (i.e., definitions, associations) could be assisted by the use of informatics methods. We envisioned a novel and simple algorithm that exploits validated knowledge repositories and generates concrete-knowledge questions by leveraging concepts' relationships. In this manuscript we present the development and validation of a prototype which successfully produced meaningful concrete-knowledge questions, opening new applications for existing knowledge repositories, potentially benefiting students of all biomedical sciences disciplines.

Design and protocol of a randomized multiple behavior change trial: Make Better Choices 2 (MBC2).

PubMed

Pellegrini, Christine A; Steglitz, Jeremy; Johnston, Winter; Warnick, Jennifer; Adams, Tiara; McFadden, H G; Siddique, Juned; Hedeker, Donald; Spring, Bonnie

2015-03-01

Suboptimal diet and inactive lifestyle are among the most prevalent preventable causes of premature death. Interventions that target multiple behaviors are potentially efficient; however the optimal way to initiate and maintain multiple health behavior changes is unknown. The Make Better Choices 2 (MBC2) trial aims to examine whether sustained healthful diet and activity change are best achieved by targeting diet and activity behaviors simultaneously or sequentially. Study design approximately 250 inactive adults with poor quality diet will be randomized to 3 conditions examining the best way to prescribe healthy diet and activity change. The 3 intervention conditions prescribe: 1) an increase in fruit and vegetable consumption (F/V+), decrease in sedentary leisure screen time (Sed-), and increase in physical activity (PA+) simultaneously (Simultaneous); 2) F/V+ and Sed- first, and then sequentially add PA+ (Sequential); or 3) Stress Management Control that addresses stress, relaxation, and sleep. All participants will receive a smartphone application to self-monitor behaviors and regular coaching calls to help facilitate behavior change during the 9 month intervention. Healthy lifestyle change in fruit/vegetable and saturated fat intakes, sedentary leisure screen time, and physical activity will be assessed at 3, 6, and 9 months. MBC2 is a randomized m-Health intervention examining methods to maximize initiation and maintenance of multiple healthful behavior changes. Results from this trial will provide insight about an optimal technology supported approach to promote improvement in diet and physical activity. Copyright © 2015 Elsevier Inc. All rights reserved.

Testing to the Top: Everything But the Kitchen Sink?

ERIC Educational Resources Information Center

Dietel, Ron

2011-01-01

Two tests intended to measure student achievement of the Common Core State Standards will face intense scrutiny, but the test makers say they will include performance assessments and other items that are not multiple-choice questions. Incorporating performance items on this tests will bring up issues over scoring, costs, and validity.

Contextual and social influences on valuation and choice.

PubMed

Engelmann, Jan B; Hein, Grit

2013-01-01

To survive in our complex environment, we have to adapt to changing contexts. Prior research that investigated how contextual changes are processed in the human brain has demonstrated important modulatory influences on multiple cognitive processes underlying decision-making, including perceptual judgments, working memory, as well as cognitive and attentional control. However, in everyday life, the importance of context is even more obvious during economic and social interactions, which often have implicit rule sets that need to be recognized by a decision-maker. Here, we review recent evidence from an increasing number of studies in the fields of Neuroeconomics and Social Neuroscience that investigate the neurobiological basis of contextual effects on valuation and social choice. Contrary to the assumptions of rational choice theory, multiple contextual factors, such as the availability of alternative choice options, shifts in reference point, and social context, have been shown to modulate behavior, as well as signals in task-relevant neural networks. A consistent picture that emerges from neurobiological results is that valuation-related activity in striatum and ventromedial prefrontal cortex is highly context dependent during both social and nonsocial choice. Alternative approaches to model and explain choice behavior, such as comparison-based choice models, as well as implications for future research are discussed. Copyright © 2013 Elsevier B.V. All rights reserved.

Electronics. Criterion-Referenced Test (CRT) Item Bank.

ERIC Educational Resources Information Center

Davis, Diane, Ed.

This document contains 519 criterion-referenced multiple choice and true or false test items for a course in electronics. The test item bank is designed to work with both the Vocational Instructional Management System (VIMS) and the Vocational Administrative Management System (VAMS) in Missouri. The items are grouped into 15 units covering the…

Do Students Know What They Know and What They Don't Know? Using a Four-Tier Diagnostic Test to Assess the Nature of Students' Alternative Conceptions

ERIC Educational Resources Information Center

Caleon, Imelda S.; Subramaniam, R.

2010-01-01

This study reports on the development and application of a four-tier multiple-choice (4TMC) diagnostic instrument, which has not been reported in the literature. It is an enhanced version of the two-tier multiple-choice (2TMC) test. As in 2TMC tests, its answer and reason tiers measure students' content knowledge and explanatory knowledge,…

Food Service Supervisor. Dietetic Support Personnel Achievement Test.

ERIC Educational Resources Information Center

Oklahoma State Dept. of Vocational and Technical Education, Stillwater.

This guide contains a series of multiple-choice items and guidelines to assist instructors in composing criterion-referenced tests for use in the food service supervisor component of Oklahoma's Dietetic Support Personnel training program. Test items addressing each of the following occupational duty areas are provided: human relations; nutrient…

Food Service Worker. Dietetic Support Personnel Achievement Test.

ERIC Educational Resources Information Center

Oklahoma State Dept. of Vocational and Technical Education, Stillwater.

This guide contains a series of multiple-choice items and guidelines to assist instructors in composing criterion-referenced tests for use in the food service worker component of Oklahoma's Dietetic Support Personnel training program. Test items addressing each of the following occupational duty areas are provided: human relations; personal…

Hierarchical competitions subserving multi-attribute choice

PubMed Central

Hunt, Laurence T; Dolan, Raymond J; Behrens, Timothy EJ

2015-01-01

Valuation is a key tenet of decision neuroscience, where it is generally assumed that different attributes of competing options are assimilated into unitary values. Such values are central to current neural models of choice. By contrast, psychological studies emphasize complex interactions between choice and valuation. Principles of neuronal selection also suggest competitive inhibition may occur in early valuation stages, before option selection. Here, we show behavior in multi-attribute choice is best explained by a model involving competition at multiple levels of representation. This hierarchical model also explains neural signals in human brain regions previously linked to valuation, including striatum, parietal and prefrontal cortex, where activity represents competition within-attribute, competition between attributes, and option selection. This multi-layered inhibition framework challenges the assumption that option values are computed before choice. Instead our results indicate a canonical competition mechanism throughout all stages of a processing hierarchy, not simply at a final choice stage. PMID:25306549

Science Library of Test Items. Volume Twenty. A Collection of Multiple Choice Test Items Relating Mainly to Physics, 1.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Science Library of Test Items. Volume Twenty-Two. A Collection of Multiple Choice Test Items Relating Mainly to Skills.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

The Effects of Georgia's Choice Curricular Reform Model on Third Grade Science Scores on the Georgia Criterion Referenced Competency Test

ERIC Educational Resources Information Center

Phemister, Art W.

2010-01-01

The purpose of this study was to evaluate the effectiveness of the Georgia's Choice reading curriculum on third grade science scores on the Georgia Criterion Referenced Competency Test from 2002 to 2008. In assessing the effectiveness of the Georgia's Choice curriculum model this causal comparative study examined the 105 elementary schools that…

Will Choice Hurt? Compared to What? A School Choice Experiment in Estonia

ERIC Educational Resources Information Center

Põder, Kaire; Lauri, Triin

2014-01-01

This article presents the empirical analysis of the effects of a school choice policy in Estonia. The article shows that relying on markets and giving autonomy to the schools over student selection will produce admission tests, even at the elementary school level. This article's contribution is to show that a school choice policy experiment with…

A gatekeeping procedure to test a primary and a secondary endpoint in a group sequential design with multiple interim looks.

PubMed

Tamhane, Ajit C; Gou, Jiangtao; Jennison, Christopher; Mehta, Cyrus R; Curto, Teresa

2018-03-01

Glimm et al. (2010) and Tamhane et al. (2010) studied the problem of testing a primary and a secondary endpoint, subject to a gatekeeping constraint, using a group sequential design (GSD) with K=2 looks. In this article, we greatly extend the previous results to multiple (K>2) looks. If the familywise error rate (FWER) is to be controlled at a preassigned α level then it is clear that the primary boundary must be of level α. We show under what conditions one α-level primary boundary is uniformly more powerful than another. Based on this result, we recommend the choice of the O'Brien and Fleming (1979) boundary over the Pocock (1977) boundary for the primary endpoint. For the secondary endpoint the choice of the boundary is more complicated since under certain conditions the secondary boundary can be refined to have a nominal level α'>α, while still controlling the FWER at level α, thus boosting the secondary power. We carry out secondary power comparisons via simulation between different choices of primary-secondary boundary combinations. The methodology is applied to the data from the RALES study (Pitt et al., 1999; Wittes et al., 2001). An R library package gsrsb to implement the proposed methodology is made available on CRAN. © 2017, The International Biometric Society.

Multiple Phenotype Association Tests Using Summary Statistics in Genome-Wide Association Studies

PubMed Central

Liu, Zhonghua; Lin, Xihong

2017-01-01

Summary We study in this paper jointly testing the associations of a genetic variant with correlated multiple phenotypes using the summary statistics of individual phenotype analysis from Genome-Wide Association Studies (GWASs). We estimated the between-phenotype correlation matrix using the summary statistics of individual phenotype GWAS analyses, and developed genetic association tests for multiple phenotypes by accounting for between-phenotype correlation without the need to access individual-level data. Since genetic variants often affect multiple phenotypes differently across the genome and the between-phenotype correlation can be arbitrary, we proposed robust and powerful multiple phenotype testing procedures by jointly testing a common mean and a variance component in linear mixed models for summary statistics. We computed the p-values of the proposed tests analytically. This computational advantage makes our methods practically appealing in large-scale GWASs. We performed simulation studies to show that the proposed tests maintained correct type I error rates, and to compare their powers in various settings with the existing methods. We applied the proposed tests to a GWAS Global Lipids Genetics Consortium summary statistics data set and identified additional genetic variants that were missed by the original single-trait analysis. PMID:28653391

Choice Orientations, Discussions, and Prospects.

ERIC Educational Resources Information Center

Raywid, Mary Anne

1992-01-01

Examining the contemporary school choice debate yields arguments that are education, economics, governance, and policy driven. To "break the exclusive franchise," school districts are increasingly sponsoring school operation and education services supplied by multiple sources, and states are discussing sponsorship of schools by entities…

Intertemporal choice in lemurs.

PubMed

Stevens, Jeffrey R; Mühlhoff, Nelly

2012-02-01

Different species vary in their ability to wait for delayed rewards in intertemporal choice tasks. Models of rate maximization account for part of this variation, but other factors such as social structure and feeding ecology seem to underly some species differences. Though studies have evaluated intertemporal choice in several primate species, including Old World monkeys, New World monkeys, and apes, prosimians have not been tested. This study investigated intertemporal choices in three species of lemur (black-and-white ruffed lemurs, Varecia variegata, red ruffed lemurs, Varecia rubra, and black lemurs, Eulemur macaco) to assess how they compare to other primate species and whether their choices are consistent with rate maximization. We offered lemurs a choice between two food items available immediately and six food items available after a delay. We found that by adjusting the delay to the larger reward, the lemurs were indifferent between the two options at a mean delay of 17 s, ranging from 9 to 25 s. These data are comparable to data collected from common marmosets (Callithrix jacchus). The lemur data were not consistent with models of rate maximization. The addition of lemurs to the list of species tested in these tasks will help uncover the role of life history and socio-ecological factors influencing intertemporal choices. Copyright © 2011 Elsevier B.V. All rights reserved.

Testing process predictions of models of risky choice: a quantitative model comparison approach

PubMed Central

Pachur, Thorsten; Hertwig, Ralph; Gigerenzer, Gerd; Brandstätter, Eduard

2013-01-01

This article presents a quantitative model comparison contrasting the process predictions of two prominent views on risky choice. One view assumes a trade-off between probabilities and outcomes (or non-linear functions thereof) and the separate evaluation of risky options (expectation models). Another view assumes that risky choice is based on comparative evaluation, limited search, aspiration levels, and the forgoing of trade-offs (heuristic models). We derived quantitative process predictions for a generic expectation model and for a specific heuristic model, namely the priority heuristic (Brandstätter et al., 2006), and tested them in two experiments. The focus was on two key features of the cognitive process: acquisition frequencies (i.e., how frequently individual reasons are looked up) and direction of search (i.e., gamble-wise vs. reason-wise). In Experiment 1, the priority heuristic predicted direction of search better than the expectation model (although neither model predicted the acquisition process perfectly); acquisition frequencies, however, were inconsistent with both models. Additional analyses revealed that these frequencies were primarily a function of what Rubinstein (1988) called “similarity.” In Experiment 2, the quantitative model comparison approach showed that people seemed to rely more on the priority heuristic in difficult problems, but to make more trade-offs in easy problems. This finding suggests that risky choice may be based on a mental toolbox of strategies. PMID:24151472

Food Production Worker. Dietetic Support Personnel Achievement Test.

ERIC Educational Resources Information Center

Oklahoma State Dept. of Vocational and Technical Education, Stillwater.

This guide contains a series of multiple-choice items and guidelines to assist instructors in composing criterion-referenced tests for use in the food production worker component of Oklahoma's Dietetic Support Personnel training program. Test items addressing each of the following occupational duty areas are provided: human relations; hygiene and…

Effects of Nicotine on Olfactogustatory Incentives: Preference, Palatability, and Operant Choice Tests

PubMed Central

2013-01-01

Introduction: The use of additives in tobacco may capitalize on the incentive motivational properties of tastes and scents such as mint (menthol), vanilla, and strawberry. These incentives are intended to increase tobacco experimentation, but their salience may also be enhanced by the incentive amplifying effects of nicotine (NIC). The goal of the present studies was to investigate the potential interaction between the incentive amplifying effects of NIC and gustatory incentives. Methods: One of two discriminable tastes (grape or cherry Kool-Aid®; 0.05% wt/vol; unsweetened) was paired with sucrose (20% wt/vol; conditioned stimulus [CS+]) in deionized water, whereas the other taste (CS−) was presented in deionized water. Experiment 1 investigated the effects of NIC pretreatment on preference for the CS+ versus CS− in 2-bottle choice tests. Experiment 2 investigated the effects of NIC on palatability of the CS+ and CS− using orofacial taste reactions. Experiment 3 investigated the effects of NIC on reinforcement by the CS+ and CS− using a concurrent choice operant task. Results: NIC pretreatment robustly increased operant responding for the CS+ but did not alter responding for the CS− in the operant choice task (Experiment 3). However, NIC pretreatment did not alter intake or palatability of the CS+ or CS− (Experiments 1 and 2). Conclusions: NIC increases the reinforcing effects of gustatory incentive stimuli, even though these stimuli were not paired with NIC administration. The findings suggest that adding taste incentives to tobacco products may increase the attractiveness of these products to consumers and the probability of repeated use. PMID:23430737

Best (but oft-forgotten) practices: the multiple problems of multiplicity-whether and how to correct for many statistical tests.

PubMed

Streiner, David L

2015-10-01

Testing many null hypotheses in a single study results in an increased probability of detecting a significant finding just by chance (the problem of multiplicity). Debates have raged over many years with regard to whether to correct for multiplicity and, if so, how it should be done. This article first discusses how multiple tests lead to an inflation of the α level, then explores the following different contexts in which multiplicity arises: testing for baseline differences in various types of studies, having >1 outcome variable, conducting statistical tests that produce >1 P value, taking multiple "peeks" at the data, and unplanned, post hoc analyses (i.e., "data dredging," "fishing expeditions," or "P-hacking"). It then discusses some of the methods that have been proposed for correcting for multiplicity, including single-step procedures (e.g., Bonferroni); multistep procedures, such as those of Holm, Hochberg, and Šidák; false discovery rate control; and resampling approaches. Note that these various approaches describe different aspects and are not necessarily mutually exclusive. For example, resampling methods could be used to control the false discovery rate or the family-wise error rate (as defined later in this article). However, the use of one of these approaches presupposes that we should correct for multiplicity, which is not universally accepted, and the article presents the arguments for and against such "correction." The final section brings together these threads and presents suggestions with regard to when it makes sense to apply the corrections and how to do so. © 2015 American Society for Nutrition.

Modeling hurricane evacuation traffic : testing the gravity and intervening opportunity models as models of destination choice in hurricane evacuation.

DOT National Transportation Integrated Search

2006-09-01

The test was conducted by estimating the models on a portion of evacuation data from South Carolina following Hurricane Floyd, and then observing how well the models reproduced destination choice at the county level on the remaining data. The tests s...

ACER Chemistry Test Item Collection. ACER Chemtic Year 12.

ERIC Educational Resources Information Center

Australian Council for Educational Research, Hawthorn.

The chemistry test item banks contains 225 multiple-choice questions suitable for diagnostic and achievement testing; a three-page teacher's guide; answer key with item facilities; an answer sheet; and a 45-item sample achievement test. Although written for the new grade 12 chemistry course in Victoria, Australia, the items are widely applicable.…

Teaching in the Time of Testing: What Have You Lost?

ERIC Educational Resources Information Center

McCracken, Nancy Mellin; McCracken, Hugh Thomas

2001-01-01

Asks several teachers what they have lost from their teaching or their classroom since the growth in mandated, standardized testing. Considers the ill effects of mandated testing, and names some educational essentials at risk of being lost while testing rules. Discusses what is lost in high-stakes multiple-choice testing of new teachers. (SG)

Computational Precision of Mental Inference as Critical Source of Human Choice Suboptimality.

PubMed

Drugowitsch, Jan; Wyart, Valentin; Devauchelle, Anne-Dominique; Koechlin, Etienne

2016-12-21

Making decisions in uncertain environments often requires combining multiple pieces of ambiguous information from external cues. In such conditions, human choices resemble optimal Bayesian inference, but typically show a large suboptimal variability whose origin remains poorly understood. In particular, this choice suboptimality might arise from imperfections in mental inference rather than in peripheral stages, such as sensory processing and response selection. Here, we dissociate these three sources of suboptimality in human choices based on combining multiple ambiguous cues. Using a novel quantitative approach for identifying the origin and structure of choice variability, we show that imperfections in inference alone cause a dominant fraction of suboptimal choices. Furthermore, two-thirds of this suboptimality appear to derive from the limited precision of neural computations implementing inference rather than from systematic deviations from Bayes-optimal inference. These findings set an upper bound on the accuracy and ultimate predictability of human choices in uncertain environments. Copyright © 2016 Elsevier Inc. All rights reserved.

Auto Mechanics. Criterion-Referenced Test (CRT) Item Bank.

ERIC Educational Resources Information Center

Tannehill, Dana, Ed.

This document contains 546 criterion-referenced multiple choice and true or false test items for a course in auto mechanics. The test item bank is designed to work with both the Vocational Instructional Management System (VIMS) and Vocational Administrative Management System (VAMS) in Missouri. The items are grouped into 35 units covering the…

Multiple phenotype association tests using summary statistics in genome-wide association studies.

PubMed

Liu, Zhonghua; Lin, Xihong

2018-03-01

We study in this article jointly testing the associations of a genetic variant with correlated multiple phenotypes using the summary statistics of individual phenotype analysis from Genome-Wide Association Studies (GWASs). We estimated the between-phenotype correlation matrix using the summary statistics of individual phenotype GWAS analyses, and developed genetic association tests for multiple phenotypes by accounting for between-phenotype correlation without the need to access individual-level data. Since genetic variants often affect multiple phenotypes differently across the genome and the between-phenotype correlation can be arbitrary, we proposed robust and powerful multiple phenotype testing procedures by jointly testing a common mean and a variance component in linear mixed models for summary statistics. We computed the p-values of the proposed tests analytically. This computational advantage makes our methods practically appealing in large-scale GWASs. We performed simulation studies to show that the proposed tests maintained correct type I error rates, and to compare their powers in various settings with the existing methods. We applied the proposed tests to a GWAS Global Lipids Genetics Consortium summary statistics data set and identified additional genetic variants that were missed by the original single-trait analysis. © 2017, The International Biometric Society.

Geography Students Assess Their Learning Using Computer-Marked Tests.

ERIC Educational Resources Information Center

Hogg, Jim

1997-01-01

Reports on a pilot study designed to assess the potential of computer-marked tests for allowing students to monitor their learning. Students' answers to multiple choice tests were fed into a computer that provided a full analysis of their strengths and weaknesses. Students responded favorably to the feedback. (MJP)