Developing multiple-choices test items as tools for measuring the scientific-generic skills on solar system

NASA Astrophysics Data System (ADS)

Bhakti, Satria Seto; Samsudin, Achmad; Chandra, Didi Teguh; Siahaan, Parsaoran

2017-05-01

The aim of research is developing multiple-choices test items as tools for measuring the scientific of generic skills on solar system. To achieve the aim that the researchers used the ADDIE model consisting Of: Analyzing, Design, Development, Implementation, dan Evaluation, all of this as a method research. While The scientific of generic skills limited research to five indicator including: (1) indirect observation, (2) awareness of the scale, (3) inference logic, (4) a causal relation, and (5) mathematical modeling. The participants are 32 students at one of junior high schools in Bandung. The result shown that multiple-choices that are constructed test items have been declared valid by the expert validator, and after the tests show that the matter of developing multiple-choices test items be able to measuring the scientific of generic skills on solar system.

Using Distractor-Driven Standards-Based Multiple-Choice Assessments and Rasch Modeling to Investigate Hierarchies of Chemistry Misconceptions and Detect Structural Problems with Individual Items

ERIC Educational Resources Information Center

Herrmann-Abell, Cari F.; DeBoer, George E.

2011-01-01

Distractor-driven multiple-choice assessment items and Rasch modeling were used as diagnostic tools to investigate students' understanding of middle school chemistry ideas. Ninety-one items were developed according to a procedure that ensured content alignment to the targeted standards and construct validity. The items were administered to 13360…

Dynamic Testing of Analogical Reasoning in 5- to 6-Year-Olds: Multiple-Choice versus Constructed-Response Training Items

ERIC Educational Resources Information Center

Stevenson, Claire E.; Heiser, Willem J.; Resing, Wilma C. M.

2016-01-01

Multiple-choice (MC) analogy items are often used in cognitive assessment. However, in dynamic testing, where the aim is to provide insight into potential for learning and the learning process, constructed-response (CR) items may be of benefit. This study investigated whether training with CR or MC items leads to differences in the strategy…

An Empirical Comparison of DDF Detection Methods for Understanding the Causes of DIF in Multiple-Choice Items

ERIC Educational Resources Information Center

Suh, Youngsuk; Talley, Anna E.

2015-01-01

This study compared and illustrated four differential distractor functioning (DDF) detection methods for analyzing multiple-choice items. The log-linear approach, two item response theory-model-based approaches with likelihood ratio tests, and the odds ratio approach were compared to examine the congruence among the four DDF detection methods.…

Sex Differences in the Tendency to Omit Items on Multiple-Choice Tests: 1980-2000

ERIC Educational Resources Information Center

von Schrader, Sarah; Ansley, Timothy

2006-01-01

Much has been written concerning the potential group differences in responding to multiple-choice achievement test items. This discussion has included references to possible disparities in tendency to omit such test items. When test scores are used for high-stakes decision making, even small differences in scores and rankings that arise from male…

The Effect of the Multiple-Choice Item Format on the Measurement of Knowledge of Language Structure

ERIC Educational Resources Information Center

Currie, Michael; Chiramanee, Thanyapa

2010-01-01

Noting the widespread use of multiple-choice items in tests in English language education in Thailand, this study compared their effect against that of constructed-response items. One hundred and fifty-two university undergraduates took a test of English structure first in constructed-response format, and later in three, stem-equivalent…

Measuring more than we know? An examination of the motivational and situational influences in science achievement

NASA Astrophysics Data System (ADS)

Haydel, Angela Michelle

The purpose of this dissertation was to advance theoretical understanding about fit between the personal resources of individuals and the characteristics of science achievement tasks. Testing continues to be pervasive in schools, yet we know little about how students perceive tests and what they think and feel while they are actually working on test items. This study focused on both the personal (cognitive and motivational) and situational factors that may contribute to individual differences in achievement-related outcomes. 387 eighth grade students first completed a survey including measures of science achievement goals, capability beliefs, efficacy related to multiple-choice items and performance assessments, validity beliefs about multiple-choice items and performance assessments, and other perceptions of these item formats. Students then completed science achievement tests including multiple-choice items and two performance assessments. A sample of students was asked to verbalize both thoughts and feelings as they worked through the test items. These think-alouds were transcribed and coded for evidence of cognitive, metacognitive and motivational engagement. Following each test, all students completed measures of effort, mood, energy level and strategy use during testing. Students reported that performance assessments were more challenging, authentic, interesting and valid than multiple-choice tests. They also believed that comparisons between students were easier using multiple-choice items. Overall, students tried harder, felt better, had higher levels of energy and used more strategies while working on performance assessments. Findings suggested that performance assessments might be more congruent with a mastery achievement goal orientation, while multiple-choice tests might be more congruent with a performance achievement goal orientation. A variable-centered analytic approach including regression analyses provided information about how students, on average, who differed in terms of their teachers' ratings of their science ability, achievement goals, capability beliefs and experiences with science achievement tasks perceived, engaged in, and performed on multiple-choice items and performance assessments. Person-centered analyses provided information about the perceptions, engagement and performance of subgroups of individuals who had different motivational characteristics. Generally, students' personal goals and capability beliefs related more strongly to test perceptions, but not performance, while teacher ratings of ability and test-specific beliefs related to performance.

Assessment of item-writing flaws in multiple-choice questions.

PubMed

Nedeau-Cayo, Rosemarie; Laughlin, Deborah; Rus, Linda; Hall, John

2013-01-01

This study evaluated the quality of multiple-choice questions used in a hospital's e-learning system. Constructing well-written questions is fraught with difficulty, and item-writing flaws are common. Study results revealed that most items contained flaws and were written at the knowledge/comprehension level. Few items had linked objectives, and no association was found between the presence of objectives and flaws. Recommendations include education for writing test questions.

Psychometrics of Multiple Choice Questions with Non-Functioning Distracters: Implications to Medical Education.

PubMed

Deepak, Kishore K; Al-Umran, Khalid Umran; AI-Sheikh, Mona H; Dkoli, B V; Al-Rubaish, Abdullah

2015-01-01

The functionality of distracters in a multiple choice question plays a very important role. We examined the frequency and impact of functioning and non-functioning distracters on psychometric properties of 5-option items in clinical disciplines. We analyzed item statistics of 1115 multiple choice questions from 15 summative assessments of undergraduate medical students and classified the items into five groups by their number of non-functioning distracters. We analyzed the effect of varying degree of non-functionality ranging from 0 to 4, on test reliability, difficulty index, discrimination index and point biserial correlation. The non-functionality of distracters inversely affected the test reliability and quality of items in a predictable manner. The non-functioning distracters made the items easier and lowered the discrimination index significantly. Three non-functional distracters in a 5-option MCQ significantly affected all psychometric properties (p < 0.5). The corrected point biserial correlation revealed that the items with 3 functional options were psychometrically as effective as 5-option items. Our study reveals that a multiple choice question with 3 functional options provides lower most limit of item format that has adequate psychometric property. The test containing items with less number of functioning options have significantly lower reliability. The distracter function analysis and revision of nonfunctioning distracters can serve as important methods to improve the psychometrics and reliability of assessment.

The Effect of SSM Grading on Reliability When Residual Items Have No Discriminating Power.

ERIC Educational Resources Information Center

Kane, Michael T.; Moloney, James M.

Gilman and Ferry have shown that when the student's score on a multiple choice test is the total number of responses necessary to get all items correct, substantial increases in reliability can occur. In contrast, similar procedures giving partial credit on multiple choice items have resulted in relatively small gains in reliability. The analysis…

Are Faculty Predictions or Item Taxonomies Useful for Estimating the Outcome of Multiple-Choice Examinations?

ERIC Educational Resources Information Center

Kibble, Jonathan D.; Johnson, Teresa

2011-01-01

The purpose of this study was to evaluate whether multiple-choice item difficulty could be predicted either by a subjective judgment by the question author or by applying a learning taxonomy to the items. Eight physiology faculty members teaching an upper-level undergraduate human physiology course consented to participate in the study. The…

A Method for Imputing Response Options for Missing Data on Multiple-Choice Assessments

ERIC Educational Resources Information Center

Wolkowitz, Amanda A.; Skorupski, William P.

2013-01-01

When missing values are present in item response data, there are a number of ways one might impute a correct or incorrect response to a multiple-choice item. There are significantly fewer methods for imputing the actual response option an examinee may have provided if he or she had not omitted the item either purposely or accidentally. This…

The Development of Multiple-Choice Items Consistent with the AP Chemistry Curriculum Framework to More Accurately Assess Deeper Understanding

ERIC Educational Resources Information Center

Domyancich, John M.

2014-01-01

Multiple-choice questions are an important part of large-scale summative assessments, such as the advanced placement (AP) chemistry exam. However, past AP chemistry exam items often lacked the ability to test conceptual understanding and higher-order cognitive skills. The redesigned AP chemistry exam shows a distinctive shift in item types toward…

Pick-N Multiple Choice-Exams: A Comparison of Scoring Algorithms

ERIC Educational Resources Information Center

Bauer, Daniel; Holzer, Matthias; Kopp, Veronika; Fischer, Martin R.

2011-01-01

To compare different scoring algorithms for Pick-N multiple correct answer multiple-choice (MC) exams regarding test reliability, student performance, total item discrimination and item difficulty. Data from six 3rd year medical students' end of term exams in internal medicine from 2005 to 2008 at Munich University were analysed (1,255 students,…

Developing Multiple Choice Tests: Tips & Techniques

ERIC Educational Resources Information Center

McCowan, Richard J.

1999-01-01

Item writing is a major responsibility of trainers. Too often, qualified staff who prepare lessons carefully and teach conscientiously use inadequate tests that do not validly reflect the true level of trainee achievement. This monograph describes techniques for constructing multiple-choice items that measure student performance accurately. It…

Set of Criteria for Efficiency of the Process Forming the Answers to Multiple-Choice Test Items

ERIC Educational Resources Information Center

Rybanov, Alexander Aleksandrovich

2013-01-01

Is offered the set of criteria for assessing efficiency of the process forming the answers to multiple-choice test items. To increase accuracy of computer-assisted testing results, it is suggested to assess dynamics of the process of forming the final answer using the following factors: loss of time factor and correct choice factor. The model…

Modeling Incorrect Responses to Multiple-Choice Items with Multilinear Formula Score Theory.

ERIC Educational Resources Information Center

Drasgow, Fritz; And Others

This paper addresses the information revealed in incorrect option selection on multiple choice items. Multilinear Formula Scoring (MFS), a theory providing methods for solving psychological measurement problems of long standing, is first used to estimate option characteristic curves for the Armed Services Vocational Aptitude Battery Arithmetic…

Cognitive Diagnostic Models for Tests with Multiple-Choice and Constructed-Response Items

ERIC Educational Resources Information Center

Kuo, Bor-Chen; Chen, Chun-Hua; Yang, Chih-Wei; Mok, Magdalena Mo Ching

2016-01-01

Traditionally, teachers evaluate students' abilities via their total test scores. Recently, cognitive diagnostic models (CDMs) have begun to provide information about the presence or absence of students' skills or misconceptions. Nevertheless, CDMs are typically applied to tests with multiple-choice (MC) items, which provide less diagnostic…

Difficulty and Discriminability of Introductory Psychology Test Items.

ERIC Educational Resources Information Center

Scialfa, Charles; Legare, Connie; Wenger, Larry; Dingley, Louis

2001-01-01

Analyzes multiple-choice questions provided in test banks for introductory psychology textbooks. Study 1 offered a consistent picture of the objective difficulty of multiple-choice tests for introductory psychology students, while both studies 1 and 2 indicated that test items taken from commercial test banks have poor psychometric properties.…

Cognitive Validity: Can Multiple-Choice Items Tap Historical Thinking Processes?

ERIC Educational Resources Information Center

Smith, Mark D.

2017-01-01

Cognitive validity examines the relationship between what an assessment aims to measure and what it actually elicits from test takers. The present study examined whether multiple-choice items from the National Assessment of Educational Progress (NAEP) grade 12 U.S. history exam elicited the historical thinking processes they were designed to…

Application of a Multidimensional Nested Logit Model to Multiple-Choice Test Items

ERIC Educational Resources Information Center

Bolt, Daniel M.; Wollack, James A.; Suh, Youngsuk

2012-01-01

Nested logit models have been presented as an alternative to multinomial logistic models for multiple-choice test items (Suh and Bolt in "Psychometrika" 75:454-473, 2010) and possess a mathematical structure that naturally lends itself to evaluating the incremental information provided by attending to distractor selection in scoring. One potential…

English 30, Part B: Reading. Questions Booklet. Grade 12 Diploma Examination, January 1997.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton. Student Evaluation Branch.

Intended for students taking the Grade 12 Diploma Examinations in English 30, this "questions booklet" presents 70 multiple choice test items based on 8 reading selections in the accompanying readings booklet. After instructions for students, the booklet presents the multiple choice items which test students' comprehension of the poetry,…

The Effect of Images on Item Statistics in Multiple Choice Anatomy Examinations

ERIC Educational Resources Information Center

Notebaert, Andrew J.

2017-01-01

Although multiple choice examinations are often used to test anatomical knowledge, these often forgo the use of images in favor of text-based questions and answers. Because anatomy is reliant on visual resources, examinations using images should be used when appropriate. This study was a retrospective analysis of examination items that were text…

Free-Response and Multiple-Choice Items: Measures of the Same Ability?

ERIC Educational Resources Information Center

Bennett, Randy Elliot; And Others

This study examined the relationship of multiple-choice and free-response items contained on the College Board's Advanced Placement Computer Science (APCS) examination. Subjects were two samples of 1,000 randomly drawn from the population of 7,372 high school students taking the 1988 examination of the APCS "AB" form. Most were high…

Writing Multiple Choice Outcome Questions to Assess Knowledge and Competence.

PubMed

Brady, Erik D

2015-11-01

Few articles contemplate the need for good guidance in question item-writing in the continuing education (CE) space. Although many of the core principles of sound item design translate to the CE health education team, the need exists for specific examples for nurse educators that clearly describe how to measure changes in competence and knowledge using multiple choice items. In this article, some keys points and specific examples for nursing CE providers are shared. Copyright 2015, SLACK Incorporated.

Understanding Rasch Measurement: Distractors with Information in Multiple Choice Items: A Rationale Based on the Rasch Model

ERIC Educational Resources Information Center

Andrich, David; Styles, Irene

2011-01-01

There is a substantial literature on attempts to obtain information on the proficiency of respondents from distractors in multiple choice items. Information in a distractor implies that a person who chooses that distractor has greater proficiency than if the person chose another distractor with no information. A further implication is that the…

The Testing Methods and Gender Differences in Multiple-Choice Assessment

NASA Astrophysics Data System (ADS)

Ng, Annie W. Y.; Chan, Alan H. S.

2009-10-01

This paper provides a comprehensive review of the multiple-choice assessment in the past two decades for facilitating people to conduct effective testing in various subject areas. It was revealed that a variety of multiple-choice test methods viz. conventional multiple-choice, liberal multiple-choice, elimination testing, confidence marking, probability testing, and order-of-preference scheme are available for use in assessing subjects' knowledge and decision ability. However, the best multiple-choice test method for use has not yet been identified. The review also indicated that the existence of gender differences in multiple-choice task performance might be due to the test area, instruction/scoring condition, and item difficulty.

Measures of Partial Knowledge and Unexpected Responses in Multiple-Choice Tests

ERIC Educational Resources Information Center

Chang, Shao-Hua; Lin, Pei-Chun; Lin, Zih-Chuan

2007-01-01

This study investigates differences in the partial scoring performance of examinees in elimination testing and conventional dichotomous scoring of multiple-choice tests implemented on a computer-based system. Elimination testing that uses the same set of multiple-choice items rewards examinees with partial knowledge over those who are simply…

On the Equivalence of Constructed-Response and Multiple-Choice Tests.

ERIC Educational Resources Information Center

Traub, Ross E.; Fisher, Charles W.

Two sets of mathematical reasoning and two sets of verbal comprehension items were cast into each of three formats--constructed response, standard multiple-choice, and Coombs multiple-choice--in order to assess whether tests with indentical content but different formats measure the same attribute, except for possible differences in error variance…

Comparison of Difficulties and Reliabilities of Math-Completion and Multiple-Choice Item Formats.

ERIC Educational Resources Information Center

Oosterhof, Albert C.; Coats, Pamela K.

Instructors who develop classroom examinations that require students to provide a numerical response to a mathematical problem are often very concerned about the appropriateness of the multiple-choice format. The present study augments previous research relevant to this concern by comparing the difficulty and reliability of multiple-choice and…

Does the Position of Response Options in Multiple-Choice Tests Matter?

ERIC Educational Resources Information Center

Hohensinn, Christine; Baghaei, Purya

2017-01-01

In large scale multiple-choice (MC) tests alternate forms of a test may be developed to prevent cheating by changing the order of items or by changing the position of the response options. The assumption is that since the content of the test forms are the same the order of items or the positions of the response options do not have any effect on…

A Stratified Study of Students' Understanding of Basic Optics Concepts in Different Contexts Using Two-Tier Multiple-Choice Items

ERIC Educational Resources Information Center

Chu, Hye-Eun; Treagust, David F.; Chandrasegaran, A. L.

2009-01-01

A large scale study involving 1786 year 7-10 Korean students from three school districts in Seoul was undertaken to evaluate their understanding of basic optics concepts using a two-tier multiple-choice diagnostic instrument consisting of four pairs of items, each of which evaluated the same concept in two different contexts. The instrument, which…

Mixed-Format Test Score Equating: Effect of Item-Type Multidimensionality, Length and Composition of Common-Item Set, and Group Ability Difference

ERIC Educational Resources Information Center

Wang, Wei

2013-01-01

Mixed-format tests containing both multiple-choice (MC) items and constructed-response (CR) items are now widely used in many testing programs. Mixed-format tests often are considered to be superior to tests containing only MC items although the use of multiple item formats leads to measurement challenges in the context of equating conducted under…

Preliminary Findings on the Computer-Administered Multiple-Choice Online Causal Comprehension Assessment, a Diagnostic Reading Comprehension Test

ERIC Educational Resources Information Center

Davison, Mark L.; Biancarosa, Gina; Carlson, Sarah E.; Seipel, Ben; Liu, Bowen

2018-01-01

The computer-administered Multiple-Choice Online Causal Comprehension Assessment (MOCCA) for Grades 3 to 5 has an innovative, 40-item multiple-choice structure in which each distractor corresponds to a comprehension process upon which poor comprehenders have been shown to rely. This structure requires revised thinking about measurement issues…

Sustainable Assessment for Large Science Classes: Non-Multiple Choice, Randomised Assignments through a Learning Management System

ERIC Educational Resources Information Center

Schultz, Madeleine

2011-01-01

This paper reports on the development of a tool that generates randomised, non-multiple choice assessment within the BlackBoard Learning Management System interface. An accepted weakness of multiple-choice assessment is that it cannot elicit learning outcomes from upper levels of Biggs' SOLO taxonomy. However, written assessment items require…

Models for Scoring Missing Responses to Multiple-Choice Items. Program Statistics Research Technical Report No. 94-1.

ERIC Educational Resources Information Center

Longford, Nicholas T.

This study is a critical evaluation of the roles for coding and scoring of missing responses to multiple-choice items in educational tests. The focus is on tests in which the test-takers have little or no motivation; in such tests omitting and not reaching (as classified by the currently adopted operational rules) is quite frequent. Data from the…

Modeling Polytomous Item Responses Using Simultaneously Estimated Multinomial Logistic Regression Models

ERIC Educational Resources Information Center

Anderson, Carolyn J.; Verkuilen, Jay; Peyton, Buddy L.

2010-01-01

Survey items with multiple response categories and multiple-choice test questions are ubiquitous in psychological and educational research. We illustrate the use of log-multiplicative association (LMA) models that are extensions of the well-known multinomial logistic regression model for multiple dependent outcome variables to reanalyze a set of…

Reducing the Need for Guesswork in Multiple-Choice Tests

ERIC Educational Resources Information Center

Bush, Martin

2015-01-01

The humble multiple-choice test is very widely used within education at all levels, but its susceptibility to guesswork makes it a suboptimal assessment tool. The reliability of a multiple-choice test is partly governed by the number of items it contains; however, longer tests are more time consuming to take, and for some subject areas, it can be…

Does Linking Mixed-Format Tests Using a Multiple-Choice Anchor Produce Comparable Results for Male and Female Subgroups? Research Report. ETS RR-11-44

ERIC Educational Resources Information Center

Kim, Sooyeon; Walker, Michael E.

2011-01-01

This study examines the use of subpopulation invariance indices to evaluate the appropriateness of using a multiple-choice (MC) item anchor in mixed-format tests, which include both MC and constructed-response (CR) items. Linking functions were derived in the nonequivalent groups with anchor test (NEAT) design using an MC-only anchor set for 4…

Evaluation of five guidelines for option development in multiple-choice item-writing.

PubMed

Martínez, Rafael J; Moreno, Rafael; Martín, Irene; Trigo, M Eva

2009-05-01

This paper evaluates certain guidelines for writing multiple-choice test items. The analysis of the responses of 5013 subjects to 630 items from 21 university classroom achievement tests suggests that an option should not differ in terms of heterogeneous content because such error has a slight but harmful effect on item discrimination. This also occurs with the "None of the above" option when it is the correct one. In contrast, results do not show the supposedly negative effects of a different-length option, the use of specific determiners, or the use of the "All of the above" option, which not only decreases difficulty but also improves discrimination when it is the correct option.

Multiple choice questions can be designed or revised to challenge learners' critical thinking.

PubMed

Tractenberg, Rochelle E; Gushta, Matthew M; Mulroney, Susan E; Weissinger, Peggy A

2013-12-01

Multiple choice (MC) questions from a graduate physiology course were evaluated by cognitive-psychology (but not physiology) experts, and analyzed statistically, in order to test the independence of content expertise and cognitive complexity ratings of MC items. Integration of higher order thinking into MC exams is important, but widely known to be challenging-perhaps especially when content experts must think like novices. Expertise in the domain (content) may actually impede the creation of higher-complexity items. Three cognitive psychology experts independently rated cognitive complexity for 252 multiple-choice physiology items using a six-level cognitive complexity matrix that was synthesized from the literature. Rasch modeling estimated item difficulties. The complexity ratings and difficulty estimates were then analyzed together to determine the relative contributions (and independence) of complexity and difficulty to the likelihood of correct answers on each item. Cognitive complexity was found to be statistically independent of difficulty estimates for 88 % of items. Using the complexity matrix, modifications were identified to increase some item complexities by one level, without affecting the item's difficulty. Cognitive complexity can effectively be rated by non-content experts. The six-level complexity matrix, if applied by faculty peer groups trained in cognitive complexity and without domain-specific expertise, could lead to improvements in the complexity targeted with item writing and revision. Targeting higher order thinking with MC questions can be achieved without changing item difficulties or other test characteristics, but this may be less likely if the content expert is left to assess items within their domain of expertise.

Evaluating the Psychometric Characteristics of Generated Multiple-Choice Test Items

ERIC Educational Resources Information Center

Gierl, Mark J.; Lai, Hollis; Pugh, Debra; Touchie, Claire; Boulais, André-Philippe; De Champlain, André

2016-01-01

Item development is a time- and resource-intensive process. Automatic item generation integrates cognitive modeling with computer technology to systematically generate test items. To date, however, items generated using cognitive modeling procedures have received limited use in operational testing situations. As a result, the psychometric…

Accommodations for Multiple Choice Tests

ERIC Educational Resources Information Center

Trammell, Jack

2011-01-01

Students with learning or learning-related disabilities frequently struggle with multiple choice assessments due to difficulty discriminating between items, filtering out distracters, and framing a mental best answer. This Practice Brief suggests accommodations and strategies that disability service providers can utilize in conjunction with…

Developing an Array Binary Code Assessment Rubric for Multiple- Choice Questions Using Item Arrays and Binary-Coded Responses

ERIC Educational Resources Information Center

Haro, Elizabeth K.; Haro, Luis S.

2014-01-01

The multiple-choice question (MCQ) is the foundation of knowledge assessment in K-12, higher education, and standardized entrance exams (including the GRE, MCAT, and DAT). However, standard MCQ exams are limited with respect to the types of questions that can be asked when there are only five choices. MCQs offering additional choices more…

The Australian Science Item Bank Project

ERIC Educational Resources Information Center

Kings, Clive B.; Cropley, Murray C.

1974-01-01

Describes the development of multiple-choice test item bank for grade ten science by the Australian Council for Educational Research. Other item banks are also being developed at the grade ten level in mathematics and social science. (RH)

The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments.

PubMed

Tarrant, Marie; Knierim, Aimee; Hayes, Sasha K; Ware, James

2006-12-01

Multiple-choice questions are a common assessment method in nursing examinations. Few nurse educators, however, have formal preparation in constructing multiple-choice questions. Consequently, questions used in baccalaureate nursing assessments often contain item-writing flaws, or violations to accepted item-writing guidelines. In one nursing department, 2770 MCQs were collected from tests and examinations administered over a five-year period from 2001 to 2005. Questions were evaluated for 19 frequently occurring item-writing flaws, for cognitive level, for question source, and for the distribution of correct answers. Results show that almost half (46.2%) of the questions contained violations of item-writing guidelines and over 90% were written at low cognitive levels. Only a small proportion of questions were teacher generated (14.1%), while 36.2% were taken from testbanks and almost half (49.4%) had no source identified. MCQs written at a lower cognitive level were significantly more likely to contain item-writing flaws. While there was no relationship between the source of the question and item-writing flaws, teacher-generated questions were more likely to be written at higher cognitive levels (p<0.001). Correct answers were evenly distributed across all four options and no bias was noted in the placement of correct options. Further training in item-writing is recommended for all faculty members who are responsible for developing tests. Pre-test review and quality assessment is also recommended to reduce the occurrence of item-writing flaws and to improve the quality of test questions.

Exploring problem solving strategies on multiple-choice science items: Comparing native Spanish-speaking English Language Learners and mainstream monolinguals

NASA Astrophysics Data System (ADS)

Kachchaf, Rachel Rae

The purpose of this study was to compare how English language learners (ELLs) and monolingual English speakers solved multiple-choice items administered with and without a new form of testing accommodation---vignette illustration (VI). By incorporating theories from second language acquisition, bilingualism, and sociolinguistics, this study was able to gain more accurate and comprehensive input into the ways students interacted with items. This mixed methods study used verbal protocols to elicit the thinking processes of thirty-six native Spanish-speaking English language learners (ELLs), and 36 native-English speaking non-ELLs when solving multiple-choice science items. Results from both qualitative and quantitative analyses show that ELLs used a wider variety of actions oriented to making sense of the items than non-ELLs. In contrast, non-ELLs used more problem solving strategies than ELLs. There were no statistically significant differences in student performance based on the interaction of presence of illustration and linguistic status or the main effect of presence of illustration. However, there were significant differences based on the main effect of linguistic status. An interaction between the characteristics of the students, the items, and the illustrations indicates considerable heterogeneity in the ways in which students from both linguistic groups think about and respond to science test items. The results of this study speak to the need for more research involving ELLs in the process of test development to create test items that do not require ELLs to carry out significantly more actions to make sense of the item than monolingual students.

V-TECS Criterion-Referenced Test Item Bank for Radiologic Technology Occupations.

ERIC Educational Resources Information Center

Reneau, Fred; And Others

This Vocational-Technical Education Consortium of States (V-TECS) criterion-referenced test item bank provides 696 multiple-choice items and 33 matching items for radiologic technology occupations. These job titles are included: radiologic technologist, chief; radiologic technologist; nuclear medicine technologist; radiation therapy technologist;…

Item Analysis in Introductory Economics Testing.

ERIC Educational Resources Information Center

Tinari, Frank D.

1979-01-01

Computerized analysis of multiple choice test items is explained. Examples of item analysis applications in the introductory economics course are discussed with respect to three objectives: to evaluate learning; to improve test items; and to help improve classroom instruction. Problems, costs and benefits of the procedures are identified. (JMD)

Item Estimates under Low-Stakes Conditions: How Should Omits Be Treated?

ERIC Educational Resources Information Center

DeMars, Christine

Using data from a pilot test of science and math from students in 30 high schools, item difficulties were estimated with a one-parameter model (partial-credit model for the multi-point items). Some items were multiple-choice items, and others were constructed-response items (open-ended). Four sets of estimates were obtained: estimates for males…

ACER Chemistry Test Item Collection. ACER Chemtic Year 12.

ERIC Educational Resources Information Center

Australian Council for Educational Research, Hawthorn.

The chemistry test item banks contains 225 multiple-choice questions suitable for diagnostic and achievement testing; a three-page teacher's guide; answer key with item facilities; an answer sheet; and a 45-item sample achievement test. Although written for the new grade 12 chemistry course in Victoria, Australia, the items are widely applicable.…

The memorial consequences of multiple-choice testing.

PubMed

Marsh, Elizabeth J; Roediger, Henry L; Bjork, Robert A; Bjork, Elizabeth L

2007-04-01

The present article addresses whether multiple-choice tests may change knowledge even as they attempt to measure it. Overall, taking a multiple-choice test boosts performance on later tests, as compared with non-tested control conditions. This benefit is not limited to simple definitional questions, but holds true for SAT II questions and for items designed to tap concepts at a higher level in Bloom's (1956) taxonomy of educational objectives. Students, however, can also learn false facts from multiple-choice tests; testing leads to persistence of some multiple-choice lures on later general knowledge tests. Such persistence appears due to faulty reasoning rather than to an increase in the familiarity of lures. Even though students may learn false facts from multiple-choice tests, the positive effects of testing outweigh this cost.

Fixed or mixed: a comparison of three, four and mixed-option multiple-choice tests in a Fetal Surveillance Education Program

PubMed Central

2013-01-01

Background Despite the widespread use of multiple-choice assessments in medical education assessment, current practice and published advice concerning the number of response options remains equivocal. This article describes an empirical study contrasting the quality of three 60 item multiple-choice test forms within the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG) Fetal Surveillance Education Program (FSEP). The three forms are described below. Methods The first form featured four response options per item. The second form featured three response options, having removed the least functioning option from each item in the four-option counterpart. The third test form was constructed by retaining the best performing version of each item from the first two test forms. It contained both three and four option items. Results Psychometric and educational factors were taken into account in formulating an approach to test construction for the FSEP. The four-option test performed better than the three-option test overall, but some items were improved by the removal of options. The mixed-option test demonstrated better measurement properties than the fixed-option tests, and has become the preferred test format in the FSEP program. The criteria used were reliability, errors of measurement and fit to the item response model. Conclusions The position taken is that decisions about the number of response options be made at the item level, with plausible options being added to complete each item on both psychometric and educational grounds rather than complying with a uniform policy. The point is to construct the better performing item in providing the best psychometric and educational information. PMID:23453056

Fixed or mixed: a comparison of three, four and mixed-option multiple-choice tests in a Fetal Surveillance Education Program.

PubMed

Zoanetti, Nathan; Beaves, Mark; Griffin, Patrick; Wallace, Euan M

2013-03-04

Despite the widespread use of multiple-choice assessments in medical education assessment, current practice and published advice concerning the number of response options remains equivocal. This article describes an empirical study contrasting the quality of three 60 item multiple-choice test forms within the Royal Australian and New Zealand College of Obstetricians and Gynaecologists (RANZCOG) Fetal Surveillance Education Program (FSEP). The three forms are described below. The first form featured four response options per item. The second form featured three response options, having removed the least functioning option from each item in the four-option counterpart. The third test form was constructed by retaining the best performing version of each item from the first two test forms. It contained both three and four option items. Psychometric and educational factors were taken into account in formulating an approach to test construction for the FSEP. The four-option test performed better than the three-option test overall, but some items were improved by the removal of options. The mixed-option test demonstrated better measurement properties than the fixed-option tests, and has become the preferred test format in the FSEP program. The criteria used were reliability, errors of measurement and fit to the item response model. The position taken is that decisions about the number of response options be made at the item level, with plausible options being added to complete each item on both psychometric and educational grounds rather than complying with a uniform policy. The point is to construct the better performing item in providing the best psychometric and educational information.

The Effects of Judgment-Based Stratum Classifications on the Efficiency of Stratum Scored CATs.

ERIC Educational Resources Information Center

Finney, Sara J.; Smith, Russell W.; Wise, Steven L.

Two operational item pools were used to investigate the performance of stratum computerized adaptive tests (CATs) when items were assigned to strata based on empirical estimates of item difficulty or human judgments of item difficulty. Items from the first data set consisted of 54 5-option multiple choice items from a form of the ACT mathematics…

High School Students' Concepts of Acids and Bases.

ERIC Educational Resources Information Center

Ross, Bertram H. B.

An investigation of Ontario high school students' understanding of acids and bases with quantitative and qualitative methods revealed misconceptions. A concept map, based on the objectives of the Chemistry Curriculum Guideline, generated multiple-choice items and interview questions. The multiple-choice test was administered to 34 grade 12…

Guide to Developing High-Quality, Reliable, and Valid Multiple-Choice Assessments

ERIC Educational Resources Information Center

Towns, Marcy H.

2014-01-01

Chemistry faculty members are highly skilled in obtaining, analyzing, and interpreting physical measurements, but often they are less skilled in measuring student learning. This work provides guidance for chemistry faculty from the research literature on multiple-choice item development in chemistry. Areas covered include content, stem, and…

Australian Chemistry Test Item Bank: Years 11 & 12. Volume 1.

ERIC Educational Resources Information Center

Commons, C., Ed.; Martin, P., Ed.

Volume 1 of the Australian Chemistry Test Item Bank, consisting of two volumes, contains nearly 2000 multiple-choice items related to the chemistry taught in Year 11 and Year 12 courses in Australia. Items which were written during 1979 and 1980 were initially published in the "ACER Chemistry Test Item Collection" and in the "ACER…

Australian Chemistry Test Item Bank: Years 11 and 12. Volume 2.

ERIC Educational Resources Information Center

Commons, C., Ed.; Martin, P., Ed.

The second volume of the Australian Chemistry Test Item Bank, consisting of two volumes, contains nearly 2000 multiple-choice items related to the chemistry taught in Year 11 and Year 12 courses in Australia. Items which were written during 1979 and 1980 were initially published in the "ACER Chemistry Test Item Collection" and in the…

Getting Lucky: How Guessing Threatens the Validity of Performance Classifications

ERIC Educational Resources Information Center

Foley, Brett P.

2016-01-01

There is always a chance that examinees will answer multiple choice (MC) items correctly by guessing. Design choices in some modern exams have created situations where guessing at random through the full exam--rather than only for a subset of items where the examinee does not know the answer--can be an effective strategy to pass the exam. This…

Application of a Utility Analysis to Evaluate a Novel Assessment Tool for Clinically Oriented Physiology and Pharmacology

ERIC Educational Resources Information Center

Cramer, Nicholas; Asmar, Abdo; Gorman, Laurel; Gros, Bernard; Harris, David; Howard, Thomas; Hussain, Mujtaba; Salazar, Sergio; Kibble, Jonathan D.

2016-01-01

Multiple-choice questions are a gold-standard tool in medical school for assessment of knowledge and are the mainstay of licensing examinations. However, multiple-choice questions items can be criticized for lacking the ability to test higher-order learning or integrative thinking across multiple disciplines. Our objective was to develop a novel…

Aligning Items and Achievement Levels: A Study Comparing Expert Judgments

ERIC Educational Resources Information Center

Kaliski, Pamela; Huff, Kristen; Barry, Carol

2011-01-01

For educational achievement tests that employ multiple-choice (MC) items and aim to reliably classify students into performance categories, it is critical to design MC items that are capable of discriminating student performance according to the stated achievement levels. This is accomplished, in part, by clearly understanding how item design…

The Applicability of Interactive Item Templates in Varied Knowledge Types

ERIC Educational Resources Information Center

Koong, Chorng-Shiuh; Wu, Chi-Ying

2011-01-01

A well-edited assessment can enhance student's learning motives. Applicability of items, which includes item content and template, plays a crucial role in authoring a good assessment. Templates in discussion contain not only conventional true & false, multiple choice, completion item and short answer but also of those interactive ones. Methods…

ACER Chemistry Test Item Collection (ACER CHEMTIC Year 12 Supplement).

ERIC Educational Resources Information Center

Australian Council for Educational Research, Hawthorn.

This publication contains 317 multiple-choice chemistry test items related to topics covered in the Victorian (Australia) Year 12 chemistry course. It allows teachers access to a range of items suitable for diagnostic and achievement purposes, supplementing the ACER Chemistry Test Item Collection--Year 12 (CHEMTIC). The topics covered are: organic…

Electronics. Criterion-Referenced Test (CRT) Item Bank.

ERIC Educational Resources Information Center

Davis, Diane, Ed.

This document contains 519 criterion-referenced multiple choice and true or false test items for a course in electronics. The test item bank is designed to work with both the Vocational Instructional Management System (VIMS) and the Vocational Administrative Management System (VAMS) in Missouri. The items are grouped into 15 units covering the…

Auto Mechanics. Criterion-Referenced Test (CRT) Item Bank.

ERIC Educational Resources Information Center

Tannehill, Dana, Ed.

This document contains 546 criterion-referenced multiple choice and true or false test items for a course in auto mechanics. The test item bank is designed to work with both the Vocational Instructional Management System (VIMS) and Vocational Administrative Management System (VAMS) in Missouri. The items are grouped into 35 units covering the…

Missouri Assessment Program (MAP), Spring 2000: Elementary Health/Physical Education, Released Items, Grade 5.

ERIC Educational Resources Information Center

Missouri State Dept. of Elementary and Secondary Education, Jefferson City.

This document presents 10 released items from the Health/Physical Education Missouri Assessment Program (MAP) test given in the spring of 2000 to fifth graders. Items from the test sessions include: selected-response (multiple choice), constructed-response, and a performance event. The selected-response items consist of individual questions…

Diagnostic Opportunities Using Rasch Measurement in the Context of a Misconceptions-Based Physical Science Assessment

ERIC Educational Resources Information Center

Wind, Stefanie A.; Gale, Jessica D.

2015-01-01

Multiple-choice (MC) items that are constructed such that distractors target known misconceptions for a particular domain provide useful diagnostic information about student misconceptions (Herrmann-Abell & DeBoer, 2011, 2014; Sadler, 1998). Item response theory models can be used to examine misconceptions distractor-driven multiple-choice…

Development of multiple choice pictorial test for measuring the dimensions of knowledge

NASA Astrophysics Data System (ADS)

Nahadi, Siswaningsih, Wiwi; Erna

2017-05-01

This study aims to develop a multiple choice pictorial test as a tool to measure dimension of knowledge in chemical equilibrium subject. The method used is Research and Development and validation that was conducted in the preliminary studies and model development. The product is multiple choice pictorial test. The test was developed by 22 items and tested to 64 high school students in XII grade. The quality of test was determined by value of validity, reliability, difficulty index, discrimination power, and distractor effectiveness. The validity of test was determined by CVR calculation using 8 validators (4 university teachers and 4 high school teachers) with average CVR value 0,89. The reliability of test has very high category with value 0,87. Discrimination power of items with a very good category is 32%, 59% as good category, and 20% as sufficient category. This test has a varying level of difficulty, item with difficult category is 23%, the medium category is 50%, and the easy category is 27%. The distractor effectiveness of items with a very poor category is 1%, poor category is 1%, medium category is 4%, good category is 39%, and very good category is 55%. The dimension of knowledge that was measured consist of factual knowledge, conceptual knowledge, and procedural knowledge. Based on the questionnaire, students responded quite well to the developed test and most of the students like this kind of multiple choice pictorial test that include picture as evaluation tool compared to the naration tests was dominated by text.

Assessing Scientific Practices Using Machine-Learning Methods: How Closely Do They Match Clinical Interview Performance?

NASA Astrophysics Data System (ADS)

Beggrow, Elizabeth P.; Ha, Minsu; Nehm, Ross H.; Pearl, Dennis; Boone, William J.

2014-02-01

The landscape of science education is being transformed by the new Framework for Science Education (National Research Council, A framework for K-12 science education: practices, crosscutting concepts, and core ideas. The National Academies Press, Washington, DC, 2012), which emphasizes the centrality of scientific practices—such as explanation, argumentation, and communication—in science teaching, learning, and assessment. A major challenge facing the field of science education is developing assessment tools that are capable of validly and efficiently evaluating these practices. Our study examined the efficacy of a free, open-source machine-learning tool for evaluating the quality of students' written explanations of the causes of evolutionary change relative to three other approaches: (1) human-scored written explanations, (2) a multiple-choice test, and (3) clinical oral interviews. A large sample of undergraduates (n = 104) exposed to varying amounts of evolution content completed all three assessments: a clinical oral interview, a written open-response assessment, and a multiple-choice test. Rasch analysis was used to compute linear person measures and linear item measures on a single logit scale. We found that the multiple-choice test displayed poor person and item fit (mean square outfit >1.3), while both oral interview measures and computer-generated written response measures exhibited acceptable fit (average mean square outfit for interview: person 0.97, item 0.97; computer: person 1.03, item 1.06). Multiple-choice test measures were more weakly associated with interview measures (r = 0.35) than the computer-scored explanation measures (r = 0.63). Overall, Rasch analysis indicated that computer-scored written explanation measures (1) have the strongest correspondence to oral interview measures; (2) are capable of capturing students' normative scientific and naive ideas as accurately as human-scored explanations, and (3) more validly detect understanding than the multiple-choice assessment. These findings demonstrate the great potential of machine-learning tools for assessing key scientific practices highlighted in the new Framework for Science Education.

Item Reliabilities for a Family of Answer-Until-Correct (AUC) Scoring Rules.

ERIC Educational Resources Information Center

Kane, Michael T.; Moloney, James M.

The Answer-Until-Correct (AUC) procedure has been proposed in order to increase the reliability of multiple-choice items. A model for examinees' behavior when they must respond to each item until they answer it correctly is presented. An expression for the reliability of AUC items, as a function of the characteristics of the item and the scoring…

A Model-Based Method for Content Validation of Automatically Generated Test Items

ERIC Educational Resources Information Center

Zhang, Xinxin; Gierl, Mark

2016-01-01

The purpose of this study is to describe a methodology to recover the item model used to generate multiple-choice test items with a novel graph theory approach. Beginning with the generated test items and working backward to recover the original item model provides a model-based method for validating the content used to automatically generate test…

Developing a Placement Exam for Spanish Heritage Language Learners: Item Analysis and Learner Characteristics

ERIC Educational Resources Information Center

Wilson, Damian Vergara

2012-01-01

This paper illustrates a method of item analysis used to identify discriminating multiple-choice items in placement data. The data come from two rounds of pilots given to both SHL students and Spanish as a Second Language (SSL) students. In the first round, 104 items were administered to 507 students. After discarding poor items, the second round…

Development and Preliminary Testing of the Food Choice Priorities Survey (FCPS): Assessing the Importance of Multiple Factors on College Students' Food Choices.

PubMed

Vilaro, Melissa J; Zhou, Wenjun; Colby, Sarah E; Byrd-Bredbenner, Carol; Riggsbee, Kristin; Olfert, Melissa D; Barnett, Tracey E; Mathews, Anne E

2017-12-01

Understanding factors that influence food choice may help improve diet quality. Factors that commonly affect adults' food choices have been described, but measures that identify and assess food choice factors specific to college students are lacking. This study developed and tested the Food Choice Priorities Survey (FCPS) among college students. Thirty-seven undergraduates participated in two focus groups ( n = 19; 11 in the male-only group, 8 in the female-only group) and interviews ( n = 18) regarding typical influences on food choice. Qualitative data informed the development of survey items with a 5-point Likert-type scale (1 = not important, 5 = extremely important). An expert panel rated FCPS items for clarity, relevance, representativeness, and coverage using a content validity form. To establish test-retest reliability, 109 first-year college students completed the 14-item FCPS at two time points, 0-48 days apart ( M = 13.99, SD = 7.44). Using Cohen's weighted κ for responses within 20 days, 11 items demonstrated moderate agreement and 3 items had substantial agreement. Factor analysis revealed a three-factor structure (9 items). The FCPS is designed for college students and provides a way to determine the factors of greatest importance regarding food choices among this population. From a public health perspective, practical applications include using the FCPS to tailor health communications and behavior change interventions to factors most salient for food choices of college students.

Validation and Structural Analysis of the Kinematics Concept Test

ERIC Educational Resources Information Center

Lichtenberger, A.; Wagner, C.; Hofer, S. I.; Stem, E.; Vaterlaus, A.

2017-01-01

The kinematics concept test (KCT) is a multiple-choice test designed to evaluate students' conceptual understanding of kinematics at the high school level. The test comprises 49 multiple-choice items about velocity and acceleration, which are based on seven kinematic concepts and which make use of three different representations. In the first part…

Multiple-Choice Test Bias Due to Answering Strategy Variation.

ERIC Educational Resources Information Center

Frary, Robert B.; Giles, Mary B.

This paper describes the development and investigation of a new approach to determining the existence of bias in multiple-choice test scores. Previous work in this area has concentrated almost exclusively on bias attributable to specific test items or to differences in test score distributions across racial or ethnic groups. In contrast, the…

To Show or Not to Show: The Effects of Item Stems and Answer Options on Performance on a Multiple-Choice Listening Comprehension Test

ERIC Educational Resources Information Center

Yanagawa, Kozo; Green, Anthony

2008-01-01

The purpose of this study is to examine whether the choice between three multiple-choice listening comprehension test formats results in any difference in listening comprehension test performance. The three formats entail (a) allowing test takers to preview both the question stem and answer options prior to listening; (b) allowing test takers to…

Student Questionnaire. [Harvard Project Physics

ERIC Educational Resources Information Center

Welch, Wayne W.; Ahlgren, Andrew

This 60-item questionnaire was designed to gather general background information from students who had used the Harvard Project Physics curriculum. The instrument includes three 20-item subscales: (1) attitude toward physics, (2) career interest, and (3) student characteristics. Items are multiple choice (5 options), and the introductory material…

Missouri Assessment Program (MAP), Spring 2000: High School Health/Physical Education, Released Items, Grade 9.

ERIC Educational Resources Information Center

Missouri State Dept. of Elementary and Secondary Education, Jefferson City.

This document presents 10 released items from the Health/Physical Education Missouri Assessment Program (MAP) test given in the spring of 2000 to ninth graders. Items from the test sessions include: selected-response (multiple choice), constructed-response, and a performance event. The selected-response items consist of individual questions…

Measuring University students' understanding of the greenhouse effect - a comparison of multiple-choice, short answer and concept sketch assessment tools with respect to students' mental models

NASA Astrophysics Data System (ADS)

Gold, A. U.; Harris, S. E.

2013-12-01

The greenhouse effect comes up in most discussions about climate and is a key concept related to climate change. Existing studies have shown that students and adults alike lack a detailed understanding of this important concept or might hold misconceptions. We studied the effectiveness of different interventions on University-level students' understanding of the greenhouse effect. Introductory level science students were tested for their pre-knowledge of the greenhouse effect using validated multiple-choice questions, short answers and concept sketches. All students participated in a common lesson about the greenhouse effect and were then randomly assigned to one of two lab groups. One group explored an existing simulation about the greenhouse effect (PhET-lesson) and the other group worked with absorption spectra of different greenhouse gases (Data-lesson) to deepen the understanding of the greenhouse effect. All students completed the same assessment including multiple choice, short answers and concept sketches after participation in their lab lesson. 164 students completed all the assessments, 76 completed the PhET lesson and 77 completed the data lesson. 11 students missed the contrasting lesson. In this presentation we show the comparison between the multiple-choice questions, short answer questions and the concept sketches of students. We explore how well each of these assessment types represents student's knowledge. We also identify items that are indicators of the level of understanding of the greenhouse effect as measured in correspondence of student answers to an expert mental model and expert responses. Preliminary data analysis shows that student who produce concept sketch drawings that come close to expert drawings also choose correct multiple-choice answers. However, correct multiple-choice answers are not necessarily an indicator that a student produces an expert-like correlating concept sketch items. Multiple-choice questions that require detailed knowledge of the greenhouse effect (e.g. direction of re-emission of infrared energy from greenhouse gas) are significantly more likely to be answered correctly by students who also produce expert-like concept sketch items than by students who don't include this aspect in their sketch and don't answer the multiple choice questions correctly. This difference is not as apparent for less technical multiple-choice questions (e.g. type of radiation emitted by Sun). Our findings explore the formation of student's mental models throughout different interventions and how well the different assessment techniques used in this study represent the student understanding of the overall concept.

Science Library of Test Items. Volume Nineteen. A Collection of Multiple Choice Test Items Relating Mainly to Geology.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Science Library of Test Items. Volume Seventeen. A Collection of Multiple Choice Test Items Relating Mainly to Biology.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Science Library of Test Items. Volume Eighteen. A Collection of Multiple Choice Test Items Relating Mainly to Chemistry.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Comparisons of mathematics achievement of grade 8 students in the United States and the Russian Federation.

PubMed

Bazarova, Saodat I; Engelhard, George

2004-01-01

Using the Mantel-Haenszel (MH) Procedure, we analyzed data for 7,087 American and 4,022 Russian Grade 8 students from the Third International Mathematics and Science Study (TIMSS) to compare mathematics achievement in the two countries on each of the 124 multiple-choice items. The results of the analyses indicate that the performance of the students on individual multiple-choice mathematics items vary by country. The results also suggest that the relationship between country and item performance differ as a function of content area. A total score of a country's achievement does not provide the whole picture of achievement dynamics; it averages out potentially important information on student achievement and the causes of their performance relative to other countries. The dynamics of achievement across countries will not be revealed unless the analyses are done at the item level.

The "None of the Above" Option in Multiple-Choice Testing: An Experimental Study

ERIC Educational Resources Information Center

DiBattista, David; Sinnige-Egger, Jo-Anne; Fortuna, Glenda

2014-01-01

The authors assessed the effects of using "none of the above" as an option in a 40-item, general-knowledge multiple-choice test administered to undergraduate students. Examinees who selected "none of the above" were given an incentive to write the correct answer to the question posed. Using "none of the above" as the…

Some Effects of Changes in Question Structure and Sequence on Performance in a Multiple Choice Chemistry Test.

ERIC Educational Resources Information Center

Hodson, D.

1984-01-01

Investigated the effect on student performance of changes in question structure and sequence on a GCE 0-level multiple-choice chemistry test. One finding noted is that there was virtually no change in test reliability on reducing the number of options (from five to per test item). (JN)

The Effect of Position and Format on the Difficulty of Assessment Exercises.

ERIC Educational Resources Information Center

Burton, Nancy W.; And Others

Assessment exercises (items) in three different formats--multiple-choice with an "I don't know" (IDK) option, multiple-choice without the IDK, and open-ended--were placed at the beginning, middle and end of 45-minute assessment packages (instruments). A balanced incomplete blocks analysis of variance was computed to determine the biasing…

Multiple-Choice versus Constructed-Response Tests in the Assessment of Mathematics Computation Skills.

ERIC Educational Resources Information Center

Gadalla, Tahany M.

The equivalence of multiple-choice (MC) and constructed response (discrete) (CR-D) response formats as applied to mathematics computation at grade levels two to six was tested. The difference between total scores from the two response formats was tested for statistical significance, and the factor structure of items in both response formats was…

Multiple Choice Questions Can Be Designed or Revised to Challenge Learners' Critical Thinking

ERIC Educational Resources Information Center

Tractenberg, Rochelle E.; Gushta, Matthew M.; Mulroney, Susan E.; Weissinger, Peggy A.

2013-01-01

Multiple choice (MC) questions from a graduate physiology course were evaluated by cognitive-psychology (but not physiology) experts, and analyzed statistically, in order to test the independence of content expertise and cognitive complexity ratings of MC items. Integration of higher order thinking into MC exams is important, but widely known to…

High time for a change: psychometric analysis of multiple-choice questions in nursing.

PubMed

Redmond, Sandra P; Hartigan-Rogers, Jackie A; Cobbett, Shelley

2012-11-26

Nurse educators teach students to develop an informed nursing practice but can educators claim the same grounding in the available evidence when formulating multiple-choice assessment tools to evaluate student learning? Multiple-choice questions are a popular assessment format within nursing education. While widely accepted as a credible format to assess student knowledge across disciplines, debate exists among educators regarding the number of options necessary to adequately test cognitive reasoning and optimal discrimination between student abilities. The purpose of this quasi-experimental between groups study was to examine the psychometric properties of three option multiple-choice questions when compared to the more traditional four option questions. Data analysis revealed that there were no statistically significant differences in the item discrimination, difficulty or the mean examination scores when multiple-choice test questions were administered with three versus four option answer choices. This study provides additional guidance for nurse educators to assist in improving multiple-choice question writing and test design.

Effect of response format on cognitive reflection: Validating a two- and four-option multiple choice question version of the Cognitive Reflection Test.

PubMed

Sirota, Miroslav; Juanchich, Marie

2018-03-27

The Cognitive Reflection Test, measuring intuition inhibition and cognitive reflection, has become extremely popular because it reliably predicts reasoning performance, decision-making, and beliefs. Across studies, the response format of CRT items sometimes differs, based on the assumed construct equivalence of tests with open-ended versus multiple-choice items (the equivalence hypothesis). Evidence and theoretical reasons, however, suggest that the cognitive processes measured by these response formats and their associated performances might differ (the nonequivalence hypothesis). We tested the two hypotheses experimentally by assessing the performance in tests with different response formats and by comparing their predictive and construct validity. In a between-subjects experiment (n = 452), participants answered stem-equivalent CRT items in an open-ended, a two-option, or a four-option response format and then completed tasks on belief bias, denominator neglect, and paranormal beliefs (benchmark indicators of predictive validity), as well as on actively open-minded thinking and numeracy (benchmark indicators of construct validity). We found no significant differences between the three response formats in the numbers of correct responses, the numbers of intuitive responses (with the exception of the two-option version, which had a higher number than the other tests), and the correlational patterns of the indicators of predictive and construct validity. All three test versions were similarly reliable, but the multiple-choice formats were completed more quickly. We speculate that the specific nature of the CRT items helps build construct equivalence among the different response formats. We recommend using the validated multiple-choice version of the CRT presented here, particularly the four-option CRT, for practical and methodological reasons. Supplementary materials and data are available at https://osf.io/mzhyc/ .

Automatically Scoring Short Essays for Content. CRESST Report 836

ERIC Educational Resources Information Center

Kerr, Deirdre; Mousavi, Hamid; Iseli, Markus R.

2013-01-01

The Common Core assessments emphasize short essay constructed response items over multiple choice items because they are more precise measures of understanding. However, such items are too costly and time consuming to be used in national assessments unless a way is found to score them automatically. Current automatic essay scoring techniques are…

Are Learning Disabled Students "Test-Wise?": An Inquiry into Reading Comprehension Test Items.

ERIC Educational Resources Information Center

Scruggs, Thomas E.; Lifson, Steve

The ability to correctly answer reading comprehension test items, without having read the accompanying reading passage, was compared for third grade learning disabled students and their peers from a regular classroom. In the first experiment, fourteen multiple choice items were selected from the Stanford Achievement Test. No reading passages were…

A Two-Parameter Latent Trait Model. Methodology Project.

ERIC Educational Resources Information Center

Choppin, Bruce

On well-constructed multiple-choice tests, the most serious threat to measurement is not variation in item discrimination, but the guessing behavior that may be adopted by some students. Ways of ameliorating the effects of guessing are discussed, especially for problems in latent trait models. A new item response model, including an item parameter…

Investigating the Stability of Four Methods for Estimating Item Bias.

ERIC Educational Resources Information Center

Perlman, Carole L.; And Others

The reliability of item bias estimates was studied for four methods: (1) the transformed delta method; (2) Shepard's modified delta method; (3) Rasch's one-parameter residual analysis; and (4) the Mantel-Haenszel procedure. Bias statistics were computed for each sample using all methods. Data were from administration of multiple-choice items from…

Science Library of Test Items. Volume Twenty. A Collection of Multiple Choice Test Items Relating Mainly to Physics, 1.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Science Library of Test Items. Volume Twenty-One. A Collection of Multiple Choice Test Items Relating Mainly to Physics, 2.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Science Library of Test Items. Volume Twenty-Two. A Collection of Multiple Choice Test Items Relating Mainly to Skills.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

As one in a series of test item collections developed by the Assessment and Evaluation Unit of the Directorate of Studies, items are made available to teachers for the construction of unit tests or term examinations or as a basis for class discussion. Each collection was reviewed for content validity and reliability. The test items meet syllabus…

Developing Achievement Test: A Research for Assessment of 5th Grade Biology Subject

ERIC Educational Resources Information Center

Sener, Nilay; Tas, Erol

2017-01-01

The purpose of this study is to prepare a multiple-choice achievement test with high reliability and validity for the "Let's Solve the Puzzle of Our Body" unit. For this purpose, a multiple choice achievement test consisting of 46 items was applied to 178 fifth grade students in total. As a result of the test and material analysis…

Asymmetry in Student Achievement on Multiple-Choice and Constructed-Response Items in Reversible Mathematics Processes

ERIC Educational Resources Information Center

Sangwin, Christopher J.; Jones, Ian

2017-01-01

In this paper we report the results of an experiment designed to test the hypothesis that when faced with a question involving the inverse direction of a reversible mathematical process, students solve a multiple-choice version by verifying the answers presented to them by the direct method, not by undertaking the actual inverse calculation.…

Two systems drive attention to rewards.

PubMed

Kovach, Christopher K; Sutterer, Matthew J; Rushia, Sara N; Teriakidis, Adrianna; Jenison, Rick L

2014-01-01

How options are framed can dramatically influence choice preference. While salience of information plays a central role in this effect, precisely how it is mediated by attentional processes remains unknown. Current models assume a simple relationship between attention and choice, according to which preference should be uniformly biased towards the attended item over the whole time-course of a decision between similarly valued items. To test this prediction we considered how framing alters the orienting of gaze during a simple choice between two options, using eye movements as a sensitive online measure of attention. In one condition participants selected the less preferred item to discard and in the other, the more preferred item to keep. We found that gaze gravitates towards the item ultimately selected, but did not observe the effect to be uniform over time. Instead, we found evidence for distinct early and late processes that guide attention according to preference in the first case and task demands in the second. We conclude that multiple time-dependent processes govern attention during choice, and that these may contribute to framing effects in different ways.

Two systems drive attention to rewards

PubMed Central

Kovach, Christopher K.; Sutterer, Matthew J.; Rushia, Sara N.; Teriakidis, Adrianna; Jenison, Rick L.

2014-01-01

How options are framed can dramatically influence choice preference. While salience of information plays a central role in this effect, precisely how it is mediated by attentional processes remains unknown. Current models assume a simple relationship between attention and choice, according to which preference should be uniformly biased towards the attended item over the whole time-course of a decision between similarly valued items. To test this prediction we considered how framing alters the orienting of gaze during a simple choice between two options, using eye movements as a sensitive online measure of attention. In one condition participants selected the less preferred item to discard and in the other, the more preferred item to keep. We found that gaze gravitates towards the item ultimately selected, but did not observe the effect to be uniform over time. Instead, we found evidence for distinct early and late processes that guide attention according to preference in the first case and task demands in the second. We conclude that multiple time-dependent processes govern attention during choice, and that these may contribute to framing effects in different ways. PMID:24550868

Analysis Test of Understanding of Vectors with the Three-Parameter Logistic Model of Item Response Theory and Item Response Curves Technique

ERIC Educational Resources Information Center

Rakkapao, Suttida; Prasitpong, Singha; Arayathanitkul, Kwan

2016-01-01

This study investigated the multiple-choice test of understanding of vectors (TUV), by applying item response theory (IRT). The difficulty, discriminatory, and guessing parameters of the TUV items were fit with the three-parameter logistic model of IRT, using the parscale program. The TUV ability is an ability parameter, here estimated assuming…

The Relationship of Item-Level Response Times with Test-Taker and Item Variables in an Operational CAT Environment. LSAC Research Report Series.

ERIC Educational Resources Information Center

Swygert, Kimberly A.

In this study, data from an operational computerized adaptive test (CAT) were examined in order to gather information concerning item response times in a CAT environment. The CAT under study included multiple-choice items measuring verbal, quantitative, and analytical reasoning. The analyses included the fitting of regression models describing the…

Do the Guideline Violations Influence Test Difficulty of High-Stake Test?: An Investigation on University Entrance Examination in Turkey

ERIC Educational Resources Information Center

Atalmis, Erkan Hasan

2016-01-01

Multiple-choice (MC) items are commonly used in high-stake tests. Thus, each item of such tests should be meticulously constructed to increase the accuracy of decisions based on test results. Haladyna and his colleagues (2002) addressed the valid item-writing guidelines to construct high quality MC items in order to increase test reliability and…

Analyzing Multiple-Choice Questions by Model Analysis and Item Response Curves

NASA Astrophysics Data System (ADS)

Wattanakasiwich, P.; Ananta, S.

2010-07-01

In physics education research, the main goal is to improve physics teaching so that most students understand physics conceptually and be able to apply concepts in solving problems. Therefore many multiple-choice instruments were developed to probe students' conceptual understanding in various topics. Two techniques including model analysis and item response curves were used to analyze students' responses from Force and Motion Conceptual Evaluation (FMCE). For this study FMCE data from more than 1000 students at Chiang Mai University were collected over the past three years. With model analysis, we can obtain students' alternative knowledge and the probabilities for students to use such knowledge in a range of equivalent contexts. The model analysis consists of two algorithms—concentration factor and model estimation. This paper only presents results from using the model estimation algorithm to obtain a model plot. The plot helps to identify a class model state whether it is in the misconception region or not. Item response curve (IRC) derived from item response theory is a plot between percentages of students selecting a particular choice versus their total score. Pros and cons of both techniques are compared and discussed.

An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis.

PubMed

Tarrant, Marie; Ware, James; Mohammed, Ahmed M

2009-07-07

Four- or five-option multiple choice questions (MCQs) are the standard in health-science disciplines, both on certification-level examinations and on in-house developed tests. Previous research has shown, however, that few MCQs have three or four functioning distractors. The purpose of this study was to investigate non-functioning distractors in teacher-developed tests in one nursing program in an English-language university in Hong Kong. Using item-analysis data, we assessed the proportion of non-functioning distractors on a sample of seven test papers administered to undergraduate nursing students. A total of 514 items were reviewed, including 2056 options (1542 distractors and 514 correct responses). Non-functioning options were defined as ones that were chosen by fewer than 5% of examinees and those with a positive option discrimination statistic. The proportion of items containing 0, 1, 2, and 3 functioning distractors was 12.3%, 34.8%, 39.1%, and 13.8% respectively. Overall, items contained an average of 1.54 (SD = 0.88) functioning distractors. Only 52.2% (n = 805) of all distractors were functioning effectively and 10.2% (n = 158) had a choice frequency of 0. Items with more functioning distractors were more difficult and more discriminating. The low frequency of items with three functioning distractors in the four-option items in this study suggests that teachers have difficulty developing plausible distractors for most MCQs. Test items should consist of as many options as is feasible given the item content and the number of plausible distractors; in most cases this would be three. Item analysis results can be used to identify and remove non-functioning distractors from MCQs that have been used in previous tests.

Australian Item Bank Program: Science Item Bank. Book 3: Biology.

ERIC Educational Resources Information Center

Australian Council for Educational Research, Hawthorn.

The Australian Science Item Bank consists of three volumes of multiple-choice questions. Book 3 contains questions on the biological sciences. The questions are designed to be suitable for high school students (year 8 to year 12 in Australian schools). The questions are classified by the subject content of the question, the cognitive skills…

Detecting a Gender-Related Differential Item Functioning Using Transformed Item Difficulty

ERIC Educational Resources Information Center

Abedalaziz, Nabeel; Leng, Chin Hai; Alahmadi, Ahlam

2014-01-01

The purpose of the study was to examine gender differences in performance on multiple-choice mathematical ability test, administered within the context of high school graduation test that was designed to match eleventh grade curriculum. The transformed item difficulty (TID) was used to detect a gender related DIF. A random sample of 1400 eleventh…

Guide to an Assessment of Consumer Skills.

ERIC Educational Resources Information Center

Education Commission of the States, Denver, CO.

This guide is intended to assist those interested in developing and/or assessing consumer skills. It is an accompanyment to a separate collection of survey items (mostly in a multiple choice format) designed to assess seventeen-year-olds' consumer skills. It is suggested that the items can be used as part of an item pool, as an instructional tool,…

An Explanatory Item Response Theory Approach for a Computer-Based Case Simulation Test

ERIC Educational Resources Information Center

Kahraman, Nilüfer

2014-01-01

Problem: Practitioners working with multiple-choice tests have long utilized Item Response Theory (IRT) models to evaluate the performance of test items for quality assurance. The use of similar applications for performance tests, however, is often encumbered due to the challenges encountered in working with complicated data sets in which local…

Of Small Beauties and Large Beasts: The Quality of Distractors on Multiple-Choice Tests Is More Important than Their Quantity

ERIC Educational Resources Information Center

Papenberg, Martin; Musch, Jochen

2017-01-01

In multiple-choice tests, the quality of distractors may be more important than their number. We therefore examined the joint influence of distractor quality and quantity on test functioning by providing a sample of 5,793 participants with five parallel test sets consisting of items that differed in the number and quality of distractors.…

Student certainty answering misconception question: study of Three-Tier Multiple-Choice Diagnostic Test in Acid-Base and Solubility Equilibrium

NASA Astrophysics Data System (ADS)

Ardiansah; Masykuri, M.; Rahardjo, S. B.

2018-04-01

Students’ concept comprehension in three-tier multiple-choice diagnostic test related to student confidence level. The confidence level related to certainty and student’s self-efficacy. The purpose of this research was to find out students’ certainty in misconception test. This research was quantitative-qualitative research method counting students’ confidence level. The research participants were 484 students that were studying acid-base and equilibrium solubility subject. Data was collected using three-tier multiple-choice (3TMC) with thirty questions and students’ questionnaire. The findings showed that #6 item gives the highest misconception percentage and high student confidence about the counting of ultra-dilute solution’s pH. Other findings were that 1) the student tendency chosen the misconception answer is to increase over item number, 2) student certainty decreased in terms of answering the 3TMC, and 3) student self-efficacy and achievement were related each other in the research. The findings suggest some implications and limitations for further research.

Feedback-related brain activity predicts learning from feedback in multiple-choice testing.

PubMed

Ernst, Benjamin; Steinhauser, Marco

2012-06-01

Different event-related potentials (ERPs) have been shown to correlate with learning from feedback in decision-making tasks and with learning in explicit memory tasks. In the present study, we investigated which ERPs predict learning from corrective feedback in a multiple-choice test, which combines elements from both paradigms. Participants worked through sets of multiple-choice items of a Swahili-German vocabulary task. Whereas the initial presentation of an item required the participants to guess the answer, corrective feedback could be used to learn the correct response. Initial analyses revealed that corrective feedback elicited components related to reinforcement learning (FRN), as well as to explicit memory processing (P300) and attention (early frontal positivity). However, only the P300 and early frontal positivity were positively correlated with successful learning from corrective feedback, whereas the FRN was even larger when learning failed. These results suggest that learning from corrective feedback crucially relies on explicit memory processing and attentional orienting to corrective feedback, rather than on reinforcement learning.

Seafarers Knowledge Inventory.

ERIC Educational Resources Information Center

Hounshell, Paul B.

This 60-item, multiple-choice Seafarers Knowledge Inventory was developed for use in marine vocational classes (grades 9-12) to measure a student's knowledge of information that "seafarers" should know. Items measure knowledge of various aspects of boating operation, weather, safety, winds, and oceanography. Steps in the construction of…

Using Tests as Learning Opportunities.

ERIC Educational Resources Information Center

Foos, Paul W.; Fisher, Ronald P.

1988-01-01

A study involving 105 undergraduates assessed the value of testing as a means of increasing, rather than simply monitoring, learning. Results indicate that fill-in-the-blank and items requiring student inferences were more effective, respectively, than multiple-choice tests and verbatim items in furthering student learning. (TJH)

Identification of technical item flaws leads to improvement of the quality of single best Multiple Choice Questions.

PubMed

Fayyaz Khan, Humaira; Farooq Danish, Khalid; Saeed Awan, Azra; Anwar, Masood

2013-05-01

The purpose of the study was to identify technical item flaws in the multiple choice questions submitted for the final exams for the years 2009, 2010 and 2011. This descriptive analytical study was carried out in Islamic International Medical College (IIMC). The Data was collected from the MCQ's submitted by the faculty for the final exams for the year 2009, 2010 and 2011. The data was compiled and evaluated by a three member assessment committee. The data was analyzed for frequency and percentages the categorical data was analyzed by chi-square test. Overall percentage of flawed item was 67% for the year 2009 of which 21% were for testwiseness and 40% were for irrelevant difficulty. In year 2010 the total item flaws were 36% and 11% testwiseness and 22% were for irrelevant difficulty. The year 2011 data showed decreased overall flaws of 21%. The flaws of testwisness were 7%, irrelevant difficulty were 11%. Technical item flaws are frequently encountered during MCQ construction, and the identification of flaws leads to improved quality of the single best MCQ's.

A hybrid heuristic for the multiple choice multidimensional knapsack problem

NASA Astrophysics Data System (ADS)

Mansi, Raïd; Alves, Cláudio; Valério de Carvalho, J. M.; Hanafi, Saïd

2013-08-01

In this article, a new solution approach for the multiple choice multidimensional knapsack problem is described. The problem is a variant of the multidimensional knapsack problem where items are divided into classes, and exactly one item per class has to be chosen. Both problems are NP-hard. However, the multiple choice multidimensional knapsack problem appears to be more difficult to solve in part because of its choice constraints. Many real applications lead to very large scale multiple choice multidimensional knapsack problems that can hardly be addressed using exact algorithms. A new hybrid heuristic is proposed that embeds several new procedures for this problem. The approach is based on the resolution of linear programming relaxations of the problem and reduced problems that are obtained by fixing some variables of the problem. The solutions of these problems are used to update the global lower and upper bounds for the optimal solution value. A new strategy for defining the reduced problems is explored, together with a new family of cuts and a reformulation procedure that is used at each iteration to improve the performance of the heuristic. An extensive set of computational experiments is reported for benchmark instances from the literature and for a large set of hard instances generated randomly. The results show that the approach outperforms other state-of-the-art methods described so far, providing the best known solution for a significant number of benchmark instances.

Item Order, Response Format, and Examinee Sex and Handedness and Performance on a Multiple-Choice Test.

ERIC Educational Resources Information Center

Kleinke, David J.

Four forms of a 36-item adaptation of the Stanford Achievement Test were administered to 484 fourth graders. External factors potentially influencing test performance were examined, namely: (1) item order (easy-to-difficult vs. uniform); (2) response location (left column vs. right column); (3) handedness which may interact with response location;…

Test Design Project: Studies in Test Bias. Annual Report.

ERIC Educational Resources Information Center

McArthur, David

Item bias in a multiple-choice test can be detected by appropriate analyses of the persons x items scoring matrix. This permits comparison of groups of examinees tested with the same instrument. The test may be biased if it is not measuring the same thing in comparable groups, if groups are responding to different aspects of the test items, or if…

Can Item Analysis of MCQs Accomplish the Need of a Proper Assessment Strategy for Curriculum Improvement in Medical Education?

ERIC Educational Resources Information Center

Pawade, Yogesh R.; Diwase, Dipti S.

2016-01-01

Item analysis of Multiple Choice Questions (MCQs) is the process of collecting, summarizing and utilizing information from students' responses to evaluate the quality of test items. Difficulty Index (p-value), Discrimination Index (DI) and Distractor Efficiency (DE) are the parameters which help to evaluate the quality of MCQs used in an…

Physics 300 Provincial Examination.

ERIC Educational Resources Information Center

Manitoba Dept. of Education and Training, Winnipeg.

This document consists of the physics 300 provincial examination (English version), a separate "provincial summary report" on the results of giving the test, and a separate French language version of the examination. This physics examination contains a 53-item multiple choice section and an 12 item free response section. Subsections of…

Marine Education Knowledge Inventory.

ERIC Educational Resources Information Center

Hounshell, Paul B.; Hampton, Carolyn

This 35-item, multiple-choice Marine Education Knowledge Inventory was developed for use in upper elementary/middle schools to measure a student's knowledge of marine science. Content of test items is drawn from oceanography, ecology, earth science, navigation, and the biological sciences (focusing on marine animals). Steps in the construction of…

Examining Two Strategies to Link Mixed-Format Tests Using Multiple-Choice Anchors. Research Report. ETS RR-10-18

ERIC Educational Resources Information Center

Walker, Michael E.; Kim, Sooyeon

2010-01-01

This study examined the use of an all multiple-choice (MC) anchor for linking mixed format tests containing both MC and constructed-response (CR) items, in a nonequivalent groups design. An MC-only anchor could effectively link two such test forms if either (a) the MC and CR portions of the test measured the same construct, so that the MC anchor…

An Item Response Theory Analysis of Palmore's Facts on Aging Quiz (FAQ) Using the Three Parameter Model.

ERIC Educational Resources Information Center

Obiekwe, Jerry C.

Palmore's Facts on Aging Quiz (FAQ) (E. Palmore, 1977) is an instrument that is used to educate, to measure learning, to test knowledge, to measure attitudes toward aging, and in research. A comparative analysis was performed between the FAQ I and its multiple choice version and the FAQ II and its multiple choice version in terms of their item…

Measuring ability to assess claims about treatment effects: a latent trait analysis of items from the ‘Claim Evaluation Tools’ database using Rasch modelling

PubMed Central

Austvoll-Dahlgren, Astrid; Guttersrud, Øystein; Nsangi, Allen; Semakula, Daniel; Oxman, Andrew D

2017-01-01

Background The Claim Evaluation Tools database contains multiple-choice items for measuring people’s ability to apply the key concepts they need to know to be able to assess treatment claims. We assessed items from the database using Rasch analysis to develop an outcome measure to be used in two randomised trials in Uganda. Rasch analysis is a form of psychometric testing relying on Item Response Theory. It is a dynamic way of developing outcome measures that are valid and reliable. Objectives To assess the validity, reliability and responsiveness of 88 items addressing 22 key concepts using Rasch analysis. Participants We administrated four sets of multiple-choice items in English to 1114 people in Uganda and Norway, of which 685 were children and 429 were adults (including 171 health professionals). We scored all items dichotomously. We explored summary and individual fit statistics using the RUMM2030 analysis package. We used SPSS to perform distractor analysis. Results Most items conformed well to the Rasch model, but some items needed revision. Overall, the four item sets had satisfactory reliability. We did not identify significant response dependence between any pairs of items and, overall, the magnitude of multidimensionality in the data was acceptable. The items had a high level of difficulty. Conclusion Most of the items conformed well to the Rasch model’s expectations. Following revision of some items, we concluded that most of the items were suitable for use in an outcome measure for evaluating the ability of children or adults to assess treatment claims. PMID:28550019

Project Physics Tests 1, Concepts of Motion.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 1 are presented in this booklet, consisting of 70 multiple-choice and 20 problem-and-essay questions. Concepts of motion are examined with respect to velocities, acceleration, forces, vectors, Newton's laws, and circular motion. Suggestions are made for time consumption in answering some items. Besides…

Meatcutting Testbook, Part 2.

ERIC Educational Resources Information Center

California State Dept. of Education, Sacramento. Bureau of Publications.

This document contains objective tests for each topic in the Meatcutting Workbook, Part 2, which is designed for apprenticeship meatcutting programs in California. Each of the 30 tests consists of from 5 to 65 multiple-choice items with most tests containing approximately 10 items. The tests are grouped according to the eight units of the…

New York Community Environment Study Questionnaire.

ERIC Educational Resources Information Center

Glaser, Daniel; Snow, Mary

This questionnaire assesses neighborhood drug problem concern, drug use practices, knowledge of drugs and agencies dealing with drugs, and views on drug education in persons aged 13 or older. The questionnaire has 31 items (multiple-choice or free response), most with several parts. The items deal with demographic and personal data, problems in…

Food Service Supervisor. Dietetic Support Personnel Achievement Test.

ERIC Educational Resources Information Center

Oklahoma State Dept. of Vocational and Technical Education, Stillwater.

This guide contains a series of multiple-choice items and guidelines to assist instructors in composing criterion-referenced tests for use in the food service supervisor component of Oklahoma's Dietetic Support Personnel training program. Test items addressing each of the following occupational duty areas are provided: human relations; nutrient…

Food Production Worker. Dietetic Support Personnel Achievement Test.

ERIC Educational Resources Information Center

Oklahoma State Dept. of Vocational and Technical Education, Stillwater.

This guide contains a series of multiple-choice items and guidelines to assist instructors in composing criterion-referenced tests for use in the food production worker component of Oklahoma's Dietetic Support Personnel training program. Test items addressing each of the following occupational duty areas are provided: human relations; hygiene and…

Handbook for Driving Knowledge Testing.

ERIC Educational Resources Information Center

Pollock, William T.; McDole, Thomas L.

Materials intended for driving knowledge test development for use by operational licensing and education agencies are presented. A pool of 1,313 multiple choice test items is included, consisting of sets of specially developed and tested items covering principles of safe driving, legal regulations, and traffic control device knowledge pertinent to…

Food Service Worker. Dietetic Support Personnel Achievement Test.

ERIC Educational Resources Information Center

Oklahoma State Dept. of Vocational and Technical Education, Stillwater.

This guide contains a series of multiple-choice items and guidelines to assist instructors in composing criterion-referenced tests for use in the food service worker component of Oklahoma's Dietetic Support Personnel training program. Test items addressing each of the following occupational duty areas are provided: human relations; personal…

Fundamentals of Marketing Core Curriculum. Test Items and Assessment Techniques.

ERIC Educational Resources Information Center

Smith, Clifton L.; And Others

This document contains multiple choice test items and assessment techniques for Missouri's fundamentals of marketing core curriculum. The core curriculum is divided into these nine occupational duties: (1) communications in marketing; (2) economics and marketing; (3) employment and advancement; (4) human relations in marketing; (5) marketing…

The Performance of IRT Model Selection Methods with Mixed-Format Tests

ERIC Educational Resources Information Center

Whittaker, Tiffany A.; Chang, Wanchen; Dodd, Barbara G.

2012-01-01

When tests consist of multiple-choice and constructed-response items, researchers are confronted with the question of which item response theory (IRT) model combination will appropriately represent the data collected from these mixed-format tests. This simulation study examined the performance of six model selection criteria, including the…

Automatic Short Essay Scoring Using Natural Language Processing to Extract Semantic Information in the Form of Propositions. CRESST Report 831

ERIC Educational Resources Information Center

Kerr, Deirdre; Mousavi, Hamid; Iseli, Markus R.

2013-01-01

The Common Core assessments emphasize short essay constructed-response items over multiple-choice items because they are more precise measures of understanding. However, such items are too costly and time consuming to be used in national assessments unless a way to score them automatically can be found. Current automatic essay-scoring techniques…

Comparison between three option, four option and five option multiple choice question tests for quality parameters: A randomized study.

PubMed

Vegada, Bhavisha; Shukla, Apexa; Khilnani, Ajeetkumar; Charan, Jaykaran; Desai, Chetna

2016-01-01

Most of the academic teachers use four or five options per item of multiple choice question (MCQ) test as formative and summative assessment. Optimal number of options in MCQ item is a matter of considerable debate among academic teachers of various educational fields. There is a scarcity of the published literature regarding the optimum number of option in each item of MCQ in the field of medical education. To compare three options, four options, and five options MCQs test for the quality parameters - reliability, validity, item analysis, distracter analysis, and time analysis. Participants were 3 rd semester M.B.B.S. students. Students were divided randomly into three groups. Each group was given one set of MCQ test out of three options, four options, and five option randomly. Following the marking of the multiple choice tests, the participants' option selections were analyzed and comparisons were conducted of the mean marks, mean time, validity, reliability and facility value, discrimination index, point biserial value, distracter analysis of three different option formats. Students score more ( P = 0.000) and took less time ( P = 0.009) for the completion of three options as compared to four options and five options groups. Facility value was more ( P = 0.004) in three options group as compared to four and five options groups. There was no significant difference between three groups for the validity, reliability, and item discrimination. Nonfunctioning distracters were more in the four and five options group as compared to three option group. Assessment based on three option MCQs is can be preferred over four option and five option MCQs.

[Development of critical thinking skill evaluation scale for nursing students].

PubMed

You, So Young; Kim, Nam Cho

2014-04-01

To develop a Critical Thinking Skill Test for Nursing Students. The construct concepts were drawn from a literature review and in-depth interviews with hospital nurses and surveys were conducted among students (n=607) from nursing colleges. The data were collected from September 13 to November 23, 2012 and analyzed using the SAS program, 9.2 version. The KR 20 coefficient for reliability, difficulty index, discrimination index, item-total correlation and known group technique for validity were performed. Four domains and 27 skills were identified and 35 multiple choice items were developed. Thirty multiple choice items which had scores higher than .80 on the content validity index were selected for the pre test. From the analysis of the pre test data, a modified 30 items were selected for the main test. In the main test, the KR 20 coefficient was .70 and Corrected Item-Total Correlations range was .11-.38. There was a statistically significant difference between two academic systems (p=.001). The developed instrument is the first critical thinking skill test reflecting nursing perspectives in hospital settings and is expected to be utilized as a tool which contributes to improvement of the critical thinking ability of nursing students.

Testing to the Top: Everything But the Kitchen Sink?

ERIC Educational Resources Information Center

Dietel, Ron

2011-01-01

Two tests intended to measure student achievement of the Common Core State Standards will face intense scrutiny, but the test makers say they will include performance assessments and other items that are not multiple-choice questions. Incorporating performance items on this tests will bring up issues over scoring, costs, and validity.

Configural Frequency Analysis as a Statistical Tool for Developmental Research.

ERIC Educational Resources Information Center

Lienert, Gustav A.; Oeveste, Hans Zur

1985-01-01

Configural frequency analysis (CFA) is suggested as a technique for longitudinal research in developmental psychology. Stability and change in answers to multiple choice and yes-no item patterns obtained with repeated measurements are identified by CFA and illustrated by developmental analysis of an item from Gorham's Proverb Test. (Author/DWH)

Effects of Repeated Testing on Short- and Long-Term Memory Performance across Different Test Formats

ERIC Educational Resources Information Center

Stenlund, Tova; Sundström, Anna; Jonsson, Bert

2016-01-01

This study examined whether practice testing with short-answer (SA) items benefits learning over time compared to practice testing with multiple-choice (MC) items, and rereading the material. More specifically, the aim was to test the hypotheses of "retrieval effort" and "transfer appropriate processing" by comparing retention…

A Quantum Chemistry Concept Inventory for Physical Chemistry Classes

ERIC Educational Resources Information Center

Dick-Perez, Marilu; Luxford, Cynthia J.; Windus, Theresa L.; Holme, Thomas

2016-01-01

A 14-item, multiple-choice diagnostic assessment tool, the quantum chemistry concept inventory or QCCI, is presented. Items were developed based on published student misconceptions and content coverage and then piloted and used in advanced physical chemistry undergraduate courses. In addition to the instrument itself, data from both a pretest,…

Appropriateness Measurement with Polychotomous Item Response Models and Standardized Indices. Measurement Series, 84-1.

ERIC Educational Resources Information Center

Drasgow, Fritz; And Others

The test scores of some examinees on a multiple-choice test may not provide adequate measures of their abilities. The goal of appropriateness measurement is to identify such individuals. Earlier theoretical and experimental work considered examinees answering all, or almost all, test items. This article reports research that extends…

The Effects of Item by Item Feedback Given during an Ability Test.

ERIC Educational Resources Information Center

Whetton, C.; Childs, R.

1981-01-01

Answer-until-correct (AUC) is a procedure for providing feedback during a multiple-choice test, giving an increased range of scores. The performance of secondary students on a verbal ability test using AUC procedures was compared with a group using conventional instructions. AUC scores considerably enhanced reliability but not validity.…

Advanced Marketing Core Curriculum. Test Items and Assessment Techniques.

ERIC Educational Resources Information Center

Smith, Clifton L.; And Others

This document contains duties and tasks, multiple-choice test items, and other assessment techniques for Missouri's advanced marketing core curriculum. The core curriculum begins with a list of 13 suggested textbook resources. Next, nine duties with their associated tasks are given. Under each task appears one or more citations to appropriate…

Measuring ability to assess claims about treatment effects: a latent trait analysis of items from the 'Claim Evaluation Tools' database using Rasch modelling.

PubMed

Austvoll-Dahlgren, Astrid; Guttersrud, Øystein; Nsangi, Allen; Semakula, Daniel; Oxman, Andrew D

2017-05-25

The Claim Evaluation Tools database contains multiple-choice items for measuring people's ability to apply the key concepts they need to know to be able to assess treatment claims. We assessed items from the database using Rasch analysis to develop an outcome measure to be used in two randomised trials in Uganda. Rasch analysis is a form of psychometric testing relying on Item Response Theory. It is a dynamic way of developing outcome measures that are valid and reliable. To assess the validity, reliability and responsiveness of 88 items addressing 22 key concepts using Rasch analysis. We administrated four sets of multiple-choice items in English to 1114 people in Uganda and Norway, of which 685 were children and 429 were adults (including 171 health professionals). We scored all items dichotomously. We explored summary and individual fit statistics using the RUMM2030 analysis package. We used SPSS to perform distractor analysis. Most items conformed well to the Rasch model, but some items needed revision. Overall, the four item sets had satisfactory reliability. We did not identify significant response dependence between any pairs of items and, overall, the magnitude of multidimensionality in the data was acceptable. The items had a high level of difficulty. Most of the items conformed well to the Rasch model's expectations. Following revision of some items, we concluded that most of the items were suitable for use in an outcome measure for evaluating the ability of children or adults to assess treatment claims. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Final Sampling Bias in Haptic Judgments: How Final Touch Affects Decision-Making.

PubMed

Mitsuda, Takashi; Yoshioka, Yuichi

2018-01-01

When people make a choice between multiple items, they usually evaluate each item one after the other repeatedly. The effect of the order and number of evaluating items on one's choices is essential to understanding the decision-making process. Previous studies have shown that when people choose a favorable item from two items, they tend to choose the item that they evaluated last. This tendency has been observed regardless of sensory modalities. This study investigated the origin of this bias by using three experiments involving two-alternative forced-choice tasks using handkerchiefs. First, the bias appeared in a smoothness discrimination task, which indicates that the bias was not based on judgments of preference. Second, the handkerchief that was touched more often tended to be chosen more frequently in the preference task, but not in the smoothness discrimination task, indicating that a mere exposure effect enhanced the bias. Third, in the condition where the number of touches did not differ between handkerchiefs, the bias appeared when people touched a handkerchief they wanted to touch last, but not when people touched the handkerchief that was predetermined. This finding suggests a direct coupling between final voluntary touching and judgment.

Fighting bias with statistics: Detecting gender differences in responses to items on a preschool science assessment

NASA Astrophysics Data System (ADS)

Greenberg, Ariela Caren

Differential item functioning (DIF) and differential distractor functioning (DDF) are methods used to screen for item bias (Camilli & Shepard, 1994; Penfield, 2008). Using an applied empirical example, this mixed-methods study examined the congruency and relationship of DIF and DDF methods in screening multiple-choice items. Data for Study I were drawn from item responses of 271 female and 236 male low-income children on a preschool science assessment. Item analyses employed a common statistical approach of the Mantel-Haenszel log-odds ratio (MH-LOR) to detect DIF in dichotomously scored items (Holland & Thayer, 1988), and extended the approach to identify DDF (Penfield, 2008). Findings demonstrated that the using MH-LOR to detect DIF and DDF supported the theoretical relationship that the magnitude and form of DIF and are dependent on the DDF effects, and demonstrated the advantages of studying DIF and DDF in multiple-choice items. A total of 4 items with DIF and DDF and 5 items with only DDF were detected. Study II incorporated an item content review, an important but often overlooked and under-published step of DIF and DDF studies (Camilli & Shepard). Interviews with 25 female and 22 male low-income preschool children and an expert review helped to interpret the DIF and DDF results and their comparison, and determined that a content review process of studied items can reveal reasons for potential item bias that are often congruent with the statistical results. Patterns emerged and are discussed in detail. The quantitative and qualitative analyses were conducted in an applied framework of examining the validity of the preschool science assessment scores for evaluating science programs serving low-income children, however, the techniques can be generalized for use with measures across various disciplines of research.

Relationship between item difficulty and discrimination indices in true/false-type multiple choice questions of a para-clinical multidisciplinary paper.

PubMed

Sim, Si-Mui; Rasiah, Raja Isaiah

2006-02-01

This paper reports the relationship between the difficulty level and the discrimination power of true/false-type multiple-choice questions (MCQs) in a multidisciplinary paper for the para-clinical year of an undergraduate medical programme. MCQ items in papers taken from Year II Parts A, B and C examinations for Sessions 2001/02, and Part B examinations for 2002/03 and 2003/04, were analysed to obtain their difficulty indices and discrimination indices. Each paper consisted of 250 true/false items (50 questions of 5 items each) on topics drawn from different disciplines. The questions were first constructed and vetted by the individual departments before being submitted to a central committee, where the final selection of the MCQs was made, based purely on the academic judgement of the committee. There was a wide distribution of item difficulty indices in all the MCQ papers analysed. Furthermore, the relationship between the difficulty index (P) and discrimination index (D) of the MCQ items in a paper was not linear, but more dome-shaped. Maximal discrimination (D = 51% to 71%) occurred with moderately easy/difficult items (P = 40% to 74%). On average, about 38% of the MCQ items in each paper were "very easy" (P > or =75%), while about 9% were "very difficult" (P <25%). About two-thirds of these very easy/difficult items had "very poor" or even negative discrimination (D < or =20%). MCQ items that demonstrate good discriminating potential tend to be moderately difficult items, and the moderately-to-very difficult items are more likely to show negative discrimination. There is a need to evaluate the effectiveness of our MCQ items.

Test of Achievement in Quantitative Economics for Secondary Schools: Construction and Validation Using Item Response Theory

ERIC Educational Resources Information Center

Eleje, Lydia I.; Esomonu, Nkechi P. M.

2018-01-01

A Test to measure achievement in quantitative economics among secondary school students was developed and validated in this study. The test is made up 20 multiple choice test items constructed based on quantitative economics sub-skills. Six research questions guided the study. Preliminary validation was done by two experienced teachers in…

Developing Form Assembly Specifications for Exams with Multiple Choice and Constructed Response Items: Balancing Reliability and Validity Concerns

ERIC Educational Resources Information Center

Hendrickson, Amy; Patterson, Brian; Ewing, Maureen

2010-01-01

The psychometric considerations and challenges associated with including constructed response items on tests are discussed along with how these issues affect the form assembly specifications for mixed-format exams. Reliability and validity, security and fairness, pretesting, content and skills coverage, test length and timing, weights, statistical…

Examining Gender DIF on a Multiple-Choice Test of Mathematics: A Confirmatory Approach.

ERIC Educational Resources Information Center

Ryan, Katherine E.; Fan, Meichu

1996-01-01

Results for 3,244 female and 3,033 male junior high school students from the Second International Mathematics Study show that applied items in algebra, geometry, and computation were easier for males but arithmetic items were differentially easier for females. Implications of these findings for assessment and instruction are discussed. (SLD)

Test-Wiseness Cues in the Options of Mathematics Items.

ERIC Educational Resources Information Center

Kuntz, Patricia

The quality of mathematics multiple choice items and their susceptibility to test wiseness were examined. Test wiseness was defined as "a subject's capacity to utilize the characteristics and formats of the test and/or test taking situation to receive a high score." The study used results of the Graduate Record Examinations Aptitude Test (GRE) and…

"But I Thought I Knew That!" Student Confidence Judgments on Course Examinations in Introductory Psychology

ERIC Educational Resources Information Center

Nevid, Jeffrey S.; Cheney, Brianna; Thompson, Clarissa

2015-01-01

Students in an introductory psychology class rated their level of confidence in their answers to exam questions on four multiple-choice exams through the course of a semester. Correlations between confidence judgments and accuracy (correct vs. incorrect) at the individual item level showed modest but significant relationships for item sets scaled…

Effects of Test Format, Self Concept and Anxiety on Item Response Changing Behaviour

ERIC Educational Resources Information Center

Afolabi, E. R. I.

2007-01-01

The study examined the effects of item format, self-concept and anxiety on response changing behaviour. Four hundred undergraduate students who offered a counseling psychology course in a Nigerian university participated in the study. Students' answers in multiple--choice and true--false formats of an achievement test were observed for response…

The Impact of Kentucky's Educational Reform Act on Writing throughout the Commonwealth.

ERIC Educational Resources Information Center

Harnack, Andrew; And Others

1994-01-01

The central role of writing in Kentucky's Education Reform Act is most evident in Kentucky's new assessment system, which employs writing on all levels. Even tests that have recently included multiple-choice items may be replaced by response items that require students to apply knowledge, concepts, and skills in a writing format. Writing itself is…

Technical flaws in multiple-choice questions in the access exam to medical specialties ("examen MIR") in Spain (2009-2013).

PubMed

Rodríguez-Díez, María Cristina; Alegre, Manuel; Díez, Nieves; Arbea, Leire; Ferrer, Marta

2016-02-03

The main factor that determines the selection of a medical specialty in Spain after obtaining a medical degree is the MIR ("médico interno residente", internal medical resident) exam. This exam consists of 235 multiple-choice questions with five options, some of which include images provided in a separate booklet. The aim of this study was to analyze the technical quality of the multiple-choice questions included in the MIR exam over the last five years. All the questions included in the exams from 2009 to 2013 were analyzed. We studied the proportion of questions including clinical vignettes, the number of items related to an image and the presence of technical flaws in the questions. For the analysis of technical flaws, we adapted the National Board of Medical Examiners (NBME) guidelines. We looked for 18 different issues included in the manual, grouped into two categories: issues related to testwiseness and issues related to irrelevant difficulties. The final number of questions analyzed was 1,143. The percentage of items based on clinical vignettes increased from 50% in 2009 to 56-58% in the following years (2010-2013). The percentage of items based on an image increased progressively from 10% in 2009 to 15% in 2012 and 2013. The percentage of items with at least one technical flaw varied between 68 and 72%. We observed a decrease in the percentage of items with flaws related to testwiseness, from 30% in 2009 to 20% in 2012 and 2013. While most of these issues decreased dramatically or even disappeared (such as the imbalance in the correct option numbers), the presence of non-plausible options remained frequent. With regard to technical flaws related to irrelevant difficulties, no improvement was observed; this is especially true with respect to negative stem questions and "hinged" questions. The formal quality of the MIR exam items has improved over the last five years with regard to testwiseness. A more detailed revision of the items submitted, checking systematically for the presence of technical flaws, could improve the validity and discriminatory power of the exam, without increasing its difficulty.

Student perception and post-exam analysis of one best MCQs and one correct MCQs: A comparative study.

PubMed

Adhi, Mohammad Idrees; Aly, Syed Moyn

2018-04-01

To find differences between One-Correct and One-Best multiple-choice questions with relation to student scores, post-exam item analyses results and student perception. This comparative cross-sectional study was conducted at the Dow University of Health Sciences, Karachi, from November 2010 to April 2011, and comprised medical students. Data was analysed using SPSS 18. Of the 207 participants, 16(7.7%) were boys and 191(92.3%) were girls. The mean score in Paper I was 18.62±4.7, while in Paper II it was 19.58±6.1. One-Best multiple-choice questions performed better than One-Correct. There was no statistically significant difference in the mean scores of the two papers or in the difficulty indices. Difficulty and discrimination indices correlated well in both papers. Cronbach's alpha of paper I was 0.584 and that of paper II was 0.696. Point-biserial values were better for paper II than for paper I. Most students expressed dissatisfaction with paper II. One-Best multiple-choice questions showed better scores, higher reliability, better item performance and correlation values.

An Investigation of the Accuracy of Alternative Methods of True Score Estimation in High-Stakes Mixed-Format Examinations.

ERIC Educational Resources Information Center

Klinger, Don A.; Rogers, W. Todd

2003-01-01

The estimation accuracy of procedures based on classical test score theory and item response theory (generalized partial credit model) were compared for examinations consisting of multiple-choice and extended-response items. Analysis of British Columbia Scholarship Examination results found an error rate of about 10 percent for both methods, with…

Attainment of Selected Earth Science Concepts by Texas High School Seniors.

ERIC Educational Resources Information Center

Rollins, Mavis M.; And Others

The purpose of this study was to determine whether high school seniors (N=492) had attained each of five selected earth science concepts and if said attainment was influenced by the number of science courses completed. A 72-item, multiple-choice format test (12 items for each concept) was developed and piloted previous to this study to measure…

An Alternative to the 3PL: Using Asymmetric Item Characteristic Curves to Address Guessing Effects

ERIC Educational Resources Information Center

Lee, Sora; Bolt, Daniel M.

2018-01-01

Both the statistical and interpretational shortcomings of the three-parameter logistic (3PL) model in accommodating guessing effects on multiple-choice items are well documented. We consider the use of a residual heteroscedasticity (RH) model as an alternative, and compare its performance to the 3PL with real test data sets and through simulation…

Assessing the Life Science Knowledge of Students and Teachers Represented by the K-8 National Science Standards

ERIC Educational Resources Information Center

Sadler, Philip M.; Coyle, Harold; Cook Smith, Nancy; Miller, Jaimie; Mintzes, Joel; Tanner, Kimberly; Murray, John

2013-01-01

We report on the development of an item test bank and associated instruments based on the National Research Council (NRC) K-8 life sciences content standards. Utilizing hundreds of studies in the science education research literature on student misconceptions, we constructed 476 unique multiple-choice items that measure the degree to which test…

Translation of P = kT into a Pictorial External Representation by High School Seniors

ERIC Educational Resources Information Center

Matijaševic, Igor; Korolija, Jasminka N.; Mandic, Ljuba M.

2016-01-01

This paper describes the results achieved by high school seniors on an item which involves translation of the equation P = kT into a corresponding pictorial external representation. The majority of students (the classes of 2011, 2012 and 2013) did not give the correct answer to the multiple choice part of the translation item. They chose pictorial…

Comparing Two Types of Diagnostic Items to Evaluate Understanding of Heat and Temperature Concepts

ERIC Educational Resources Information Center

Chu, Hye-Eun; Chandrasegaran, A. L.; Treagust, David F.

2018-01-01

The purpose of this research was to investigate an efficient method to assess year 8 (age 13-14) students' conceptual understanding of heat and temperature concepts. Two different types of instruments were used in this study: Type 1, consisting of multiple-choice items with open-ended justifications; and Type 2, consisting of two-tier…

Interpreting Secondary Students' Performance on a Timed, Multiple-Choice Reading Comprehension Assessment: The Prevalence and Impact of Non-Attempted Items

ERIC Educational Resources Information Center

Clemens, Nathan H.; Davis, John L.; Simmons, Leslie E.; Oslund, Eric L.; Simmons, Deborah C.

2015-01-01

Standardized measures are often used as an index of students' reading comprehension and scores have important implications, particularly for students who perform below expectations. This study examined secondary-level students' patterns of responding and the prevalence and impact of non-attempted items on a timed, group-administered,…

An Instrument to Predict Job Performance of Home Health Aides--Testing the Reliability and Validity.

ERIC Educational Resources Information Center

Sturges, Jack; Quina, Patricia

The development of four paper-and-pencil tests, useful in assessing the effectiveness of inservice training provided to either nurses aides or home health aides, was described. These tests were designed for utilization in employment selection and case assignment. Two tests of 37 multiple-choice items and two tests of 10 matching items were…

An Investigation of the Impact of Guessing on Coefficient α and Reliability

PubMed Central

2014-01-01

Guessing is known to influence the test reliability of multiple-choice tests. Although there are many studies that have examined the impact of guessing, they used rather restrictive assumptions (e.g., parallel test assumptions, homogeneous inter-item correlations, homogeneous item difficulty, and homogeneous guessing levels across items) to evaluate the relation between guessing and test reliability. Based on the item response theory (IRT) framework, this study investigated the extent of the impact of guessing on reliability under more realistic conditions where item difficulty, item discrimination, and guessing levels actually vary across items with three different test lengths (TL). By accommodating multiple item characteristics simultaneously, this study also focused on examining interaction effects between guessing and other variables entered in the simulation to be more realistic. The simulation of the more realistic conditions and calculations of reliability and classical test theory (CTT) item statistics were facilitated by expressing CTT item statistics, coefficient α, and reliability in terms of IRT model parameters. In addition to the general negative impact of guessing on reliability, results showed interaction effects between TL and guessing and between guessing and test difficulty.

Force Concept Inventory-Based Multiple-Choice Test for Investigating Students' Representational Consistency

ERIC Educational Resources Information Center

Nieminen, Pasi; Savinainen, Antti; Viiri, Jouni

2010-01-01

This study investigates students' ability to interpret multiple representations consistently (i.e., representational consistency) in the context of the force concept. For this purpose we developed the Representational Variant of the Force Concept Inventory (R-FCI), which makes use of nine items from the 1995 version of the Force Concept Inventory…

Multiple Imputation of Multilevel Missing Data-Rigor versus Simplicity

ERIC Educational Resources Information Center

Drechsler, Jörg

2015-01-01

Multiple imputation is widely accepted as the method of choice to address item-nonresponse in surveys. However, research on imputation strategies for the hierarchical structures that are typically found in the data in educational contexts is still limited. While a multilevel imputation model should be preferred from a theoretical point of view if…

Practical Implications of Test Dimensionality for Item Response Theory Calibration of the Medical College Admission Test. MCAT Monograph.

ERIC Educational Resources Information Center

Childs, Ruth A.; Oppler, Scott H.

The use of item response theory (IRT) in the Medical College Admission Test (MCAT) testing program has been limited. This study provides a basis for future IRT analyses of the MCAT by exploring the dimensionality of each of the MCAT's three multiple-choice test sections (Verbal Reasoning, Physical Sciences, and Biological Sciences) and the…

Fundamental Fraction Knowledge of Preservice Elementary Teachers: A Cross-National Study in the United States and Taiwan

ERIC Educational Resources Information Center

Luo, Fenqjen; Lo, Jane-Jane; Leu, Yuh-Chyn

2011-01-01

The purpose of this paper is to show the similarities as well as the differences of fundamental fraction knowledge owned by preservice elementary teachers from the United States (N = 89) and Taiwan (N = 85). To this end, we examined and compared their performance on an instrument including 15 multiple-choice test items. The items were categorized…

An Investigation into the Relationship between Students' Conceptions of the Particulate Nature of Matter and Their Understanding of Chemical Bonding

ERIC Educational Resources Information Center

Othman, Jazilah; Treagust, David F.; Chandrasegaran, A. L.

2008-01-01

A thorough understanding of chemical bonding requires familiarity with the particulate nature of matter. In this study, a two-tier multiple-choice diagnostic instrument consisting of ten items (five items involving each of the two concepts) was developed to assess students' understanding of the particulate nature of matter and chemical bonding so…

PubMed Central

PANATTO, D.; ARATA, L.; BEVILACQUA, I.; APPRATO, L.; GASPARINI, R.; AMICIZIA, D.

2015-01-01

Summary Introduction. Health-related knowledge is often assessed through multiple-choice tests. Among the different types of formats, researchers may opt to use multiple-mark items, i.e. with more than one correct answer. Although multiple-mark items have long been used in the academic setting – sometimes with scant or inconclusive results – little is known about the implementation of this format in research on in-field health education and promotion. Methods. A study population of secondary school students completed a survey on nutrition-related knowledge, followed by a single- lecture intervention. Answers were scored by means of eight different scoring algorithms and analyzed from the perspective of classical test theory. The same survey was re-administered to a sample of the students in order to evaluate the short-term change in their knowledge. Results. In all, 286 questionnaires were analyzed. Partial scoring algorithms displayed better psychometric characteristics than the dichotomous rule. In particular, the algorithm proposed by Ripkey and the balanced rule showed greater internal consistency and relative efficiency in scoring multiple-mark items. A penalizing algorithm in which the proportion of marked distracters was subtracted from that of marked correct answers was the only one that highlighted a significant difference in performance between natives and immigrants, probably owing to its slightly better discriminatory ability. This algorithm was also associated with the largest effect size in the pre-/post-intervention score change. Discussion. The choice of an appropriate rule for scoring multiple- mark items in research on health education and promotion should consider not only the psychometric properties of single algorithms but also the study aims and outcomes, since scoring rules differ in terms of biasness, reliability, difficulty, sensitivity to guessing and discrimination. PMID:26900331

Influence of multiple categories on the prediction of unknown properties

PubMed Central

Verde, Michael F.; Murphy, Gregory L.; Ross, Brian H.

2006-01-01

Knowing an item's category helps us predict its unknown properties. Previous studies suggest that when asked to evaluate the probability of an unknown property, people tend to consider only an item's most likely category, ignoring alternative categories. In the present study, property prediction took the form of either a probability rating or a speeded, binary-choice judgment. Consistent with past findings, subjects ignored alternative categories in their probability ratings. However, their binary-choice judgments were influenced by alternative categories. This novel finding suggests that the way category knowledge is used in prediction depends critically on the form of the prediction. PMID:16156183

An item response curves analysis of the Force Concept Inventory

NASA Astrophysics Data System (ADS)

Morris, Gary A.; Harshman, Nathan; Branum-Martin, Lee; Mazur, Eric; Mzoughi, Taha; Baker, Stephen D.

2012-09-01

Several years ago, we introduced the idea of item response curves (IRC), a simplistic form of item response theory (IRT), to the physics education research community as a way to examine item performance on diagnostic instruments such as the Force Concept Inventory (FCI). We noted that a full-blown analysis using IRT would be a next logical step, which several authors have since taken. In this paper, we show that our simple approach not only yields similar conclusions in the analysis of the performance of items on the FCI to the more sophisticated and complex IRT analyses but also permits additional insights by characterizing both the correct and incorrect answer choices. Our IRC approach can be applied to a variety of multiple-choice assessments but, as applied to a carefully designed instrument such as the FCI, allows us to probe student understanding as a function of ability level through an examination of each answer choice. We imagine that physics teachers could use IRC analysis to identify prominent misconceptions and tailor their instruction to combat those misconceptions, fulfilling the FCI authors' original intentions for its use. Furthermore, the IRC analysis can assist test designers to improve their assessments by identifying nonfunctioning distractors that can be replaced with distractors attractive to students at various ability levels.

An Exploration of Positional Response Sets in Disadvantaged Children and a Technique for Reduction of Such Sets. Final Report.

ERIC Educational Resources Information Center

Victor, Jack

This study is concerned with an examination of tendencies among individuals or groups to vary in their selection of certain types of responses when the same choice is presented in some other form, the tendencies being termed "response sets." Positional response sets (PRS), to which multiple-choice type items are prone, have reportedly…

Performance of Certification and Recertification Examinees on Multiple Choice Test Items: Does Physician Age Have an Impact?

PubMed

Shen, Linjun; Juul, Dorthea; Faulkner, Larry R

2016-01-01

The development of recertification programs (now referred to as Maintenance of Certification or MOC) by the members of the American Board of Medical Specialties provides the opportunity to study knowledge base across the professional lifespan of physicians. Research results to date are mixed with some studies finding negative associations between age and various measures of competency and others finding no or minimal relationships. Four groups of multiple choice test items that were independently developed for certification and MOC examinations in psychiatry and neurology were administered to certification and MOC examinees within each specialty. Percent correct scores were calculated for each examinee. Differences between certification and MOC examinees were compared using unpaired t tests, and logistic regression was used to compare MOC and certification examinee performance on the common test items. Except for the neurology certification test items that addressed basic neurology concepts, the performance of the certification and MOC examinees was similar. The differences in performance on individual test items did not consistently favor one group or the other and could not be attributed to any distinguishable content or format characteristics of those items. The findings of this study are encouraging in that physicians who had recently completed residency training possessed clinical knowledge that was comparable to that of experienced physicians, and the experienced physicians' clinical knowledge was equivalent to that of recent residency graduates. The role testing can play in enhancing expertise is described.

Multimodal Likelihoods in Educational Assessment: Will the Real Maximum Likelihood Score Please Stand up?

ERIC Educational Resources Information Center

Wothke, Werner; Burket, George; Chen, Li-Sue; Gao, Furong; Shu, Lianghua; Chia, Mike

2011-01-01

It has been known for some time that item response theory (IRT) models may exhibit a likelihood function of a respondent's ability which may have multiple modes, flat modes, or both. These conditions, often associated with guessing of multiple-choice (MC) questions, can introduce uncertainty and bias to ability estimation by maximum likelihood…

Force Concept Inventory-based multiple-choice test for investigating students' representational consistency

NASA Astrophysics Data System (ADS)

Nieminen, Pasi; Savinainen, Antti; Viiri, Jouni

2010-07-01

This study investigates students’ ability to interpret multiple representations consistently (i.e., representational consistency) in the context of the force concept. For this purpose we developed the Representational Variant of the Force Concept Inventory (R-FCI), which makes use of nine items from the 1995 version of the Force Concept Inventory (FCI). These original FCI items were redesigned using various representations (such as motion map, vectorial and graphical), yielding 27 multiple-choice items concerning four central concepts underpinning the force concept: Newton’s first, second, and third laws, and gravitation. We provide some evidence for the validity and reliability of the R-FCI; this analysis is limited to the student population of one Finnish high school. The students took the R-FCI at the beginning and at the end of their first high school physics course. We found that students’ (n=168) representational consistency (whether scientifically correct or not) varied considerably depending on the concept. On average, representational consistency and scientifically correct understanding increased during the instruction, although in the post-test only a few students performed consistently both in terms of representations and scientifically correct understanding. We also compared students’ (n=87) results of the R-FCI and the FCI, and found that they correlated quite well.

Will a Short Training Session Improve Multiple-Choice Item-Writing Quality by Dental School Faculty? A Pilot Study.

PubMed

Dellinges, Mark A; Curtis, Donald A

2017-08-01

Faculty members are expected to write high-quality multiple-choice questions (MCQs) in order to accurately assess dental students' achievement. However, most dental school faculty members are not trained to write MCQs. Extensive faculty development programs have been used to help educators write better test items. The aim of this pilot study was to determine if a short workshop would result in improved MCQ item-writing by dental school faculty at one U.S. dental school. A total of 24 dental school faculty members who had previously written MCQs were randomized into a no-intervention group and an intervention group in 2015. Six previously written MCQs were randomly selected from each of the faculty members and given an item quality score. The intervention group participated in a training session of one-hour duration that focused on reviewing standard item-writing guidelines to improve in-house MCQs. The no-intervention group did not receive any training but did receive encouragement and an explanation of why good MCQ writing was important. The faculty members were then asked to revise their previously written questions, and these were given an item quality score. The item quality scores for each faculty member were averaged, and the difference from pre-training to post-training scores was evaluated. The results showed a significant difference between pre-training and post-training MCQ difference scores for the intervention group (p=0.04). This pilot study provides evidence that the training session of short duration was effective in improving the quality of in-house MCQs.

Do item-writing flaws reduce examinations psychometric quality?

PubMed

Pais, João; Silva, Artur; Guimarães, Bruno; Povo, Ana; Coelho, Elisabete; Silva-Pereira, Fernanda; Lourinho, Isabel; Ferreira, Maria Amélia; Severo, Milton

2016-08-11

The psychometric characteristics of multiple-choice questions (MCQ) changed when taking into account their anatomical sites and the presence of item-writing flaws (IWF). The aim is to understand the impact of the anatomical sites and the presence of IWF in the psychometric qualities of the MCQ. 800 Clinical Anatomy MCQ from eight examinations were classified as standard or flawed items and according to one of the eight anatomical sites. An item was classified as flawed if it violated at least one of the principles of item writing. The difficulty and discrimination indices of each item were obtained. 55.8 % of the MCQ were flawed items. The anatomical site of the items explained 6.2 and 3.2 % of the difficulty and discrimination parameters and the IWF explained 2.8 and 0.8 %, respectively. The impact of the IWF was heterogeneous, the Writing the Stem and Writing the Choices categories had a negative impact (higher difficulty and lower discrimination) while the other categories did not have any impact. The anatomical site effect was higher than IWF effect in the psychometric characteristics of the examination. When constructing MCQ, the focus should be in the topic/area of the items and only after in the presence of IWF.

Is It Working? Distractor Analysis Results from the Test Of Astronomy STandards (TOAST) Assessment Instrument

NASA Astrophysics Data System (ADS)

Slater, Stephanie

2009-05-01

The Test Of Astronomy STandards (TOAST) assessment instrument is a multiple-choice survey tightly aligned to the consensus learning goals stated by the American Astronomical Society - Chair's Conference on ASTRO 101, the American Association of the Advancement of Science's Project 2061 Benchmarks, and the National Research Council's National Science Education Standards. Researchers from the Cognition in Astronomy, Physics and Earth sciences Research (CAPER) Team at the University of Wyoming's Science and Math Teaching Center (UWYO SMTC) have been conducting a question-by-question distractor analysis procedure to determine the sensitivity and effectiveness of each item. In brief, the frequency each possible answer choice, known as a foil or distractor on a multiple-choice test, is determined and compared to the existing literature on the teaching and learning of astronomy. In addition to having statistical difficulty and discrimination values, a well functioning assessment item will show students selecting distractors in the relative proportions to how we expect them to respond based on known misconceptions and reasoning difficulties. In all cases, our distractor analysis suggests that all items are functioning as expected. These results add weight to the validity of the Test Of Astronomy STandards (TOAST) assessment instrument, which is designed to help instructors and researchers measure the impact of course-length duration instructional strategies for undergraduate science survey courses with learning goals tightly aligned to the consensus goals of the astronomy education community.

Attention! Can choices for low value food over high value food be trained?

PubMed

Zoltak, Michael J; Veling, Harm; Chen, Zhang; Holland, Rob W

2018-05-01

People choose high value food items over low value food items, because food choices are guided by the comparison of values placed upon choice alternatives. This value comparison process is also influenced by the amount of attention people allocate to different items. Recent research shows that choices for food items can be increased by training attention toward these items, with a paradigm named cued-approach training (CAT). However, previous work till now has only examined the influence of CAT on choices between two equally valued items. It has remained unclear whether CAT can increase choices for low value items when people choose between a low and high value food item. To address this question in the current study participants were cued to make rapid responses in CAT to certain low and high value items. Next, they made binary choices between low and high value items, where we systematically varied whether the low and high value items were cued or uncued. In two experiments, we found that participants overall preferred high over low value food items for real consumption. More important, their choices for low value items increased when only the low value item had been cued in CAT compared to when both low and high value items had not been cued. Exploratory analyses revealed that this effect was more pronounced for participants with a relatively small value difference between low and high value items. The present research thus suggests that CAT may be used to boost the choice and consumption of low value items via enhanced attention toward these items, as long as the value difference is not too large. Implications for facilitating choices for healthy food are discussed. Copyright © 2017 Elsevier Ltd. All rights reserved.

Assessment of High-school Students Engaged in the EarthLabs Climate Modules using the Climate Concept Inventory

NASA Astrophysics Data System (ADS)

McNeal, K.; Libarkin, J. C.; Ledley, T. S.; Gold, A. U.; Lynds, S. E.; Haddad, N.; Ellins, K.; Dunlap, C.; Bardar, E. W.; Youngman, E.

2015-12-01

Instructors must have on hand appropriate assessments that align with their teaching and learning goals in order to provide evidence of student learning. We have worked with curriculum developers and scientists to develop the Climate Concept Inventory (CCI), which meets goals of the EarthLabs Climate on-line curriculum. The developed concept inventory includes 19 content-driven multiple choice questions, six affective-based multiple choice questions, one confidence question, three open-ended questions, and eight demographic questions. Our analysis of the instrument applies item response theory and uses item characteristic curves. We have assessed over 500 students in nearly twenty high school classrooms in Mississippi and Texas that have engaged in the implementation of the EarthLabs curriculum and completed the CCI. Results indicate that students had pre-post gains on 9 out of 10 of the content-based multiple choice questions with positive gains in answer choice selection ranging from 1.72% to 42%. Students significantly reported increased confidence with 15% more students reporting that they were either very or fairly confident with their answers. Of the six affective questions posed, 5 out of 6 showed significant shifts towards gains in knowledge, awareness, and information about Earth's climate system. The research has resulted in a robust and validated climate concept inventory for use with advanced high school students, where we have been able to apply its use within the EarthLabs project.

Decision making under internal uncertainty: the case of multiple-choice tests with different scoring rules.

PubMed

Bereby-Meyer, Yoella; Meyer, Joachim; Budescu, David V

2003-02-01

This paper assesses framing effects on decision making with internal uncertainty, i.e., partial knowledge, by focusing on examinees' behavior in multiple-choice (MC) tests with different scoring rules. In two experiments participants answered a general-knowledge MC test that consisted of 34 solvable and 6 unsolvable items. Experiment 1 studied two scoring rules involving Positive (only gains) and Negative (only losses) scores. Although answering all items was the dominating strategy for both rules, the results revealed a greater tendency to answer under the Negative scoring rule. These results are in line with the predictions derived from Prospect Theory (PT) [Econometrica 47 (1979) 263]. The second experiment studied two scoring rules, which allowed respondents to exhibit partial knowledge. Under the Inclusion-scoring rule the respondents mark all answers that could be correct, and under the Exclusion-scoring rule they exclude all answers that might be incorrect. As predicted by PT, respondents took more risks under the Inclusion rule than under the Exclusion rule. The results illustrate that the basic process that underlies choice behavior under internal uncertainty and especially the effect of framing is similar to the process of choice under external uncertainty and can be described quite accurately by PT. Copyright 2002 Elsevier Science B.V.

Physics Achievement Test.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

This document is an evaluation instrument developed as a part of Harvard Project Physics (HPP). It consists of a 36-item, multiple choice (five options) Physics Achievement Test (PAT) designed to measure general knowledge of physics as well as the material emphasized in HPP. (PEB)

The Vitamin D Endocrine System.

ERIC Educational Resources Information Center

Norman, Anthony W.

1985-01-01

Discusses the physiology and biochemistry of the vitamin D endocrine system, including role of biological calcium and phosphorus, vitamin D metabolism, and related diseases. A 10-item, multiple-choice test which can be used to obtain continuing medical education credit is included. (JN)

Calibrating the Medical Council of Canada's Qualifying Examination Part I using an integrated item response theory framework: a comparison of models and designs.

PubMed

De Champlain, Andre F; Boulais, Andre-Philippe; Dallas, Andrew

2016-01-01

The aim of this research was to compare different methods of calibrating multiple choice question (MCQ) and clinical decision making (CDM) components for the Medical Council of Canada's Qualifying Examination Part I (MCCQEI) based on item response theory. Our data consisted of test results from 8,213 first time applicants to MCCQEI in spring and fall 2010 and 2011 test administrations. The data set contained several thousand multiple choice items and several hundred CDM cases. Four dichotomous calibrations were run using BILOG-MG 3.0. All 3 mixed item format (dichotomous MCQ responses and polytomous CDM case scores) calibrations were conducted using PARSCALE 4. The 2-PL model had identical numbers of items with chi-square values at or below a Type I error rate of 0.01 (83/3,499 or 0.02). In all 3 polytomous models, whether the MCQs were either anchored or concurrently run with the CDM cases, results suggest very poor fit. All IRT abilities estimated from dichotomous calibration designs correlated very highly with each other. IRT-based pass-fail rates were extremely similar, not only across calibration designs and methods, but also with regard to the actual reported decision to candidates. The largest difference noted in pass rates was 4.78%, which occurred between the mixed format concurrent 2-PL graded response model (pass rate= 80.43%) and the dichotomous anchored 1-PL calibrations (pass rate= 85.21%). Simpler calibration designs with dichotomized items should be implemented. The dichotomous calibrations provided better fit of the item response matrix than more complex, polytomous calibrations.

Improving Measures via Examining the Behavior of Distractors in Multiple-Choice Tests

PubMed Central

Sideridis, Georgios; Tsaousis, Ioannis; Al Harbi, Khaleel

2017-01-01

The purpose of the present article was to illustrate, using an example from a national assessment, the value from analyzing the behavior of distractors in measures that engage the multiple-choice format. A secondary purpose of the present article was to illustrate four remedial actions that can potentially improve the measurement of the construct(s) under study. Participants were 2,248 individuals who took a national examination of chemistry. The behavior of the distractors was analyzed by modeling their behavior within the Rasch model. Potentially informative distractors were (a) further modeled using the partial credit model, (b) split onto separate items and retested for model fit and parsimony, (c) combined to form a “super” item or testlet, and (d) reexamined after deleting low-ability individuals who likely guessed on those informative, albeit erroneous, distractors. Results indicated that all but the item split strategies were associated with better model fit compared with the original model. The best fitted model, however, involved modeling and crediting informative distractors via the partial credit model or eliminating the responses of low-ability individuals who likely guessed on informative distractors. The implications, advantages, and disadvantages of modeling informative distractors for measurement purposes are discussed. PMID:29795904

Multicategorical Spline Model for Item Response Theory.

ERIC Educational Resources Information Center

Abrahamowicz, Michal; Ramsay, James O.

1992-01-01

A nonparametric multicategorical model for multiple-choice data is proposed as an extension of the binary spline model of J. O. Ramsay and M. Abrahamowicz (1989). Results of two Monte Carlo studies illustrate the model, which approximates probability functions by rational splines. (SLD)

Development of the Newtonian Gravity Concept Inventory

ERIC Educational Resources Information Center

Williamson, Kathryn E.; Willoughby, Shannon; Prather, Edward E.

2013-01-01

We introduce the Newtonian Gravity Concept Inventory (NGCI), a 26-item multiple-choice instrument to assess introductory general education college astronomy ("Astro 101") student understanding of Newtonian gravity. This paper describes the development of the NGCI through four phases: Planning, Construction, Quantitative Analysis, and…

Test item linguistic complexity and assessments for deaf students.

PubMed

Cawthon, Stephanie

2011-01-01

Linguistic complexity of test items is one test format element that has been studied in the context of struggling readers and their participation in paper-and-pencil tests. The present article presents findings from an exploratory study on the potential relationship between linguistic complexity and test performance for deaf readers. A total of 64 students completed 52 multiple-choice items, 32 in mathematics and 20 in reading. These items were coded for linguistic complexity components of vocabulary, syntax, and discourse. Mathematics items had higher linguistic complexity ratings than reading items, but there were no significant relationships between item linguistic complexity scores and student performance on the test items. The discussion addresses issues related to the subject area, student proficiency levels in the test content, factors to look for in determining a "linguistic complexity effect," and areas for further research in test item development and deaf students.

Validation of an Instrument for Measuring Students' Understanding of Interdisciplinary Science in Grades 4-8 over Multiple Semesters: A Rasch Measurement Study

ERIC Educational Resources Information Center

Yang, Yang; He, Peng; Liu, Xiufeng

2018-01-01

So far, not enough effort has been invested in developing reliable, valid, and engaging assessments in school science, especially assessment of interdisciplinary science based on the new Next Generation Science Standards (NGSS). Furthermore, previous tools rely mostly on multiple-choice items and evaluation of student outcome is linked only to…

Get it while it's hot: a peak-first bias in self-generated choice order in rhesus macaques.

PubMed

Jung, Kanghoon; Kralik, Jerald D

2013-01-01

Animals typically must make a number of successive choices to achieve a goal: e.g., eating multiple food items before becoming satiated. However, it is unclear whether choosing the best first or saving the best for last represents the best choice strategy to maximize overall reward. Specifically, since outcomes can be evaluated prospectively (with future rewards discounted and more immediate rewards preferred) or retrospectively (with prior rewards discounted and more recent rewards preferred), the conditions under which each are used remains unclear. On the one hand, humans and non-human animals clearly discount future reward, preferring immediate rewards to delayed ones, suggesting prospective evaluation; on the other hand, it has also been shown that a sequence that ends well, i.e., with the best event or item last, is often preferred, suggesting retrospective evaluation. Here we hypothesized that when individuals are allowed to build the sequence themselves they are more likely to evaluate each item individually and therefore build a sequence using prospective evaluation. We examined the relationship between self-generated choice order and preference in rhesus monkeys in two experiments in which the distinctiveness of options were relatively high and low, respectively. We observed a positive linear relationship between choice order and preference among highly distinct options, indicating that the rhesus monkeys chose their preferred food first: i.e., a peak-first order preference. Overall, choice order depended on the degree of relative preference among alternatives and a peak-first bias, providing evidence for prospective evaluation when choice order is self-generated.

Developing a Better Understanding of the Process of Fat Absorption.

ERIC Educational Resources Information Center

Yip, Din Yan

2001-01-01

Performance on a multiple-choice item in a public examination indicates that most students do not understand how fat is absorbed through villi. A teaching strategy is suggested to overcome this problem by helping students review their own ideas critically. (Author/MM)

Wesleyan University Student Questionnaire.

ERIC Educational Resources Information Center

Haagen, C. Hess

This questionnaire assesses marijuana use practices in college students. The 30 items (multiple choice or free response) are concerned with personal and demographic data, marijuana smoking practices, use history, effects from smoking marijuana, present attitude toward the substance, and use of other drugs. The Questionnaire is untimed and…

Investigating High School Students' Understanding of Chemical Equilibrium Concepts

ERIC Educational Resources Information Center

Karpudewan, Mageswary; Treagust, David F.; Mocerino, Mauro; Won, Mihye; Chandrasegaran, A. L.

2015-01-01

This study investigated the year 12 students' (N = 56) understanding of chemical equilibrium concepts after instruction using two conceptual tests, the "Chemical Equilibrium Conceptual Test 1" ("CECT-1") consisting of nine two-tier multiple-choice items and the "Chemical Equilibrium Conceptual Test 2"…

Full Inclusion: The Least Restrictive Environment

ERIC Educational Resources Information Center

Mullings, Shirley E.

2011-01-01

The purpose of the phenomenological study was to examine elementary educators' perceptions of full inclusion as the least restrictive environment for students with disabilities. Thirty-six teachers and administrators participated in interviews and responded to multiple-choice survey items. The recorded data from the interviews were…

Comparing Item Performance on Three- Versus Four-Option Multiple Choice Questions in a Veterinary Toxicology Course.

PubMed

Royal, Kenneth; Dorman, David

2018-06-09

The number of answer options is an important element of multiple-choice questions (MCQs). Many MCQs contain four or more options despite the limited literature suggesting that there is little to no benefit beyond three options. The purpose of this study was to evaluate item performance on 3-option versus 4-option MCQs used in a core curriculum course in veterinary toxicology at a large veterinary medical school in the United States. A quasi-experimental, crossover design was used in which students in each class were randomly assigned to take one of two versions (A or B) of two major exams. Both the 3-option and 4-option MCQs resulted in similar psychometric properties. The findings of our study support earlier research in other medical disciplines and settings that likewise concluded there was no significant change in the psychometric properties of three option MCQs when compared to the traditional MCQs with four or more options.

Mathematics Assessment Sampler 3-5

ERIC Educational Resources Information Center

National Council of Teachers of Mathematics, 2005

2005-01-01

The sample assessment items in this volume are sorted according to the strands of number and operations, algebra, geometry, measurement, and data analysis and probability. Because one goal of assessment is to determine students' abilities to communicate mathematically, the writing team suggests ways to extend or modify multiple-choice and…

The Impact of Disclosure of Nutrition Information on Consumers' Behavioral Intention in Korea.

PubMed

Choi, Jinkyung

2015-01-01

To investigate the effect of nutritional information disclosure on consumers' nutritional perception, attitude, and behavioral intention to purchase the food item. Questionnaires were distributed measuring nutritional perception, attitude, and behavioral intention with different nutritional information about the food (no information, calories only, and six nutritional content information items: food weight(g), calories(kcal), protein(g), sugar(g), sodium(g), and saturated fat(g)). Food items shown to the respondents were hamburgers and bibimbap. Descriptive analysis, analysis of variance, and multiple regression were used in order to examine the effects of nutritional information levels and different food items on consumers' behavioral intentions. Nutritional perception, food attitude, and food choice intention were all affected by levels of nutritional information and different food items. Also, food attitude was a predictor of food choice behavioral intention and was affected by different food items as well. However, results of the study found that objective and subjective knowledge of individuals are not related to their nutritional perception, attitude, and behavioral intention. Results of this study would help restaurant managers to prepare for consumers' demand on disclosure of nutritional information and adjust their menu ingredients for consumers' healthy food inquiries in order to respond to consumers' interests in nutritional information and ensure consumers satisfaction with the perceived nutritional value of food.

Project Physics Tests 6, The Nucleus.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 6 are presented in this booklet. Included are 70 multiple-choice and 24 problem-and-essay questions. Nuclear physics fundamentals are examined with respect to the shell model, isotopes, neutrons, protons, nuclides, charge-to-mass ratios, alpha particles, Becquerel's discovery, gamma rays, cyclotrons,…

Item Readability and Science Achievement in TIMSS 2003 in South Africa

ERIC Educational Resources Information Center

Dempster, Edith R.; Reddy, Vijay

2007-01-01

This study investigated the relationship between readability of 73 text-only multiple-choice questions from Trends in International Mathematics and Science Study (TIMSS) 2003 and performance of two groups of South African learners: those with limited English-language proficiency (learners attending African schools) and those with better…

Constructing Multiple-Choice Items to Measure Higher-Order Thinking

ERIC Educational Resources Information Center

Scully, Darina

2017-01-01

Across education, certification and licensure, there are repeated calls for the development of assessments that target "higher-order thinking," as opposed to mere recall of facts. A common assumption is that this necessitates the use of constructed response or essay-style test questions; however, empirical evidence suggests that this may…

Utah Drug Use Questionnaire.

ERIC Educational Resources Information Center

Governor's Citizen Advisory Committee on Drugs, Salt Lake City, UT.

This questionnaire assesses drug use practices in junior and senior high school students. The 21 multiple choice items pertain to drug use practices, use history, available of drugs, main reason for drug use, and demographic data. The questionnaire is untimed, group administered, and may be given by the classroom teacher in about 10 minutes. Item…

Meatcutting Testbook, Part I.

ERIC Educational Resources Information Center

Strazicich, Mirko, Ed.

This document contains objective tests for each lesson in the Meatcutting Workbook, Part I (see note), which is designed for apprenticeship programs in meatcutting in California. Each of the 36 tests contains from 10 to 45 multiple-choice items. The tests are grouped according to the eight units of the workbook: the apprentice meatcutter; applied…

Project Physics Tests 5, Models of the Atom.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 5 are presented in this booklet. Included are 70 multiple-choice and 23 problem-and-essay questions. Concepts of atomic model are examined on aspects of relativistic corrections, electron emission, photoelectric effects, Compton effect, quantum theories, electrolysis experiments, atomic number and mass,…

Utah Drop-Out Drug Use Questionnaire.

ERIC Educational Resources Information Center

Governor's Citizen Advisory Committee on Drugs, Salt Lake City, UT.

This questionnaire assesses drug use practices in high school drop-outs. The 79 items (multiple choice or apply/not apply) are concerned with demographic data and use, use history, reasons for use/nonuse, attitudes toward drugs, availability of drugs, and drug information with respect to narcotics, amphetamines, LSD, Marijuana, and barbiturates.…

Heubach Smoking Habits and Attitudes Questionnaire.

ERIC Educational Resources Information Center

Heubach, Philip Gilbert

This Questionnaire, consisting of 74 yes/no, multiple choice, and completion items, is designed to assess smoking practices and attitudes toward smoking in high school students. Questions pertain to personal data, family smoking practices and attitudes, personal smoking habits, reasons for smoking or not smoking, and opinions on smoking. Detailed…

Beta-Blockers and the Kidney: Implications for Renal Function and Renin Release.

ERIC Educational Resources Information Center

Epstein, Murray; And Others

1985-01-01

Reviews and discusses current information on the human renal response as related to beta-blockers (antihypertension agents). Topic areas considered include cardioselectivity, renal hemodynamics, systemic hemodynamics, changes with acute and chronic administration, influence of dose, and others. Implications and an 11-item multiple-choice self-quiz…

Using Web-Based Practice to Enhance Mathematics Learning and Achievement

ERIC Educational Resources Information Center

Nguyen, Diem M.; Kulm, Gerald

2005-01-01

This article describes 1) the special features and accessibility of an innovative web-based practice instrument (WebMA) designed with randomized short-answer, matching and multiple choice items incorporated with automatically adapted feedback for middle school students; and 2) an exploratory study that compares the effects and contributions of…

Project Physics Tests 4, Light and Electromagnetism.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 4 are presented in this booklet. Included are 70 multiple-choice and 22 problem-and-essay questions. Concepts of light and electromagnetism are examined on charges, reflection, electrostatic forces, electric potential, speed of light, electromagnetic waves and radiations, Oersted's and Faraday's work,…

Michigan High School Student Drug Attitudes and Behavior Questionnaire.

ERIC Educational Resources Information Center

Bogg, Richard A.; And Others

This questionnaire assesses drug use practices and attitudes toward drugs in high school students. The instrument has 59 items (multiple choice or completion), some with several parts. The question pertain to aspirations for the future, general attitudes and opinions, biographic and demographic data, family background and relationships, alcohol…

Next-Generation Environments for Assessing and Promoting Complex Science Learning

ERIC Educational Resources Information Center

Quellmalz, Edys S.; Davenport, Jodi L.; Timms, Michael J.; DeBoer, George E.; Jordan, Kevin A.; Huang, Chun-Wei; Buckley, Barbara C.

2013-01-01

How can assessments measure complex science learning? Although traditional, multiple-choice items can effectively measure declarative knowledge such as scientific facts or definitions, they are considered less well suited for providing evidence of science inquiry practices such as making observations or designing and conducting investigations.…

Latent Image Processing Can Bolster the Value of Quizzes.

ERIC Educational Resources Information Center

Singer, David

1985-01-01

Latent image processing is a method which reveals hidden ink when marked with a special pen. Using multiple-choice items with commercially available latent image transfers can provide immediate feedback on take-home quizzes. Students benefitted from formative evaluation and were challenged to search for alternative solutions and explain unexpected…

Science Competencies That Go Unassessed

ERIC Educational Resources Information Center

Gilmer, Penny J.; Sherdan, Danielle M.; Oosterhof, Albert; Rohani, Faranak; Rouby, Aaron

2011-01-01

Present large-scale assessments require the use of item formats, such as multiple choice, that can be administered and scored efficiently. This limits competencies that can be measured by these assessments. An alternative approach to large-scale assessments is being investigated that would include the use of complex performance assessments. As…

Investigating Urban Eighth-Grade Students' Knowledge of Energy Resources

ERIC Educational Resources Information Center

Bodzin, Alec

2012-01-01

This study investigated urban eighth-grade students' knowledge of energy resources and associated issues including energy acquisition, energy generation, storage and transport, and energy consumption and conservation. A 39 multiple-choice-item energy resources knowledge assessment was completed by 1043 eighth-grade students in urban schools in two…

Nuclear Energy Assessment Battery. Form C.

ERIC Educational Resources Information Center

Showers, Dennis Edward

This publication consists of a nuclear energy assessment battery for secondary level students. The test contains 44 multiple choice items and is organized into four major sections. Parts include: (1) a knowledge scale; (2) attitudes toward nuclear energy; (3) a behaviors and intentions scale; and (4) an anxiety scale. Directions are provided for…

Development and Validation of the Homeostasis Concept Inventory

ERIC Educational Resources Information Center

McFarland, Jenny L.; Price, Rebecca M.; Wenderoth, Mary Pat; Martinková, Patrícia; Cliff, William; Michael, Joel; Modell, Harold; Wright, Ann

2017-01-01

We present the Homeostasis Concept Inventory (HCI), a 20-item multiple-choice instrument that assesses how well undergraduates understand this critical physiological concept. We used an iterative process to develop a set of questions based on elements in the Homeostasis Concept Framework. This process involved faculty experts and undergraduate…

Project Physics Tests 3, The Triumph of Mechanics.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 3 are presented in this booklet. Included are 70 multiple-choice and 20 problem-and-essay questions. Concepts of mechanics are examined on energy, momentum, kinetic theory of gases, pulse analyses, "heat death," water waves, power, conservation laws, normal distribution, thermodynamic laws, and…

Recall in older cancer patients: measuring memory for medical information.

PubMed

Jansen, Jesse; van Weert, Julia; van der Meulen, Nienke; van Dulmen, Sandra; Heeren, Thea; Bensing, Jozien

2008-04-01

Remembering medical treatment information may be particularly taxing for older cancer patients, but to our knowledge this ability has never been assessed in this specific age group only. Our purpose in this study was to investigate older cancer patients' recall of information after patient education preceding chemotherapy. We constructed a recall questionnaire consisting of multiple-choice questions, completion items, and open-ended questions related to information about treatment and recommendations on how to handle side effects. Immediately after a nursing consultation preceding chemotherapy treatment, 69 older patients (M = 71.8 years, SD = 4.1) completed the questionnaire. We checked recall against the actual communication in video recordings of the consultations. On average, 82.2 items were discussed during the consultations. The mean percentage of information recalled correctly was 23.2% for open-ended questions, 68.0% for completion items, and 80.2% for multiple-choice questions. Older cancer patients are confronted with a lot of information. Recall of information strongly depended on question format; especially active reproduction appeared to be poor. To improve treatment outcomes, it is important that cancer patients are able to actively retrieve knowledge about how to prevent and recognize adverse side effects and that this is checked by the health professional. We make suggestions on how to make information more memorable for older cancer patients.

High confidence in falsely recognizing prototypical faces.

PubMed

Sampaio, Cristina; Reinke, Victoria; Mathews, Jeffrey; Swart, Alexandra; Wallinger, Stephen

2018-06-01

We applied a metacognitive approach to investigate confidence in recognition of prototypical faces. Participants were presented with sets of faces constructed digitally as deviations from prototype/base faces. Participants were then tested with a simple recognition task (Experiment 1) or a multiple-choice task (Experiment 2) for old and new items plus new prototypes, and they showed a high rate of confident false alarms to the prototypes. Confidence and accuracy relationship in this face recognition paradigm was found to be positive for standard items but negative for the prototypes; thus, it was contingent on the nature of the items used. The data have implications for lineups that employ match-to-suspect strategies.

Exploring undergraduates' understanding of photosynthesis using diagnostic question clusters.

PubMed

Parker, Joyce M; Anderson, Charles W; Heidemann, Merle; Merrill, John; Merritt, Brett; Richmond, Gail; Urban-Lurain, Mark

2012-01-01

We present a diagnostic question cluster (DQC) that assesses undergraduates' thinking about photosynthesis. This assessment tool is not designed to identify individual misconceptions. Rather, it is focused on students' abilities to apply basic concepts about photosynthesis by reasoning with a coordinated set of practices based on a few scientific principles: conservation of matter, conservation of energy, and the hierarchical nature of biological systems. Data on students' responses to the cluster items and uses of some of the questions in multiple-choice, multiple-true/false, and essay formats are compared. A cross-over study indicates that the multiple-true/false format shows promise as a machine-gradable format that identifies students who have a mixture of accurate and inaccurate ideas. In addition, interviews with students about their choices on three multiple-choice questions reveal the fragility of students' understanding. Collectively, the data show that many undergraduates lack both a basic understanding of the role of photosynthesis in plant metabolism and the ability to reason with scientific principles when learning new content. Implications for instruction are discussed.

Exploring Undergraduates' Understanding of Photosynthesis Using Diagnostic Question Clusters

PubMed Central

Parker, Joyce M.; Anderson, Charles W.; Heidemann, Merle; Merrill, John; Merritt, Brett; Richmond, Gail; Urban-Lurain, Mark

2012-01-01

We present a diagnostic question cluster (DQC) that assesses undergraduates' thinking about photosynthesis. This assessment tool is not designed to identify individual misconceptions. Rather, it is focused on students' abilities to apply basic concepts about photosynthesis by reasoning with a coordinated set of practices based on a few scientific principles: conservation of matter, conservation of energy, and the hierarchical nature of biological systems. Data on students' responses to the cluster items and uses of some of the questions in multiple-choice, multiple-true/false, and essay formats are compared. A cross-over study indicates that the multiple-true/false format shows promise as a machine-gradable format that identifies students who have a mixture of accurate and inaccurate ideas. In addition, interviews with students about their choices on three multiple-choice questions reveal the fragility of students' understanding. Collectively, the data show that many undergraduates lack both a basic understanding of the role of photosynthesis in plant metabolism and the ability to reason with scientific principles when learning new content. Implications for instruction are discussed. PMID:22383617

Item analysis of university-wide multiple choice objective examinations: the experience of a Nigerian private university.

PubMed

Odukoya, Jonathan A; Adekeye, Olajide; Igbinoba, Angie O; Afolabi, A

2018-01-01

Teachers and Students worldwide often dance to the tune of tests and examinations. Assessments are powerful tools for catalyzing the achievement of educational goals, especially if done rightly. One of the tools for 'doing it rightly' is item analysis. The core objectives for this study, therefore, were: ascertaining the item difficulty and distractive indices of the university wide courses. A range of 112-1956 undergraduate students participated in this study. With the use of secondary data, the ex-post facto design was adopted for this project. In virtually all cases, majority of the items (ranging between 65% and 97% of the 70 items fielded in each course) did not meet psychometric standard in terms of difficulty and distractive indices and consequently needed to be moderated or deleted. Considering the importance of these courses, the need to apply item analyses when developing these tests was emphasized.

The promise and challenge of including multimedia items in medical licensure examinations: some insights from an empirical trial.

PubMed

Shen, Linjun; Li, Feiming; Wattleworth, Roberta; Filipetto, Frank

2010-10-01

The Comprehensive Osteopathic Medical Licensing Examination conducted a trial of multimedia items in the 2008-2009 Level 3 testing cycle to determine (1) if multimedia items were able to test additional elements of medical knowledge and skills and (2) how to develop effective multimedia items. Forty-four content-matched multimedia and text multiple-choice items were randomly delivered to Level 3 candidates. Logistic regression and paired-samples t tests were used for pairwise and group-level comparisons, respectively. Nine pairs showed significant differences in either difficulty or/and discrimination. Content analysis found that, if text narrations were less direct, multimedia materials could make items easier. When textbook terminologies were replaced by multimedia presentations, multimedia items could become more difficult. Moreover, a multimedia item was found not uniformly difficult for candidates at different ability levels, possibly because multimedia and text items tested different elements of a same concept. Multimedia items may be capable of measuring some constructs different from what text items can measure. Effective multimedia items with reasonable psychometric properties can be intentionally developed.

Student Opinion Inventory. Instructions for Use. Part A. Part B.

ERIC Educational Resources Information Center

National Study of School Evaluation, Arlington, VA.

An important part of any school's self-evaluation is student input or feedback. This inventory was developed in order to accomplish two goals: assessing student attitudes toward many facets of the school, and providing an opportunity for students to make recommendations for improvement. Thirty-four multiple choice items collect information on…

Opinions of Female Juvenile Delinquents on Language-Based Literacy Activities

ERIC Educational Resources Information Center

Sanger, Dixie; Ritzman, Mitzi; Stremlau, Aliza; Fairchild, Lindsey; Brunken, Cindy

2009-01-01

A mixed methods study was conducted to examine female juvenile delinquents' opinions and reactions on nine language-based literacy activities. Forty-one participants ranging in age from 13 to 18 years responded to a survey consisting of nine multiple-choice items and one open-ended question concerning the usefulness of activities. Quantitative and…

The Precalculus Concept Assessment: A Tool for Assessing Students' Reasoning Abilities and Understandings

ERIC Educational Resources Information Center

Carlson, Marilyn; Oehrtman, Michael; Engelke, Nicole

2010-01-01

This article describes the development of the Precalculus Concept Assessment (PCA) instrument, a 25-item multiple-choice exam. The reasoning abilities and understandings central to precalculus and foundational for beginning calculus were identified and characterized in a series of research studies and are articulated in the PCA Taxonomy. These…

A Study of Students' Readiness to Learn Calculus

ERIC Educational Resources Information Center

Carlson, Marilyn P.; Madison, Bernard; West, Richard D.

2015-01-01

The Calculus Concept Readiness (CCR) instrument assesses foundational understandings and reasoning abilities that have been documented to be essential for learning calculus. The CCR Taxonomy describes the understandings and reasoning abilities assessed by CCR. The CCR is a 25-item multiple-choice instrument that can be used as a placement test for…

The Influence of Distractor Strength and Response Order on MCQ Responding

ERIC Educational Resources Information Center

Kiat, John Emmanuel; Ong, Ai Rene; Ganesan, Asha

2018-01-01

Multiple-choice questions (MCQs) play a key role in standardised testing and in-class assessment. Research into the influence of within-item response order on MCQ characteristics has been mixed. While some researchers have shown preferential selection of response options presented earlier in the answer list, others have failed to replicate these…

Modeling Incorrect Responses to Multiple-Choice Items with Multilinear Formula Score Theory.

DTIC Science & Technology

1987-08-01

Eisenhower Avenue University of Leyden Alexandria, VA 22333 Education Research Center Boerhaavelaan 2 Dr. John M. Eddins 2334 EN Leyden University of...22302-0268 Dr. William Montague NPRDC Code 13 Dr. William L. Maloy San Diego, CA 92152-6800 Chief of Naval Education and Training Ms. Kathleen Moreno

Toward the Development of a Model to Estimate the Readability of Credentialing-Examination Materials

ERIC Educational Resources Information Center

Badgett, Barbara A.

2010-01-01

The purpose of this study was to develop a set of procedures to establish readability, including an equation, that accommodates the multiple-choice item format and occupational-specific language related to credentialing examinations. The procedures and equation should be appropriate for learning materials, examination materials, and occupational…

University of Michigan Drug Education Questionnaire.

ERIC Educational Resources Information Center

Francis, John Bruce; Patch, David J.

This questionnaire assesses attitudes toward potential drug education programs and drug use practices in college students. The 87 items (multiple choice or free response) pertain to the history and extent of usage of 27 different drugs, including two non-existent drugs which may be utilized as a validity check; attitude toward the content, format,…

The Instructional Effects of Matching or Mismatching Lesson and Posttest Screen Color

ERIC Educational Resources Information Center

Clariana, Roy B.

2004-01-01

This investigation considers the instructional effects of color as an over-arching context variable when learning from computer displays. The purpose of this investigation is to examine the posttest retrieval effects of color as a local, extra-item non-verbal lesson context variable for constructed-response versus multiple-choice posttest…

Applying Item Response Theory Methods to Examine the Impact of Different Response Formats

ERIC Educational Resources Information Center

Hohensinn, Christine; Kubinger, Klaus D.

2011-01-01

In aptitude and achievement tests, different response formats are usually used. A fundamental distinction must be made between the class of multiple-choice formats and the constructed response formats. Previous studies have examined the impact of different response formats applying traditional statistical approaches, but these influences can also…

Changing Ideas about the Periodic Table of Elements and Students' Alternative Concepts of Isotopes and Allotropes.

ERIC Educational Resources Information Center

Schmidt, Hans-Jurgen; Baumgartner, Tim; Eybe, Holger

2003-01-01

Investigates secondary school students' concepts of isotopes and allotropes and how the concepts are linked to the Periodic Table of Elements (PTE). Questions senior high school students with multiple choice items and interviews. Shows that students actively tried to make sense of what they had experienced. (KHR)

Pursuing the Qualities of a "Good" Test

ERIC Educational Resources Information Center

Coniam, David

2014-01-01

This article examines the issue of the quality of teacher-produced tests, limiting itself in the current context to objective, multiple-choice tests. The article investigates a short, two-part 20-item English language test. After a brief overview of the key test qualities of reliability and validity, the article examines the two subtests in terms…

Research in the Automation of Teaching. Technical Report.

ERIC Educational Resources Information Center

Zuckerman, Carl B.; And Others

An experiment was designed to compare the value of the Skinner Teaching Machine with more traditional teaching methods and to compare various means of presenting material via the teaching machine. Material from the United States Navy Basic Electricity course was programed into three series of items: one completion, one multiple choice, and one…

Understanding Misconceptions: Teaching and Learning in Middle School Physical Science

ERIC Educational Resources Information Center

Sadler, Philip M.; Sonnert, Gerhard

2016-01-01

In this study the authors set out to better understand the relationship between teacher knowledge of science and student learning. The authors administered identical multiple-choice assessment items both to teachers of middle school physical science and to their students throughout the school year. The authors found that teachers who have strong…

It’s the season! Seasonal changes of MyPyramid food groups in weekly Sunday grocery store sale advertisements

USDA-ARS?s Scientific Manuscript database

Background: Faced with tens of thousands of food choices, consumers frequently turn to promotional advertising, such as Sunday sales circulars, to make purchasing decisions. To date, little research has examined the content of sales circulars over multiple seasons. Methods: Food items from 12 months...

Development of a State-Wide Competency Test for Marketing Education. Final Report.

ERIC Educational Resources Information Center

Smith, Clifton L.

A project was conducted to develop a valid, competency-referenced test on the core competencies identified for the Missouri Fundamentals of Marketing curriculum. During the project: (1) multiple-choice test items based on the core competencies in the Fundamentals of Marketing curriculum were developed; (2) instructions for onsite administration of…

Selected Test Items in American History. Bulletin Number 6, Fifth Edition.

ERIC Educational Resources Information Center

Anderson, Howard R.; Lindquist, E. F.

Designed for high school students, this bulletin provides an extensive file of 1,062 multiple-choice questions in American history. Taken largely from the Iowa Every-Pupil Program and the Cooperative Test Service standardized examinations, the questions are chronologically divided into 16 topic areas. They include exploration and discovery;…

SCHOOL ANXIETY AND THE FACILITATION OF PERFORMANCE.

ERIC Educational Resources Information Center

DUNN, JAMES A.; SCHELKUN, RUTH F.

THE RELATIONSHIPS BETWEEN SCHOOL GENERATED ANXIETY AND VARIOUS INDICES OF SCHOOL ACHIEVEMENT, CREATIVITY, AGE, AND IQ, ARE INVESTIGATED. A 160 ITEM, MULTIPLE-CHOICE, MULTI-SCALE, SCHOOL ANXIETY QUESTIONNAIRE WAS ADMINISTERED TO 56 FOURTH, FIFTH, AND SIXTH GRADE CHILDREN WITH A MEAN STANFORD BINET IQ OF 126 FROM AN UPPER MIDDLE CLASS COMMUNITY.…

Assessment of Electrochemical Concepts: A Comparative Study Involving Senior High-School Students in Indonesia and Japan

ERIC Educational Resources Information Center

Rahayu, Sri; Treagust, David F.; Chandrasegaran, A. L.; Kita, Masakazu; Ibnu, Suhadi

2011-01-01

Background and purpose: This study investigated Indonesian and Japanese senior high-school students' understanding of electrochemistry concepts. Sample: The questionnaire was administered to 244 Indonesian and 189 Japanese public senior high-school students. Design and methods: An 18-item multiple-choice questionnaire relating to five conceptual…

How Much Detail Needs to Be Elucidated in Self-Harm Research?

ERIC Educational Resources Information Center

Stanford, Sarah; Jones, Michael P.

2010-01-01

Assessing self-harm through brief multiple choice items is simple and less invasive than more detailed methods of assessment. However, there is currently little validation for brief methods of self-harm assessment. This study evaluates the extent to which adolescents' perceptions of self-harm agree with definitions in the literature, and what…

An Introduction to Multilinear Formula Score Theory. Measurement Series 84-4.

ERIC Educational Resources Information Center

Levine, Michael V.

Formula score theory (FST) associates each multiple choice test with a linear operator and expresses all of the real functions of item response theory as linear combinations of the operator's eigenfunctions. Hard measurement problems can then often be reformulated as easier, standard mathematical problems. For example, the problem of estimating…

Progress Monitoring in Grade 5 Science for Low Achievers

ERIC Educational Resources Information Center

Vannest, Kimberly J.; Parker, Richard; Dyer, Nicole

2011-01-01

This article presents procedures and results from a 2-year project developing science key vocabulary (KV) short tests suitable for progress monitoring Grade 5 science in Texas public schools using computer-generated, -administered, and -scored assessments. KV items included KV definitions and important usages in a multiple-choice cloze format. A…

The None-of-the-Above Option: An Empirical Study.

ERIC Educational Resources Information Center

Frary, Robert B.

1991-01-01

The use of the "none-of-the-above" option (NOTA) in 20 college-level multiple-choice tests was evaluated for classes with 100 or more students. Eight academic disciplines were represented, and 295 NOTA and 724 regular test items were used. It appears that the NOTA can be compatible with good classroom measurement. (TJH)

Technical Adequacy of the easyCBM Grade 2 Reading Measures. Technical Report #1004

ERIC Educational Resources Information Center

Jamgochian, Elisa; Park, Bitnara Jasmine; Nese, Joseph F. T.; Lai, Cheng-Fei; Saez, Leilani; Anderson, Daniel; Alonzo, Julie; Tindal, Gerald

2010-01-01

In this technical report, we provide reliability and validity evidence for the easyCBM[R] Reading measures for grade 2 (word and passage reading fluency and multiple choice reading comprehension). Evidence for reliability includes internal consistency and item invariance. Evidence for validity includes concurrent, predictive, and construct…

Project Physics Tests 2, Motion in the Heavens.

ERIC Educational Resources Information Center

Harvard Univ., Cambridge, MA. Harvard Project Physics.

Test items relating to Project Physics Unit 2 are presented in this booklet. Included are 70 multiple-choice and 22 problem-and-essay questions. Concepts of motion in the heavens are examined for planetary motions, heliocentric theory, forces exerted on the planets, Kepler's laws, gravitational force, Galileo's work, satellite orbits, Jupiter's…

Relationship between affective determinants and achievement in science for seventeen-year-olds

NASA Astrophysics Data System (ADS)

Napier, John D.; Riley, Joseph P.

Data collected in the 1976-1977 NAEP survey of seventeen-year-olds was used to reanalyze the hypothesis that there are affective determinates of science achievement. Factor and item analysis procedures were used to examine affective and cognitive items from Booklet 4. Eight affective scales and one cognitive achievement scale were identified. Using stepwise multiple regression procedures, the four affective scales of Motivation, Anxiety, Student Choice, and Teacher Support were found to account for the majority of the correlation between the affective determinants and achievement.

Efficient Methods of Estimating the Operating Characteristics of Item Response Categories and Challenge to a New Model for the Multiple-Choice Item

DTIC Science & Technology

1981-11-01

i very little effort has been put upon the model validation, which is essential in any scientific research. T’-- -rientation we aim at in the present...better than the former to the target function. This implies that, although the interval of ability e of our interest is even a little smaller than [-3.0...approaches turned out to be similar, with some deviations, i.e., some of them are a little closer to the theoretical density function, and some of

The Environment Makes a Difference: The Impact of Explicit and Implicit Attitudes as Precursors in Different Food Choice Tasks

PubMed Central

König, Laura M.; Giese, Helge; Schupp, Harald T.; Renner, Britta

2016-01-01

Studies show that implicit and explicit attitudes influence food choice. However, precursors of food choice often are investigated using tasks offering a very limited number of options despite the comparably complex environment surrounding real life food choice. In the present study, we investigated how the assortment impacts the relationship between implicit and explicit attitudes and food choice (confectionery and fruit), assuming that a more complex choice architecture is more taxing on cognitive resources. Specifically, a binary and a multiple option choice task based on the same stimulus set (fake food items) were presented to ninety-seven participants. Path modeling revealed that both explicit and implicit attitudes were associated with relative food choice (confectionery vs. fruit) in both tasks. In the binary option choice task, both explicit and implicit attitudes were significant precursors of food choice, with explicit attitudes having a greater impact. Conversely, in the multiple option choice task, the additive impact of explicit and implicit attitudes was qualified by an interaction indicating that, even if explicit and implicit attitudes toward confectionery were inconsistent, more confectionery was chosen than fruit if either was positive. This compensatory ‘one is sufficient’-effect indicates that the structure of the choice environment modulates the relationship between attitudes and choice. The study highlights that environmental constraints, such as the number of choice options, are an important boundary condition that need to be included when investigating the relationship between psychological precursors and behavior. PMID:27621719

The Environment Makes a Difference: The Impact of Explicit and Implicit Attitudes as Precursors in Different Food Choice Tasks.

PubMed

König, Laura M; Giese, Helge; Schupp, Harald T; Renner, Britta

2016-01-01

Studies show that implicit and explicit attitudes influence food choice. However, precursors of food choice often are investigated using tasks offering a very limited number of options despite the comparably complex environment surrounding real life food choice. In the present study, we investigated how the assortment impacts the relationship between implicit and explicit attitudes and food choice (confectionery and fruit), assuming that a more complex choice architecture is more taxing on cognitive resources. Specifically, a binary and a multiple option choice task based on the same stimulus set (fake food items) were presented to ninety-seven participants. Path modeling revealed that both explicit and implicit attitudes were associated with relative food choice (confectionery vs. fruit) in both tasks. In the binary option choice task, both explicit and implicit attitudes were significant precursors of food choice, with explicit attitudes having a greater impact. Conversely, in the multiple option choice task, the additive impact of explicit and implicit attitudes was qualified by an interaction indicating that, even if explicit and implicit attitudes toward confectionery were inconsistent, more confectionery was chosen than fruit if either was positive. This compensatory 'one is sufficient'-effect indicates that the structure of the choice environment modulates the relationship between attitudes and choice. The study highlights that environmental constraints, such as the number of choice options, are an important boundary condition that need to be included when investigating the relationship between psychological precursors and behavior.

Evaluation of the flipped classroom approach in a veterinary professional skills course

PubMed Central

Moffett, Jenny; Mill, Aileen C

2014-01-01

Background The flipped classroom is an educational approach that has had much recent coverage in the literature. Relatively few studies, however, use objective assessment of student performance to measure the impact of the flipped classroom on learning. The purpose of this study was to evaluate the use of a flipped classroom approach within a medical education setting to the first two levels of Kirkpatrick and Kirkpatrick’s effectiveness of training framework. Methods This study examined the use of a flipped classroom approach within a professional skills course offered to postgraduate veterinary students. A questionnaire was administered to two cohorts of students: those who had completed a traditional, lecture-based version of the course (Introduction to Veterinary Medicine [IVM]) and those who had completed a flipped classroom version (Veterinary Professional Foundations I [VPF I]). The academic performance of students within both cohorts was assessed using a set of multiple-choice items (n=24) nested within a written examination. Data obtained from the questionnaire were analyzed using Cronbach’s alpha, Kruskal–Wallis tests, and factor analysis. Data obtained from student performance in the written examination were analyzed using the nonparametric Wilcoxon rank sum test. Results A total of 133 IVM students and 64 VPF I students (n=197) agreed to take part in the study. Overall, study participants favored the flipped classroom approach over the traditional classroom approach. With respect to student academic performance, the traditional classroom students outperformed the flipped classroom students on a series of multiple-choice items (IVM mean =21.4±1.48 standard deviation; VPF I mean =20.25±2.20 standard deviation; Wilcoxon test, w=7,578; P<0.001). Conclusion This study demonstrates that learners seem to prefer a flipped classroom approach. The flipped classroom was rated more positively than the traditional classroom on many different characteristics. This preference, however, did not translate into improved student performance, as assessed by a series of multiple-choice items delivered during a written examination. PMID:25419164

Evaluation of the flipped classroom approach in a veterinary professional skills course.

PubMed

Moffett, Jenny; Mill, Aileen C

2014-01-01

The flipped classroom is an educational approach that has had much recent coverage in the literature. Relatively few studies, however, use objective assessment of student performance to measure the impact of the flipped classroom on learning. The purpose of this study was to evaluate the use of a flipped classroom approach within a medical education setting to the first two levels of Kirkpatrick and Kirkpatrick's effectiveness of training framework. This study examined the use of a flipped classroom approach within a professional skills course offered to postgraduate veterinary students. A questionnaire was administered to two cohorts of students: those who had completed a traditional, lecture-based version of the course (Introduction to Veterinary Medicine [IVM]) and those who had completed a flipped classroom version (Veterinary Professional Foundations I [VPF I]). The academic performance of students within both cohorts was assessed using a set of multiple-choice items (n=24) nested within a written examination. Data obtained from the questionnaire were analyzed using Cronbach's alpha, Kruskal-Wallis tests, and factor analysis. Data obtained from student performance in the written examination were analyzed using the nonparametric Wilcoxon rank sum test. A total of 133 IVM students and 64 VPF I students (n=197) agreed to take part in the study. Overall, study participants favored the flipped classroom approach over the traditional classroom approach. With respect to student academic performance, the traditional classroom students outperformed the flipped classroom students on a series of multiple-choice items (IVM mean =21.4±1.48 standard deviation; VPF I mean =20.25±2.20 standard deviation; Wilcoxon test, w=7,578; P<0.001). This study demonstrates that learners seem to prefer a flipped classroom approach. The flipped classroom was rated more positively than the traditional classroom on many different characteristics. This preference, however, did not translate into improved student performance, as assessed by a series of multiple-choice items delivered during a written examination.

Training impulsive choices for healthy and sustainable food.

PubMed

Veling, Harm; Chen, Zhang; Tombrock, Merel C; Verpaalen, Iris A M; Schmitz, Laura I; Dijksterhuis, Ap; Holland, Rob W

2017-06-01

Many people find it hard to change their dietary choices. Food choice often occurs impulsively, without deliberation, and it has been unclear whether impulsive food choice can be experimentally created. Across 3 exploratory and 2 confirmatory preregistered experiments we examined whether impulsive food choice can be trained. Participants were cued to make motor responses upon the presentation of, among others, healthy and sustainable food items. They subsequently selected these food items more often for actual consumption when they needed to make their choices impulsively as a result of time pressure. This effect disappeared when participants were asked to think about their choices, merely received more time to make their choices, or when choosing required attention to alternatives. Participants preferred high to low valued food items under time pressure and without time pressure, suggesting that the impulsive choices reflect valid preferences. These findings demonstrate that it is possible to train impulsive choices for food items while leaving deliberative choices for these items unaffected, and connect research on attention training to dual-process theories of decision making. The present research suggests that attention training may lead to behavioral change only when people behave impulsively. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

Should essays and other "open-ended"-type questions retain a place in written summative assessment in clinical medicine?

PubMed

Hift, Richard J

2014-11-28

Written assessments fall into two classes: constructed-response or open-ended questions, such as the essay and a number of variants of the short-answer question, and selected-response or closed-ended questions; typically in the form of multiple-choice. It is widely believed that constructed response written questions test higher order cognitive processes in a manner that multiple-choice questions cannot, and consequently have higher validity. An extensive review of the literature suggests that in summative assessment neither premise is evidence-based. Well-structured open-ended and multiple-choice questions appear equivalent in their ability to assess higher cognitive functions, and performance in multiple-choice assessments may correlate more highly than the open-ended format with competence demonstrated in clinical practice following graduation. Studies of construct validity suggest that both formats measure essentially the same dimension, at least in mathematics, the physical sciences, biology and medicine. The persistence of the open-ended format in summative assessment may be due to the intuitive appeal of the belief that synthesising an answer to an open-ended question must be both more cognitively taxing and similar to actual experience than is selecting a correct response. I suggest that cognitive-constructivist learning theory would predict that a well-constructed context-rich multiple-choice item represents a complex problem-solving exercise which activates a sequence of cognitive processes which closely parallel those required in clinical practice, hence explaining the high validity of the multiple-choice format. The evidence does not support the proposition that the open-ended assessment format is superior to the multiple-choice format, at least in exit-level summative assessment, in terms of either its ability to test higher-order cognitive functioning or its validity. This is explicable using a theory of mental models, which might predict that the multiple-choice format will have higher validity, a statement for which some empiric support exists. Given the superior reliability and cost-effectiveness of the multiple-choice format consideration should be given to phasing out open-ended format questions in summative assessment. Whether the same applies to non-exit-level assessment and formative assessment is a question which remains to be answered; particularly in terms of the educational effect of testing, an area which deserves intensive study.

Comprehension of confidence intervals - development and piloting of patient information materials for people with multiple sclerosis: qualitative study and pilot randomised controlled trial.

PubMed

Rahn, Anne C; Backhus, Imke; Fuest, Franz; Riemann-Lorenz, Karin; Köpke, Sascha; van de Roemer, Adrianus; Mühlhauser, Ingrid; Heesen, Christoph

2016-09-20

Presentation of confidence intervals alongside information about treatment effects can support informed treatment choices in people with multiple sclerosis. We aimed to develop and pilot-test different written patient information materials explaining confidence intervals in people with relapsing-remitting multiple sclerosis. Further, a questionnaire on comprehension of confidence intervals was developed and piloted. We developed different patient information versions aiming to explain confidence intervals. We used an illustrative example to test three different approaches: (1) short version, (2) "average weight" version and (3) "worm prophylaxis" version. Interviews were conducted using think-aloud and teach-back approaches to test feasibility and analysed using qualitative content analysis. To assess comprehension of confidence intervals, a six-item multiple choice questionnaire was developed and tested in a pilot randomised controlled trial using the online survey software UNIPARK. Here, the average weight version (intervention group) was tested against a standard patient information version on confidence intervals (control group). People with multiple sclerosis were invited to take part using existing mailing-lists of people with multiple sclerosis in Germany and were randomised using the UNIPARK algorithm. Participants were blinded towards group allocation. Primary endpoint was comprehension of confidence intervals, assessed with the six-item multiple choice questionnaire with six points representing perfect knowledge. Feasibility of the patient information versions was tested with 16 people with multiple sclerosis. For the pilot randomised controlled trial, 64 people with multiple sclerosis were randomised (intervention group: n = 36; control group: n = 28). More questions were answered correctly in the intervention group compared to the control group (mean 4.8 vs 3.8, mean difference 1.1 (95 % CI 0.42-1.69), p = 0.002). The questionnaire's internal consistency was moderate (Cronbach's alpha = 0.56). The pilot-phase shows promising results concerning acceptability and feasibility. Pilot randomised controlled trial results indicate that the patient information is well understood and that knowledge gain on confidence intervals can be assessed with a set of six questions. German Clinical Trials Register: DRKS00008561 . Registered 8th of June 2015.

How do STEM-interested students pursue multiple interests in their higher educational choice?

NASA Astrophysics Data System (ADS)

Vulperhorst, Jonne Pieter; Wessels, Koen Rens; Bakker, Arthur; Akkerman, Sanne Floor

2018-05-01

Interest in science, technology, engineering and mathematics (STEM) has lately received attention in research due to a gap between the number of STEM students and the needs of the labour market. As interest seems to be one of the most important factors in deciding what to study, we focus in the present study on how STEM-interested students weigh multiple interests in making educational choices. A questionnaire with both open-ended and closed-ended items was administered to 91 STEM-interested students enrolled in a STEM programme of a Dutch University for secondary school students. Results indicate that students find it important that a study programme allows them to pursue multiple interests. Some students pursued multiple interests by choosing to enrol in two programmes at the same time. Most students chose one programme that enabled them to combine multiple interests. Combinations of pursued interests were dependent on the disciplinary range of interests of students. Students who were interested in diverse domains combined interests in an educational programme across academic and non-academic domains, whilst students who were mainly interested in STEM combined only STEM-focused interests. Together these findings stress the importance of taking a multiple interest perspective on interest development and educational choice.

Using a MaxEnt Classifier for the Automatic Content Scoring of Free-Text Responses

NASA Astrophysics Data System (ADS)

Sukkarieh, Jana Z.

2011-03-01

Criticisms against multiple-choice item assessments in the USA have prompted researchers and organizations to move towards constructed-response (free-text) items. Constructed-response (CR) items pose many challenges to the education community—one of which is that they are expensive to score by humans. At the same time, there has been widespread movement towards computer-based assessment and hence, assessment organizations are competing to develop automatic content scoring engines for such items types—which we view as a textual entailment task. This paper describes how MaxEnt Modeling is used to help solve the task. MaxEnt has been used in many natural language tasks but this is the first application of the MaxEnt approach to textual entailment and automatic content scoring.

Revisiting the role of recollection in item versus forced-choice recognition memory.

PubMed

Cook, Gabriel I; Marsh, Richard L; Hicks, Jason L

2005-08-01

Many memory theorists have assumed that forced-choice recognition tests can rely more on familiarity, whereas item (yes-no) tests must rely more on recollection. In actuality, several studies have found no differences in the contributions of recollection and familiarity underlying the two different test formats. Using word frequency to manipulate stimulus characteristics, the present study demonstrated that the contributions of recollection to item versus forced-choice tests is variable. Low word frequency resulted in significantly more recollection in an item test than did a forced-choice procedure, but high word frequency produced the opposite result. These results clearly constrain any uniform claim about the degree to which recollection supports responding in item versus forced-choice tests.

A multi-level differential item functioning analysis of trends in international mathematics and science study: Potential sources of gender and minority difference among U.S. eighth graders' science achievement

NASA Astrophysics Data System (ADS)

Qian, Xiaoyu

Science is an area where a large achievement gap has been observed between White and minority, and between male and female students. The science minority gap has continued as indicated by the National Assessment of Educational Progress and the Trends in International Mathematics and Science Studies (TIMSS). TIMSS also shows a gender gap favoring males emerging at the eighth grade. Both gaps continue to be wider in the number of doctoral degrees and full professorships awarded (NSF, 2008). The current study investigated both minority and gender achievement gaps in science utilizing a multi-level differential item functioning (DIF) methodology (Kamata, 2001) within fully Bayesian framework. All dichotomously coded items from TIMSS 2007 science assessment at eighth grade were analyzed. Both gender DIF and minority DIF were studied. Multi-level models were employed to identify DIF items and sources of DIF at both student and teacher levels. The study found that several student variables were potential sources of achievement gaps. It was also found that gender DIF favoring male students was more noticeable in the content areas of physics and earth science than biology and chemistry. In terms of item type, the majority of these gender DIF items were multiple choice than constructed response items. Female students also performed less well on items requiring visual-spatial ability. Minority students performed significantly worse on physics and earth science items as well. A higher percentage of minority DIF items in earth science and biology were constructed response than multiple choice items, indicating that literacy may be the cause of minority DIF. Three-level model results suggested that some teacher variables may be the cause of DIF variations from teacher to teacher. It is essential for both middle school science teachers and science educators to find instructional methods that work more effectively to improve science achievement of both female and minority students. Physics and earth science are two areas to be improved for both groups. Curriculum and instruction need to enhance female students' learning interests and give them opportunities to improve their visual perception skills. Science instruction should address improving minority students' literacy skills while teaching science.

The Impact of Television on Public Environmental Knowledge Concerning the Great Lakes.

ERIC Educational Resources Information Center

Brothers, Christine C.

The purpose of this study was to collect baseline information about public knowledge of and opinions toward the Great Lakes and to measure the impact of a television news program in educating adults about the Great Lakes. Survey questionnaires containing multiple-choice knowledge items and Likert scale opinion statements were completed by 570…

Preference of Students on the Format of Options in a Multiple-Choice Test

ERIC Educational Resources Information Center

Oyzon, Voltaire Q.; Bendulo, Hermabeth O.; Tibus, Erlinda D.; Bande, Rhodora A.; Macalinao, Myrna L.

2016-01-01

Schools in the Philippines, especially those that are offering teacher education programs, are advised to construct examinations that are Licensure Examination for Teachers (LET)-like test items. This is because "if any aspect of a test is unfamiliar to candidates, they are likely to perform less well than they would do otherwise on…

An Odds Ratio Approach for Detecting DDF under the Nested Logit Modeling Framework

ERIC Educational Resources Information Center

Terzi, Ragip; Suh, Youngsuk

2015-01-01

An odds ratio approach (ORA) under the framework of a nested logit model was proposed for evaluating differential distractor functioning (DDF) in multiple-choice items and was compared with an existing ORA developed under the nominal response model. The performances of the two ORAs for detecting DDF were investigated through an extensive…

Developing Information Skills Test for Malaysian Youth Students Using Rasch Analysis

ERIC Educational Resources Information Center

Karim, Aidah Abdul; Shah, Parilah M.; Din, Rosseni; Ahmad, Mazalah; Lubis, Maimun Aqhsa

2014-01-01

This study explored the psychometric properties of a locally developed information skills test for youth students in Malaysia using Rasch analysis. The test was a combination of 24 structured and multiple choice items with a 4-point grading scale. The test was administered to 72 technical college students and 139 secondary school students. The…

Construction of Valid and Reliable Test for Assessment of Students

ERIC Educational Resources Information Center

Osadebe, P. U.

2015-01-01

The study was carried out to construct a valid and reliable test in Economics for secondary school students. Two research questions were drawn to guide the establishment of validity and reliability for the Economics Achievement Test (EAT). It is a multiple choice objective test of five options with 100 items. A sample of 1000 students was randomly…

Construction of Economics Achievement Test for Assessment of Students

ERIC Educational Resources Information Center

Osadebe, P. U.

2014-01-01

The study was carried out to construct a valid and reliable test in Economics for secondary school students. Two research questions were drawn to guide the establishment of validity and reliability for the Economics Achievement Test (EAT). It is a multiple choice objective test of five options with 100 items. A sample of 1000 students was randomly…

Validation of a Standardized Multiple-Choice Multicultural Competence Test: Implications for Training, Assessment, and Practice

ERIC Educational Resources Information Center

Gillem, Angela R.; Bartoli, Eleonora; Bertsch, Kristin N.; McCarthy, Maureen A.; Constant, Kerra; Marrero-Meisky, Sheila; Robbins, Steven J.; Bellamy, Scarlett

2016-01-01

The Multicultural Counseling and Psychotherapy Test (MCPT), a measure of multicultural counseling competence (MCC), was validated in 2 phases. In Phase 1, the authors administered 451 test items derived from multicultural guidelines in counseling and psychology to 32 multicultural experts and 30 nonexperts. In Phase 2, the authors administered the…

Correction for Guessing in the Framework of the 3PL Item Response Theory

ERIC Educational Resources Information Center

Chiu, Ting-Wei

2010-01-01

Guessing behavior is an important topic with regard to assessing proficiency on multiple choice tests, particularly for examinees at lower levels of proficiency due to greater the potential for systematic error or bias which that inflates observed test scores. Methods that incorporate a correction for guessing on high-stakes tests generally rely…

Data Collection Design for Equivalent Groups Equating: Using a Matrix Stratification Framework for Mixed-Format Assessment

ERIC Educational Resources Information Center

Mbella, Kinge Keka

2012-01-01

Mixed-format assessments are increasingly being used in large scale standardized assessments to measure a continuum of skills ranging from basic recall to higher order thinking skills. These assessments are usually comprised of a combination of (a) multiple-choice items which can be efficiently scored, have stable psychometric properties, and…

Youth Risk Behavior Survey Results, 1995. Executive Summary.

ERIC Educational Resources Information Center

New Hampshire State Dept. of Education, Concord.

An 84-item multiple choice Youth Risk Behavior Survey was administered to 2,092 students in 62 public high schools in New Hampshire during the spring of 1995. The survey covered behaviors in six categories: (1) behaviors that result in unintentional or intentional injuries; (2) tobacco use; (3) alcohol and other drug use; (4) sexual behaviors that…

Development and Analysis of an Instrument to Assess Student Understanding of GOB Chemistry Knowledge Relevant to Clinical Nursing Practice

ERIC Educational Resources Information Center

Brown, Corina E.; Hyslop, Richard M.; Barbera, Jack

2015-01-01

The General, Organic, and Biological Chemistry Knowledge Assessment (GOB-CKA) is a multiple-choice instrument designed to assess students' understanding of the chemistry topics deemed important to clinical nursing practice. This manuscript describes the development process of the individual items along with a psychometric evaluation of the…

Drawing and Using Free Body Diagrams: Why It May Be Better Not to Decompose Forces

ERIC Educational Resources Information Center

Aviani, Ivica; Erceg, Nataša; Mešic, Vanes

2015-01-01

In this study we investigated how two different approaches to drawing free body diagrams influence the development of students' understanding of Newton's laws, including their ability to identify real forces. For this purpose we developed a 12-item two-tier multiple choice survey and conducted a quasiexperiment. This experiment included two groups…

A Critical Analysis of the Body of Work Method for Setting Cut-Scores

ERIC Educational Resources Information Center

Radwan, Nizam; Rogers, W. Todd

2006-01-01

The recent increase in the use of constructed-response items in educational assessment and the dissatisfaction with the nature of the decision that the judges must make using traditional standard-setting methods created a need to develop new and effective standard-setting procedures for tests that include both multiple-choice and…

The Potential Use of the Discouraging Random Guessing (DRG) Approach in Multiple-Choice Exams in Medical Education.

ERIC Educational Resources Information Center

Friedman, Miriam; And Others

1987-01-01

Test performances of sophomore medical students on a pretest and final exam (under guessing and no-guessing instructions) were compared. Discouraging random guessing produced test information with improved test reliability and less distortion of item difficulty. More able examinees were less compliant than less able examinees. (Author/RH)

Analyzing the Reliability of the easyCBM Reading Comprehension Measures: Grade 5. Technical Report #1204

ERIC Educational Resources Information Center

Park, Bitnara Jasmine; Irvin, P. Shawn; Lai, Cheng-Fei; Alonzo, Julie; Tindal, Gerald

2012-01-01

In this technical report, we present the results of a reliability study of the fifth-grade multiple choice reading comprehension measures available on the easyCBM learning system conducted in the spring of 2011. Analyses include split-half reliability, alternate form reliability, person and item reliability as derived from Rasch analysis,…

Analyzing the Reliability of the easyCBM Reading Comprehension Measures: Grade 2. Technical Report #1201

ERIC Educational Resources Information Center

Lai, Cheng-Fei; Irvin, P. Shawn; Alonzo, Julie; Park, Bitnara Jasmine; Tindal, Gerald

2012-01-01

In this technical report, we present the results of a reliability study of the second-grade multiple choice reading comprehension measures available on the easyCBM learning system conducted in the spring of 2011. Analyses include split-half reliability, alternate form reliability, person and item reliability as derived from Rasch analysis,…

Analyzing the Reliability of the easyCBM Reading Comprehension Measures: Grade 4. Technical Report #1203

ERIC Educational Resources Information Center

Park, Bitnara Jasmine; Irvin, P. Shawn; Alonzo, Julie; Lai, Cheng-Fei; Tindal, Gerald

2012-01-01

In this technical report, we present the results of a reliability study of the fourth-grade multiple choice reading comprehension measures available on the easyCBM learning system conducted in the spring of 2011. Analyses include split-half reliability, alternate form reliability, person and item reliability as derived from Rasch analysis,…

Analyzing the Reliability of the easyCBM Reading Comprehension Measures: Grade 6. Technical Report #1205

ERIC Educational Resources Information Center

Irvin, P. Shawn; Alonzo, Julie; Park, Bitnara Jasmine; Lai, Cheng-Fei; Tindal, Gerald

2012-01-01

In this technical report, we present the results of a reliability study of the sixth-grade multiple choice reading comprehension measures available on the easyCBM learning system conducted in the spring of 2011. Analyses include split-half reliability, alternate form reliability, person and item reliability as derived from Rasch analysis,…

Analyzing the Reliability of the easyCBM Reading Comprehension Measures: Grade 7. Technical Report #1206

ERIC Educational Resources Information Center

Irvin, P. Shawn; Alonzo, Julie; Lai, Cheng-Fei; Park, Bitnara Jasmine; Tindal, Gerald

2012-01-01

In this technical report, we present the results of a reliability study of the seventh-grade multiple choice reading comprehension measures available on the easyCBM learning system conducted in the spring of 2011. Analyses include split-half reliability, alternate form reliability, person and item reliability as derived from Rasch analysis,…

Analyzing the Reliability of the easyCBM Reading Comprehension Measures: Grade 3. Technical Report #1202

ERIC Educational Resources Information Center

Lai, Cheng-Fei; Irvin, P. Shawn; Park, Bitnara Jasmine; Alonzo, Julie; Tindal, Gerald

2012-01-01

In this technical report, we present the results of a reliability study of the third-grade multiple choice reading comprehension measures available on the easyCBM learning system conducted in the spring of 2011. Analyses include split-half reliability, alternate form reliability, person and item reliability as derived from Rasch analysis,…

Uncovering Students' Incorrect Ideas about Foundational Concepts for Biochemistry

ERIC Educational Resources Information Center

Villafane, Sachel M.; Loertscher, Jennifer; Minderhout, Vicky; Lewis, Jennifer E.

2011-01-01

This paper presents preliminary data on how an assessment instrument with a unique structure can be used to identify common incorrect ideas from prior coursework at the beginning of a biochemistry course, and to determine whether these ideas have changed by the end of the course. The twenty-one multiple-choice items address seven different…

Industrial Arts Test Development, Book III. Resource Items for Graphics Technology, Power Technology, Production Technology.

ERIC Educational Resources Information Center

New York State Education Dept., Albany.

This booklet is designed to assist teachers in developing examinations for classroom use. It is a collection of 955 objective test questions, mostly multiple choice, for industrial arts students in the three areas of graphics technology, power technology, and production technology. Scoring keys are provided. There are no copyright restrictions,…

PRE-COLLEGE EXPERIENCES AS PREPARATION FOR COLLEGE COURSES IN AGRONOMY.

ERIC Educational Resources Information Center

BEEKS, JOHN C.

TO DETERMINE THE KNOWLEDGE OF AGRONOMY POSSESSED BY ENTERING FRESHMEN IN THE COLLEGE OF AGRICULTURE AT THE UNIVERSITY OF MISSOURI, STUDENTS ENROLLED IN THE REQUIRED COURSE AGRICULTURE IN THE ECONOMY DURING THE YEARS 1962 AND 1963 RESPONDED TO A 100-ITEM MULTIPLE CHOICE INSTRUMENT. A TOTAL OF 310 USABLE ANSWER SHEETS FURNISHED DATA ON STUDENTS--(1)…

A Dialogue about MCQs, Reliability, and Item Response Modelling

ERIC Educational Resources Information Center

Wright, Daniel B.; Skagerberg, Elin M.

2006-01-01

Multiple choice questions (MCQs) are becoming more common in UK psychology departments and the need to assess their reliability is apparent. Having examined the reliability of MCQs in our department we faced many questions from colleagues about why we were examining reliability, what it was that we were doing, and what should be reported when…

Computer Managed Instruction Homework Modules for Calculus I.

ERIC Educational Resources Information Center

Goodman-Petrushka, Sharon; Roitberg, Yael

This booklet contains 11 modules (290 multiple-choice items) designed for use in the first course of a three-course calculus sequence using the textbook "Calculus with Analytic Geometry" (Dennis G. Zill). In each module, relevant sections of the textbook are identified for users. It can, however, be used in conjunction with any calculus textbook.…

Influence of Particle Theory Conceptions on Pre-Service Science Teachers' Understanding of Osmosis and Diffusion

ERIC Educational Resources Information Center

AlHarbi, Nawaf N. S.; Treagust, David F.; Chandrasegaran, A. L.; Won, Mihye

2015-01-01

This study investigated the understanding of diffusion, osmosis and particle theory of matter concepts among 192 pre-service science teachers in Saudi Arabia using a 17-item two-tier multiple-choice diagnostic test. The data analysis showed that the pre-service teachers' understanding of osmosis and diffusion concepts was mildly correlated with…

The Relationship Between Student Alienation and Extent of Faculty Agreement on Pupil Control Ideology.

ERIC Educational Resources Information Center

Shearin, Wiley H., Jr.

1982-01-01

Results supported hypothesis that schools with high agreement among staff on pupil control ideology would have less student alienation than those schools with low agreement. A 20-item, multiple-choice instrument was used to measure humanistic or custodial teacher orienation and the Kolesar's Pupil Attitude Questionnaire (PAQ) to measure student…

Development and Evaluation of a Questionnaire to Assess Physical Educators' Knowledge of Student Assessment

ERIC Educational Resources Information Center

Emmanouilidou, Kyriaki; Derri, Vassiliki; Aggelousis, Nicolaos; Vassiliadou, Olga

2012-01-01

The purpose of this pilot study was to develop and evaluate an instrument for measuring Greek elementary physical educators' knowledge of student assessment. A multiple-choice questionnaire comprised of items about concepts, methods, tools, and types of student assessment in physical education was designed and tested. The initial 35-item…

Using Multigroup Confirmatory Factor Analysis to Test Measurement Invariance in Raters: A Clinical Skills Examination Application

ERIC Educational Resources Information Center

Kahraman, Nilufer; Brown, Crystal B.

2015-01-01

Psychometric models based on structural equation modeling framework are commonly used in many multiple-choice test settings to assess measurement invariance of test items across examinee subpopulations. The premise of the current article is that they may also be useful in the context of performance assessment tests to test measurement invariance…

Determination of Students' Alternative Conceptions about Chemical Equilibrium: A Review of Research and the Case of Turkey

ERIC Educational Resources Information Center

Ozmen, Haluk

2008-01-01

This study aims to determine prospective science student teachers' alternative conceptions of the chemical equilibrium concept. A 13-item pencil and paper, two-tier multiple choice diagnostic instrument, the Test to Identify Students' Alternative Conceptions (TISAC), was developed and administered to 90 second-semester science student teachers…

A Comparison of Domain-Referenced and Classic Psychometric Test Construction Methods.

ERIC Educational Resources Information Center

Willoughby, Lee; And Others

This study compared a domain referenced approach with a traditional psychometric approach in the construction of a test. Results of the December, 1975 Quarterly Profile Exam (QPE) administered to 400 examinees at a university were the source of data. The 400 item QPE is a five alternative multiple choice test of information a "safe"…

American Sign Language Comprehension Test: A Tool for Sign Language Researchers

ERIC Educational Resources Information Center

Hauser, Peter C.; Paludneviciene, Raylene; Riddle, Wanda; Kurz, Kim B.; Emmorey, Karen; Contreras, Jessica

2016-01-01

The American Sign Language Comprehension Test (ASL-CT) is a 30-item multiple-choice test that measures ASL receptive skills and is administered through a website. This article describes the development and psychometric properties of the test based on a sample of 80 college students including deaf native signers, hearing native signers, deaf…

The Disaggregation of Value-Added Test Scores to Assess Learning Outcomes in Economics Courses

ERIC Educational Resources Information Center

Walstad, William B.; Wagner, Jamie

2016-01-01

This study disaggregates posttest, pretest, and value-added or difference scores in economics into four types of economic learning: positive, retained, negative, and zero. The types are derived from patterns of student responses to individual items on a multiple-choice test. The micro and macro data from the "Test of Understanding in College…

Decision making: rational or hedonic?

PubMed Central

Cabanac, Michel; Bonniot-Cabanac, Marie-Claude

2007-01-01

Three experiments studied the hedonicity of decision making. Participants rated their pleasure/displeasure while reading item-sentences describing political and social problems followed by different decisions (Questionnaire 1). Questionnaire 2 was multiple-choice, grouping the items from Questionnaire 1. In Experiment 1, participants answered Questionnaire 2 rapidly or slowly. Both groups selected what they had rated as pleasant, but the 'leisurely' group maximized pleasure less. In Experiment 2, participants selected the most rational responses. The selected behaviors were pleasant but less than spontaneous behaviors. In Experiment 3, Questionnaire 2 was presented once with items grouped by theme, and once with items shuffled. Participants maximized the pleasure of their decisions, but the items selected on Questionnaires 2 were different when presented in different order. All groups maximized pleasure equally in their decisions. These results support that decisions are made predominantly in the hedonic dimension of consciousness. PMID:17848195

Set-fit effects in choice.

PubMed

Evers, Ellen R K; Inbar, Yoel; Zeelenberg, Marcel

2014-04-01

In 4 experiments, we investigate how the "fit" of an item with a set of similar items affects choice. We find that people have a notion of a set that "fits" together--one where all items are the same, or all items differ, on salient attributes. One consequence of this notion is that in addition to preferences over the set's individual items, choice reflects set-fit. This leads to predictable shifts in preferences, sometimes even resulting in people choosing normatively inferior options over superior ones.

Comparing narrative and multiple-choice formats in online communication skill assessment.

PubMed

Kim, Sara; Spielberg, Freya; Mauksch, Larry; Farber, Stu; Duong, Cuong; Fitch, Wes; Greer, Tom

2009-06-01

We compared multiple-choice and open-ended responses collected from a web-based tool designated 'Case for Change', which had been developed for assessing and teaching medical students in the skills involved in integrating sexual risk assessment and behaviour change discussions into patient-centred primary care visits. A total of 111 Year 3 students completed the web-based tool. A series of videos from one patient encounter illustrated how a clinician uses patient-centred communication and health behaviour change skills while caring for a patient presenting with a urinary tract infection. Each video clip was followed by a request for students to respond in two ways to the question: 'What would you do next?' Firstly, students typed their statements of what they would say to the patient. Secondly, students selected from a multiple-choice list the statements that most closely resembled their free text entries. These two modes of students' answers were analysed and compared. When articulating what they would say to the patient in a narrative format, students frequently used doctor-centred approaches that focused on premature diagnostic questioning or neglected to elicit patient perspectives. Despite the instruction to select a matching statement from the multiple-choice list, students tended to choose the most exemplary patient-centred statement, which was contrary to the doctor-centred approaches reflected in their narrative responses. Open-ended questions facilitate in-depth understanding of students' educational needs, although the scoring of narrative responses is time-consuming. Multiple-choice questions allow efficient scoring and individualised feedback associated with question items but do not fully elicit students' thought processes.

Online evaluation of novel choices by simultaneous representation of multiple memories

PubMed Central

Barron, Helen C; Dolan, Raymond J; Behrens, Timothy E J

2014-01-01

Prior experience plays a critical role in decision making. It enables explicit representation of potential outcomes and provides training to valuation mechanisms. However, we can also make choices in the absence of prior experience, by merely imagining the consequences of a new experience. Here, using fMRI repetition suppression in humans, we show how neuronal representations of novel rewards can be constructed and evaluated. A likely novel experience is constructed by invoking multiple independent memories within hippocampus and medial prefrontal cortex. This construction persists for only a short time period, during which new associations are observed between the memories for component items. Together these findings suggest that in the absence of direct experience, co-activation of multiple relevant memories can provide a training signal to the valuation system which allows the consequences of new experiences to be imagined and acted upon. PMID:24013592

Influence of Context on Item Parameters in Forced-Choice Personality Assessments

ERIC Educational Resources Information Center

Lin, Yin; Brown, Anna

2017-01-01

A fundamental assumption in computerized adaptive testing is that item parameters are invariant with respect to context--items surrounding the administered item. This assumption, however, may not hold in forced-choice (FC) assessments, where explicit comparisons are made between items included in the same block. We empirically examined the…

Defining value through quantity and quality-Chimpanzees (Pan troglodytes) undervalue food quantities when items are broken.

PubMed

Parrish, Audrey E; Evans, Theodore A; Beran, Michael J

2015-02-01

Decision-making largely is influenced by the relative value of choice options, and the value of such options can be determined by a combination of different factors (e.g., the quantity, size, or quality of a stimulus). In this study, we examined the competing influences of quantity (i.e., the number of food items in a set) and quality (i.e., the original state of a food item) of choice items on chimpanzees' food preferences in a two-option natural choice paradigm. In Experiment 1, chimpanzees chose between sets of food items that were either entirely whole or included items that were broken into pieces before being shown to the chimpanzees. Chimpanzees exhibited a bias for whole food items even when such choice options consisted of a smaller overall quantity of food than the sets containing broken items. In Experiment 2, chimpanzees chose between sets of entirely whole food items and sets of initially whole items that were subsequently broken in view of the chimpanzees just before choice time. Chimpanzees continued to exhibit a bias for sets of whole items. In Experiment 3, chimpanzees chose between sets of new food items that were initially discrete but were subsequently transformed into a larger cohesive unit. Here, chimpanzees were biased to choose the discrete sets that retained their original qualitative state rather than toward the cohesive or clumped sets. These results demonstrate that beyond a food set's quantity (i.e., the value dimension that accounts for maximization in terms of caloric intake), other seemingly non-relevant features (i.e., quality in terms of a set's original state) affect how chimpanzees assign value to their choice options. Copyright © 2014 Elsevier B.V. All rights reserved.

Analysis test of understanding of vectors with the three-parameter logistic model of item response theory and item response curves technique

NASA Astrophysics Data System (ADS)

Rakkapao, Suttida; Prasitpong, Singha; Arayathanitkul, Kwan

2016-12-01

This study investigated the multiple-choice test of understanding of vectors (TUV), by applying item response theory (IRT). The difficulty, discriminatory, and guessing parameters of the TUV items were fit with the three-parameter logistic model of IRT, using the parscale program. The TUV ability is an ability parameter, here estimated assuming unidimensionality and local independence. Moreover, all distractors of the TUV were analyzed from item response curves (IRC) that represent simplified IRT. Data were gathered on 2392 science and engineering freshmen, from three universities in Thailand. The results revealed IRT analysis to be useful in assessing the test since its item parameters are independent of the ability parameters. The IRT framework reveals item-level information, and indicates appropriate ability ranges for the test. Moreover, the IRC analysis can be used to assess the effectiveness of the test's distractors. Both IRT and IRC approaches reveal test characteristics beyond those revealed by the classical analysis methods of tests. Test developers can apply these methods to diagnose and evaluate the features of items at various ability levels of test takers.

Cognitive task analysis for teaching technical skills in an inanimate surgical skills laboratory.

PubMed

Velmahos, George C; Toutouzas, Konstantinos G; Sillin, Lelan F; Chan, Linda; Clark, Richard E; Theodorou, Demetrios; Maupin, Fredric

2004-01-01

The teaching of surgical skills is based mostly on the traditional "see one, do one, teach one" resident-to-resident method. Surgical skills laboratories provide a new environment for teaching skills but their effectiveness has not been adequately tested. Cognitive task analysis is an innovative method to teach skills, used successfully in nonmedical fields. The objective of this study is to evaluate the effectiveness of a 3-hour surgical skills laboratory course on central venous catheterization (CVC), taught by the principles of cognitive task analysis to surgical interns. Upon arrival to the Department of Surgery, 26 new interns were randomized to either receive a surgical skills laboratory course on CVC ("course" group, n = 12) or not ("traditional" group, n = 14). The course consisted mostly of hands-on training on inanimate CVC models. All interns took a 15-item multiple-choice question test on CVC at the beginning of the study. Within two and a half months all interns performed CVC on critically ill patients. The outcome measures were cognitive knowledge and technical-skill competence on CVC. These outcomes were assessed by a 14-item checklist evaluating the interns while performing CVC on a patient and by the 15-item multiple-choice-question test, which was repeated at that time. There were no differences between the two groups in the background characteristics of the interns or the patients having CVC. The scores at the initial multiple-choice test were similar (course: 7.33 +/- 1.07, traditional: 8 +/- 2.15, P = 0.944). However, the course interns scored significantly higher in the repeat test compared with the traditional interns (11 +/- 1.86 versus 8.64 +/- 1.82, P = 0.03). Also, the course interns achieved a higher score on the 14-item checklist (12.6 +/- 1.1 versus 7.5 +/- 2.2, P <0.001). They required fewer attempts to find the vein (3.3 +/- 2.2 versus 6.4 +/- 4.2, P = 0.046) and showed a trend toward less time to complete the procedure (15.4 +/- 9.5 versus 20.6 +/- 9.1 minutes, P = 0.149). A surgical skills laboratory course on CVC, taught by the principles of cognitive task analysis and using inanimate models, improves the knowledge and technical skills of new surgical interns on this task.

The Relationship Between College Zoology Students' Religious Beliefs and Their Ability to Objectively View the Scientific Evidence Supporting Evolutionary Theory.

ERIC Educational Resources Information Center

Sinclair, Anne; Baldwin, Beatrice

An anonymous 12-item, multiple-choice questionnaire was administered to 218 southern college, introductory zoology students prior to and following a study of evolutionary theory to assess their understanding and acceptance of the credibility of the evidence supporting the theory. Key topics addressed were the history of evolutionary thought, basic…

Clientele Recognition of Library Terms and Concepts Used by Librarians: A Case of an Academic Library in the Philippines

ERIC Educational Resources Information Center

Cana, Mercy B.; Cueto, Quiza Lynn Grace G.; De Guzman, Allan B.; Fuchigami, Kaori B.; Manalo, Leona Rica T.; Yu, Jake Cathleen U.

2005-01-01

Using a 30-item multiple-choice type test, this investigation focused on the ability of college students to recognise terms and concepts used by librarians. A total of 447 respondents representing the fields of Education, Nutrition, Food Technology, Tourism and Hotel and Restaurant Management took part in this investigation. Data were gathered…

For Internet Knowledge, Should You Ask Ol' Blue Eyes or the Brown-Eyed Girl?

ERIC Educational Resources Information Center

Boshier, Roger; Kolpakova, Yulia; Klinkhamer, Sooz

2004-01-01

The digital divide is generally thought to arise from socio-economic disparities. However, there is more to it. Eye colour is a factor. In this study, the 16 multiple-choice item Internet Quiz was administered to 3,208 respondents in the Lower Mainland (Vancouver) of British Columbia, Canada. Blue and hazel-eyed people knew significantly more…

An Empirical Comparison of Five Linear Equating Methods for the NEAT Design

ERIC Educational Resources Information Center

Suh, Youngsuk; Mroch, Andrew A.; Kane, Michael T.; Ripkey, Douglas R.

2009-01-01

In this study, a data base containing the responses of 40,000 candidates to 90 multiple-choice questions was used to mimic data sets for 50-item tests under the "nonequivalent groups with anchor test" (NEAT) design. Using these smaller data sets, we evaluated the performance of five linear equating methods for the NEAT design with five levels of…

Force, Velocity, and Work: The Effects of Different Contexts on Students' Understanding of Vector Concepts Using Isomorphic Problems

ERIC Educational Resources Information Center

Barniol, Pablo; Zavala, Genaro

2014-01-01

In this article we compare students' understanding of vector concepts in problems with no physical context, and with three mechanics contexts: force, velocity, and work. Based on our "Test of Understanding of Vectors," a multiple-choice test presented elsewhere, we designed two isomorphic shorter versions of 12 items each: a test with no…

Science Library of Test Items. Volume Three. Mastery Testing Programme. Introduction and Manual.

ERIC Educational Resources Information Center

New South Wales Dept. of Education, Sydney (Australia).

A set of short tests aimed at measuring student mastery of specific skills in the natural sciences are presented with a description of the mastery program's purposes, development, and methods. Mastery learning, criterion-referenced testing, and the scope of skills to be tested are defined. Each of the multiple choice tests for grades 7 through 10…

Reliability and Validity of a Procedure to Measure Diagnostic Reasoning and Problem-Solving Skills Taught in Predoctoral Orthodontic Education.

ERIC Educational Resources Information Center

Albanese, Mark A.; Jacobs, Richard M.

1990-01-01

The reliability and validity of a procedure to measure diagnostic-reasoning and problem-solving skills taught in predoctoral orthodontic education were studied using 68 second year dental students. The procedure includes stimulus material and 33 multiple-choice items. It is a feasible way of assessing problem-solving skills in dentistry education…

Unraveling Vocabulary Learning: Reader and Item-Level Predictors of Vocabulary Learning within Comprehension Instruction for Fifth and Sixth Graders

ERIC Educational Resources Information Center

Goodwin, Amanda P.; Cho, Sun-Joo

2016-01-01

This study explores reader, word, and learning activity characteristics related to vocabulary learning for 202 fifth and sixth graders (N = 118 and 84, respectively) learning 16 words. Three measures of word knowledge were used: multiple-choice definition knowledge, self-report of meaning knowledge, and production of morphologically related words.…

Using Two-Tier Test to Identify Primary Students' Conceptual Understanding and Alternative Conceptions in Acid Base

ERIC Educational Resources Information Center

Bayrak, Beyza Karadeniz

2013-01-01

The purpose of this study was to identify primary students' conceptual understanding and alternative conceptions in acid-base. For this reason, a 15 items two-tier multiple choice test administered 56 eighth grade students in spring semester 2009-2010. Data for this study were collected using a conceptual understanding scale prepared to include…

catcher: A Software Program to Detect Answer Copying in Multiple-Choice Tests Based on Nominal Response Model

ERIC Educational Resources Information Center

Kalender, Ilker

2012-01-01

catcher is a software program designed to compute the [omega] index, a common statistical index for the identification of collusions (cheating) among examinees taking an educational or psychological test. It requires (a) responses and (b) ability estimations of individuals, and (c) item parameters to make computations and outputs the results of…

Computerized Classification Testing under the One-Parameter Logistic Response Model with Ability-Based Guessing

ERIC Educational Resources Information Center

Wang, Wen-Chung; Huang, Sheng-Yun

2011-01-01

The one-parameter logistic model with ability-based guessing (1PL-AG) has been recently developed to account for effect of ability on guessing behavior in multiple-choice items. In this study, the authors developed algorithms for computerized classification testing under the 1PL-AG and conducted a series of simulations to evaluate their…

Understanding Test-Takers' Perceptions of Difficulty in EAP Vocabulary Tests: The Role of Experiential Factors

ERIC Educational Resources Information Center

Oruç Ertürk, Nesrin; Mumford, Simon E.

2017-01-01

This study, conducted by two researchers who were also multiple-choice question (MCQ) test item writers at a private English-medium university in an English as a foreign language (EFL) context, was designed to shed light on the factors that influence test-takers' perceptions of difficulty in English for academic purposes (EAP) vocabulary, with the…

Applied Reading Test--Forms A and B, Interim Manual, and Answer Sheets.

ERIC Educational Resources Information Center

Australian Council for Educational Research, Hawthorn.

Designed for use in the selection of apprentices, trainees, technical and trade personnel, and any other persons who need to read and understand text of a technical nature, this Applied Reading Test specimen set contains six passages and 32 items, has a 30-minute time limit, and is presented in a reusable multiple choice test booklet. The specimen…

Exploring Different Types of Assessment Items to Measure Linguistically Diverse Students' Understanding of Energy and Matter in Chemistry

ERIC Educational Resources Information Center

Ryoo, Kihyun; Toutkoushian, Emily; Bedell, Kristin

2018-01-01

Energy and matter are fundamental, yet challenging concepts in middle school chemistry due to their abstract, unobservable nature. Although it is important for science teachers to elicit a range of students' ideas to design and revise their instruction, capturing such varied ideas using traditional assessments consisting of multiple-choice items…

Development of a Microcomputer-Based Adaptive Testing System. Phase I. Specification of Requirements and Preliminary Design.

DTIC Science & Technology

1982-06-30

treatments, and cure (or kill ) a patient. Administratively, the items were in a multiple-choice format and the simulation proceeded by branching...Discs: dual 5 1/4 inch floppies (IM) Bus: N/A Operating System: CP/M, MmmOST Price: $3,495 -14 ~-174- - ’i~ Model 820 Xerox 1341 West Mockingbird Lane

A Normalized Direct Approach for Estimating the Parameters of the Normal Ogive Three-Parameter Model for Ability Tests.

ERIC Educational Resources Information Center

Gugel, John F.

A new method for estimating the parameters of the normal ogive three-parameter model for multiple-choice test items--the normalized direct (NDIR) procedure--is examined. The procedure is compared to a more commonly used estimation procedure, Lord's LOGIST, using computer simulations. The NDIR procedure uses the normalized (mid-percentile)…

Grade 9 Pilot Test. Mathematics. June 1988 = 9e Annee Test Pilote. Mathematiques. Juin 1988.

ERIC Educational Resources Information Center

Alberta Dept. of Education, Edmonton.

This pilot test for ninth grade mathematics is written in both French and English. The test consists of 75 multiple-choice items. Students are given 90 minutes to complete the examination and the use of a calculator is highly recommended. The test content covers a wide range of mathematical topics including: decimals; exponents; arithmetic word…

Immediate vs. Delayed Feedback in a Computer-Managed Test: Effects on Long-Term Retention. Technical Report, March 1976-August 1976.

ERIC Educational Resources Information Center

Sturges, Persis T.

This experiment was designed to test the effect of immediate and delayed feedback on retention of learning in an educational situation. Four groups of college undergraduates took a multiple-choice computer-managed test. Three of these groups received informative feedback (the entire item with the correct answer identified) either: (1) immediately…

Evaluation of an Intervention Instructional Program to Facilitate Understanding of Basic Particle Concepts among Students Enrolled in Several Levels of Study

ERIC Educational Resources Information Center

Treagust, David F.; Chandrasegaran, A. L.; Zain, Ahmad N. M.; Ong, Eng Tek; Karpudewan, Mageswary; Halim, Lilia

2011-01-01

The efficacy of an intervention instructional program was evaluated to facilitate understanding of particle theory concepts among students (N = 190) using a diagnostic instrument consisting of eleven two-tier multiple-choice items in a pre-test--post-test design. The students involved were high school students, undergraduates and postgraduates…

Pursuing Higher Education: Are There Gender Differences in the Factors That Influence Individuals To Pursue Higher Education?

ERIC Educational Resources Information Center

Harris, Sandra McMeans

This study investigated whether gender differences exist in the factors thought to influence a person's desire to pursue higher education. A 152-item multiple choice questionnaire, completed by 346 students enrolled at a large university during 1998, was the source of the data. The independent variable was gender; dependent variables were…

A Diagnostic Assessment for Introductory Molecular and Cell Biology

PubMed Central

Wood, William B.; Martin, Jennifer M.; Guild, Nancy A.; Vicens, Quentin; Knight, Jennifer K.

2010-01-01

We have developed and validated a tool for assessing understanding of a selection of fundamental concepts and basic knowledge in undergraduate introductory molecular and cell biology, focusing on areas in which students often have misconceptions. This multiple-choice Introductory Molecular and Cell Biology Assessment (IMCA) instrument is designed for use as a pre- and posttest to measure student learning gains. To develop the assessment, we first worked with faculty to create a set of learning goals that targeted important concepts in the field and seemed likely to be emphasized by most instructors teaching these subjects. We interviewed students using open-ended questions to identify commonly held misconceptions, formulated multiple-choice questions that included these ideas as distracters, and reinterviewed students to establish validity of the instrument. The assessment was then evaluated by 25 biology experts and modified based on their suggestions. The complete revised assessment was administered to more than 1300 students at three institutions. Analysis of statistical parameters including item difficulty, item discrimination, and reliability provides evidence that the IMCA is a valid and reliable instrument with several potential uses in gauging student learning of key concepts in molecular and cell biology. PMID:21123692

Biased predecisional processing of leading and nonleading alternatives.

PubMed

Blanchard, Simon J; Carlson, Kurt A; Meloy, Margaret G

2014-03-01

When people obtain information about choice alternatives in a set one attribute at a time, they rapidly identify a leading alternative. Although previous research has established that people then distort incoming information, it is unclear whether distortion occurs through favoring of the leading alternative, disfavoring of the trailing alternative, or both. Prior examinations have not explored the predecisional treatment of the nonleading alternative (or alternatives) because they conceptualized distortion as a singular construct in binary choice and measured it using a relative item comparing the evaluation of both alternatives simultaneously. In this article, we introduce a measure of distortion at the level of the alternative, which allows for measuring whether predecisional distortion favors or disfavors every alternative being considered in choice sets of various sizes. We report that both proleader and antitrailer distortion occur and that the use of antitrailer processing differs between binary choices and multiple-options choices.

Cognitive dissonance resolution depends on episodic memory.

PubMed

Chammat, Mariam; Karoui, Imen El; Allali, Sébastien; Hagège, Joshua; Lehongre, Katia; Hasboun, Dominique; Baulac, Michel; Epelbaum, Stéphane; Michon, Agnès; Dubois, Bruno; Navarro, Vincent; Salti, Moti; Naccache, Lionel

2017-01-23

The notion that past choices affect preferences is one of the most influential concepts of social psychology since its first report in the 50 s, and its theorization within the cognitive dissonance framework. In the free-choice paradigm (FCP) after choosing between two similarly rated items, subjects reevaluate chosen items as more attractive and rejected items as less attractive. However the relations prevailing between episodic memory and choice-induced preference change (CIPC) remain highly debated: is this phenomenon dependent or independent from memory of past choices? We solve this theoretical debate by demonstrating that CIPC occurs exclusively for items which were correctly remembered as chosen or rejected during the choice stage. We used a combination of fMRI and intra-cranial electrophysiological recordings to reveal a modulation of left hippocampus activity, a hub of episodic memory retrieval, immediately before the occurrence of CIPC during item reevaluation. Finally, we show that contrarily to a previous influential report flawed by a statistical artifact, this phenomenon is absent in amnesic patients for forgotten items. These results demonstrate the dependence of cognitive dissonance on conscious episodic memory. This link between current preferences and previous choices suggests a homeostatic function of this regulative process, aiming at preserving subjective coherence.

Cognitive dissonance resolution depends on episodic memory

PubMed Central

Chammat, Mariam; Karoui, Imen El; Allali, Sébastien; Hagège, Joshua; Lehongre, Katia; Hasboun, Dominique; Baulac, Michel; Epelbaum, Stéphane; Michon, Agnès; Dubois, Bruno; Navarro, Vincent; Salti, Moti; Naccache, Lionel

2017-01-01

The notion that past choices affect preferences is one of the most influential concepts of social psychology since its first report in the 50 s, and its theorization within the cognitive dissonance framework. In the free-choice paradigm (FCP) after choosing between two similarly rated items, subjects reevaluate chosen items as more attractive and rejected items as less attractive. However the relations prevailing between episodic memory and choice-induced preference change (CIPC) remain highly debated: is this phenomenon dependent or independent from memory of past choices? We solve this theoretical debate by demonstrating that CIPC occurs exclusively for items which were correctly remembered as chosen or rejected during the choice stage. We used a combination of fMRI and intra-cranial electrophysiological recordings to reveal a modulation of left hippocampus activity, a hub of episodic memory retrieval, immediately before the occurrence of CIPC during item reevaluation. Finally, we show that contrarily to a previous influential report flawed by a statistical artifact, this phenomenon is absent in amnesic patients for forgotten items. These results demonstrate the dependence of cognitive dissonance on conscious episodic memory. This link between current preferences and previous choices suggests a homeostatic function of this regulative process, aiming at preserving subjective coherence. PMID:28112261

Attentional priority determines working memory precision.

PubMed

Klyszejko, Zuzanna; Rahmati, Masih; Curtis, Clayton E

2014-12-01

Visual working memory is a system used to hold information actively in mind for a limited time. The number of items and the precision with which we can store information has limits that define its capacity. How much control do we have over the precision with which we store information when faced with these severe capacity limitations? Here, we tested the hypothesis that rank-ordered attentional priority determines the precision of multiple working memory representations. We conducted two psychophysical experiments that manipulated the priority of multiple items in a two-alternative forced choice task (2AFC) with distance discrimination. In Experiment 1, we varied the probabilities with which memorized items were likely to be tested. To generalize the effects of priority beyond simple cueing, in Experiment 2, we manipulated priority by varying monetary incentives contingent upon successful memory for items tested. Moreover, we illustrate our hypothesis using a simple model that distributed attentional resources across items with rank-ordered priorities. Indeed, we found evidence in both experiments that priority affects the precision of working memory in a monotonic fashion. Our results demonstrate that representations of priority may provide a mechanism by which resources can be allocated to increase the precision with which we encode and briefly store information. Copyright © 2014 Elsevier Ltd. All rights reserved.

Controlling Guessing Bias in the Dichotomous Rasch Model Applied to a Large-Scale, Vertically Scaled Testing Program

ERIC Educational Resources Information Center

Andrich, David; Marais, Ida; Humphry, Stephen Mark

2016-01-01

Recent research has shown how the statistical bias in Rasch model difficulty estimates induced by guessing in multiple-choice items can be eliminated. Using vertical scaling of a high-profile national reading test, it is shown that the dominant effect of removing such bias is a nonlinear change in the unit of scale across the continuum. The…

Definite Integral Automatic Analysis Mechanism Research and Development Using the "Find the Area by Integration" Unit as an Example

ERIC Educational Resources Information Center

Ting, Mu Yu

2017-01-01

Using the capabilities of expert knowledge structures, the researcher prepared test questions on the university calculus topic of "finding the area by integration." The quiz is divided into two types of multiple choice items (one out of four and one out of many). After the calculus course was taught and tested, the results revealed that…

Analysis of the Alternative Conceptions of Preservice Teachers and High School Students Concerning Atomic Size

ERIC Educational Resources Information Center

Eymur, Guluzar; Çetin, Pinar; Geban, Ömer

2013-01-01

The purpose of this study was to analyze and compare the alternative conceptions of high school students and preservice teachers on the concept of atomic size. The Atomic Size Diagnostic Instrument was developed; it is composed of eight, two-tier multiple-choice items. The results of the study showed that as a whole 56.2% of preservice teachers…

Constructing a Criterion Reference Test to Measure the Research and Statistical Competencies of Graduate Students at the Jordanian Governmental Universities

ERIC Educational Resources Information Center

Al-Habashneh, Maher Hussein; Najjar, Nabil Juma

2017-01-01

This study aimed at constructing a criterion-reference test to measure the research and statistical competencies of graduate students at the Jordanian governmental universities, the test has to be in its first form of (50) multiple choice items, then the test was introduced to (5) arbitrators with competence in measurement and evaluation to…

Middle School Students' Conceptual Learning from the Implementation of a New NSF Supported Curriculum: Interactions in Physical Science[TM

ERIC Educational Resources Information Center

Eick, Charles J.; Dias, Michael; Smith, Nancy R. Cook

2009-01-01

A new National Science Foundation supported curriculum, Interactions in Physical Science[TM], was evaluated on students' conceptual change in the twelve concept areas of the national physical science content standard (B) for grades 5-8. Eighth grade students (N = 66) were evaluated pre and post on a 31-item multiple-choice test of conceptual…

Effect of Gender on Students' Academic Performance in Computer Studies in Secondary Schools in New Bussa, Borgu Local Government of Niger State

ERIC Educational Resources Information Center

Adigun, Joseph; Onihunwa, John; Irunokhai, Eric; Sada, Yusuf; Adesina, Olubunmi

2015-01-01

This research studied the relationship between student's gender and academic performance in computer science in New Bussa, Borgu local government of Niger state. Questionnaire which consisted of 30 multiple-choice items drawn from Senior School Certificate Examination past questions as set by the West Africa Examination Council in 2014 multiple…

Bilingual Test as a Test Accommodation to Determine the Mathematics Achievement of Mainstream Students with Limited English Proficiency

ERIC Educational Resources Information Center

Shanmugam, S. Kanageswari Suppiah; Lan, Ong Saw

2013-01-01

Purpose: This study aims to investigate the validity of using bilingual test to measure the mathematics achievement of students who have limited English proficiency (LEP). The bilingual test and the English-only test consist of 20 computation and 20 word problem multiple-choice questions (from TIMSS 2003 and 2007 released items. The bilingual test…

Designing Adaptive Instructional Environments: Insights from Empirical Evidence

DTIC Science & Technology

2011-10-01

theorems. Cohen’s f effect size for pretest to posttest gain, averaged across different problems = 0.46. 7 Basis for Adaptation Ability of...problems and took a posttest . Measures of Learning 26-item multiple choice pretest and posttest . Effect size on posttest scores as measured by...solving algebraic equations. Measures of Learning Pretest and posttest using rapid diagnostic testing procedure: Student had to provide their

Health-Related Fitness Knowledge and Its Relation to Student Physical Activity Patterns at a Large U.S. Southern State University

ERIC Educational Resources Information Center

Keating, Xiaofen D.; Castro-Pinero, Jose; Centeio, Erin; Harrison, Louis, Jr.; Ramirez, Tere; Chen, Li

2010-01-01

This study examined student health-related fitness (HRF) knowledge and its relationship to physical activity (PA). The participants were undergraduate students from a large U.S. state university. HRF knowledge was assessed using a test consisting of 150 multiple choice items. Differences in HRF knowledge scores by sex, ethnicity, and years in…

When Listening Is Better Than Reading: Performance Gains on Cardiac Auscultation Test Questions.

PubMed

Short, Kathleen; Bucak, S Deniz; Rosenthal, Francine; Raymond, Mark R

2018-05-01

In 2007, the United States Medical Licensing Examination embedded multimedia simulations of heart sounds into multiple-choice questions. This study investigated changes in item difficulty as determined by examinee performance over time. The data reflect outcomes obtained following initial use of multimedia items from 2007 through 2012, after which an interface change occurred. A total of 233,157 examinees responded to 1,306 cardiology test items over the six-year period; 138 items included multimedia simulations of heart sounds, while 1,168 text-based items without multimedia served as controls. The authors compared changes in difficulty of multimedia items over time with changes in difficulty of text-based cardiology items over time. Further, they compared changes in item difficulty for both groups of items between graduates of Liaison Committee on Medical Education (LCME)-accredited and non-LCME-accredited (i.e., international) medical schools. Examinee performance on cardiology test items with multimedia heart sounds improved by 12.4% over the six-year period, while performance on text-based cardiology items improved by approximately 1.4%. These results were similar for graduates of LCME-accredited and non-LCME-accredited medical schools. Examinees' ability to interpret auscultation findings in test items that include multimedia presentations increased from 2007 to 2012.

Forced-Choice Assessment of Work-Related Maladaptive Personality Traits: Preliminary Evidence From an Application of Thurstonian Item Response Modeling.

PubMed

Guenole, Nigel; Brown, Anna A; Cooper, Andrew J

2018-06-01

This article describes an investigation of whether Thurstonian item response modeling is a viable method for assessment of maladaptive traits. Forced-choice responses from 420 working adults to a broad-range personality inventory assessing six maladaptive traits were considered. The Thurstonian item response model's fit to the forced-choice data was adequate, while the fit of a counterpart item response model to responses to the same items but arranged in a single-stimulus design was poor. Monotrait heteromethod correlations indicated corresponding traits in the two formats overlapped substantially, although they did not measure equivalent constructs. A better goodness of fit and higher factor loadings for the Thurstonian item response model, coupled with a clearer conceptual alignment to the theoretical trait definitions, suggested that the single-stimulus item responses were influenced by biases that the independent clusters measurement model did not account for. Researchers may wish to consider forced-choice designs and appropriate item response modeling techniques such as Thurstonian item response modeling for personality questionnaire applications in industrial psychology, especially when assessing maladaptive traits. We recommend further investigation of this approach in actual selection situations and with different assessment instruments.

Looking Closer at the Effects of Framing on Risky Choice: An Item Response Theory Analysis.

PubMed

Sickar; Highhouse

1998-07-01

Item response theory (IRT) methodology allowed an in-depth examination of several issues that would be difficult to explore using traditional methodology. IRT models were estimated for 4 risky-choice items, answered by students under either a gain or loss frame. Results supported the typical framing finding of risk-aversion for gains and risk-seeking for losses but also suggested that a latent construct we label preference for risk was influential in predicting risky choice. Also, the Asian Disease item, most often used in framing research, was found to have anomalous statistical properties when compared to other framing items. Copyright 1998 Academic Press.

The role of unconscious memory errors in judgments of confidence for sentence recognition.

PubMed

Sampaio, Cristina; Brewer, William F

2009-03-01

The present experiment tested the hypothesis that unconscious reconstructive memory processing can lead to the breakdown of the relationship between memory confidence and memory accuracy. Participants heard deceptive schema-inference sentences and nondeceptive sentences and were tested with either simple or forced-choice recognition. The nondeceptive items showed a positive relation between confidence and accuracy in both simple and forced-choice recognition. However, the deceptive items showed a strong negative confidence/accuracy relationship in simple recognition and a low positive relationship in forced choice. The mean levels of confidence for erroneous responses for deceptive items were inappropriately high in simple recognition but lower in forced choice. These results suggest that unconscious reconstructive memory processes involved in memory for the deceptive schema-inference items led to inaccurate confidence judgments and that, when participants were made aware of the deceptive nature of the schema-inference items through the use of a forced-choice procedure, they adjusted their confidence accordingly.

Effect of price and information on the food choices of women university students in Saudi Arabia: An experimental study.

PubMed

Halimic, Aida; Gage, Heather; Raats, Monique; Williams, Peter

2018-04-01

To explore the impact of price manipulation and healthy eating information on intended food choices. Health information was provided to a random half of subjects (vs. information on Saudi agriculture). Each subject chose from the same lunch menu, containing two healthy and two unhealthy entrees, deserts and beverages, on five occasions. Reference case prices were 5, 3 and 2 Saudi Arabian Reals (SARs). Prices of healthy and unhealthy items were manipulated up (taxed) and down (subsidized) by 1 SAR in four menu variations (random order); subjects were given a budget enabling full choice within any menu. The number of healthy food choices were compared with different price combinations, and between information groups. Linear regression modelling explored the effect of relative prices of healthy/unhealthy options and information on number of healthy choices controlling for dietary behaviours and hunger levels. University campus, Saudi Arabia, 2013. 99 women students. In the reference case, 49.5% of choices were for healthy items. When the price of healthy items was reduced, 58.5% of selections were healthy; 57.2% when the price of unhealthy items rose. In regression modelling, reducing the price of healthy items and increasing the price of unhealthy items increased the number of healthy choices by 5% and 6% respectively. Students reporting a less healthy usual diet selected significantly fewer healthy items. Providing healthy eating information was not a significant influence. Price manipulation offers potential for altering behaviours to combat rising youth obesity in Saudi Arabia. Copyright © 2018 Elsevier Ltd. All rights reserved.

A Five-Year Evaluation of Examination Structure in a Cardiovascular Pharmacotherapy Course

PubMed Central

Kolar, Claire; Janke, Kristin K.

2015-01-01

Objective. To evaluate the composition and effectiveness as an assessment tool of a criterion-referenced examination comprised of clinical cases tied to practice decisions, to examine the effect of varying audience response system (ARS) questions on student examination preparation, and to articulate guidelines for structuring examinations to maximize evaluation of student learning. Design. Multiple-choice items developed over 5 years were evaluated using Bloom’s Taxonomy classification, point biserial correlation, item difficulty, and grade distribution. In addition, examination items were classified into categories based on similarity to items used in ARS preparation. Assessment. As the number of items directly tied to clinical practice rose, Bloom’s Taxonomy level and item difficulty also rose. In examination years where Bloom’s levels were high but preparation was minimal, average grade distribution was lower compared with years in which student preparation was higher. Conclusion. Criterion-referenced examinations can benefit from systematic evaluation of their composition and effectiveness as assessment tools. Calculated design and delivery of classroom preparation is an asset in improving examination performance on rigorous, practice-relevant examinations. PMID:27168611

Robust sampling of decision information during perceptual choice

PubMed Central

Vandormael, Hildward; Herce Castañón, Santiago; Balaguer, Jan; Li, Vickie; Summerfield, Christopher

2017-01-01

Humans move their eyes to gather information about the visual world. However, saccadic sampling has largely been explored in paradigms that involve searching for a lone target in a cluttered array or natural scene. Here, we investigated the policy that humans use to overtly sample information in a perceptual decision task that required information from across multiple spatial locations to be combined. Participants viewed a spatial array of numbers and judged whether the average was greater or smaller than a reference value. Participants preferentially sampled items that were less diagnostic of the correct answer (“inlying” elements; that is, elements closer to the reference value). This preference to sample inlying items was linked to decisions, enhancing the tendency to give more weight to inlying elements in the final choice (“robust averaging”). These findings contrast with a large body of evidence indicating that gaze is directed preferentially to deviant information during natural scene viewing and visual search, and suggest that humans may sample information “robustly” with their eyes during perceptual decision-making. PMID:28223519

An Empirical Test of a Strategy for Training Examinees in the Use of Partial Information in Taking Multiple Choice Tests.

ERIC Educational Resources Information Center

Bliss, Leonard B.

The aim of this study was to show that the superiority of corrected-for-guessing scores over number right scores as true score estimates depends on the ability of examinees to recognize situations where they can eliminate one or more alternatives as incorrect and to omit items where they would only be guessing randomly. Previous investigations…

Constructing objective tests

NASA Astrophysics Data System (ADS)

Aubrecht, Gordon J.; Aubrecht, Judith D.

1983-07-01

True-false or multiple-choice tests can be useful instruments for evaluating student progress. We examine strategies for planning objective tests which serve to test the material covered in science (physics) courses. We also examine strategies for writing questions for tests within a test blueprint. The statistical basis for judging the quality of test items are discussed. Reliability, difficulty, and discrimination indices are defined and examples presented. Our recommendation are rather easily put into practice.

An Examination of the Perceived Importance of Technical Competence in Acquisition Project Management

DTIC Science & Technology

1991-09-01

Develop (First Draft) Instructions Critique (Revision) Answerability Pilot Test (Second Draft) Analysis Response Mode Revision Useability Preparation...appropriate questionnaire items. Initially, the set of questions developed for the study reflected a few shortcomings. A pilot test of the first draft among...resulted. First, feedback from the pilot test indicated a need to reduce the completion time. Because the multiple choice format required several

Music lessons are associated with increased verbal memory in individuals with Williams syndrome.

PubMed

Dunning, Brittany A; Martens, Marilee A; Jungers, Melissa K

2014-11-16

Williams syndrome (WS) is a genetic disorder characterized by intellectual delay and an affinity for music. It has been previously shown that familiar music can enhance verbal memory in individuals with WS who have had music training. There is also evidence that unfamiliar, or novel, music may also improve cognitive recall. This study was designed to examine if a novel melody could also enhance verbal memory in individuals with WS, and to more fully characterize music training in this population. We presented spoken or sung sentences that described an animal and its group name to 44 individuals with WS, and then tested their immediate and delayed memory using both recall and multiple choice formats. Those with formal music training (average duration of training 4½ years) scored significantly higher on both the spoken and sung recall items, as well as on the spoken multiple choice items, than those with no music training. Music therapy, music enjoyment, age, and Verbal IQ did not impact performance on the memory tasks. These findings provide further evidence that formal music lessons may impact the neurological pathways associated with verbal memory in individuals with WS, consistent with findings in typically developing individuals. Copyright © 2014 Elsevier Ltd. All rights reserved.

Development of the Exam of GeoloGy Standards, EGGS, to Measure Students' Conceptual Understanding of Geology Concepts

NASA Astrophysics Data System (ADS)

Guffey, S. K.; Slater, T. F.; Slater, S. J.

2017-12-01

Discipline-based geoscience education researchers have considerable need for criterion-referenced, easy-to-administer and easy-to-score, conceptual diagnostic surveys for undergraduates taking introductory science survey courses in order for faculty to better be able to monitor the learning impacts of various interactive teaching approaches. To support ongoing discipline-based science education research to improve teaching and learning across the geosciences, this study establishes the reliability and validity of a 28-item, multiple-choice, pre- and post- Exam of GeoloGy Standards, hereafter simply called EGGS. The content knowledge EGGS addresses is based on 11 consensus concepts derived from a systematic, thematic analysis of the overlapping ideas presented in national science education reform documents including the Next Generation Science Standards, the AAAS Benchmarks for Science Literacy, the Earth Science Literacy Principles, and the NRC National Science Education Standards. Using community agreed upon best-practices for creating, field-testing, and iteratively revising modern multiple-choice test items using classical item analysis techniques, EGGS emphasizes natural student language over technical scientific vocabulary, leverages illustrations over students' reading ability, specifically targets students' misconceptions identified in the scholarly literature, and covers the range of topics most geology educators expect general education students to know at the end of their formal science learning experiences. The current version of EGGS is judged to be valid and reliable with college-level, introductory science survey students based on both standard quantitative and qualitative measures, including extensive clinical interviews with targeted students and systematic expert review.

The results of STEM education methods for enhancing critical thinking and problem solving skill in physics the 10th grade level

NASA Astrophysics Data System (ADS)

Soros, P.; Ponkham, K.; Ekkapim, S.

2018-01-01

This research aimed to: 1) compare the critical think and problem solving skills before and after learning using STEM Education plan, 2) compare student achievement before and after learning about force and laws of motion using STEM Education plan, and 3) the satisfaction of learning by using STEM Education. The sample used were 37 students from grade 10 at Borabu School, Borabu District, Mahasarakham Province, semester 2, Academic year 2016. Tools used in this study consist of: 1) STEM Education plan about the force and laws of motion for grade 10 students of 1 schemes with total of 14 hours, 2) The test of critical think and problem solving skills with multiple-choice type of 5 options and 2 option of 30 items, 3) achievement test on force and laws of motion with multiple-choice of 4 options of 30 items, 4) satisfaction learning with 5 Rating Scale of 20 items. The statistics used in data analysis were percentage, mean, standard deviation, and t-test (Dependent). The results showed that 1) The student with learning using STEM Education plan have score of critical think and problem solving skills on post-test higher than pre-test with statistically significant level .01. 2) The student with learning using STEM Education plan have achievement score on post-test higher than pre-test with statistically significant level of .01. 3) The student'level of satisfaction toward the learning by using STEM Education plan was at a high level (X ¯ = 4.51, S.D=0.56).

Investigating the potential influence of established multiple-choice test-taking cues on item response in a pharmacotherapy board certification examination preparatory manual: a pilot study.

PubMed

Gettig, Jacob P

2006-04-01

To determine the prevalence of established multiple-choice test-taking correct and incorrect answer cues in the American College of Clinical Pharmacy's Updates in Therapeutics: The Pharmacotherapy Preparatory Course, 2005 Edition, as an equal or lesser surrogate indication of the prevalence of such cues in the Pharmacotherapy board certification examination. All self-assessment and patient case question-and-answer sets were assessed individually to determine if they were subject to selected correct and incorrect answer cues commonly seen in multiple-choice question writing. If the question was considered evaluable, correct answer cues-longest answer, mid-range number, one of two similar choices, and one of two opposite choices-were tallied. In addition, incorrect answer cues- inclusionary language and grammatical mismatch-were also tallied. Each cue was counted if it did what was expected or did the opposite of what was expected. Multiple cues could be identified in each question. A total of 237 (47.7%) of 497 questions in the manual were deemed evaluable. A total of 325 correct answer cues and 35 incorrect answer cues were identified in the 237 evaluable questions. Most evaluable questions contained one to two correct and/or incorrect answer cue(s). Longest answer was the most frequently identified correct answer cue; however, it was the least likely to identify the correct answer. Inclusionary language was the most frequently identified incorrect answer cue. Incorrect answer cues were considerably more likely to identify incorrect answer choices than correct answer cues were able to identify correct answer choices. The use of established multiple-choice test-taking cues is unlikely to be of significant help when taking the Pharmacotherapy board certification examination, primarily because of the lack of questions subject to such cues and the inability of correct answer cues to accurately identify correct answers. Incorrect answer cues, especially the use of inclusionary language, almost always will accurately identify an incorrect answer choice. Assuming that questions in the preparatory course manual were equal or lesser surrogates of those in the board certification examination, it is unlikely that intuition alone can replace adequate preparation and studying as the sole determinant of examination success.

Testing primary-school children's understanding of the nature of science.

PubMed

Koerber, Susanne; Osterhaus, Christopher; Sodian, Beate

2015-03-01

Understanding the nature of science (NOS) is a critical aspect of scientific reasoning, yet few studies have investigated its developmental beginnings and initial structure. One contributing reason is the lack of an adequate instrument. Two studies assessed NOS understanding among third graders using a multiple-select (MS) paper-and-pencil test. Study 1 investigated the validity of the MS test by presenting the items to 68 third graders (9-year-olds) and subsequently interviewing them on their underlying NOS conception of the items. All items were significantly related between formats, indicating that the test was valid. Study 2 applied the same instrument to a larger sample of 243 third graders, and their performance was compared to a multiple-choice (MC) version of the test. Although the MC format inflated the guessing probability, there was a significant relation between the two formats. In summary, the MS format was a valid method revealing third graders' NOS understanding, thereby representing an economical test instrument. A latent class analysis identified three groups of children with expertise in qualitatively different aspects of NOS, suggesting that there is not a single common starting point for the development of NOS understanding; instead, multiple developmental pathways may exist. © 2014 The British Psychological Society.

Ontology-Based Multiple Choice Question Generation

PubMed Central

Al-Yahya, Maha

2014-01-01

With recent advancements in Semantic Web technologies, a new trend in MCQ item generation has emerged through the use of ontologies. Ontologies are knowledge representation structures that formally describe entities in a domain and their relationships, thus enabling automated inference and reasoning. Ontology-based MCQ item generation is still in its infancy, but substantial research efforts are being made in the field. However, the applicability of these models for use in an educational setting has not been thoroughly evaluated. In this paper, we present an experimental evaluation of an ontology-based MCQ item generation system known as OntoQue. The evaluation was conducted using two different domain ontologies. The findings of this study show that ontology-based MCQ generation systems produce satisfactory MCQ items to a certain extent. However, the evaluation also revealed a number of shortcomings with current ontology-based MCQ item generation systems with regard to the educational significance of an automatically constructed MCQ item, the knowledge level it addresses, and its language structure. Furthermore, for the task to be successful in producing high-quality MCQ items for learning assessments, this study suggests a novel, holistic view that incorporates learning content, learning objectives, lexical knowledge, and scenarios into a single cohesive framework. PMID:24982937

Measuring sexual orientation in adolescent health surveys: evaluation of eight school-based surveys.

PubMed

Saewyc, Elizabeth M; Bauer, Greta R; Skay, Carol L; Bearinger, Linda H; Resnick, Michael D; Reis, Elizabeth; Murphy, Aileen

2004-10-01

To examine the performance of various items measuring sexual orientation within 8 school-based adolescent health surveys in the United States and Canada from 1986 through 1999. Analyses examined nonresponse and unsure responses to sexual orientation items compared with other survey items, demographic differences in responses, tests for response set bias, and congruence of responses to multiple orientation items; analytical methods included frequencies, contingency tables with Chi-square, and ANOVA with least significant differences (LSD)post hoc tests; all analyses were conducted separately by gender. In all surveys, nonresponse rates for orientation questions were similar to other sexual questions, but not higher; younger students, immigrants, and students with learning disabilities were more likely to skip items or select "unsure." Sexual behavior items had the lowest nonresponse, but fewer than half of all students reported sexual behavior, limiting its usefulness for indicating orientation. Item placement in the survey, wording, and response set bias all appeared to influence nonresponse and unsure rates. Specific recommendations include standardizing wording across future surveys, and pilot testing items with diverse ages and ethnic groups of teens before use. All three dimensions of orientation should be assessed where possible; when limited to single items, sexual attraction may be the best choice. Specific wording suggestions are offered for future surveys.

A trans-Atlantic examination of haddock Melanogrammus aeglefinus food habits.

PubMed

Tam, J C; Link, J S; Large, S I; Bogstad, B; Bundy, A; Cook, A M; Dingsør, G E; Dolgov, A V; Howell, D; Kempf, A; Pinnegar, J K; Rindorf, A; Schückel, S; Sell, A F; Smith, B E

2016-06-01

The food habits of Melanogrammus aeglefinus were explored and contrasted across multiple north-eastern and north-western Atlantic Ocean ecosystems, using databases that span multiple decades. The results show that among all ecosystems, echinoderms are a consistent part of M. aeglefinus diet, but patterns emerge regarding where and when M. aeglefinus primarily eat fishes v. echinoderms. Melanogrammus aeglefinus does not regularly exhibit the increase in piscivory with ontogeny that other gadoids often show, and in several ecosystems there is a lower occurrence of piscivory. There is an apparent inverse relationship between the consumption of fishes and echinoderms in M. aeglefinus over time, where certain years show high levels of one prey item and low levels of the other. This apparent binary choice can be viewed as part of a gradient of prey options, contingent upon a suite of factors external to M. aeglefinus dynamics. The energetic consequences of this prey choice are discussed, noting that in some instances it may not be a choice at all. © 2016 The Fisheries Society of the British Isles.

How IRT Can Solve Problems of Ipsative Data in Forced-Choice Questionnaires

ERIC Educational Resources Information Center

Brown, Anna; Maydeu-Olivares, Alberto

2013-01-01

In multidimensional forced-choice (MFC) questionnaires, items measuring different attributes are presented in blocks, and participants have to rank order the items within each block (fully or partially). Such comparative formats can reduce the impact of numerous response biases often affecting single-stimulus items (aka rating or Likert scales).…

Algorithms for Developing Test Questions from Sentences in Instructional Materials: an Extension of an Earlier Study

DTIC Science & Technology

1980-01-01

Silverfish, Canine, and Cicadas . b. ’Mgorithinically--Sdverfish, Females, Individuals, and Wasps. This process resulted in 16.0 multiple-choice items: 20...and Their Text Frequency Nouns Adjectives Rare Singleton Keyword Rare Singleton Keyword Instars Insect (8) Plant-feeding Immature (3) Cicadas ...Sllverflsh (c) Caniru’S (d) Cicadas c. Foils Produced Algorl ..nii-allv: Sllverflsh Fenvilea ItuliviJu.i Is Wanps ?. Kevword Adjective—Immature

An Exploratory Study of the Relationships between Reported Imagery and the Comprehension and Recall of a Story in Fifth Graders. Instructional Research Laboratory Technical Paper # R82007.

ERIC Educational Resources Information Center

Sadoski, Mark C.

A study investigated the role of visual imagery in the comprehension and retention of prose. Subjects were 48 fifth grade students who orally read a story and then completed three comprehension tasks directly related to the story: a retelling, an oral reading cloze test, and a multiple choice question test comprised of items demonstrated to be…

A New Family of Models for the Multiple-Choice Item.

DTIC Science & Technology

1979-12-19

analysis of the verbal scholastic aptitude test using Birnhaum’s three-parameter logistic model. Educational and Psychological Measurement, 28, 989-1020...16. [8] McBride, J. R. Some properties of a Bayesian adaptive ability testing strategy. Applied Psychological Measurement, 1, 121-140, 1977. [9...University of Michigan Ann Arbor, MI 48106 ’~KL -137- Non Govt Mon Govt 1 Dr. Earl Hunt 1 Dr. Frederick N. Lord Dept. of Psychology Educational Testing

The development of a computer assisted instruction and assessment system in pharmacology.

PubMed

Madsen, B W; Bell, R C

1977-01-01

We describe the construction of a computer based system for instruction and assessment in pharmacology, utilizing a large bank of multiple choice questions. Items were collected from many sources, edited and coded for student suitability, topic, taxonomy and difficulty and text references. Students reserve a time during the day, specify the type of test desired and questions are presented randomly from the subset satisfying their criteria. Answers are scored after each question and a summary given at the end of every test; details on item performance are recorded automatically. The biggest hurdle in implementation was the assembly, review, classification and editing of items, while the programming was relatively straight-forward. A number of modifications had to be made to the initial plans and changes will undoubtedly continue with further experience. When fully operational the system will possess a number of advantages including: elimination of test preparation, editing and marking; facilitated item review opportunities; increased objectivity, feedback, flexibility and descreased anxiety in students.

Advertising influences on young children's food choices and parental influence.

PubMed

Ferguson, Christopher J; Muñoz, Monica E; Medrano, Maria R

2012-03-01

To evaluate whether advertising for food influences choices made by children, the strength of these influences, and whether they might be easily undone by parental influences. Children between 3 and 8 years of age (n=75) were randomized to watch a series of programs with embedded commercials. Some children watched a commercial for a relatively healthy food item, the other children watched a commercial for a less healthy item, both from the same fast-food company. Children were also randomized either to receive parental encouragement to choose the healthy item or to choose whichever item they preferred. Results indicated that children were more likely to choose the advertised item, despite parental input. Parental input only slightly moderated this influence. Although advertising impact on children's food choices is moderate in size, it appears resilient to parental efforts to intervene. Food advertisements directed at children may have a small but meaningful effect on their healthy food choices. Copyright © 2012 Mosby, Inc. All rights reserved.

M-OSCE as a method to measure dental hygiene students' critical thinking: a pilot study.

PubMed

McComas, Martha J; Wright, Rebecca A; Mann, Nancy K; Cooper, Mary D; Jacks, Mary E

2013-04-01

Educators in all academic disciplines have been encouraged to utilize assessment strategies to evaluate students' critical thinking. The purpose of this study was to assess the viability of the modified objective structured clinical examination (m-OSCE) to evaluate critical thinking in dental hygiene education. This evaluation utilized a convenience sample of senior dental hygiene students. Students participated in the m-OSCE in which portions of a patient case were revealed at four stations. The exam consisted of multiple-choice questions intended to measure students' ability to utilize critical thinking skills. Additionally, there was one fill-in-the-blank question and a treatment plan that was completed at the fifth station. The results of this study revealed that the m-OSCE did not reliably measure dental hygiene students' critical thinking. Statistical analysis found no satisfactory reliability within the multiple-choice questions and moderately reliable results within the treatment planning portion of the examination. In addition, the item analysis found gaps in students' abilities to transfer clinical evidence/data to basic biomedical knowledge as demonstrated through the multiple-choice questioning results. This outcome warrants further investigation of the utility of the m-OSCE, with a focus on modifications to the evaluation questions, grading rubric, and patient case.

There’s more to food store choice than proximity: a questionnaire development study

PubMed Central

2013-01-01

Background Proximity of food stores is associated with dietary intake and obesity; however, individuals frequently shop at stores that are not the most proximal. Little is known about other factors that influence food store choice. The current research describes the development of the Food Store Selection Questionnaire (FSSQ) and describes preliminary results of field testing the questionnaire. Methods Development of the FSSQ involved a multidisciplinary literature review, qualitative analysis of focus group transcripts, and expert and community reviews. Field testing consisted of 100 primary household food shoppers (93% female, 64% African American), in rural and urban Arkansas communities, rating FSSQ items as to their importance in store choice and indicating their top two reasons. After eliminating 14 items due to low mean importance scores and high correlations with other items, the final FSSQ questionnaire consists of 49 items. Results Items rated highest in importance were: meat freshness; store maintenance; store cleanliness; meat varieties; and store safety. Items most commonly rated as top reasons were: low prices; proximity to home; fruit/vegetable freshness; fruit/vegetable variety; and store cleanliness. Conclusions The FSSQ is a comprehensive questionnaire for detailing key reasons in food store choice. Although proximity to home was a consideration for participants, there were clearly other key factors in their choice of a food store. Understanding the relative importance of these different dimensions driving food store choice in specific communities may be beneficial in informing policies and programs designed to support healthy dietary intake and obesity prevention. PMID:23773428

There's more to food store choice than proximity: a questionnaire development study.

PubMed

Krukowski, Rebecca A; Sparks, Carla; DiCarlo, Marisha; McSweeney, Jean; West, Delia Smith

2013-06-17

Proximity of food stores is associated with dietary intake and obesity; however, individuals frequently shop at stores that are not the most proximal. Little is known about other factors that influence food store choice. The current research describes the development of the Food Store Selection Questionnaire (FSSQ) and describes preliminary results of field testing the questionnaire. Development of the FSSQ involved a multidisciplinary literature review, qualitative analysis of focus group transcripts, and expert and community reviews. Field testing consisted of 100 primary household food shoppers (93% female, 64% African American), in rural and urban Arkansas communities, rating FSSQ items as to their importance in store choice and indicating their top two reasons. After eliminating 14 items due to low mean importance scores and high correlations with other items, the final FSSQ questionnaire consists of 49 items. Items rated highest in importance were: meat freshness; store maintenance; store cleanliness; meat varieties; and store safety. Items most commonly rated as top reasons were: low prices; proximity to home; fruit/vegetable freshness; fruit/vegetable variety; and store cleanliness. The FSSQ is a comprehensive questionnaire for detailing key reasons in food store choice. Although proximity to home was a consideration for participants, there were clearly other key factors in their choice of a food store. Understanding the relative importance of these different dimensions driving food store choice in specific communities may be beneficial in informing policies and programs designed to support healthy dietary intake and obesity prevention.

Development and Validation of the Homeostasis Concept Inventory

PubMed Central

McFarland, Jenny L.; Price, Rebecca M.; Wenderoth, Mary Pat; Martinková, Patrícia; Cliff, William; Michael, Joel; Modell, Harold; Wright, Ann

2017-01-01

We present the Homeostasis Concept Inventory (HCI), a 20-item multiple-choice instrument that assesses how well undergraduates understand this critical physiological concept. We used an iterative process to develop a set of questions based on elements in the Homeostasis Concept Framework. This process involved faculty experts and undergraduate students from associate’s colleges, primarily undergraduate institutions, regional and research-intensive universities, and professional schools. Statistical results provided strong evidence for the validity and reliability of the HCI. We found that graduate students performed better than undergraduates, biology majors performed better than nonmajors, and students performed better after receiving instruction about homeostasis. We used differential item analysis to assess whether students from different genders, races/ethnicities, and English language status performed differently on individual items of the HCI. We found no evidence of differential item functioning, suggesting that the items do not incorporate cultural or gender biases that would impact students’ performance on the test. Instructors can use the HCI to guide their teaching and student learning of homeostasis, a core concept of physiology. PMID:28572177

Multi-step routes of capuchin monkeys in a laser pointer traveling salesman task.

PubMed

Howard, Allison M; Fragaszy, Dorothy M

2014-09-01

Prior studies have claimed that nonhuman primates plan their routes multiple steps in advance. However, a recent reexamination of multi-step route planning in nonhuman primates indicated that there is no evidence for planning more than one step ahead. We tested multi-step route planning in capuchin monkeys using a pointing device to "travel" to distal targets while stationary. This device enabled us to determine whether capuchins distinguish the spatial relationship between goals and themselves and spatial relationships between goals and the laser dot, allocentrically. In Experiment 1, two subjects were presented with identical food items in Near-Far (one item nearer to subject) and Equidistant (both items equidistant from subject) conditions with a laser dot visible between the items. Subjects moved the laser dot to the items using a joystick. In the Near-Far condition, one subject demonstrated a bias for items closest to self but the other subject chose efficiently. In the second experiment, subjects retrieved three food items in similar Near-Far and Equidistant arrangements. Both subjects preferred food items nearest the laser dot and showed no evidence of multi-step route planning. We conclude that these capuchins do not make choices on the basis of multi-step look ahead strategies. © 2014 Wiley Periodicals, Inc.

Mechanisms of Choice Behavior Shift Using Cue-approach Training.

PubMed

Bakkour, Akram; Leuker, Christina; Hover, Ashleigh M; Giles, Nathan; Poldrack, Russell A; Schonberg, Tom

2016-01-01

Cue-approach training has been shown to effectively shift choices for snack food items by associating a cued button-press motor response to particular food items. Furthermore, attention was biased toward previously cued items, even when the cued item is not chosen for real consumption during a choice phase. However, the exact mechanism by which preferences shift during cue-approach training is not entirely clear. In three experiments, we shed light on the possible underlying mechanisms at play during this novel paradigm: (1) Uncued, wholly predictable motor responses paired with particular food items were not sufficient to elicit a preference shift; (2) Cueing motor responses early - concurrently with food item onset - and thus eliminating the need for heightened top-down attention to the food stimulus in preparation for a motor response also eliminated the shift in food preferences. This finding reinforces our hypothesis that heightened attention at behaviorally relevant points in time is key to changing choice behavior in the cue-approach task; (3) Crucially, indicating choice using eye movements rather than manual button presses preserves the effect, thus demonstrating that the shift in preferences is not governed by a learned motor response but more likely via modulation of subjective value in higher associative regions, consistent with previous neuroimaging results. Cue-approach training drives attention at behaviorally relevant points in time to modulate the subjective value of individual items, providing a mechanism for behavior change that does not rely on external reinforcement and that holds great promise for developing real world behavioral interventions.

Action and Valence Modulate Choice and Choice-Induced Preference Change

PubMed Central

Koster, Raphael; Duzel, Emrah; Dolan, Raymond J.

2015-01-01

Choices are not only communicated via explicit actions but also passively through inaction. In this study we investigated how active or passive choice impacts upon the choice process itself as well as a preference change induced by choice. Subjects were tasked to select a preference for unfamiliar photographs by action or inaction, before and after they gave valuation ratings for all photographs. We replicate a finding that valuation increases for chosen items and decreases for unchosen items compared to a control condition in which the choice was made post re-evaluation. Whether choice was expressed actively or passively affected the dynamics of revaluation differently for positive and negatively valenced items. Additionally, the choice itself was biased towards action such that subjects tended to choose a photograph obtained by action more often than a photographed obtained through inaction. These results highlight intrinsic biases consistent with a tight coupling of action and reward and add to an emerging understanding of how the mode of action itself, and not just an associated outcome, modulates the decision making process. PMID:25747703

The recall of information from working memory. Insights from behavioural and chronometric perspectives.

PubMed

Towse, John N; Cowan, Nelson; Hitch, Graham J; Horton, Neil J

2008-01-01

We describe and evaluate a recall reconstruction hypothesis for working memory (WM), according to which items can be recovered from multiple memory representations. Across four experiments, participants recalled memoranda that were either integrated with or independent of the sentence content. We found consistently longer pauses accompanying the correct recall of integrated compared with independent words, supporting the argument that sentence memory could scaffold the access of target items. Integrated words were also more likely to be recalled correctly, dependent on the details of the task. Experiment 1 investigated the chronometry of spoken recall for word span and reading span, with participants completing an unfinished sentence in the latter case. Experiments 2 and 3 confirm recall time differences without using word generation requirements, while Experiment 4 used an item and order response choice paradigm with nonspoken responses. Data emphasise the value of recall timing in constraining theories of WM functioning.

Evaluation of diagnostic tools that tertiary teachers can apply to profile their students' conceptions

NASA Astrophysics Data System (ADS)

Schultz, Madeleine; Lawrie, Gwendolyn A.; Bailey, Chantal H.; Bedford, Simon B.; Dargaville, Tim R.; O'Brien, Glennys; Tasker, Roy; Thompson, Christopher D.; Williams, Mark; Wright, Anthony H.

2017-03-01

A multi-institution collaborative team of Australian chemistry education researchers, teaching a total of over 3000 first year chemistry students annually, has explored a tool for diagnosing students' prior conceptions as they enter tertiary chemistry courses. Five core topics were selected and clusters of diagnostic items were assembled linking related concepts in each topic together. An ordered multiple choice assessment strategy was adopted to enable provision of formative feedback to students through combination of the specific distractors that they chose. Concept items were either sourced from existing research instruments or developed by the project team. The outcome is a diagnostic tool consisting of five topic clusters of five concept items that has been delivered in large introductory chemistry classes at five Australian institutions. Statistical analysis of data has enabled exploration of the composition and validity of the instrument including a comparison between delivery of the complete 25 item instrument with subsets of five items, clustered by topic. This analysis revealed that most items retained their validity when delivered in small clusters. Tensions between the assembly, validation and delivery of diagnostic instruments for the purposes of acquiring robust psychometric research data versus their pragmatic use are considered in this study.

The Effects of Item Format and Cognitive Domain on Students' Science Performance in TIMSS 2011

NASA Astrophysics Data System (ADS)

Liou, Pey-Yan; Bulut, Okan

2017-12-01

The purpose of this study was to examine eighth-grade students' science performance in terms of two test design components, item format, and cognitive domain. The portion of Taiwanese data came from the 2011 administration of the Trends in International Mathematics and Science Study (TIMSS), one of the major international large-scale assessments in science. The item difficulty analysis was initially applied to show the proportion of correct items. A regression-based cumulative link mixed modeling (CLMM) approach was further utilized to estimate the impact of item format, cognitive domain, and their interaction on the students' science scores. The results of the proportion-correct statistics showed that constructed-response items were more difficult than multiple-choice items, and that the reasoning cognitive domain items were more difficult compared to the items in the applying and knowing domains. In terms of the CLMM results, students tended to obtain higher scores when answering constructed-response items as well as items in the applying cognitive domain. When the two predictors and the interaction term were included together, the directions and magnitudes of the predictors on student science performance changed substantially. Plausible explanations for the complex nature of the effects of the two test-design predictors on student science performance are discussed. The results provide practical, empirical-based evidence for test developers, teachers, and stakeholders to be aware of the differential function of item format, cognitive domain, and their interaction in students' science performance.

Hunger enhances consistent economic choices in non-human primates.

PubMed

Yamada, Hiroshi

2017-05-24

Hunger and thirst are fundamental biological processes that drive consumption behavior in humans and non-human animals. While the existing literature in neuroscience suggests that these satiety states change how consumable rewards are represented in the brain, it remains unclear as to how they change animal choice behavior and the underlying economic preferences. Here, I used combined techniques from experimental economics, psychology, and neuroscience to measure food preferences of marmoset monkeys (Callithrix jacchus), a recently developed primate model for neuroscience. Hunger states of animals were manipulated by scheduling feeding intervals, resulting in three different conditions: sated, non-sated, and hungry. During these hunger states, animals performed pairwise choices of food items, which included all possible pairwise combinations of five different food items except for same-food pairs. Results showed that hunger enhanced economic rationality, evident as a decrease of transitivity violations (item A was preferred to item B, and B to C, but C was preferred to A). Further analysis demonstrated that hungry monkeys chose more-preferred items over less-preferred items in a more deterministic manner, while the individual food preferences appeared to remain stable across hunger states. These results suggest that hunger enhances consistent choice behavior and shifts animals towards efficient outcome maximization.

An algorithm for calculating exam quality as a basis for performance-based allocation of funds at medical schools.

PubMed

Kirschstein, Timo; Wolters, Alexander; Lenz, Jan-Hendrik; Fröhlich, Susanne; Hakenberg, Oliver; Kundt, Günther; Darmüntzel, Martin; Hecker, Michael; Altiner, Attila; Müller-Hilke, Brigitte

2016-01-01

The amendment of the Medical Licensing Act (ÄAppO) in Germany in 2002 led to the introduction of graded assessments in the clinical part of medical studies. This, in turn, lent new weight to the importance of written tests, even though the minimum requirements for exam quality are sometimes difficult to reach. Introducing exam quality as a criterion for the award of performance-based allocation of funds is expected to steer the attention of faculty members towards more quality and perpetuate higher standards. However, at present there is a lack of suitable algorithms for calculating exam quality. In the spring of 2014, the students' dean commissioned the "core group" for curricular improvement at the University Medical Center in Rostock to revise the criteria for the allocation of performance-based funds for teaching. In a first approach, we developed an algorithm that was based on the results of the most common type of exam in medical education, multiple choice tests. It included item difficulty and discrimination, reliability as well as the distribution of grades achieved. This algorithm quantitatively describes exam quality of multiple choice exams. However, it can also be applied to exams involving short assay questions and the OSCE. It thus allows for the quantitation of exam quality in the various subjects and - in analogy to impact factors and third party grants - a ranking among faculty. Our algorithm can be applied to all test formats in which item difficulty, the discriminatory power of the individual items, reliability of the exam and the distribution of grades are measured. Even though the content validity of an exam is not considered here, we believe that our algorithm is suitable as a general basis for performance-based allocation of funds.

Dual process theory and intermediate effect: are faculty and residents' performance on multiple-choice, licensing exam questions different?

PubMed

Dong, Ting; Durning, Steven J; Artino, Anthony R; van der Vleuten, Cees; Holmboe, Eric; Lipner, Rebecca; Schuwirth, Lambert

2015-04-01

Clinical reasoning is essential for the practice of medicine. Dual process theory conceptualizes reasoning as falling into two general categories: nonanalytic reasoning (pattern recognition) and analytic reasoning (active comparing and contrasting of alternatives). The debate continues regarding how expert performance develops and how individuals make the best use of analytic and nonanalytic processes. Several investigators have identified the unexpected finding that intermediates tend to perform better on licensing examination items than experts, which has been termed the "intermediate effect." We explored differences between faculty and residents on multiple-choice questions (MCQs) using dual process measures (both reading and answering times) to inform this ongoing debate. Faculty (board-certified internists; experts) and residents (internal medicine interns; intermediates) answered live licensing examination MCQs (U.S. Medical Licensing Examination Step 2 Clinical Knowledge and American Board of Internal Medicine Certifying Examination) while being timed. We conducted repeated analysis of variance to compare the 2 groups on average reading time, answering time, and accuracy on various types of items. Faculty and residents did not differ significantly in reading time [F (1,35) = 0.01, p = 0.93], answering time [F (1,35) = 0.60, p = 0.44], or accuracy [F (1,35) = 0.24, p = 0.63] regardless of easy or hard items. Dual process theory was not evidenced in this study. However, this lack of difference between faculty and residents may have been affected by the small sample size of participants and MCQs may not reflect how physicians made decisions in actual practice setting. Reprint & Copyright © 2015 Association of Military Surgeons of the U.S.

Value of freedom to choose encoded by the human brain

PubMed Central

Fujiwara, Juri; Usui, Nobuo; Park, Soyoung Q.; Williams, Tony; Iijima, Toshio; Taira, Masato; Tsutsui, Ken-Ichiro

2013-01-01

Humans and animals value the opportunity to choose by preferring alternatives that offer more rather than fewer choices. This preference for choice may arise not only from an increased probability of obtaining preferred outcomes but also from the freedom it provides. We used human neuroimaging to investigate the neural basis of the preference for choice as well as for the items that could be chosen. In each trial, participants chose between two options, a monetary amount option and a “choice option.” The latter consisted of a number that corresponded to the number of everyday items participants would subsequently be able to choose from. We found that the opportunity to choose from a larger number of items was equivalent to greater amounts of money, indicating that participants valued having more choice; moreover, participants varied in the degree to which they valued having the opportunity to choose, with some valuing it more than the increased probability of obtaining preferred items. Neural activations in the mid striatum increased with the value of the opportunity to choose. The same region also coded the value of the items. Conversely, activation in the dorsolateral striatum was not related to the value of the items but was elevated when participants were offered more choices, particularly in those participants who overvalued the opportunity to choose. These data suggest a functional dissociation of value representations within the striatum, with general representations in mid striatum and specific representations of the value of freedom provided by the opportunity to choose in dorsolateral striatum. PMID:23864380

Mechanisms of Choice Behavior Shift Using Cue-approach Training

PubMed Central

Bakkour, Akram; Leuker, Christina; Hover, Ashleigh M.; Giles, Nathan; Poldrack, Russell A.; Schonberg, Tom

2016-01-01

Cue-approach training has been shown to effectively shift choices for snack food items by associating a cued button-press motor response to particular food items. Furthermore, attention was biased toward previously cued items, even when the cued item is not chosen for real consumption during a choice phase. However, the exact mechanism by which preferences shift during cue-approach training is not entirely clear. In three experiments, we shed light on the possible underlying mechanisms at play during this novel paradigm: (1) Uncued, wholly predictable motor responses paired with particular food items were not sufficient to elicit a preference shift; (2) Cueing motor responses early – concurrently with food item onset – and thus eliminating the need for heightened top–down attention to the food stimulus in preparation for a motor response also eliminated the shift in food preferences. This finding reinforces our hypothesis that heightened attention at behaviorally relevant points in time is key to changing choice behavior in the cue-approach task; (3) Crucially, indicating choice using eye movements rather than manual button presses preserves the effect, thus demonstrating that the shift in preferences is not governed by a learned motor response but more likely via modulation of subjective value in higher associative regions, consistent with previous neuroimaging results. Cue-approach training drives attention at behaviorally relevant points in time to modulate the subjective value of individual items, providing a mechanism for behavior change that does not rely on external reinforcement and that holds great promise for developing real world behavioral interventions. PMID:27047435

A 2-phase labeling and choice architecture intervention to improve healthy food and beverage choices.

PubMed

Thorndike, Anne N; Sonnenberg, Lillian; Riis, Jason; Barraclough, Susan; Levy, Douglas E

2012-03-01

We assessed whether a 2-phase labeling and choice architecture intervention would increase sales of healthy food and beverages in a large hospital cafeteria. Phase 1 was a 3-month color-coded labeling intervention (red = unhealthy, yellow = less healthy, green = healthy). Phase 2 added a 3-month choice architecture intervention that increased the visibility and convenience of some green items. We compared relative changes in 3-month sales from baseline to phase 1 and from phase 1 to phase 2. At baseline (977,793 items, including 199,513 beverages), 24.9% of sales were red and 42.2% were green. Sales of red items decreased in both phases (P < .001), and green items increased in phase 1 (P < .001). The largest changes occurred among beverages. Red beverages decreased 16.5% during phase 1 (P < .001) and further decreased 11.4% in phase 2 (P < .001). Green beverages increased 9.6% in phase 1 (P < .001) and further increased 4.0% in phase 2 (P < .001). Bottled water increased 25.8% during phase 2 (P < .001) but did not increase at 2 on-site comparison cafeterias (P < .001). A color-coded labeling intervention improved sales of healthy items and was enhanced by a choice architecture intervention.

A Analysis of Saudi Arabian High School Students' Misconceptions about Physics Concepts.

NASA Astrophysics Data System (ADS)

Al-Rubayea, Abdullah A. M.

This study was conducted to explore Saudi high students' misconceptions in selected physics concepts. It also detected the effects of gender, grade level and location of school on Saudi high school students' misconceptions. In addition, a further analysis of students' misconceptions in each question was investigated and a correlation between students' responses, confidence in answers and sensibleness was conducted. There was an investigation of sources of students' answers in this study. Finally, this study included an analysis of students' selection of reasons only in the instrument. The instrument used to detect the students' misconceptions was a modified form of the Misconception Identification in Science Questionnaire (MISQ). This instrument was developed by Franklin (1992) to detected students' misconceptions in selected physics concepts. This test is a two-tier multiple choice test that examines four areas of physics: Force and motion, heat and temperature, light and color and electricity and magnetism. This study included a sample of 1080 Saudi high school students who were randomly selected from six Saudi educational districts. This study also included both genders, the three grade levels of Saudi high schools, six different educational districts, and a city and a town in each educational district. The sample was equally divided between genders, grade levels, and educational districts. The result of this study revealed that Saudi Arabian high school students hold numerous misconceptions about selected physics concepts. It also showed that tenth grade students were significantly different than the other grades. The result also showed that different misconceptions are held by the students for each concept in the MISQ. A positive correlation between students' responses, confidence in answers and sensibleness in many questions was shown. In addition, it showed that guessing was the most dominant source of misconceptions. The result revealed that gender and grade level had an affect on students' choice of decision on the MISQ items. A positive change in the means of gender and grade levels in the multiple choice test and gender differences in selection of reason may be associated with specific concepts. No significant difference in frequencies of the reasons chosen by the student to justify their answers were found in most of the items (10 items).

State Test Programs Mushroom as NCLB Mandate Kicks in: Nearly Half of States Are Expanding Their Testing Programs to Additional Grades This School Year to Comply with the Federal No Child Left Behind Act

ERIC Educational Resources Information Center

Olson, Lynn

2005-01-01

Twenty-three states are expanding their testing programs to additional grades this school year to comply with the federal No Child Left Behind Act. In devising the new tests, most states have defied predictions and chosen to go beyond multiple-choice items, by including questions that ask students to construct their own responses. But many state…

Test of understanding of vectors: A reliable multiple-choice vector concept test

NASA Astrophysics Data System (ADS)

Barniol, Pablo; Zavala, Genaro

2014-06-01

In this article we discuss the findings of our research on students' understanding of vector concepts in problems without physical context. First, we develop a complete taxonomy of the most frequent errors made by university students when learning vector concepts. This study is based on the results of several test administrations of open-ended problems in which a total of 2067 students participated. Using this taxonomy, we then designed a 20-item multiple-choice test [Test of understanding of vectors (TUV)] and administered it in English to 423 students who were completing the required sequence of introductory physics courses at a large private Mexican university. We evaluated the test's content validity, reliability, and discriminatory power. The results indicate that the TUV is a reliable assessment tool. We also conducted a detailed analysis of the students' understanding of the vector concepts evaluated in the test. The TUV is included in the Supplemental Material as a resource for other researchers studying vector learning, as well as instructors teaching the material.

Trends in computer applications in science assessment

NASA Astrophysics Data System (ADS)

Kumar, David D.; Helgeson, Stanley L.

1995-03-01

Seven computer applications to science assessment are reviewed. Conventional test administration includes record keeping, grading, and managing test banks. Multiple-choice testing involves forced selection of an answer from a menu, whereas constructed-response testing involves options for students to present their answers within a set standard deviation. Adaptive testing attempts to individualize the test to minimize the number of items and time needed to assess a student's knowledge. Figurai response testing assesses science proficiency in pictorial or graphic mode and requires the student to construct a mental image rather than selecting a response from a multiple choice menu. Simulations have been found useful for performance assessment on a large-scale basis in part because they make it possible to independently specify different aspects of a real experiment. An emerging approach to performance assessment is solution pathway analysis, which permits the analysis of the steps a student takes in solving a problem. Virtually all computer-based testing systems improve the quality and efficiency of record keeping and data analysis.

Climbing Bloom's taxonomy pyramid: Lessons from a graduate histology course.

PubMed

Zaidi, Nikki B; Hwang, Charles; Scott, Sara; Stallard, Stefanie; Purkiss, Joel; Hortsch, Michael

2017-09-01

Bloom's taxonomy was adopted to create a subject-specific scoring tool for histology multiple-choice questions (MCQs). This Bloom's Taxonomy Histology Tool (BTHT) was used to analyze teacher- and student-generated quiz and examination questions from a graduate level histology course. Multiple-choice questions using histological images were generally assigned a higher BTHT level than simple text questions. The type of microscopy technique (light or electron microscopy) used for these image-based questions did not result in any significant differences in their Bloom's taxonomy scores. The BTHT levels for teacher-generated MCQs correlated positively with higher discrimination indices and inversely with the percent of students answering these questions correctly (difficulty index), suggesting that higher-level Bloom's taxonomy questions differentiate well between higher- and lower-performing students. When examining BTHT scores for MCQs that were written by students in a Multiple-Choice Item Development Assignment (MCIDA) there was no significant correlation between these scores and the students' ability to answer teacher-generated MCQs. This suggests that the ability to answer histology MCQs relies on a different skill set than the aptitude to construct higher-level Bloom's taxonomy questions. However, students significantly improved their average BTHT scores from the midterm to the final MCIDA task, which indicates that practice, experience and feedback increased their MCQ writing proficiency. Anat Sci Educ 10: 456-464. © 2017 American Association of Anatomists. © 2017 American Association of Anatomists.

Assessing the Life Science Knowledge of Students and Teachers Represented by the K–8 National Science Standards

PubMed Central

Sadler, Philip M.; Coyle, Harold; Smith, Nancy Cook; Miller, Jaimie; Mintzes, Joel; Tanner, Kimberly; Murray, John

2013-01-01

We report on the development of an item test bank and associated instruments based on the National Research Council (NRC) K–8 life sciences content standards. Utilizing hundreds of studies in the science education research literature on student misconceptions, we constructed 476 unique multiple-choice items that measure the degree to which test takers hold either a misconception or an accepted scientific view. Tested nationally with 30,594 students, following their study of life science, and their 353 teachers, these items reveal a range of interesting results, particularly student difficulties in mastering the NRC standards. Teachers also answered test items and demonstrated a high level of subject matter knowledge reflecting the standards of the grade level at which they teach, but exhibiting few misconceptions of their own. In addition, teachers predicted the difficulty of each item for their students and which of the wrong answers would be the most popular. Teachers were found to generally overestimate their own students’ performance and to have a high level of awareness of the particular misconceptions that their students hold on the K–4 standards, but a low level of awareness of misconceptions related to the 5–8 standards. PMID:24006402

[A self administered survey to assess bullying in schools].

PubMed

Lecannelier, Felipe; Varela, Jorge; Rodríguez, Jorge; Hoffmann, Marianela; Flores, Fernanda; Ascanio, Lorena

2011-04-01

Bullying is common in schools and has negative consequences. It can be assessed using a self-reported instrument. To validate a Spanish self-reporting tool called "Survey of High School Bullying Abuse of Power" (MIAP). The instrument has 13 questions, of which 7 are multiple choice, rendering a total of 49 items. It was applied to 2.341 children of seventh and eighth grade attending private, subsidized and municipal schools in the city of Concepción, Chile. Expert judge analysis and estimated reliability using the Cronbach Alpha were used to validate the survey. The instrument obtained a Cronbach Alpha coefficient of 0.8892, classified as good. This analysis generated four scales that explained 30.9% of the variance. They were called "Witness Bullying" with 18 items, accounting for 11.4% of the variance, "Bullying Victim" with 12 items, accounting for 7.5% of the variance, "Bullying Perpetrator and Severe bullying Victim", with 10 items explaining 6.4% of the variance and "Aggressor Bullying" with 6 items accounting for 5.7% of the variance. The MIAP can recognize four basic factors that facilitate the analysis and understanding of bullying, with good levels of reliability and validity. The remaining questions also deliver valuable information.

Assessing the life science knowledge of students and teachers represented by the K-8 national science standards.

PubMed

Sadler, Philip M; Coyle, Harold; Smith, Nancy Cook; Miller, Jaimie; Mintzes, Joel; Tanner, Kimberly; Murray, John

2013-01-01

We report on the development of an item test bank and associated instruments based on the National Research Council (NRC) K-8 life sciences content standards. Utilizing hundreds of studies in the science education research literature on student misconceptions, we constructed 476 unique multiple-choice items that measure the degree to which test takers hold either a misconception or an accepted scientific view. Tested nationally with 30,594 students, following their study of life science, and their 353 teachers, these items reveal a range of interesting results, particularly student difficulties in mastering the NRC standards. Teachers also answered test items and demonstrated a high level of subject matter knowledge reflecting the standards of the grade level at which they teach, but exhibiting few misconceptions of their own. In addition, teachers predicted the difficulty of each item for their students and which of the wrong answers would be the most popular. Teachers were found to generally overestimate their own students' performance and to have a high level of awareness of the particular misconceptions that their students hold on the K-4 standards, but a low level of awareness of misconceptions related to the 5-8 standards.

Assessing learning in small sized physics courses

NASA Astrophysics Data System (ADS)

Ene, Emanuela; Ackerson, Bruce J.

2018-01-01

We describe the construction, validation, and testing of a concept inventory for an Introduction to Physics of Semiconductors course offered by the department of physics to undergraduate engineering students. By design, this inventory addresses both content knowledge and the ability to interpret content via different cognitive processes outlined in Bloom's revised taxonomy. The primary challenge comes from the low number of test takers. We describe the Rasch modeling analysis for this concept inventory, and the results of the calibration on a small sample size, with the intention of providing a useful blueprint to other instructors. Our study involved 101 students from Oklahoma State University and fourteen faculty teaching or doing research in the field of semiconductors at seven universities. The items were written in four-option multiple-choice format. It was possible to calibrate a 30-item unidimensional scale precisely enough to characterize the student population enrolled each semester and, therefore, to allow the tailoring of the learning activities of each class. We show that this scale can be employed as an item bank from which instructors could extract short testlets and where we can add new items fitting the existing calibration.

Preference index supported by motivation tests in Nile tilapia

PubMed Central

2017-01-01

The identification of animal preferences is assumed to provide better rearing environments for the animals in question. Preference tests focus on the frequency of approaches or the time an animal spends in proximity to each item of the investigated resource during a multiple-choice trial. Recently, a preference index (PI) was proposed to differentiate animal preferences from momentary responses (Sci Rep, 2016, 6:28328, DOI: 10.1038/srep28328). This index also quantifies the degree of preference for each item. Each choice response is also weighted, with the most recent responses weighted more heavily, but the index includes the entire bank of tests, and thus represents a history-based approach. In this study, we compared this PI to motivation tests, which consider how much effort is expended to access a resource. We performed choice tests over 7 consecutive days for 34 Nile tilapia fish that presented with different colored compartments in each test. We first detected the preferred and non-preferred colors of each fish using the PI and then tested their motivation to reach these compartments. We found that fish preferences varied individually, but the results were consistent with the motivation profiles, as individual fish were more motivated (the number of touches made on transparent, hinged doors that prevented access to the resource) to access their preferred items. On average, most of the 34 fish avoided the color yellow and showed less motivation to reach yellow and red colors. The fish also exhibited greater motivation to access blue and green colors (the most preferred colors). These results corroborate the PI as a reliable tool for the identification of animal preferences. We recommend this index to animal keepers and researchers to identify an animal’s preferred conditions. PMID:28426689

Preference index supported by motivation tests in Nile tilapia.

PubMed

Maia, Caroline Marques; Volpato, Gilson Luiz

2017-01-01

The identification of animal preferences is assumed to provide better rearing environments for the animals in question. Preference tests focus on the frequency of approaches or the time an animal spends in proximity to each item of the investigated resource during a multiple-choice trial. Recently, a preference index (PI) was proposed to differentiate animal preferences from momentary responses (Sci Rep, 2016, 6:28328, DOI: 10.1038/srep28328). This index also quantifies the degree of preference for each item. Each choice response is also weighted, with the most recent responses weighted more heavily, but the index includes the entire bank of tests, and thus represents a history-based approach. In this study, we compared this PI to motivation tests, which consider how much effort is expended to access a resource. We performed choice tests over 7 consecutive days for 34 Nile tilapia fish that presented with different colored compartments in each test. We first detected the preferred and non-preferred colors of each fish using the PI and then tested their motivation to reach these compartments. We found that fish preferences varied individually, but the results were consistent with the motivation profiles, as individual fish were more motivated (the number of touches made on transparent, hinged doors that prevented access to the resource) to access their preferred items. On average, most of the 34 fish avoided the color yellow and showed less motivation to reach yellow and red colors. The fish also exhibited greater motivation to access blue and green colors (the most preferred colors). These results corroborate the PI as a reliable tool for the identification of animal preferences. We recommend this index to animal keepers and researchers to identify an animal's preferred conditions.

Restrictive Food Intake As A Choice – A Paradigm for Study

PubMed Central

Steinglass, Joanna; Foerde, Karin; Kostro, Katrina; Shohamy, Daphna; Timothy Walsh, B.

2014-01-01

Objective: Inadequate intake and preference for low-calorie foods are salient behavioral features of Anorexia Nervosa (AN). The neurocognitive mechanisms underlying pathological food choice have not been characterized. This study aimed to develop a new paradigm for experimentally modeling maladaptive food choice in AN. Method: Individuals with AN (n=22) and healthy controls (HC, n=20) participated in a computer-based Food Choice Task, adapted for individuals with eating disorders. Participants first rated 43 food images (including high-fat and low-fat items) for Healthiness and Tastiness; an item rated neutral on both blocks was then selected as the Reference item. On each of 42 subsequent trials participants were asked to choose between the food item presented and the Reference item. Results: The AN group was less likely to choose high-fat foods relative to HC, as evidenced both in multilevel logistic regression (z=2.59, p=0.009) and ANOVA (F(1,39)=7.80, p=0.008) analyses. Health ratings influenced choice significantly more in AN relative to HC (z=2.7, p=0.006), and were more related to Taste among AN (χ2=4.10, p=0.04). Additionally, Taste ratings declined with duration of illness(r=−0.50, p=0.02). Conclusions: The Food Choice Task captures the preference for low-fat foods among individuals with AN. The findings suggest that the experience of tastiness changes over time and may contribute to perpetuation of illness. By providing an experimental quantitative measure of food restriction, this task opens the door to new experimental investigations into the cognitive, affective and neural factors contributing to maladaptive food choices characteristic of AN. PMID:25130380

Restrictive food intake as a choice--a paradigm for study.

PubMed

Steinglass, Joanna; Foerde, Karin; Kostro, Katrina; Shohamy, Daphna; Walsh, B Timothy

2015-01-01

Inadequate intake and preference for low-calorie foods are salient behavioral features of Anorexia Nervosa (AN). The neurocognitive mechanisms underlying pathological food choice have not been characterized. This study aimed to develop a new paradigm for experimentally modeling maladaptive food choice in AN. Individuals with AN (n = 22) and healthy controls (HC, n = 20) participated in a computer-based Food Choice Task, adapted for individuals with eating disorders. Participants first rated 43 food images (including high-fat and low-fat items) for Healthiness and Tastiness; an item rated neutral on both blocks was then selected as the Reference item. On each of 42 subsequent trials participants were asked to choose between the food item presented and the Reference item. The AN group was less likely to choose high-fat foods relative to HC, as evidenced both in multilevel logistic regression (z = 2.59, p = .009) and ANOVA (F(1,39) = 7.80, p = .008) analyses. Health ratings influenced choice significantly more in AN relative to HC (z = 2.7, p = .006), and were more related to Taste among AN (χ(2) = 4.10, p = .04). Additionally, taste ratings declined with duration of illness (r = -.50, p = .02). The Food Choice Task captures the preference for low-fat foods among individuals with AN. The findings suggest that the experience of tastiness changes over time and may contribute to perpetuation of illness. By providing an experimental quantitative measure of food restriction, this task opens the door to new experimental investigations into the cognitive, affective, and neural factors contributing to maladaptive food choices characteristic of AN. © 2014 Wiley Periodicals, Inc.

Food Choices of Minority and Low-Income Employees

PubMed Central

Levy, Douglas E.; Riis, Jason; Sonnenberg, Lillian M.; Barraclough, Susan J.; Thorndike, Anne N.

2012-01-01

Background Effective strategies are needed to address obesity, particularly among minority and low-income individuals. Purpose To test whether a two-phase point-of-purchase intervention improved food choices across racial, socioeconomic (job type) groups. Design A 9-month longitudinal study from 2009 to 2010 assessing person-level changes in purchases of healthy and unhealthy foods following sequentially introduced interventions. Data were analyzed in 2011. Setting/participants Participants were 4642 employees of a large hospital in Boston MA who were regular cafeteria patrons. Interventions The first intervention was a traffic light–style color-coded labeling system encouraging patrons to purchase healthy items (labeled green) and avoid unhealthy items (labeled red). The second intervention manipulated “choice architecture” by physically rearranging certain cafeteria items, making green-labeled items more accessible, red-labeled items less accessible. Main outcome measures Proportion of green- (or red-) labeled items purchased by an employee. Subanalyses tracked beverage purchases, including calories and price per beverage. Results Employees self-identified as white (73%), black (10%), Latino (7%), and Asian (10%). Compared to white employees, Latino and black employees purchased a higher proportion of red items at baseline (18%, 28%, and 33%, respectively, p<0.001) and a lower proportion of green (48%, 38%, and 33%, p<0.001). Labeling decreased all employees’ red item purchases (−11.2% [95% CI= −13.6%, −8.9%]) and increased green purchases (6.6% [95% CI=5.2%, 7.9%]). Red beverage purchases decreased most (−23.8% [95% CI= −28.1%, −19.6%]). The choice architecture intervention further decreased red purchases after the labeling. Intervention effects were similar across all race/ethnicity and job types (p>0.05 for interaction between race or job type and intervention). Mean calories per beverage decreased similarly over the study period for all racial groups and job types, with no increase in per-beverage spending. Conclusions Despite baseline differences in healthy food purchases, a simple color-coded labeling and choice architecture intervention improved food and beverage choices among employees from all racial and socioeconomic backgrounds. PMID:22898116

Food choices of minority and low-income employees: a cafeteria intervention.

PubMed

Levy, Douglas E; Riis, Jason; Sonnenberg, Lillian M; Barraclough, Susan J; Thorndike, Anne N

2012-09-01

Effective strategies are needed to address obesity, particularly among minority and low-income individuals. To test whether a two-phase point-of-purchase intervention improved food choices across racial, socioeconomic (job type) groups. A 9-month longitudinal study from 2009 to 2010 assessing person-level changes in purchases of healthy and unhealthy foods following sequentially introduced interventions. Data were analyzed in 2011. Participants were 4642 employees of a large hospital in Boston MA who were regular cafeteria patrons. The first intervention was a traffic light-style color-coded labeling system encouraging patrons to purchase healthy items (labeled green) and avoid unhealthy items (labeled red). The second intervention manipulated "choice architecture" by physically rearranging certain cafeteria items, making green-labeled items more accessible and red-labeled items less accessible. Proportion of green- (or red-) labeled items purchased by an employee. Subanalyses tracked beverage purchases, including calories and price per beverage. Employees self-identified as white (73%); black (10%); Latino (7%); and Asian (10%). Compared to white employees, Latino and black employees purchased a higher percentage of red items at baseline (18%, 28%, and 33%, respectively, p<0.001) and a lower percentage of green (48%, 38%, and 33%, p<0.001). Labeling decreased all employees' red item purchases (-11.2%, 95% CI= -13.6%, -8.9%) and increased green purchases (6.6%, 95% CI=5.2%, 7.9%). Red beverage purchases decreased most (-23.8%, 95% CI= -28.1%, -19.6%). The choice architecture intervention further decreased red purchases after the labeling. Intervention effects were similar across all race/ethnicity and job types (p>0.05 for interaction between race or job type and intervention). Mean calories per beverage decreased similarly over the study period for all racial groups and job types, with no increase in per-beverage spending. Despite baseline differences in healthy food purchases, a simple color-coded labeling and choice architecture intervention improved food and beverage choices among employees from all racial and socioeconomic backgrounds. Copyright © 2012 American Journal of Preventive Medicine. Published by Elsevier Inc. All rights reserved.

A 2-Phase Labeling and Choice Architecture Intervention to Improve Healthy Food and Beverage Choices

PubMed Central

Sonnenberg, Lillian; Riis, Jason; Barraclough, Susan; Levy, Douglas E.

2012-01-01

Objectives. We assessed whether a 2-phase labeling and choice architecture intervention would increase sales of healthy food and beverages in a large hospital cafeteria. Methods. Phase 1 was a 3-month color-coded labeling intervention (red = unhealthy, yellow = less healthy, green = healthy). Phase 2 added a 3-month choice architecture intervention that increased the visibility and convenience of some green items. We compared relative changes in 3-month sales from baseline to phase 1 and from phase 1 to phase 2. Results. At baseline (977 793 items, including 199 513 beverages), 24.9% of sales were red and 42.2% were green. Sales of red items decreased in both phases (P < .001), and green items increased in phase 1 (P < .001). The largest changes occurred among beverages. Red beverages decreased 16.5% during phase 1 (P < .001) and further decreased 11.4% in phase 2 (P < .001). Green beverages increased 9.6% in phase 1 (P < .001) and further increased 4.0% in phase 2 (P < .001). Bottled water increased 25.8% during phase 2 (P < .001) but did not increase at 2 on-site comparison cafeterias (P < .001). Conclusions. A color-coded labeling intervention improved sales of healthy items and was enhanced by a choice architecture intervention. PMID:22390518

Assessment of representational competence in kinematics

NASA Astrophysics Data System (ADS)

Klein, P.; Müller, A.; Kuhn, J.

2017-06-01

A two-tier instrument for representational competence in the field of kinematics (KiRC) is presented, designed for a standard (1st year) calculus-based introductory mechanics course. It comprises 11 multiple choice (MC) and 7 multiple true-false (MTF) questions involving multiple representational formats, such as graphs, pictures, and formal (mathematical) expressions (1st tier). Furthermore, students express their answer confidence for selected items, providing additional information (2nd tier). Measurement characteristics of KiRC were assessed in a validation sample (pre- and post-test, N =83 and N =46 , respectively), including usefulness for measuring learning gain. Validity is checked by interviews and by benchmarking KiRC against related measures. Values for item difficulty, discrimination, and consistency are in the desired ranges; in particular, a good reliability was obtained (KR 20 =0.86 ). Confidence intervals were computed and a replication study yielded values within the latter. For practical and research purposes, KiRC as a diagnostic tool goes beyond related extant instruments both for the representational formats (e.g., mathematical expressions) and for the scope of content covered (e.g., choice of coordinate systems). Together with the satisfactory psychometric properties it appears a versatile and reliable tool for assessing students' representational competency in kinematics (and of its potential change). Confidence judgments add further information to the diagnostic potential of the test, in particular for representational misconceptions. Moreover, we present an analytic result for the question—arising from guessing correction or educational considerations—of how the total effect size (Cohen's d ) varies upon combination of two test components with known individual effect sizes, and then discuss the results in the case of KiRC (MC and MTF combination). The introduced method of test combination analysis can be applied to any test comprising two components for the purpose of finding effect size ranges.

The effect of order of dwells on the first dwell gaze bias for eventually chosen items

PubMed Central

Onuma, Takuya; Penwannakul, Yuwadee; Fuchimoto, Jun

2017-01-01

The relationship between choice and eye movement has gained marked interest. The gaze bias effect, i.e., the tendency to look longer at items that are eventually chosen, has been shown to occur in the first dwell (initial cohesion of fixations for an item). In the two-alternative forced-choice (2AFC) paradigm, participants would look at one of the items first (defined as first look; FL), and they would then move and look at another item (second look; SL). This study investigated how the order in which the chosen items were looked at modulates the first dwell gaze bias effect. Participants were asked to assert their preferences and perceptual 2AFC decisions about human faces (Experiment 1) and daily consumer products (Experiment 2), while their eye movements were recorded. The results showed that the first dwell gaze bias was found only when the eventually chosen item was looked at after another one; the chosen item was looked at for longer as compared to the not-chosen item in the SL, but not in the FL. These results indicate that participants actively allocate more time to looking at a subsequently chosen item only after they perceive both items in the SL. Therefore, the selective encoding seems to occur in the early comparison stage of visual decision making, and not in the initial encoding stage. These findings provide insight into the relationship between choice and eye movement. PMID:28723947

Development and analysis of an instrument to assess student understanding of GOB chemistry knowledge relevant to clinical nursing practice.

PubMed

Brown, Corina E; Hyslop, Richard M; Barbera, Jack

2015-01-01

The General, Organic, and Biological Chemistry Knowledge Assessment (GOB-CKA) is a multiple-choice instrument designed to assess students' understanding of the chemistry topics deemed important to clinical nursing practice. This manuscript describes the development process of the individual items along with a psychometric evaluation of the final version of the items and instrument. In developing items for the GOB-CKA, essential topics were identified through a series of expert interviews (with practicing nurses, nurse educators, and GOB chemistry instructors) and confirmed through a national survey. Individual items were tested in qualitative studies with students from the target population for clarity and wording. Data from pilot and beta studies were used to evaluate each item and narrow the total item count to 45. A psychometric analysis performed on data from the 45-item final version was used to provide evidence of validity and reliability. The final version of the instrument has a Cronbach's alpha value of 0.76. Feedback from an expert panel provided evidence of face and content validity. Convergent validity was estimated by comparing the results from the GOB-CKA with the General-Organic-Biochemistry Exam (Form 2007) of the American Chemical Society. Instructors who wish to use the GOB-CKA for teaching and research may contact the corresponding author for a copy of the instrument. © 2014 Wiley Periodicals, Inc.

Factors that influence beverage choices at meal times. An application of the food choice kaleidoscope framework.

PubMed

Mueller Loose, S; Jaeger, S R

2012-12-01

Beverages are consumed at almost every meal occasion, but knowledge about the factors that influence beverage choice is less than for food choice. The aim of this research was to characterize and quantify factors that influence beverage choices at meal times. Insights into what beverages are chosen by whom, when and where can be helpful for manufacturers, dieticians/health care providers, and health policy makers. A descriptive framework - the food choice kaleidoscope (Jaeger et al., 2011) - was applied to self-reported 24h food recall data from a sample of New Zealand consumers. Participants (n=164) described 8356 meal occasions in terms of foods and beverages consumed, and the contextual characteristics of the occasion. Beverage choice was explored with random-parameter logit regressions to reveal influences linked to food items eaten, context factors and person factors. Thereby this study contributed to the food choice kaleidoscope research approach by expressing the degree of context dependency in the form of odds ratios and according significance levels. The exploration of co-occurrence of beverages with food items suggests that beverage-meal item combinations can be meal specific. Furthermore, this study integrates psychographic variables into the 'person' mirror of the food choice kaleidoscope. A measure of habit in beverage choice was obtained from the inter-participant correlation. Copyright © 2012 Elsevier Ltd. All rights reserved.

Parameter Estimation for Thurstone Choice Models

DOE Office of Scientific and Technical Information (OSTI.GOV)

Vojnovic, Milan; Yun, Seyoung

We consider the estimation accuracy of individual strength parameters of a Thurstone choice model when each input observation consists of a choice of one item from a set of two or more items (so called top-1 lists). This model accommodates the well-known choice models such as the Luce choice model for comparison sets of two or more items and the Bradley-Terry model for pair comparisons. We provide a tight characterization of the mean squared error of the maximum likelihood parameter estimator. We also provide similar characterizations for parameter estimators defined by a rank-breaking method, which amounts to deducing one ormore » more pair comparisons from a comparison of two or more items, assuming independence of these pair comparisons, and maximizing a likelihood function derived under these assumptions. We also consider a related binary classification problem where each individual parameter takes value from a set of two possible values and the goal is to correctly classify all items within a prescribed classification error. The results of this paper shed light on how the parameter estimation accuracy depends on given Thurstone choice model and the structure of comparison sets. In particular, we found that for unbiased input comparison sets of a given cardinality, when in expectation each comparison set of given cardinality occurs the same number of times, for a broad class of Thurstone choice models, the mean squared error decreases with the cardinality of comparison sets, but only marginally according to a diminishing returns relation. On the other hand, we found that there exist Thurstone choice models for which the mean squared error of the maximum likelihood parameter estimator can decrease much faster with the cardinality of comparison sets. We report empirical evaluation of some claims and key parameters revealed by theory using both synthetic and real-world input data from some popular sport competitions and online labor platforms.« less

The establisment of an achievement test for determination of primary teachers’ knowledge level of earthquake

DOE Office of Scientific and Technical Information (OSTI.GOV)

Aydin, Süleyman, E-mail: yupul@hotmail.com; Haşiloğlu, M. Akif, E-mail: mehmet.hasiloglu@hotmail.com; Kunduraci, Ayşe, E-mail: ayse-kndrc@hotmail.com

In this study it was aimed to improve an academic achievement test to establish the students’ knowledge about the earthquake and the ways of protection from earthquakes. In the method of this study, the steps that Webb (1994) was created to improve an academic achievement test for a unit were followed. In the developmental process of multiple choice test having 25 questions, was prepared to measure the pre-service teachers’ knowledge levels about the earthquake and the ways of protection from earthquakes. The multiple choice test was presented to view of six academics (one of them was from geographic field andmore » five of them were science educator) and two expert teachers in science Prepared test was applied to 93 pre-service teachers studying in elementary education department in 2014-2015 academic years. As a result of validity and reliability of the study, the test was composed of 20 items. As a result of these applications, Pearson Moments Multiplication half-reliability coefficient was found to be 0.94. When this value is adjusted according to Spearman Brown reliability coefficient the reliability coefficient was set at 0.97.« less

Food Choice Architecture: An Intervention in a Secondary School and its Impact on Students' Plant-based Food Choices.

PubMed

Ensaff, Hannah; Homer, Matt; Sahota, Pinki; Braybrook, Debbie; Coan, Susan; McLeod, Helen

2015-06-02

With growing evidence for the positive health outcomes associated with a plant-based diet, the study's purpose was to examine the potential of shifting adolescents' food choices towards plant-based foods. Using a real world setting of a school canteen, a set of small changes to the choice architecture was designed and deployed in a secondary school in Yorkshire, England. Focussing on designated food items (whole fruit, fruit salad, vegetarian daily specials, and sandwiches containing salad) the changes were implemented for six weeks. Data collected on students' food choice (218,796 transactions) enabled students' (980 students) selections to be examined. Students' food choice was compared for three periods: baseline (29 weeks); intervention (six weeks); and post-intervention (three weeks). Selection of designated food items significantly increased during the intervention and post-intervention periods, compared to baseline (baseline, 1.4%; intervention 3.0%; post-intervention, 2.2%) χ(2)(2) = 68.1, p < 0.001. Logistic regression modelling also revealed the independent effect of the intervention, with students 2.5 times as likely (p < 0.001) to select the designated food items during the intervention period, compared to baseline. The study's results point to the influence of choice architecture within secondary school settings, and its potential role in improving adolescents' daily food choices.

Applying Item Response Theory methods to design a learning progression-based science assessment

NASA Astrophysics Data System (ADS)

Chen, Jing

Learning progressions are used to describe how students' understanding of a topic progresses over time and to classify the progress of students into steps or levels. This study applies Item Response Theory (IRT) based methods to investigate how to design learning progression-based science assessments. The research questions of this study are: (1) how to use items in different formats to classify students into levels on the learning progression, (2) how to design a test to give good information about students' progress through the learning progression of a particular construct and (3) what characteristics of test items support their use for assessing students' levels. Data used for this study were collected from 1500 elementary and secondary school students during 2009--2010. The written assessment was developed in several formats such as the Constructed Response (CR) items, Ordered Multiple Choice (OMC) and Multiple True or False (MTF) items. The followings are the main findings from this study. The OMC, MTF and CR items might measure different components of the construct. A single construct explained most of the variance in students' performances. However, additional dimensions in terms of item format can explain certain amount of the variance in student performance. So additional dimensions need to be considered when we want to capture the differences in students' performances on different types of items targeting the understanding of the same underlying progression. Items in each item format need to be improved in certain ways to classify students more accurately into the learning progression levels. This study establishes some general steps that can be followed to design other learning progression-based tests as well. For example, first, the boundaries between levels on the IRT scale can be defined by using the means of the item thresholds across a set of good items. Second, items in multiple formats can be selected to achieve the information criterion at all the defined boundaries. This ensures the accuracy of the classification. Third, when item threshold parameters vary a bit, the scoring rubrics and the items need to be reviewed to make the threshold parameters similar across items. This is because one important design criterion of the learning progression-based items is that ideally, a student should be at the same level across items, which means that the item threshold parameters (d1, d 2 and d3) should be similar across items. To design a learning progression-based science assessment, we need to understand whether the assessment measures a single construct or several constructs and how items are associated with the constructs being measured. Results from dimension analyses indicate that items of different carbon transforming processes measure different aspects of the carbon cycle construct. However, items of different practices assess the same construct. In general, there are high correlations among different processes or practices. It is not clear whether the strong correlations are due to the inherent links among these process/practice dimensions or due to the fact that the student sample does not show much variation in these process/practice dimensions. Future data are needed to examine the dimensionalities in terms of process/practice in detail. Finally, based on item characteristics analysis, recommendations are made to write more discriminative CR items and better OMC, MTF options. Item writers can follow these recommendations to write better learning progression-based items.

Further examination of factors that influence preference for positive versus negative reinforcement.

PubMed

Kodak, Tiffany; Lerman, Dorothea C; Volkert, Valerie M; Trosclair, Nicole

2007-01-01

Factors that influence choice between qualitatively different reinforcers (e.g., a food item or a break from work) are important to consider when arranging treatments for problem behavior. Previous findings indicate that children who engage in problem behavior maintained by escape from demands may choose a food item over the functional reinforcer during treatment (DeLeon, Neidert, Anders, & Rodriguez-Catter, 2001; Lalli et al., 1999). However, a number of variables may influence choice between concurrently available forms of reinforcement. An analogue for treatment situations in which positive reinforcement for compliance is in direct competition with negative reinforcement for problem behavior was used in the current study to evaluate several variables that may influence choice. Participants were 5 children who had been diagnosed with developmental disabilities and who engaged in problem behavior maintained by escape from demands. In the first phase, the effects of task preference and schedule of reinforcement on choice between a 30-s break and a high-preference food item were evaluated. The food item was preferred over the break, regardless of the preference level of the task or the reinforcement schedule, for all but 1 participant. In the second phase, the quality of the break was manipulated by combining escape with toys, attention, or both. Only 1 participant showed preference for the enriched break. In the third phase, choice of a medium- or low-preference food item versus the enriched break was evaluated. Three of 4 participants showed preference for the break over the less preferred food item. Results extend previous research by identifying some of the conditions under which individuals who engage in escape-maintained behavior will prefer a food reinforcer over the functional one.

Further Examination of Factors that Influence Preference for Positive Versus Negative Reinforcement

PubMed Central

Kodak, Tiffany; Lerman, Dorothea C; Volkert, Valerie M; Trosclair, Nicole

2007-01-01

Factors that influence choice between qualitatively different reinforcers (e.g., a food item or a break from work) are important to consider when arranging treatments for problem behavior. Previous findings indicate that children who engage in problem behavior maintained by escape from demands may choose a food item over the functional reinforcer during treatment (DeLeon, Neidert, Anders, & Rodriguez-Catter, 2001; Lalli et al., 1999). However, a number of variables may influence choice between concurrently available forms of reinforcement. An analogue for treatment situations in which positive reinforcement for compliance is in direct competition with negative reinforcement for problem behavior was used in the current study to evaluate several variables that may influence choice. Participants were 5 children who had been diagnosed with developmental disabilities and who engaged in problem behavior maintained by escape from demands. In the first phase, the effects of task preference and schedule of reinforcement on choice between a 30-s break and a high-preference food item were evaluated. The food item was preferred over the break, regardless of the preference level of the task or the reinforcement schedule, for all but 1 participant. In the second phase, the quality of the break was manipulated by combining escape with toys, attention, or both. Only 1 participant showed preference for the enriched break. In the third phase, choice of a medium- or low-preference food item versus the enriched break was evaluated. Three of 4 participants showed preference for the break over the less preferred food item. Results extend previous research by identifying some of the conditions under which individuals who engage in escape-maintained behavior will prefer a food reinforcer over the functional one. PMID:17471792

Simultaneous modeling of visual saliency and value computation improves predictions of economic choice.

PubMed

Towal, R Blythe; Mormann, Milica; Koch, Christof

2013-10-01

Many decisions we make require visually identifying and evaluating numerous alternatives quickly. These usually vary in reward, or value, and in low-level visual properties, such as saliency. Both saliency and value influence the final decision. In particular, saliency affects fixation locations and durations, which are predictive of choices. However, it is unknown how saliency propagates to the final decision. Moreover, the relative influence of saliency and value is unclear. Here we address these questions with an integrated model that combines a perceptual decision process about where and when to look with an economic decision process about what to choose. The perceptual decision process is modeled as a drift-diffusion model (DDM) process for each alternative. Using psychophysical data from a multiple-alternative, forced-choice task, in which subjects have to pick one food item from a crowded display via eye movements, we test four models where each DDM process is driven by (i) saliency or (ii) value alone or (iii) an additive or (iv) a multiplicative combination of both. We find that models including both saliency and value weighted in a one-third to two-thirds ratio (saliency-to-value) significantly outperform models based on either quantity alone. These eye fixation patterns modulate an economic decision process, also described as a DDM process driven by value. Our combined model quantitatively explains fixation patterns and choices with similar or better accuracy than previous models, suggesting that visual saliency has a smaller, but significant, influence than value and that saliency affects choices indirectly through perceptual decisions that modulate economic decisions.

Simultaneous modeling of visual saliency and value computation improves predictions of economic choice

PubMed Central

Towal, R. Blythe; Mormann, Milica; Koch, Christof

2013-01-01

Many decisions we make require visually identifying and evaluating numerous alternatives quickly. These usually vary in reward, or value, and in low-level visual properties, such as saliency. Both saliency and value influence the final decision. In particular, saliency affects fixation locations and durations, which are predictive of choices. However, it is unknown how saliency propagates to the final decision. Moreover, the relative influence of saliency and value is unclear. Here we address these questions with an integrated model that combines a perceptual decision process about where and when to look with an economic decision process about what to choose. The perceptual decision process is modeled as a drift–diffusion model (DDM) process for each alternative. Using psychophysical data from a multiple-alternative, forced-choice task, in which subjects have to pick one food item from a crowded display via eye movements, we test four models where each DDM process is driven by (i) saliency or (ii) value alone or (iii) an additive or (iv) a multiplicative combination of both. We find that models including both saliency and value weighted in a one-third to two-thirds ratio (saliency-to-value) significantly outperform models based on either quantity alone. These eye fixation patterns modulate an economic decision process, also described as a DDM process driven by value. Our combined model quantitatively explains fixation patterns and choices with similar or better accuracy than previous models, suggesting that visual saliency has a smaller, but significant, influence than value and that saliency affects choices indirectly through perceptual decisions that modulate economic decisions. PMID:24019496

Dual processing theory and experts' reasoning: exploring thinking on national multiple-choice questions.

PubMed

Durning, Steven J; Dong, Ting; Artino, Anthony R; van der Vleuten, Cees; Holmboe, Eric; Schuwirth, Lambert

2015-08-01

An ongoing debate exists in the medical education literature regarding the potential benefits of pattern recognition (non-analytic reasoning), actively comparing and contrasting diagnostic options (analytic reasoning) or using a combination approach. Studies have not, however, explicitly explored faculty's thought processes while tackling clinical problems through the lens of dual process theory to inform this debate. Further, these thought processes have not been studied in relation to the difficulty of the task or other potential mediating influences such as personal factors and fatigue, which could also be influenced by personal factors such as sleep deprivation. We therefore sought to determine which reasoning process(es) were used with answering clinically oriented multiple-choice questions (MCQs) and if these processes differed based on the dual process theory characteristics: accuracy, reading time and answering time as well as psychometrically determined item difficulty and sleep deprivation. We performed a think-aloud procedure to explore faculty's thought processes while taking these MCQs, coding think-aloud data based on reasoning process (analytic, nonanalytic, guessing or combination of processes) as well as word count, number of stated concepts, reading time, answering time, and accuracy. We also included questions regarding amount of work in the recent past. We then conducted statistical analyses to examine the associations between these measures such as correlations between frequencies of reasoning processes and item accuracy and difficulty. We also observed the total frequencies of different reasoning processes in the situations of getting answers correctly and incorrectly. Regardless of whether the questions were classified as 'hard' or 'easy', non-analytical reasoning led to the correct answer more often than to an incorrect answer. Significant correlations were found between self-reported recent number of hours worked with think-aloud word count and number of concepts used in the reasoning but not item accuracy. When all MCQs were included, 19 % of the variance of correctness could be explained by the frequency of expression of these three think-aloud processes (analytic, nonanalytic, or combined). We found evidence to support the notion that the difficulty of an item in a test is not a systematic feature of the item itself but is always a result of the interaction between the item and the candidate. Use of analytic reasoning did not appear to improve accuracy. Our data suggest that individuals do not apply either System 1 or System 2 but instead fall along a continuum with some individuals falling at one end of the spectrum.

Evaluating The Influence of Postsession Reinforcement on Choice of Reinforcers

PubMed Central

Kodak, Tiffany; Lerman, Dorothea C; Call, Nathan

2007-01-01

Factors that influence reinforcer choice have been examined in a number of applied studies (e.g., Neef, Mace, Shea, & Shade, 1992; Shore, Iwata, DeLeon, Kahng, & Smith, 1997; Tustin, 1994). However, no applied studies have evaluated the effects of postsession reinforcement on choice between concurrently available reinforcers, even though basic findings indicate that this is an important factor to consider (Hursh, 1978; Zeiler, 1999). In this bridge investigation, we evaluated the influence of postsession reinforcement on choice of two food items when task responding was reinforced on progressive-ratio schedules. Participants were 3 children who had been diagnosed with developmental disabilities. Results indicated that response allocation shifted from one food item to the other food item under thinner schedules of reinforcement when no postsession reinforcement was provided. These findings suggest that the efficacy of instructional programs or treatments for problem behavior may be improved by restricting reinforcers outside treatment sessions. PMID:17970264

Evaluating the influence of postsession reinforcement on choice of reinforcers.

PubMed

Kodak, Tiffany; Lerman, Dorothea C; Call, Nathan

2007-01-01

Factors that influence reinforcer choice have been examined in a number of applied studies (e.g., Neef, Mace, Shea, & Shade, 1992; Shore, Iwata, DeLeon, Kahng, & Smith, 1997; Tustin, 1994). However, no applied studies have evaluated the effects of postsession reinforcement on choice between concurrently available reinforcers, even though basic findings indicate that this is an important factor to consider (Hursh, 1978; Zeiler, 1999). In this bridge investigation, we evaluated the influence of postsession reinforcement on choice of two food items when task responding was reinforced on progressive-ratio schedules. Participants were 3 children who had been diagnosed with developmental disabilities. Results indicated that response allocation shifted from one food item to the other food item under thinner schedules of reinforcement when no postsession reinforcement was provided. These findings suggest that the efficacy of instructional programs or treatments for problem behavior may be improved by restricting reinforcers outside treatment sessions.

Item Response Models for Examinee-Selected Items

ERIC Educational Resources Information Center

Wang, Wen-Chung; Jin, Kuan-Yu; Qiu, Xue-Lan; Wang, Lei

2012-01-01

In some tests, examinees are required to choose a fixed number of items from a set of given items to answer. This practice creates a challenge to standard item response models, because more capable examinees may have an advantage by making wiser choices. In this study, we developed a new class of item response models to account for the choice…

Product variety in Australian snacks and drinks: how can the consumer make a healthy choice?

PubMed

Walker, Karen Z; Woods, Julie L; Rickard, Cassie A; Wong, Carrie K

2008-10-01

To estimate the proportion of snack food and beverage choices available to an Australian consumer. A survey of product Nutrition Information Panels (NIP) and product labels on snack foods and beverages offered for sale. Data on nutrient content were compared with criteria from different nutrient profile systems to estimate the proportion of items conforming to a choice. A large supermarket in metropolitan Melbourne, Australia. A consumer could choose from 1,070 different snack foods and 863 different drinks. Flavour variety was more common in snacks (maximum thirteen per product) while variation in container size was more common for drinks (up to ten per product). Recommended serving size for snacks varied greatly (1822 % of snack foods presented for sale could be deemed by multiple criteria. Similarly, only 14 healthy healthier' snack foods and beverages, e.g. by reformulation of many products by the food industry and their presentation in smaller, standardised portion-size packaging.

Effect of a promotional campaign on heart-healthy menu choices in community restaurants.

PubMed

Fitzgerald, Catherine M; Kannan, Srimathi; Sheldon, Sharon; Eagle, Kim Allen

2004-03-01

The research question examined in this study was: Does a promotional campaign impact the sales of heart-healthy menu items at community restaurants? The 8-week promotional campaign used professionally developed advertisements in daily and monthly print publications and posters and table tents in local restaurants. Nine restaurants tracked the sales of selected heart-healthy menu items and comparable menu items sold before and after a promotional campaign. The percentage of heart-healthy items sold after the campaign showed a trend toward a slight increase in heart-healthy menu item selections, although it was not statistically significant. This study and others indicate that dietetics professionals must continue to develop strategies to promote heart-healthy food choices in community restaurants.

Model Choice and Sample Size in Item Response Theory Analysis of Aphasia Tests

ERIC Educational Resources Information Center

Hula, William D.; Fergadiotis, Gerasimos; Martin, Nadine

2012-01-01

Purpose: The purpose of this study was to identify the most appropriate item response theory (IRT) measurement model for aphasia tests requiring 2-choice responses and to determine whether small samples are adequate for estimating such models. Method: Pyramids and Palm Trees (Howard & Patterson, 1992) test data that had been collected from…

The development and validation of a test of science critical thinking for fifth graders.

PubMed

Mapeala, Ruslan; Siew, Nyet Moi

2015-01-01

The paper described the development and validation of the Test of Science Critical Thinking (TSCT) to measure the three critical thinking skill constructs: comparing and contrasting, sequencing, and identifying cause and effect. The initial TSCT consisted of 55 multiple choice test items, each of which required participants to select a correct response and a correct choice of critical thinking used for their response. Data were obtained from a purposive sampling of 30 fifth graders in a pilot study carried out in a primary school in Sabah, Malaysia. Students underwent the sessions of teaching and learning activities for 9 weeks using the Thinking Maps-aided Problem-Based Learning Module before they answered the TSCT test. Analyses were conducted to check on difficulty index (p) and discrimination index (d), internal consistency reliability, content validity, and face validity. Analysis of the test-retest reliability data was conducted separately for a group of fifth graders with similar ability. Findings of the pilot study showed that out of initial 55 administered items, only 30 items with relatively good difficulty index (p) ranged from 0.40 to 0.60 and with good discrimination index (d) ranged within 0.20-1.00 were selected. The Kuder-Richardson reliability value was found to be appropriate and relatively high with 0.70, 0.73 and 0.92 for identifying cause and effect, sequencing, and comparing and contrasting respectively. The content validity index obtained from three expert judgments equalled or exceeded 0.95. In addition, test-retest reliability showed good, statistically significant correlations ([Formula: see text]). From the above results, the selected 30-item TSCT was found to have sufficient reliability and validity and would therefore represent a useful tool for measuring critical thinking ability among fifth graders in primary science.

Lower-fat menu items in restaurants satisfy customers.

PubMed

Fitzpatrick, M P; Chapman, G E; Barr, S I

1997-05-01

To evaluate a restaurant-based nutrition program by measuring customer satisfaction with lower-fat menu items and assessing patrons' reactions to the program. Questionnaires to assess satisfaction with menu items were administered to patrons in eight of the nine restaurants that volunteered to participate in the nutrition program. One patron from each participating restaurant was randomly selected for a semistructured interview about nutrition programming in restaurants. Persons dining in eight participating restaurants over a 1-week period (n = 686). Independent samples t tests were used to compare respondents' satisfaction with lower-fat and regular menu items. Two-way analysis of variance tests were completed using overall satisfaction as the dependent variable and menu-item classification (ie, lower fat or regular) and one of eight other menu item and respondent characteristics as independent variables. Qualitative methods were used to analyze interview transcripts. Of 1,127 menu items rated for satisfaction, 205 were lower fat, 878 were regular, and 44 were of unknown classification. Customers were significantly more satisfied with lower-fat than with regular menu items (P < .001). Overall satisfaction did not vary by any of the other independent variables. Interview results indicate the importance of restaurant during as an indulgent experience. High satisfaction with lower-fat menu items suggests that customers will support restaurant providing such choices. Dietitians can use these findings to encourage restaurateurs to include lower-fat choices on their menus, and to assure clients that their expectations of being indulged are not incompatible with these choices.

Interactive anatomical and surgical live stream lectures improve students' academic performance in applied clinical anatomy.

PubMed

Shiozawa, Thomas; Butz, Benjamin; Herlan, Stephan; Kramer, Andreas; Hirt, Bernhard

2017-01-01

Tuebingen's Sectio Chirurgica (TSC) is an innovative, interactive, multimedia, and transdisciplinary teaching method designed to complement dissection courses. The Tuebingen's Sectio Chirurgica (TSC) allows clinical anatomy to be taught via interactive live stream surgeries moderated by an anatomist. This method aims to provide an application-oriented approach to teaching anatomy that offers students a deeper learning experience. A cohort study was devised to determine whether students who participated in the TSC were better able to solve clinical application questions than students who did not participate. A total of 365 students participated in the dissection course during the winter term of the 2012/2013 academic year. The final examination contained 40 standard multiple-choice (S-MC) and 20 clinically-applied multiple-choice (CA-MC) items. The CA-MC items referred to clinical cases but could be answered solely using anatomical knowledge. Students who regularly participated in the TSC answered the CA-MC questions significantly better than the control group (75% and 65%, respectively; P < 0.05, Mann-Whitney U test). The groups exhibited no differences on the S-MC questions (85% and 82.5%, respectively; P > 0.05). The CA-MC questions had a slightly higher level of difficulty than the S-MC questions (0.725 and 0.801, respectively; P = 0.083). The discriminatory power of the items was comparable (S-MC median Pearson correlations: 0.321; CA-MC: 0.283). The TSC successfully teaches the clinical application of anatomical knowledge. Students who attended the TSC in addition to the dissection course were able to answer CA-MC questions significantly better than students who did not attend the TSC. Thus, attending the TSC in addition to the dissection course supported students' clinical learning goals. Anat Sci Educ 10: 46-52. © 2016 American Association of Anatomists. © 2016 American Association of Anatomists.

The Geoscience Concept Test: A New Assessment Tool Based on Student Misconceptions

NASA Astrophysics Data System (ADS)

Libarkin, J.; Anderson, S. W.; Boone, W. J.; Beilfuss, M.; Dahl, J.

2002-12-01

We developed and began pilot testing of an earth science assessment tool called the geoscience concept test (GCT). The GCT uses student misconceptions as distractors in a 30 item multiple-choice instrument. Student misconceptions were first assessed through the analysis of nearly 300 questionnaires administered in introductory geology courses at three institutions. Results from the questionnaires guided the development of an interview protocol that was used by four interviewers at four different institutions. Over 100 in-depth student interviews lasting from 0.5 to 1 hour probed topics related to the Earth's interior, geologic time, and the formation of Earth surface features such as mountains and volcanoes to better define misconceptions. Thematic content analysis of the interviews identified a number of widely held misconceptions, which were then incorporated into the GCT as multiple-choice distractors (wrong answers). For content validity, the initial GCT was reviewed by seven experts (3 geoscientists and 4 science educators) and revised before pilot testing. Approximately 100 introductory and non-science major college students from four institutions were assessed with the GCT pilot in the spring of 2002. Rasch model analysis of this data showed that students found the pilot test difficult, and the level of difficulty was consistent between the four institutions. Analysis of individual items showed that students had fewer misconceptions regarding the locations of earthquakes, and many misconceptions regarding the locations of volcanoes on the Earth's surface, suggesting a disconnect in their understanding of the role of plate tectonics in these phenomena. Analysis of the misfit statistic for each item showed that none of the questions misfit, although we dropped one question and modified the wording of another for clarity in the next round of piloting. A second round of piloting scheduled for the fall of 2002 includes nearly 3000 students from 34 institutions in 19 states.

Effects of an icon-based menu labelling initiative on consumer food choice.

PubMed

Kerins, Claire; Cunningham, Katie; Finucane, Francis M; Gibson, Irene; Jones, Jenni; Kelly, Colette

2017-01-01

The purpose of this study was to examine the impact of an icon-based menu labelling initiative on consumer buying behaviour. This quasi-experimental study recruited a convenience sample of eight food service establishments, all with at least one menu item meeting the heart healthy criteria. Data from sales of all menu items sold over an 8-week period were collated 4 weeks prior to and 4 weeks during the display of information icons related to healthy food choices on menus. The absolute change in menu item sales showed a non-significant trend towards an increase in healthier menu item selections. Furthermore, there was no association between the type of food service establishment and the percentage change in labelled menu item sales. The study did not find a statistically significant influence of the icon-based menu labels on consumer food choice. Given the limited amount of research that examines alternative menu labelling formats in real-world settings, more studies are necessary to confirm these results. Further research is needed to identify the optimal format, content and impact of menu labels on consumer behaviour.

The role of source memory in gambling task decision making.

PubMed

Whitney, Paul; Hinson, John M

2012-01-01

The role of memory in the Iowa Gambling Task (IGT) was tested in two experiments that dissociated item memory (memory for losses obtained) from source memory (the deck that produced a given loss). In Experiment 1, participants observed 75 choices that had been made by controls or patients in previous research, followed by memory tests, and then 25 active choices from the participant. In Experiment 2, participants made choices for 75 trials, performed the memory tests, and then made 25 final choices. The data show that item and source memory can diverge within the IGT, and that source memory makes a significant contribution to IGT performance.

Measurement of ethical food choice motives.

PubMed

Lindeman, M; Väänänen, M

2000-02-01

The two studies describe the development of three complementary scales to the Food Choice Questionnaire developed by Steptoe, Pollard & Wardle (1995). The new items address various ethical food choice motives and were derived from previous studies on vegetarianism and ethical food choice. The items were factor analysed in Study 1 (N=281) and the factor solution was confirmed in Study 2 (N=125), in which simple validity criteria were also included. Furthermore, test-retest reliability was assessed with a separate sample of subjects (N=36). The results indicated that the three new scales, Ecological Welfare (including subscales for Animal Welfare and Environment Protection), Political Values and Religion, are reliable and valid instruments for a brief screening of ethical food choice reasons. Copyright 2000 Academic Press.

Evidence that judgments of learning are causally related to study choice.

PubMed

Metcalfe, Janet; Finn, Bridgid

2008-02-01

Three experiments investigated whether study choice was directly related to judgments of learning (JOLs) by examining people's choices in cases in which JOLs were dissociated from recall. In Experiment 1, items were given either three repetitions or one repetition on Trial 1. Items given three repetitions received one on Trial 2, and those given one repetition received three on Trial 2-equating performance at the end of Trial 2, but yielding different immediate Trial 2 JOLs. Study choice followed the "illusory" JOLs. A delayed JOL condition in Experiment 2 did not show this JOL bias and neither did study choice. Finally, using a paradigm (Koriat & Bjork, 2005) in which similar JOLs are given to forward and backward associative pairs, despite much worse performance on the backward pairs, study choice again followed the mistaken JOLs. We concluded that JOLs-what people believe they know-directly influence people's study choices.

Developing a prelicensure exam for Canada: an international collaboration.

PubMed

Hobbins, Bonnie; Bradley, Pat

2013-01-01

Nine previously conducted studies indicate that Elsevier's HESI Exit Exam (E(2)) is 96.36%-99.16% accurate in predicting success on the National Council Licensure Examination for Registered Nurses. No similar standardized exam is available in Canada to predict Canadian Registered Nurse Examination (CRNE) success. Like the E(2), such an exam could be used to evaluate Canadian nursing students' preparedness for the CRNE, and scores on the numerous subject matter categories could be used to guide students' remediation efforts so that, ultimately, they are successful on their first attempt at taking the CRNE. The international collaboration between a HESI test construction expert and a nursing faculty member from Canada, who served as the content expert, resulted in the development of a 180-item, multiple-choice/single-answer prelicensure exam (PLE) that was pilot tested with Canadian nursing students (N = 175). Item analysis data obtained from this pilot testing were used to develop a 160-item PLE, which includes an additional 20 pilot test items. The estimated reliability of this exam is 0.91, and it exhibits congruent validity with the CRNE because the PLE test blueprint mimics the CRNE test blueprint. Copyright © 2013 Elsevier Inc. All rights reserved.

Food Choice Architecture: An Intervention in a Secondary School and its Impact on Students’ Plant-based Food Choices

PubMed Central

Ensaff, Hannah; Homer, Matt; Sahota, Pinki; Braybrook, Debbie; Coan, Susan; McLeod, Helen

2015-01-01

With growing evidence for the positive health outcomes associated with a plant-based diet, the study’s purpose was to examine the potential of shifting adolescents’ food choices towards plant-based foods. Using a real world setting of a school canteen, a set of small changes to the choice architecture was designed and deployed in a secondary school in Yorkshire, England. Focussing on designated food items (whole fruit, fruit salad, vegetarian daily specials, and sandwiches containing salad) the changes were implemented for six weeks. Data collected on students’ food choice (218,796 transactions) enabled students’ (980 students) selections to be examined. Students’ food choice was compared for three periods: baseline (29 weeks); intervention (six weeks); and post-intervention (three weeks). Selection of designated food items significantly increased during the intervention and post-intervention periods, compared to baseline (baseline, 1.4%; intervention 3.0%; post-intervention, 2.2%) χ2(2) = 68.1, p < 0.001. Logistic regression modelling also revealed the independent effect of the intervention, with students 2.5 times as likely (p < 0.001) to select the designated food items during the intervention period, compared to baseline. The study’s results point to the influence of choice architecture within secondary school settings, and its potential role in improving adolescents’ daily food choices. PMID:26043039

Critical success factors in awareness of and choice towards low vision rehabilitation.

PubMed

Fraser, Sarah A; Johnson, Aaron P; Wittich, Walter; Overbury, Olga

2015-01-01

The goal of the current study was to examine the critical factors indicative of an individual's choice to access low vision rehabilitation services. Seven hundred and forty-nine visually impaired individuals, from the Montreal Barriers Study, completed a structured interview and questionnaires (on visual function, coping, depression, satisfaction with life). Seventy-five factors from the interview and questionnaires were entered into a data-driven Classification and Regression Tree Analysis in order to determine the best predictors of awareness group: positive personal choice (I knew and I went), negative personal choice (I knew and did not go), and lack of information (Nobody told me, and I did not know). Having a response of moderate to no difficulty on item 6 (reading signs) of the Visual Function Index 14 (VF-14) indicated that the person had made a positive personal choice to seek rehabilitation, whereas reporting a great deal of difficulty on this item was associated with a lack of information on low vision rehabilitation. In addition to this factor, symptom duration of under nine years, moderate difficulty or less on item 5 (seeing steps or curbs) of the VF-14, and an indication of little difficulty or less on item 3 (reading large print) of the VF-14 further identified those who were more likely to have made a positive personal choice. Individuals in the lack of information group also reported greater difficulty on items 3 and 5 of the VF-14 and were more likely to be male. The duration-of-symptoms factor suggests that, even in the positive choice group, it may be best to offer rehabilitation services early. Being male and responding moderate difficulty or greater to the VF-14 questions about far, medium-distance and near situations involving vision was associated with individuals that lack information. Consequently, these individuals may need additional education about the benefits of low vision services in order to make a positive personal choice. © 2014 The Authors Ophthalmic & Physiological Optics © 2014 The College of Optometrists.

Controlling for Response Order Effects in Ranking Items Using Latent Choice Factor Modeling

ERIC Educational Resources Information Center

Vriens, Ingrid; Moors, Guy; Gelissen, John; Vermunt, Jeroen K.

2017-01-01

Measuring values in sociological research sometimes involves the use of ranking data. A disadvantage of a ranking assignment is that the order in which the items are presented might influence the choice preferences of respondents regardless of the content being measured. The standard procedure to rule out such effects is to randomize the order of…

Middle school students' reading comprehension of mathematical texts and algebraic equations

NASA Astrophysics Data System (ADS)

Duru, Adem; Koklu, Onder

2011-06-01

In this study, middle school students' abilities to translate mathematical texts into algebraic representations and vice versa were investigated. In addition, students' difficulties in making such translations and the potential sources for these difficulties were also explored. Both qualitative and quantitative methods were used to collect data for this study: questionnaire and clinical interviews. The questionnaire consisted of two general types of items: (1) selected-response (multiple-choice) items for which the respondent selects from multiple options and (2) open-ended items for which the respondent constructs a response. In order to further investigate the students' strategies while they were translating the given mathematical texts to algebraic equations and vice versa, five randomly chosen (n = 5) students were interviewed. Data were collected in the 2007-2008 school year from 185 middle-school students in five teachers' classrooms in three different schools in the city of Adıyaman, Turkey. After the analysis of data, it was found that students who participated in this study had difficulties in translating the mathematical texts into algebraic equations by using symbols. It was also observed that these students had difficulties in translating the symbolic representations into mathematical texts because of their weak reading comprehension. In addition, finding of this research revealed that students' difficulties in translating the given mathematical texts into symbolic representations or vice versa come from different sources.

When a Stone Tries to Climb up a Slope: The Interplay between Lexical and Perceptual Animacy in Referential Choices

PubMed Central

Vogels, Jorrig; Krahmer, Emiel; Maes, Alfons

2013-01-01

Several studies suggest that referential choices are influenced by animacy. On the one hand, animate referents are more likely to be mentioned as subjects than inanimate referents. On the other hand, animate referents are more frequently pronominalized than inanimate referents. These effects have been analyzed as effects of conceptual accessibility. In this paper, we raise the question whether these effects are driven only by lexical concepts, such that referents described by animate lexical items (e.g., “toddler”) are more accessible than referents described by inanimate lexical items (e.g., “shoe”), or can also be influenced by context-derived conceptualizations, such that referents that are perceived as animate in a particular context are more accessible than referents that are not. In two animation-retelling experiments, conducted in Dutch, we investigated the influence of lexical and perceptual animacy on the choice of referent and the choice of referring expression. If the effects of animacy are context-dependent, entities that are perceived as animate should yield more subject references and more pronouns than entities that are perceived as inanimate, irrespective of their lexical animacy. If the effects are tied to lexical concepts, entities described with animate lexical items should be mentioned as the subject and pronominalized more frequently than entities described with inanimate lexical items, irrespective of their perceptual animacy. The results show that while only lexical animacy appears to affect the choice of subject referent, perceptual animacy may overrule lexical animacy in the choice of referring expression. These findings suggest that referential choices can be influenced by conceptualizations based on the perceptual context. PMID:23554600

Mining for preparatory processes of transfer learning in a blended course

NASA Astrophysics Data System (ADS)

Ng, K.; Hartman, K.; Goodkin, N.; Wai Hoong Andy, K.

2017-12-01

585 undergraduate science students enrolled in a multidisciplinary environmental sustainability course. Each week, students were given the opportunity to read online materials, answer multiple choice and short answer questions, and attend a three-hour lecture. The online materials and questions were released one week prior to the lecture. After each week, we mined the student data logs exported from the course learning management system and used a model-based clustering algorithm to divide the class into six groups according to resource access patterns. The patterns were mostly based on the frequency with which a student accessed the items in the growing set of online resources and whether those resources were relevant to the upcoming exam. Each exam was self-contained—meaning the second exam did not reference content taught during the first half of the course. The exam items themselves were intentionally designed to provide a mix of recall, application, and transfer items. Recall items referenced facts and examples provided during the lectures and course materials. Application items asked students to solve problems using the methods shown during lecture. Transfer items asked students to use what they had learned to analyze new data sets and unfamiliar problems. We then used a log-likelihood analysis to determine if there were differences in item accuracy on the exams by resource pattern clusters. We found students who deviated from the majority of student access patterns by accessing prior material during the recess break before new material had been assigned and introduced performed significantly more accurately on the transfer items than the other cluster groups. This finding fits with the concept of Preparation for Future Learning (Bransford & Schwartz, 1999) which suggests learners can be strategic about their learning to prepare themselves to complete new tasks in the future. Our findings also suggest that using learning analytics to call attention activity during expected lulls in a course might be a productive method of predicting future performance. Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: A simple proposal with multiple implications. In A. Iran-Nejad & P. D. Pearson (Eds.), Review of research in education, 24 (pp. 61-101). Washington, DC: American Educational Research Association

The Impact Analysis of Psychological Reliability of Population Pilot Study for Selection of Particular Reliable Multi-Choice Item Test in Foreign Language Research Work

ERIC Educational Resources Information Center

Fazeli, Seyed Hossein

2010-01-01

The purpose of research described in the current study is the psychological reliability, its importance, application, and more to investigate on the impact analysis of psychological reliability of population pilot study for selection of particular reliable multi-choice item test in foreign language research work. The population for subject…

Using Likert-type and ipsative/forced choice items in sequence to generate a preference.

PubMed

Ried, L Douglas

2014-01-01

Collaboration and implementation of a minimum, standardized set of core global educational and professional competencies seems appropriate given the expanding international evolution of pharmacy practice. However, winnowing down hundreds of competencies from a plethora of local, national and international competency frameworks to select the most highly preferred to be included in the core set is a daunting task. The objective of this paper is to describe a combination of strategies used to ascertain the most highly preferred items among a large number of disparate items. In this case, the items were >100 educational and professional competencies that might be incorporated as the core components of new and existing competency frameworks. Panelists (n = 30) from the European Union (EU) and United States (USA) were chosen to reflect a variety of practice settings. Each panelist completed two electronic surveys. The first survey presented competencies in a Likert-type format and the second survey presented many of the same competencies in an ipsative/forced choice format. Item mean scores were calculated for each competency, the competencies were ranked, and non-parametric statistical tests were used to ascertain the consistency in the rankings achieved by the two strategies. This exploratory study presented over 100 competencies to the panelists in the beginning. The two methods provided similar results, as indicated by the significant correlation between the rankings (Spearman's rho = 0.30, P < 0.09). A two-step strategy using Likert-type and ipsative/forced choice formats in sequence, appears to be useful in a situation where a clear preference is required from among a large number of choices. The ipsative/forced choice format resulted in some differences in the competency preferences because the panelists could not rate them equally by design. While this strategy was used for the selection of professional educational competencies in this exploratory study, it is applicable in other situations where a smaller set of highly preferred items might be selected from a large list of choices in other areas of inquiry (e.g., patient reported outcomes). Copyright © 2014 Elsevier Inc. All rights reserved.

Neural mechanisms of cue-approach training

PubMed Central

Bakkour, Akram; Lewis-Peacock, Jarrod A.; Poldrack, Russell A.; Schonberg, Tom

2016-01-01

Biasing choices may prove a useful way to implement behavior change. Previous work has shown that a simple training task (the cue-approach task), which does not rely on external reinforcement, can robustly influence choice behavior by biasing choice toward items that were targeted during training. In the current study, we replicate previous behavioral findings and explore the neural mechanisms underlying the shift in preferences following cue-approach training. Given recent successes in the development and application of machine learning techniques to task-based fMRI data, which have advanced understanding of the neural substrates of cognition, we sought to leverage the power of these techniques to better understand neural changes during cue-approach training that subsequently led to a shift in choice behavior. Contrary to our expectations, we found that machine learning techniques applied to fMRI data during non-reinforced training were unsuccessful in elucidating the neural mechanism underlying the behavioral effect. However, univariate analyses during training revealed that the relationship between BOLD and choices for Go items increases as training progresses compared to choices of NoGo items primarily in lateral prefrontal cortical areas. This new imaging finding suggests that preferences are shifted via differential engagement of task control networks that interact with value networks during cue-approach training. PMID:27677231

Multidimensional Extension of Multiple Indicators Multiple Causes Models to Detect DIF

ERIC Educational Resources Information Center

Lee, Soo; Bulut, Okan; Suh, Youngsuk

2017-01-01

A number of studies have found multiple indicators multiple causes (MIMIC) models to be an effective tool in detecting uniform differential item functioning (DIF) for individual items and item bundles. A recently developed MIMIC-interaction model is capable of detecting both uniform and nonuniform DIF in the unidimensional item response theory…

Statistically Comparing the Performance of Multiple Automated Raters across Multiple Items

ERIC Educational Resources Information Center

Kieftenbeld, Vincent; Boyer, Michelle

2017-01-01

Automated scoring systems are typically evaluated by comparing the performance of a single automated rater item-by-item to human raters. This presents a challenge when the performance of multiple raters needs to be compared across multiple items. Rankings could depend on specifics of the ranking procedure; observed differences could be due to…

Evaluation of Modified Essay Questions (MEQ) and Multiple Choice Questions (MCQ) as a tool for Assessing the Cognitive Skills of Undergraduate Medical Students

PubMed Central

Khan, Moeen-uz-Zafar; Aljarallah, Badr Muhammad

2011-01-01

Objectives: Developing and testing the cognitive skills and abstract thinking of undergraduate medical students are the main objectives of problem based learning. Modified Essay Questions (MEQ) and Multiple Choice Questions (MCQ) may both be designed to test these skills. The objectives of this study were to assess the effectiveness of both forms of questions in testing the different levels of the cognitive skills of undergraduate medical students and to detect any item writing flaws in the questions. Methods: A total of 50 MEQs and 50 MCQs were evaluated. These questions were chosen randomly from various examinations given to different batches of undergraduate medical students taking course MED 411–412 at the Department of Medicine, Qassim University from the years 2005 to 2009. The effectiveness of the questions was determined by two assessors and was defined by the question’s ability to measure higher cognitive skills, as determined by modified Bloom’s taxonomy, and its quality as determined by the presence of item writing flaws. ‘SPSS15’ and ‘Medcalc’ programs were used to tabulate and analyze the data. Results: The percentage of questions testing the level III (problem solving) cognitive skills of the students was 40% for MEQs and 60% for the MCQs; the remaining questions merely assessed the recall and comprehension. No significant difference was found between MEQ and MCQ in relation to the type of questions (recall; comprehension or problem solving x2 = 5.3, p = 0.07).The agreement between the two assessors was quite high in case of MCQ (kappa=0.609; SE 0.093; 95%CI 0.426 – 0.792) but lower in case of MEQ (kappa=0.195; SE 0.073; 95%CI 0.052 – 0.338). 16% of the MEQs and 12% of the MCQs had item writing flaws. Conclusion: A well constructed MCQ is superior to MEQ in testing the higher cognitive skills of undergraduate medical students in a problem based learning setup. Constructing an MEQ for assessing the cognitive skills of a student is not a simple task and is more frequently associated with item writing flaws. PMID:22489228

Informed choice: understanding knowledge in the context of screening uptake.

PubMed

Michie, Susan; Dormandy, Elizabeth; Marteau, Theresa M

2003-07-01

This study evaluates a scale measuring knowledge about a screening test and investigates the association between knowledge, uptake and attitudes towards screening. One thousand four hundred ninety-nine pregnant women completed the knowledge scale of the multidimensional measure of informed choice (MMIC). Three hundred forty-five of these women and 152 professionals providing antenatal care also rated the importance of the knowledge items. Item characteristic curves show that, with one exception, the knowledge items reflect a spread of difficulty and are able to discriminate between people. All items were seen as essential or helpful by both women and health professionals, with two items seen as particularly important and one as unimportant. There were some differences between health professionals, women with low risk results and women with high risk results. Knowledge was not associated with uptake, attitude, or the extent to which uptake was consistent with women's attitudes towards undergoing the test.

The development of a knowledge test of depression and its treatment for patients suffering from non-psychotic depression: a psychometric assessment

PubMed Central

Gabriel, Adel; Violato, Claudio

2009-01-01

Background To develop and psychometrically assess a multiple choice question (MCQ) instrument to test knowledge of depression and its treatments in patients suffering from depression. Methods A total of 63 depressed patients and twelve psychiatric experts participated. Based on empirical evidence from an extensive review, theoretical knowledge and in consultations with experts, 27-item MCQ knowledge of depression and its treatment test was constructed. Data collected from the psychiatry experts were used to assess evidence of content validity for the instrument. Results Cronbach's alpha of the instrument was 0.68, and there was an overall 87.8% agreement (items are highly relevant) between experts about the relevance of the MCQs to test patient knowledge on depression and its treatments. There was an overall satisfactory patients' performance on the MCQs with 78.7% correct answers. Results of an item analysis indicated that most items had adequate difficulties and discriminations. Conclusion There was adequate reliability and evidence for content and convergent validity for the instrument. Future research should employ a lager and more heterogeneous sample from both psychiatrist and community samples, than did the present study. Meanwhile, the present study has resulted in psychometrically tested instruments for measuring knowledge of depression and its treatment of depressed patients. PMID:19754944

The Effect of Response Format on the Psychometric Properties of the Narcissistic Personality Inventory: Consequences for Item Meaning and Factor Structure.

PubMed

Ackerman, Robert A; Donnellan, M Brent; Roberts, Brent W; Fraley, R Chris

2016-04-01

The Narcissistic Personality Inventory (NPI) is currently the most widely used measure of narcissism in social/personality psychology. It is also relatively unique because it uses a forced-choice response format. We investigate the consequences of changing the NPI's response format for item meaning and factor structure. Participants were randomly assigned to one of three conditions: 40 forced-choice items (n = 2,754), 80 single-stimulus dichotomous items (i.e., separate true/false responses for each item; n = 2,275), or 80 single-stimulus rating scale items (i.e., 5-point Likert-type response scales for each item; n = 2,156). Analyses suggested that the "narcissistic" and "nonnarcissistic" response options from the Entitlement and Superiority subscales refer to independent personality dimensions rather than high and low levels of the same attribute. In addition, factor analyses revealed that although the Leadership dimension was evident across formats, dimensions with entitlement and superiority were not as robust. Implications for continued use of the NPI are discussed. © The Author(s) 2015.

Changing value through cued approach: An automatic mechanism of behavior change

PubMed Central

Schonberg, Tom; Bakkour, Akram; Hover, Ashleigh M.; Mumford, Jeanette A.; Nagar, Lakshya; Perez, Jacob; Poldrack, Russell A.

2014-01-01

It is believed that choice behavior reveals the underlying value of goods. The subjective values of stimuli can be changed through reward-based learning mechanisms as well as by modifying the description of the decision problem, but it has yet to be shown that preferences can be manipulated by perturbing intrinsic values of individual items. Here we show that the value of food items can be modulated by the concurrent presentation of an irrelevant auditory cue to which subjects must make a simple motor response (i.e. cue-approach training). Follow-up tests show that the effects of this pairing on choice lasted at least two months after prolonged training. Eye-tracking during choice confirmed that cue-approach training increased attention to the cued items. Neuroimaging revealed the neural signature of a value change in the form of amplified preference-related activity in ventromedial prefrontal cortex. PMID:24609465

Dental Student Academic Integrity in U.S. Dental Schools: Current Status and Recommendations for Enhancement.

PubMed

Graham, Bruce S; Knight, G William; Graham, Linda

2016-01-01

Cheating incidents in 2006-07 led U.S. dental schools to heighten their efforts to enhance the environment of academic integrity in their institutions. The aims of this study were to document the measures being used by U.S. dental schools to discourage student cheating, determine the current incidence of reported cheating, and make recommendations for enhancing a culture of integrity in dental education. In late 2014-early 2015, an online survey was distributed to academic deans of all 61 accredited U.S. dental schools that had four classes of dental students enrolled; 50 (82%) responded. Among measures used, 98% of respondents reported having policy statements regarding student academic integrity, 92% had an Honor Code, 96% provided student orientation to integrity policies, and most used proctoring of final exams (91%) and tests (93%). Regarding disciplinary processes, 27% reported their faculty members only rarely reported suspected cheating (though required in 76% of the schools), and 40% disseminated anonymous results of disciplinary hearings. A smaller number of schools (n=36) responded to the question about student cheating than to other questions; those results suggested that reported cheating had increased almost threefold since 1998. The authors recommend that schools add cheating case scenarios to professional ethics curricula; disseminate outcomes of cheating enforcement actions; have students sign a statement attesting to compliance with academic integrity policies at every testing activity; add curricular content on correct writing techniques to avoid plagiarism; require faculty to distribute retired test items; acquire examination-authoring software programs to enable faculty to generate new multiple-choice items and different versions of the same multiple-choice tests; avoid take-home exams when assessing independent student knowledge; and utilize student assessment methods directly relevant to clinical practice.

Obstetric training in Emergency Medicine: a needs assessment.

PubMed

Janicki, Adam James; MacKuen, Courteney; Hauspurg, Alisse; Cohn, Jamieson

2016-01-01

Identification and management of obstetric emergencies is essential in emergency medicine (EM), but exposure to pregnant patients during EM residency training is frequently limited. To date, there is little data describing effective ways to teach residents this material. Current guidelines require completion of 2 weeks of obstetrics or 10 vaginal deliveries, but it is unclear whether this instills competency. We created a 15-item survey evaluating resident confidence and knowledge related to obstetric emergencies. To assess confidence, we asked residents about their exposure and comfort level regarding obstetric emergencies and eight common presentations and procedures. We assessed knowledge via multiple-choice questions addressing common obstetric presentations, pelvic ultrasound image, and cardiotocography interpretation. The survey was distributed to residency programs utilizing the Council of Emergency Medicine Residency Directors (CORD) listserv. The survey was completed by 212 residents, representing 55 of 204 (27%) programs belonging to CORD and 11.2% of 1,896 eligible residents. Fifty-six percent felt they had adequate exposure to obstetric emergencies. The overall comfort level was 2.99 (1-5 scale) and comfort levels of specific presentations and procedures ranged from 2.58 to 3.97; all increased moderately with postgraduate year (PGY) level. Mean overall percentage of items answered correctly on the multiple-choice questions was 58% with no statistical difference by PGY level. Performance on individual questions did not differ by PGY level. The identification and management of obstetric emergencies is the cornerstone of EM. We found preliminary evidence of a concerning lack of resident comfort regarding obstetric conditions and knowledge deficits on core obstetrics topics. EM residents may benefit from educational interventions to increase exposure to these topics.

Risk of error estimated from Palestine pharmacists' knowledge and certainty on the adverse effects and contraindications of active pharmaceutical ingredients and excipients.

PubMed

Shawahna, Ramzi; Al-Rjoub, Mohammed; Al-Horoub, Mohammed M; Al-Hroub, Wasif; Al-Rjoub, Bisan; Al-Nabi, Bashaaer Abd

2016-01-01

This study aimed to investigate community pharmacists' knowledge and certainty of adverse effects and contraindications of pharmaceutical products to estimate the risk of error. Factors influencing their knowledge and certainty were also investigated. The knowledge of community pharmacists was assessed in a cross-sectional design using a multiple-choice questions test on the adverse effects and contraindications of active pharmaceutical ingredients and excipients from May 2014 to March 2015. Self-rated certainty scores were also recorded for each question. Knowledge and certainty scores were combined to estimate the risk of error. Out of 315 subjects, 129 community pharmacists (41.0%) completed the 30 multiple-choice questions test on active ingredients and excipients. Knowledge on active ingredients was associated with the year of graduation and obtaining a licence to practice pharmacy. Knowledge on excipients was associated with the degree obtained. There was higher risk of error in items on excipients than those on ingredients (P<0.01). The knowledge of community pharmacists in Palestine was insufficient with high risk of errors. Knowledge of community pharmacists on the safety issues of active ingredients and excipients need to be improved.

Cognitive dissonance resolution is related to episodic memory.

PubMed

Salti, Moti; El Karoui, Imen; Maillet, Mathurin; Naccache, Lionel

2014-01-01

The notion that our past choices affect our future behavior is certainly one of the most influential concepts of social psychology since its first experimental report in the 50 s, and its initial theorization by Festinger within the "cognitive dissonance" framework. Using the free choice paradigm (FCP), it was shown that choosing between two similarly rated items made subjects reevaluate the chosen items as more attractive and the rejected items as less attractive. However, in 2010 a major work by Chen and Risen revealed a severe statistical flaw casting doubt on most previous studies. Izuma and colleagues (2010) supplemented the traditional FCP with original control conditions and concluded that the effect observed could not be solely attributed to this methodological flaw. In the present work we aimed at establishing the existence of genuine choice-induced preference change and characterizing this effect. To do so, we replicated Izuma et al.' study and added a new important control condition which was absent from the original study. Moreover, we added a memory test in order to measure the possible relation between episodic memory of choices and observed behavioral effects. In two experiments we provide experimental evidence supporting genuine choice-induced preference change obtained with FCP. We also contribute to the understanding of the phenomenon by showing that choice-induced preference change effects are strongly correlated with episodic memory.

Cognitive Dissonance Resolution Is Related to Episodic Memory

PubMed Central

Maillet, Mathurin; Naccache, Lionel

2014-01-01

The notion that our past choices affect our future behavior is certainly one of the most influential concepts of social psychology since its first experimental report in the 50 s, and its initial theorization by Festinger within the “cognitive dissonance” framework. Using the free choice paradigm (FCP), it was shown that choosing between two similarly rated items made subjects reevaluate the chosen items as more attractive and the rejected items as less attractive. However, in 2010 a major work by Chen and Risen revealed a severe statistical flaw casting doubt on most previous studies. Izuma and colleagues (2010) supplemented the traditional FCP with original control conditions and concluded that the effect observed could not be solely attributed to this methodological flaw. In the present work we aimed at establishing the existence of genuine choice-induced preference change and characterizing this effect. To do so, we replicated Izuma et al.’ study and added a new important control condition which was absent from the original study. Moreover, we added a memory test in order to measure the possible relation between episodic memory of choices and observed behavioral effects. In two experiments we provide experimental evidence supporting genuine choice-induced preference change obtained with FCP. We also contribute to the understanding of the phenomenon by showing that choice-induced preference change effects are strongly correlated with episodic memory. PMID:25264950

Using future thinking to reduce temporal discounting: Under what circumstances are the medial temporal lobes critical?

PubMed

Palombo, D J; Keane, M M; Verfaellie, M

2016-08-01

The capacity to envision the future plays an important role in many aspects of cognition, including our ability to make optimal, adaptive choices. Past work has shown that the medial temporal lobe (MTL) is necessary for decisions that draw on episodic future thinking. By contrast, little is known about the role of the MTL in decisions that draw on semantic future thinking. Accordingly, the present study investigated whether the MTL contributes to one form of decision making, namely intertemporal choice, when such decisions depend on semantic consideration of the future. In an intertemporal choice task, participants must select either a smaller amount of money that is available in the present or a larger amount of money that would be available at a future date. Amnesic individuals with MTL damage and healthy control participants performed such a task in which, prior to making a choice, they engaged in a semantic generation exercise, wherein they generated items that they would purchase with the future reward. In experiment 1, we found that, relative to a baseline condition involving standard intertemporal choice, healthy individuals were more inclined to select a larger, later reward over a smaller, present reward after engaging in semantic future thinking. By contrast, amnesic participants were paradoxically less inclined to wait for a future reward following semantic future thinking. This finding suggests that amnesics may have had difficulty "tagging" the generated item(s) as belonging to the future. Critically, experiment 2 showed that when the generated items were presented alongside the intertemporal choices, both controls and amnesic participants shifted to more patient choices. These findings suggest that the MTL is not needed for making optimal decisions that draw on semantic future thinking as long as scaffolding is provided to support accurate time tagging. Together, these findings stand to better clarify the role of the MTL in decision making. Published by Elsevier Ltd.

The Influence of Purchasing Context and Reversibility of Choice on Consumer Responses Toward Personalized Products and Standardized Products.

PubMed

Choi, Jieun; Lee, Doo-Hee; Taylor, Charles R

2016-04-01

Existing research on personalization has found that consumers generally prefer personalized products over standardized ones. This study argued that consumer preference for personalized products is dependent on purchasing context and reversibility of choice. Results of an experiment conducted in this study found that consumers preferred personalized products when purchasing an item for personal use but preferred standardized products when purchasing an item as a gift. However, the effects of purchasing context were negated when consumers were given the assurance that personalized products could be returned (reversibility of choice); when presented with reversibility of choice, consumers preferred personalized products over standardized products regardless of purchasing context. Theoretical and managerial implications of these results were discussed. © The Author(s) 2016.

Influencing food choices by training: Evidence for modulation of frontoparietal control signals

PubMed Central

Bakkour, Akram; Hover, Ashleigh M.; Mumford, Jeanette A.; Poldrack, Russell A.

2014-01-01

To overcome unhealthy behaviors, one must be able to make better choices. Changing food preferences is an important strategy in addressing the obesity epidemic and its accompanying public health risks. However, little is known about how food preferences can be effectively affected and what neural systems support such changes. In this study we investigated a novel extensive training paradigm where participants chose from specific pairs of palatable junk food items and were rewarded for choosing the items with lower subjective value over higher value ones. In a later probe phase, when choices were made for real consumption, participants chose the lower-valued item more often in the trained pairs compared to untrained pairs. We replicated the behavioral results in an independent sample of participants while they were scanned with fMRI. We found that as training progressed there was decreased recruitment of regions that have been previously associated with cognitive control, specifically left dorsolateral prefrontal cortex (dlPFC) and bilateral parietal cortices. Furthermore, we found that connectivity of the left dlPFC was greater with primary motor regions by the end of training for choices of lower-valued items that required exertion of self-control, suggesting a formation of a stronger stimulus-response association. These findings demonstrate that it is possible to influence food choices through training, and that this training is associated with a decreasing need for top-down frontoparietal control. The results suggest that training paradigms may be promising as the basis for interventions to influence real world food preferences. PMID:24116842

Influencing food choices by training: evidence for modulation of frontoparietal control signals.

PubMed

Schonberg, Tom; Bakkour, Akram; Hover, Ashleigh M; Mumford, Jeanette A; Poldrack, Russell A

2014-02-01

To overcome unhealthy behaviors, one must be able to make better choices. Changing food preferences is an important strategy in addressing the obesity epidemic and its accompanying public health risks. However, little is known about how food preferences can be effectively affected and what neural systems support such changes. In this study, we investigated a novel extensive training paradigm where participants chose from specific pairs of palatable junk food items and were rewarded for choosing the items with lower subjective value over higher value ones. In a later probe phase, when choices were made for real consumption, participants chose the lower-valued item more often in the trained pairs compared with untrained pairs. We replicated the behavioral results in an independent sample of participants while they were scanned with fMRI. We found that, as training progressed, there was decreased recruitment of regions that have been previously associated with cognitive control, specifically the left dorsolateral pFC and bilateral parietal cortices. Furthermore, we found that connectivity of the left dorsolateral pFC was greater with primary motor regions by the end of training for choices of lower-valued items that required exertion of self-control, suggesting a formation of a stronger stimulus-response association. These findings demonstrate that it is possible to influence food choices through training and that this training is associated with a decreasing need for top-down frontoparietal control. The results suggest that training paradigms may be promising as the basis for interventions to influence real-world food preferences.

Neural correlates of cognitive dissonance and choice-induced preference change

PubMed Central

Izuma, Keise; Matsumoto, Madoka; Murayama, Kou; Samejima, Kazuyuki; Sadato, Norihiro; Matsumoto, Kenji

2010-01-01

According to many modern economic theories, actions simply reflect an individual's preferences, whereas a psychological phenomenon called “cognitive dissonance” claims that actions can also create preference. Cognitive dissonance theory states that after making a difficult choice between two equally preferred items, the act of rejecting a favorite item induces an uncomfortable feeling (cognitive dissonance), which in turn motivates individuals to change their preferences to match their prior decision (i.e., reducing preference for rejected items). Recently, however, Chen and Risen [Chen K, Risen J (2010) J Pers Soc Psychol 99:573–594] pointed out a serious methodological problem, which casts a doubt on the very existence of this choice-induced preference change as studied over the past 50 y. Here, using a proper control condition and two measures of preferences (self-report and brain activity), we found that the mere act of making a choice can change self-report preference as well as its neural representation (i.e., striatum activity), thus providing strong evidence for choice-induced preference change. Furthermore, our data indicate that the anterior cingulate cortex and dorsolateral prefrontal cortex tracked the degree of cognitive dissonance on a trial-by-trial basis. Our findings provide important insights into the neural basis of how actions can alter an individual's preferences. PMID:21135218

Syntax for calculation of discounting indices from the monetary choice questionnaire and probability discounting questionnaire.

PubMed

Gray, Joshua C; Amlung, Michael T; Palmer, Abraham A; MacKillop, James

2016-09-01

The 27-item Monetary Choice Questionnaire (MCQ; Kirby, Petry, & Bickel, 1999) and 30-item Probability Discounting Questionnaire (PDQ; Madden, Petry, & Johnson, 2009) are widely used, validated measures of preferences for immediate versus delayed rewards and guaranteed versus risky rewards, respectively. The MCQ measures delayed discounting by asking individuals to choose between rewards available immediately and larger rewards available after a delay. The PDQ measures probability discounting by asking individuals to choose between guaranteed rewards and a chance at winning larger rewards. Numerous studies have implicated these measures in addiction and other health behaviors. Unlike typical self-report measures, the MCQ and PDQ generate inferred hyperbolic temporal and probability discounting functions by comparing choice preferences to arrays of functions to which the individual items are preconfigured. This article provides R and SPSS syntax for processing the MCQ and PDQ. Specifically, for the MCQ, the syntax generates k values, consistency of the inferred k, and immediate choice ratios; for the PDQ, the syntax generates h indices, consistency of the inferred h, and risky choice ratios. The syntax is intended to increase the accessibility of these measures, expedite the data processing, and reduce risk for error. © 2016 Society for the Experimental Analysis of Behavior.

Neural correlates of cognitive dissonance and choice-induced preference change.

PubMed

Izuma, Keise; Matsumoto, Madoka; Murayama, Kou; Samejima, Kazuyuki; Sadato, Norihiro; Matsumoto, Kenji

2010-12-21

According to many modern economic theories, actions simply reflect an individual's preferences, whereas a psychological phenomenon called "cognitive dissonance" claims that actions can also create preference. Cognitive dissonance theory states that after making a difficult choice between two equally preferred items, the act of rejecting a favorite item induces an uncomfortable feeling (cognitive dissonance), which in turn motivates individuals to change their preferences to match their prior decision (i.e., reducing preference for rejected items). Recently, however, Chen and Risen [Chen K, Risen J (2010) J Pers Soc Psychol 99:573-594] pointed out a serious methodological problem, which casts a doubt on the very existence of this choice-induced preference change as studied over the past 50 y. Here, using a proper control condition and two measures of preferences (self-report and brain activity), we found that the mere act of making a choice can change self-report preference as well as its neural representation (i.e., striatum activity), thus providing strong evidence for choice-induced preference change. Furthermore, our data indicate that the anterior cingulate cortex and dorsolateral prefrontal cortex tracked the degree of cognitive dissonance on a trial-by-trial basis. Our findings provide important insights into the neural basis of how actions can alter an individual's preferences.

Development and evaluation of a thermochemistry concept inventory for college-level general chemistry

NASA Astrophysics Data System (ADS)

Wren, David A.

The research presented in this dissertation culminated in a 10-item Thermochemistry Concept Inventory (TCI). The development of the TCI can be divided into two main phases: qualitative studies and quantitative studies. Both phases focused on the primary stakeholders of the TCI, college-level general chemistry instructors and students. Each phase was designed to collect evidence for the validity of the interpretations and uses of TCI testing data. A central use of TCI testing data is to identify student conceptual misunderstandings, which are represented as incorrect options of multiple-choice TCI items. Therefore, quantitative and qualitative studies focused heavily on collecting evidence at the item-level, where important interpretations may be made by TCI users. Qualitative studies included student interviews (N = 28) and online expert surveys (N = 30). Think-aloud student interviews (N = 12) were used to identify conceptual misunderstandings used by students. Novice response process validity interviews (N = 16) helped provide information on how students interpreted and answered TCI items and were the basis of item revisions. Practicing general chemistry instructors (N = 18), or experts, defined boundaries of thermochemistry content included on the TCI. Once TCI items were in the later stages of development, an online version of the TCI was used in expert response process validity survey (N = 12), to provide expert feedback on item content, format and consensus of the correct answer for each item. Quantitative studies included three phases: beta testing of TCI items (N = 280), pilot testing of the a 12-item TCI (N = 485), and a large data collection using a 10-item TCI ( N = 1331). In addition to traditional classical test theory analysis, Rasch model analysis was also used for evaluation of testing data at the test and item level. The TCI was administered in both formative assessment (beta and pilot testing) and summative assessment (large data collection), with items performing well in both. One item, item K, did not have acceptable psychometric properties when the TCI was used as a quiz (summative assessment), but was retained in the final version of the TCI based on the acceptable psychometric properties displayed in pilot testing (formative assessment).

Differential Item Functioning Detection Using the Multiple Indicators, Multiple Causes Method with a Pure Short Anchor

ERIC Educational Resources Information Center

Shih, Ching-Lin; Wang, Wen-Chung

2009-01-01

The multiple indicators, multiple causes (MIMIC) method with a pure short anchor was proposed to detect differential item functioning (DIF). A simulation study showed that the MIMIC method with an anchor of 1, 2, 4, or 10 DIF-free items yielded a well-controlled Type I error rate even when such tests contained as many as 40% DIF items. In general,…

13 CFR 121.407 - What are the size procedures for multiple item procurements?

Code of Federal Regulations, 2010 CFR

2010-01-01

... Requirements for Government Procurement § 121.407 What are the size procedures for multiple item procurements? If a procurement calls for two or more specific end items or types of services with different size... multiple item procurements? 121.407 Section 121.407 Business Credit and Assistance SMALL BUSINESS...

The Development of the Planet Formation Concept Inventory: A Preliminary Analysis of Version 1

NASA Astrophysics Data System (ADS)

Simon, Molly; Impey, Chris David; Buxner, Sanlyn

2018-01-01

The topic of planet formation is poorly represented in the educational literature, especially at the college level. As recently as 2014, when developing the Test of Astronomy Standards (TOAST), Slater (2014) noted that for two topics (formation of the Solar System and cosmology), “high quality test items that reflect our current understanding of students’ conceptions were not available [in the literature]” (Slater,2014, p. 8). Furthermore, nearly half of ASTR 101 enrollments are at 2 year/community colleges where both instructors and students have little access to current research and models of planet formation. In response, we administered six student replied response (SSR) short answer questions on the topic of planet formation to n = 1,050 students enrolled in introductory astronomy and planetary science courses at The University of Arizona in the Fall 2016 and Spring 2017 semesters. After analyzing and coding the data from the SSR questions, we developed a preliminary version of the Planet Formation Concept Inventory (PFCI). The PFCI is a multiple-choice instrument with 20 planet formation-related questions, and 4 demographic-related questions. We administered version 1 of the PFCI to six introductory astronomy and planetary science courses (n ~ 700 students) during the Fall 2017 semester. We provided students with 7-8 multiple-choice with explanation of reasoning (MCER) questions from the PFCI. Students selected an answer (similar to a traditional multiple-choice test), and then briefly explained why they chose the answer they did. We also conducted interviews with ~15 students to receive feedback on the quality of the questions and clarity of the instrument. We will present an analysis of the MCER responses and student interviews, and discuss any modifications that will be made to the instrument as a result.

Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper

PubMed Central

Palmer, Edward J; Devitt, Peter G

2007-01-01

Background Reliable and valid written tests of higher cognitive function are difficult to produce, particularly for the assessment of clinical problem solving. Modified Essay Questions (MEQs) are often used to assess these higher order abilities in preference to other forms of assessment, including multiple-choice questions (MCQs). MEQs often form a vital component of end-of-course assessments in higher education. It is not clear how effectively these questions assess higher order cognitive skills. This study was designed to assess the effectiveness of the MEQ to measure higher-order cognitive skills in an undergraduate institution. Methods An analysis of multiple-choice questions and modified essay questions (MEQs) used for summative assessment in a clinical undergraduate curriculum was undertaken. A total of 50 MCQs and 139 stages of MEQs were examined, which came from three exams run over two years. The effectiveness of the questions was determined by two assessors and was defined by the questions ability to measure higher cognitive skills, as determined by a modification of Bloom's taxonomy, and its quality as determined by the presence of item writing flaws. Results Over 50% of all of the MEQs tested factual recall. This was similar to the percentage of MCQs testing factual recall. The modified essay question failed in its role of consistently assessing higher cognitive skills whereas the MCQ frequently tested more than mere recall of knowledge. Conclusion Construction of MEQs, which will assess higher order cognitive skills cannot be assumed to be a simple task. Well-constructed MCQs should be considered a satisfactory replacement for MEQs if the MEQs cannot be designed to adequately test higher order skills. Such MCQs are capable of withstanding the intellectual and statistical scrutiny imposed by a high stakes exit examination. PMID:18045500

Concise evaluation of decision aids.

PubMed

Stalmeier, Peep F M; Roosmalen, Marielle S

2009-01-01

Decision aids purport to help patients make treatment related choices. Several instruments exist to evaluate decision aids. Our aim is to compare the responsiveness of several instruments. Two different decision aids were randomized in patients at high risk for breast and ovarian cancer. Treatment choices were between prophylactic surgery and screening. Effect sizes were calculated to compare the responsiveness of the measures. One decision aid was randomized in 390 women, the other in 91 ensuing mutation carriers. Three factors were identified related to Information, Well-being and Decision Making. Within each factor, single item measures were as responsive as multi-item measures. Four single items, 'the amount of information received for decision making,' 'strength of preference,' 'I weighed the pros and cons,' and 'General Health,' were adequately responsive to the decision aids. These items might be considered for inclusion in questionnaires to evaluate decision aids.

Controlling Guessing Bias in the Dichotomous Rasch Model Applied to a Large-Scale, Vertically Scaled Testing Program

PubMed Central

Andrich, David; Marais, Ida; Humphry, Stephen Mark

2015-01-01

Recent research has shown how the statistical bias in Rasch model difficulty estimates induced by guessing in multiple-choice items can be eliminated. Using vertical scaling of a high-profile national reading test, it is shown that the dominant effect of removing such bias is a nonlinear change in the unit of scale across the continuum. The consequence is that the proficiencies of the more proficient students are increased relative to those of the less proficient. Not controlling the guessing bias underestimates the progress of students across 7 years of schooling with important educational implications. PMID:29795871

Preference bias of head orientation in choosing between two non-durables.

PubMed

Funaya, Hiroyuki; Shibata, Tomohiro

2015-01-01

The goal of this study is to investigate how customers' gaze, head and body orientations reflect their choices. Although the relationship between human choice and gaze behavior has been well-studied, other behaviors such as head and body are unknown. We conducted a two-alternatives-forced-choice task to examine (1) whether preference bias, i.e., a positional bias in gaze, head and body toward the item that was later chosen, exists in choice, (2) when preference bias is observed and when prediction of the resulting choice becomes possible (3) whether human choice is affected when the body orientations are manipulated. We used real non-durable products (cheap snacks and clothing) on a shopping shelf. The results showed that there was a significant preference bias in head orientation at the beginning 1 s when the subjects stood straight toward the shelf, and that the head orientation was more biased toward the selected item than the gaze and the center of pressure at the ending 1 s. Manipulating body orientation did not affect the result of choice. The preference bias detected by observing the head orientation would be useful in marketing science for predicting customers' choice.

Force, velocity, and work: The effects of different contexts on students' understanding of vector concepts using isomorphic problems

NASA Astrophysics Data System (ADS)

Barniol, Pablo; Zavala, Genaro

2014-12-01

In this article we compare students' understanding of vector concepts in problems with no physical context, and with three mechanics contexts: force, velocity, and work. Based on our "Test of Understanding of Vectors," a multiple-choice test presented elsewhere, we designed two isomorphic shorter versions of 12 items each: a test with no physical context, and a test with mechanics contexts. For this study, we administered the items twice to students who were finishing an introductory mechanics course at a large private university in Mexico. The first time, we administered the two 12-item tests to 608 students. In the second, we only tested the items for which we had found differences in students' performances that were difficult to explain, and in this case, we asked them to show their reasoning in written form. In the first administration, we detected no significant difference between the medians obtained in the tests; however, we did identify significant differences in some of the items. For each item we analyze the type of difference found between the tests in the selection of the correct answer, the most common error on each of the tests, and the differences in the selection of incorrect answers. We also investigate the causes of the different context effects. Based on these analyses, we establish specific recommendations for the instruction of vector concepts in an introductory mechanics course. In the Supplemental Material we include both tests for other researchers studying vector learning, and for physics teachers who teach this material.

Development of knowledge tests for multi-disciplinary emergency training: a review and an example.

PubMed

Sørensen, J L; Thellesen, L; Strandbygaard, J; Svendsen, K D; Christensen, K B; Johansen, M; Langhoff-Roos, P; Ekelund, K; Ottesen, B; Van Der Vleuten, C

2015-01-01

The literature is sparse on written test development in a post-graduate multi-disciplinary setting. Developing and evaluating knowledge tests for use in multi-disciplinary post-graduate training is challenging. The objective of this study was to describe the process of developing and evaluating a multiple-choice question (MCQ) test for use in a multi-disciplinary training program in obstetric-anesthesia emergencies. A multi-disciplinary working committee with 12 members representing six professional healthcare groups and another 28 participants were involved. Recurrent revisions of the MCQ items were undertaken followed by a statistical analysis. The MCQ items were developed stepwise, including decisions on aims and content, followed by testing for face and content validity, construct validity, item-total correlation, and reliability. To obtain acceptable content validity, 40 out of originally 50 items were included in the final MCQ test. The MCQ test was able to distinguish between levels of competence, and good construct validity was indicated by a significant difference in the mean score between consultants and first-year trainees, as well as between first-year trainees and medical and midwifery students. Evaluation of the item-total correlation analysis in the 40 items set revealed that 11 items needed re-evaluation, four of which addressed content issues in local clinical guidelines. A Cronbach's alpha of 0.83 for reliability was found, which is acceptable. Content and construct validity and reliability were acceptable. The presented template for the development of this MCQ test could be useful to others when developing knowledge tests and may enhance the overall quality of test development. © 2014 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd.

Expanding the basic science debate: the role of physics knowledge in interpreting clinical findings.

PubMed

Goldszmidt, Mark; Minda, John Paul; Devantier, Sarah L; Skye, Aimee L; Woods, Nicole N

2012-10-01

Current research suggests a role for biomedical knowledge in learning and retaining concepts related to medical diagnosis. However, learning may be influenced by other, non-biomedical knowledge. We explored this idea using an experimental design and examined the effects of causal knowledge on the learning, retention, and interpretation of medical information. Participants studied a handout about several respiratory disorders and how to interpret respiratory exam findings. The control group received the information in standard "textbook" format and the experimental group was presented with the same information as well as a causal explanation about how sound travels through lungs in both the normal and disease states. Comprehension and memory of the information was evaluated with a multiple-choice exam. Several questions that were not related to the causal knowledge served as control items. Questions related to the interpretation of physical exam findings served as the critical test items. The experimental group outperformed the control group on the critical test items, and our study shows that a causal explanation can improve a student's memory for interpreting clinical details. We suggest an expansion of which basic sciences are considered fundamental to medical education.

Mixture Rasch model for guessing group identification

NASA Astrophysics Data System (ADS)

Siow, Hoo Leong; Mahdi, Rasidah; Siew, Eng Ling

2013-04-01

Several alternative dichotomous Item Response Theory (IRT) models have been introduced to account for guessing effect in multiple-choice assessment. The guessing effect in these models has been considered to be itemrelated. In the most classic case, pseudo-guessing in the three-parameter logistic IRT model is modeled to be the same for all the subjects but may vary across items. This is not realistic because subjects can guess worse or better than the pseudo-guessing. Derivation from the three-parameter logistic IRT model improves the situation by incorporating ability in guessing. However, it does not model non-monotone function. This paper proposes to study guessing from a subject-related aspect which is guessing test-taking behavior. Mixture Rasch model is employed to detect latent groups. A hybrid of mixture Rasch and 3-parameter logistic IRT model is proposed to model the behavior based guessing from the subjects' ways of responding the items. The subjects are assumed to simply choose a response at random. An information criterion is proposed to identify the behavior based guessing group. Results show that the proposed model selection criterion provides a promising method to identify the guessing group modeled by the hybrid model.

Fifth Graders' Learning About Simple Machines Through Engineering Design-Based Instruction Using LEGO™ Materials

NASA Astrophysics Data System (ADS)

Marulcu, Ismail; Barnett, Mike

2013-10-01

This study is part of a 5-year National Science Foundation-funded project, Transforming Elementary Science Learning Through LEGO™ Engineering Design. In this study, we report on the successes and challenges of implementing an engineering design-based and LEGO™-oriented unit in an urban classroom setting and we focus on the impact of the unit on students' content understanding of simple machines. The LEGO™ engineering-based simple machines module, which was developed for fifth graders by our research team, was implemented in an urban school in a large city in the Northeastern region of the USA. Thirty-three fifth grade students participated in the study, and they showed significant growth in content understanding. We measured students' content knowledge by using identical paper tests and semistructured interviews before and after instruction. Our paired t test analysis results showed that students significantly improved their test and interview scores (t = -3.62, p < 0.001 for multiple-choice items and t = -9.06, p < 0.000 for the open-ended items in the test and t = -12.11, p < 0.000 for the items in interviews). We also identified several alternative conceptions that are held by students on simple machines.

An experimental study of a museum-based, science PD programme's impact on teachers and their students

NASA Astrophysics Data System (ADS)

Aaron Price, C.; Chiu, A.

2018-06-01

We present results of an experimental study of an urban, museum-based science teacher PD programme. A total of 125 teachers and 1676 of their students in grades 4-8 were tested at the beginning and end of the school year in which the PD programme took place. Teachers and students were assessed on subject content knowledge and attitudes towards science, along with teacher classroom behaviour. Subject content questions were mostly taken from standardised state tests and literature, with an 'Explain:' prompt added to some items. Teachers in the treatment group showed a 7% gain in subject content knowledge over the control group. Students of teachers in the treatment group showed a 4% gain in subject content knowledge over the control group on multiple-choice items and an 11% gain on the constructed response items. There was no overall change in science attitudes of teachers or students over the control groups but we did find differences in teachers' reported self-efficacy and teaching anxiety levels, plus PD teachers reported doing more student-centered science teaching activities than the control group. All teachers came into the PD with high initial excitement, perhaps reflecting its context within an informal learning environment.

Cognitive modeling as an interface between brain and behavior: Measuring the semantic decline in mild cognitive impairment.

PubMed

Johns, Brendan T; Taler, Vanessa; Pisoni, David B; Farlow, Martin R; Hake, Ann Marie; Kareken, David A; Unverzagt, Frederick W; Jones, Michael N

2018-06-01

Mild cognitive impairment (MCI) is characterised by subjective and objective memory impairment in the absence of dementia. MCI is a strong predictor for the development of Alzheimer's disease, and may represent an early stage in the disease course in many cases. A standard task used in the diagnosis of MCI is verbal fluency, where participants produce as many items from a specific category (e.g., animals) as possible. Verbal fluency performance is typically analysed by counting the number of items produced. However, analysis of the semantic path of the items produced can provide valuable additional information. We introduce a cognitive model that uses multiple types of lexical information in conjunction with a standard memory search process. The model used a semantic representation derived from a standard semantic space model in conjunction with a memory searching mechanism derived from the Luce choice rule (Luce, 1977). The model was able to detect differences in the memory searching process of patients who were developing MCI, suggesting that the formal analysis of verbal fluency data is a promising avenue to examine the underlying changes occurring in the development of cognitive impairment. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Exploring the opportunities for food and drink purchasing and consumption by teenagers during their journeys between home and school: a feasibility study using a novel method.

PubMed

Cowburn, Gill; Matthews, Anne; Doherty, Aiden; Hamilton, Alex; Kelly, Paul; Williams, Julianne; Foster, Charlie; Nelson, Michael

2016-01-01

To investigate the feasibility and acceptability of using wearable cameras as a method to capture the opportunities for food and drink purchasing/consumption that young people encounter on their regular journeys to and from school. A qualitative study using multiple data-collection methods including wearable cameras, global positioning system units, individual interviews, food and drink purchase and consumption diaries completed by participants over four days, and an audit of food outlets located within an 800 m Euclidean buffer zone around each school. A community setting. Twenty-two students (fourteen girls and eight boys) aged 13-15 years recruited from four secondary schools in two counties of England. Wearable cameras offered a feasible and acceptable method for collecting food purchase and consumption data when used alongside traditional methods of data collection in a small number of teenagers. We found evidence of participants making deliberate choices about whether or not to purchase/consume food and drink on their journeys. These choices were influenced by priorities over money, friends, journey length, travel mode and ease of access to opportunities for purchase/consumption. Most food and drink items were purchased/consumed within an 800 m Euclidean buffer around school, with items commonly selected being high in energy, fat and sugar. Wearable camera images combined with interviews helped identify unreported items and misreporting errors. Wearable camera images prompt detailed discussion and generate contextually specific information which could offer new insights and understanding around eating behaviour patterns. The feasibility of scaling up the use of these methods requires further empirical work.

Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing.

PubMed

Butler, Andrew C; Roediger, Henry L

2008-04-01

Multiple-choice tests are used frequently in higher education without much consideration of the impact this form of assessment has on learning. Multiple-choice testing enhances retention of the material tested (the testing effect); however, unlike other tests, multiple-choice can also be detrimental because it exposes students to misinformation in the form of lures. The selection of lures can lead students to acquire false knowledge (Roediger & Marsh, 2005). The present research investigated whether feedback could be used to boost the positive effects and reduce the negative effects of multiple-choice testing. Subjects studied passages and then received a multiple-choice test with immediate feedback, delayed feedback, or no feedback. In comparison with the no-feedback condition, both immediate and delayed feedback increased the proportion of correct responses and reduced the proportion of intrusions (i.e., lure responses from the initial multiple-choice test) on a delayed cued recall test. Educators should provide feedback when using multiple-choice tests.

Building the BIKE: Development and Testing of the Biotechnology Instrument for Knowledge Elicitation (BIKE)

NASA Astrophysics Data System (ADS)

Witzig, Stephen B.; Rebello, Carina M.; Siegel, Marcelle A.; Freyermuth, Sharyn K.; Izci, Kemal; McClure, Bruce

2014-10-01

Identifying students' conceptual scientific understanding is difficult if the appropriate tools are not available for educators. Concept inventories have become a popular tool to assess student understanding; however, traditionally, they are multiple choice tests. International science education standard documents advocate that assessments should be reform based, contain diverse question types, and should align with instructional approaches. To date, no instrument of this type targeting student conceptions in biotechnology has been developed. We report here the development, testing, and validation of a 35-item Biotechnology Instrument for Knowledge Elicitation (BIKE) that includes a mix of question types. The BIKE was designed to elicit student thinking and a variety of conceptual understandings, as opposed to testing closed-ended responses. The design phase contained nine steps including a literature search for content, student interviews, a pilot test, as well as expert review. Data from 175 students over two semesters, including 16 student interviews and six expert reviewers (professors from six different institutions), were used to validate the instrument. Cronbach's alpha on the pre/posttest was 0.664 and 0.668, respectively, indicating the BIKE has internal consistency. Cohen's kappa for inter-rater reliability among the 6,525 total items was 0.684 indicating substantial agreement among scorers. Item analysis demonstrated that the items were challenging, there was discrimination among the individual items, and there was alignment with research-based design principles for construct validity. This study provides a reliable and valid conceptual understanding instrument in the understudied area of biotechnology.

The positive and negative consequences of multiple-choice testing.

PubMed

Roediger, Henry L; Marsh, Elizabeth J

2005-09-01

Multiple-choice tests are commonly used in educational settings but with unknown effects on students' knowledge. The authors examined the consequences of taking a multiple-choice test on a later general knowledge test in which students were warned not to guess. A large positive testing effect was obtained: Prior testing of facts aided final cued-recall performance. However, prior testing also had negative consequences. Prior reading of a greater number of multiple-choice lures decreased the positive testing effect and increased production of multiple-choice lures as incorrect answers on the final test. Multiple-choice testing may inadvertently lead to the creation of false knowledge.

Testing for Nonuniform Differential Item Functioning with Multiple Indicator Multiple Cause Models

ERIC Educational Resources Information Center

Woods, Carol M.; Grimm, Kevin J.

2011-01-01

In extant literature, multiple indicator multiple cause (MIMIC) models have been presented for identifying items that display uniform differential item functioning (DIF) only, not nonuniform DIF. This article addresses, for apparently the first time, the use of MIMIC models for testing both uniform and nonuniform DIF with categorical indicators. A…

Effect of Multiple Testing Adjustment in Differential Item Functioning Detection

ERIC Educational Resources Information Center

Kim, Jihye; Oshima, T. C.

2013-01-01

In a typical differential item functioning (DIF) analysis, a significance test is conducted for each item. As a test consists of multiple items, such multiple testing may increase the possibility of making a Type I error at least once. The goal of this study was to investigate how to control a Type I error rate and power using adjustment…

Willingness to Pay for Improving the Residential Waste Disposal System in Korea: A Choice Experiment Study

NASA Astrophysics Data System (ADS)

Ku, Se-Ju; Yoo, Seung-Hoon; Kwak, Seung-Jun

2009-08-01

This study attempts to apply choice experiments with regard to the residential waste disposal system (RWDS) in Korea by considering various attributes that are related to RWDS. Using data from a survey conducted on 492 households, the empirical analysis yields estimates of the willingness to pay for a clean food-waste collection facility, the collection of small items (such as obsolete mobile phones and add-ons for personal computers), and a more convenient large waste disposal system. The estimation results of multinomial logit models are quite similar to those of nested logit models. The results reveal that residents have preferences for the cleanliness of facilities and the collection of small items. In Korea, residents are required to purchase and attach stickers for the disposal of large items; they want to be able to obtain stickers at not only village offices but also supermarkets. On the other hand, the frequency of waste collection is not a significant factor in the choice of the improved waste management program.

Willingness to pay for improving the residential waste disposal system in Korea: a choice experiment study.

PubMed

Ku, Se-Ju; Yoo, Seung-Hoon; Kwak, Seung-Jun

2009-08-01

This study attempts to apply choice experiments with regard to the residential waste disposal system (RWDS) in Korea by considering various attributes that are related to RWDS. Using data from a survey conducted on 492 households, the empirical analysis yields estimates of the willingness to pay for a clean food-waste collection facility, the collection of small items (such as obsolete mobile phones and add-ons for personal computers), and a more convenient large waste disposal system. The estimation results of multinomial logit models are quite similar to those of nested logit models. The results reveal that residents have preferences for the cleanliness of facilities and the collection of small items. In Korea, residents are required to purchase and attach stickers for the disposal of large items; they want to be able to obtain stickers at not only village offices but also supermarkets. On the other hand, the frequency of waste collection is not a significant factor in the choice of the improved waste management program.

Development of a representational conceptual evaluation in the first law of thermodynamics

NASA Astrophysics Data System (ADS)

Sriyansyah, S. P.; Suhandi, A.

2016-08-01

As part of an ongoing research to investigate student consistency in understanding the first law of thermodynamics, a representational conceptual evaluation (RCET) has been developed to assess student conceptual understanding, representational consistency, and scientific consistency in the introductory physics course. Previous physics education research findings were used to develop the test. RCET items were 30 items which designed as an isomorphic multiple-choice test with three different representations concerning the concept of work, heat, first law of thermodynamics, and its application in the thermodynamic processes. Here, we present preliminary measures of the validity and reliability of the instrument, including the classical test statistics. This instrument can be used to measure the intended concept in the first law of thermodynamics and it will give the consistent results with the ability to differentiate well between high-achieving students and low-achieving students and also students at different level. As well as measuring the effectiveness of the learning process in the concept of the first law of thermodynamics.

Where There Is a Way, Is There a Will? The Effect of Future Choices on Self-Control

ERIC Educational Resources Information Center

Khan, Uzma; Dhar, Ravi

2007-01-01

Choices often involve self-control conflicts such that options that are immediately appealing are less desirable in the long run. In the current research, the authors examine how viewing such a choice as one of a series of similar future choices rather than as an isolated decision decreases the preference for items requiring self-control. The…

Factors Affecting Career Choice among Speech-Language Pathology and Audiology Students

ERIC Educational Resources Information Center

Stone, Larissa; Pellowski, Mark W.

2016-01-01

This investigation assessed the factors affecting career choice among 474 current undergraduate and graduate speech-language pathology and audiology students (from four universities). A 14-item questionnaire was developed that included questions related to general influence of career choice and whether or not the participants had previously been,…

Endogenous Formation of Preferences: Choices Systematically Change Willingness-to-Pay for Goods

ERIC Educational Resources Information Center

Voigt, Katharina; Murawski, Carsten; Bode, Stefan

2017-01-01

Standard decision theory assumes that choices result from stable preferences. This position has been challenged by claims that the act of choosing between goods may alter preferences. To test this claim, we investigated in three experiments whether choices between equally valued snack food items can systematically shape preferences. We directly…

The Effect of Guessing on Item Reliability under Answer-Until-Correct Scoring

ERIC Educational Resources Information Center

Kane, Michael; Moloney, James

1978-01-01

The answer-until-correct (AUC) procedure requires that examinees respond to a multi-choice item until they answer it correctly. Using a modified version of Horst's model for examinee behavior, this paper compares the effect of guessing on item reliability for the AUC procedure and the zero-one scoring procedure. (Author/CTM)

The language of science and the high school student: The recognition of concept definitions: A comparison between hindi speaking students in India and english speaking students in Australia

NASA Astrophysics Data System (ADS)

Lynch, P. P.; Chipman, H. H.; Pachaury, A. C.

Sixteen concept words (mass, length, area, volume, solid, liquid, gas, element, compound, mixture, electron, proton, neutron, atom, molecule, and ion) associated with the theme, the nature of matter were described as simple text book definitions after examination of classroom notes and school texts of the last three decades. Sixteen multiple-choice items all of the same form were constructed for each of the concept definitions. The English version of the sixteen item test was given to 1635 high school students in Tasmania (where the language of instruction and the home language is English) and the Hindi version of the test was given to 826 students from the Bhopal/Barwani region of India where the medium of instruction is Hindi. The English and Hindi speaking data are compared from the point of view of development, performance for individual items, and overall performance at grade 10. A number of linguistic hypotheses are examined and reported upon. Although the overall score at grade 10 was identical (10.8/16) for both groups there are differences in development overall and for individual items which are of interest. Overall, the science specificity of the Hindi words does not appear to confer any clearly defined advantage or disadvantage though again there are some interesting individual anomolies.