Feedback-based probabilistic category learning is selectively impaired in attention/hyperactivity deficit disorder.

PubMed

Gabay, Yafit; Goldfarb, Liat

2017-07-01

Although Attention-Deficit Hyperactivity Disorder (ADHD) is closely linked to executive function deficits, it has recently been attributed to procedural learning impairments that are quite distinct from the former. These observations challenge the ability of the executive function framework solely to account for the diverse range of symptoms observed in ADHD. A recent neurocomputational model emphasizes the role of striatal dopamine (DA) in explaining ADHD's broad range of deficits, but the link between this model and procedural learning impairments remains unclear. Significantly, feedback-based procedural learning is hypothesized to be disrupted in ADHD because of the involvement of striatal DA in this type of learning. In order to test this assumption, we employed two variants of a probabilistic category learning task known from the neuropsychological literature. Feedback-based (FB) and paired associate-based (PA) probabilistic category learning were employed in a non-medicated sample of ADHD participants and neurotypical participants. In the FB task, participants learned associations between cues and outcomes initially by guessing and subsequently through feedback indicating the correctness of the response. In the PA learning task, participants viewed the cue and its associated outcome simultaneously without receiving an overt response or corrective feedback. In both tasks, participants were trained across 150 trials. Learning was assessed in a subsequent test without a presentation of the outcome or corrective feedback. Results revealed an interesting disassociation in which ADHD participants performed as well as control participants in the PA task, but were impaired compared with the controls in the FB task. The learning curve during FB training differed between the two groups. Taken together, these results suggest that the ability to incrementally learn by feedback is selectively disrupted in ADHD participants. These results are discussed in relation to both the ADHD dopaminergic dysfunction model and recent findings implicating procedural learning impairments in those with ADHD. Copyright © 2017 Elsevier Inc. All rights reserved.

Visuo-Motor and Cognitive Procedural Learning in Children with Basal Ganglia Pathology

ERIC Educational Resources Information Center

Mayor-Dubois, C.; Maeder, P.; Zesiger, P.; Roulet-Perez, E.

2010-01-01

We investigated procedural learning in 18 children with basal ganglia (BG) lesions or dysfunctions of various aetiologies, using a visuo-motor learning test, the Serial Reaction Time (SRT) task, and a cognitive learning test, the Probabilistic Classification Learning (PCL) task. We compared patients with early (less than 1 year old, n=9), later…

Sequence Learning and Selection Difficulty

ERIC Educational Resources Information Center

Rowland, Lee A.; Shanks, David R.

2006-01-01

The authors studied the role of attention as a selection mechanism in implicit learning by examining the effect on primary sequence learning of performing a demanding target-selection task. Participants were trained on probabilistic sequences in a novel version of the serial reaction time (SRT) task, with dual- and triple-stimulus participants…

Impaired sequential and partially compensated probabilistic skill learning in Parkinson's disease.

PubMed

Kemény, Ferenc; Demeter, Gyula; Racsmány, Mihály; Valálik, István; Lukács, Ágnes

2018-06-08

The striatal dopaminergic dysfunction in Parkinson's disease (PD) has been associated with deficits in skill learning in numerous studies, but some of the findings remain controversial. Our aim was to explore the generality of the learning deficit using two widely reported skill learning tasks in the same group of Parkinson's patients. Thirty-four patients with PD (mean age: 62.83 years, SD: 7.67) were compared to age-matched healthy adults. Two tasks were employed: the Serial Reaction Time Task (SRT), testing the learning of motor sequences, and the Weather Prediction (WP) task, testing non-sequential probabilistic category learning. On the SRT task, patients with PD showed no significant evidence for sequence learning. These results support and also extend previous findings, suggesting that motor skill learning is vulnerable in PD. On the WP task, the PD group showed the same amount of learning as controls, but they exploited qualitatively different strategies in predicting the target categories. While controls typically combined probabilities from multiple predicting cues, patients with PD instead focused on individual cues. We also found moderate to high correlations between the different measures of skill learning. These findings support our hypothesis that skill learning is generally impaired in PD, and can in some cases be compensated by relying on alternative learning strategies. © 2018 The Authors. Journal of Neuropsychology published by John Wiley & Sons Ltd on behalf of British Psychological Society.

Procedural learning: A developmental study of motor sequence learning and probabilistic classification learning in school-aged children.

PubMed

Mayor-Dubois, Claire; Zesiger, Pascal; Van der Linden, Martial; Roulet-Perez, Eliane

2016-01-01

In this study, we investigated motor and cognitive procedural learning in typically developing children aged 8-12 years with a serial reaction time (SRT) task and a probabilistic classification learning (PCL) task. The aims were to replicate and extend the results of previous SRT studies, to investigate PCL in school-aged children, to explore the contribution of declarative knowledge to SRT and PCL performance, to explore the strategies used by children in the PCL task via a mathematical model, and to see whether performances obtained in motor and cognitive tasks correlated. The results showed similar learning effects in the three age groups in the SRT and in the first half of the PCL tasks. Participants did not develop explicit knowledge in the SRT task whereas declarative knowledge of the cue-outcome associations correlated with the performances in the second half of the PCL task, suggesting a participation of explicit knowledge after some time of exposure in PCL. An increasing proportion of the optimal strategy use with increasing age was observed in the PCL task. Finally, no correlation appeared between cognitive and motor performance. In conclusion, we extended the hypothesis of age invariance from motor to cognitive procedural learning, which had not been done previously. The ability to adopt more efficient learning strategies with age may rely on the maturation of the fronto-striatal loops. The lack of correlation between performance in the SRT task and the first part of the PCL task suggests dissociable developmental trajectories within the procedural memory system.

Magnetic Tunnel Junction Mimics Stochastic Cortical Spiking Neurons

NASA Astrophysics Data System (ADS)

Sengupta, Abhronil; Panda, Priyadarshini; Wijesinghe, Parami; Kim, Yusung; Roy, Kaushik

2016-07-01

Brain-inspired computing architectures attempt to mimic the computations performed in the neurons and the synapses in the human brain in order to achieve its efficiency in learning and cognitive tasks. In this work, we demonstrate the mapping of the probabilistic spiking nature of pyramidal neurons in the cortex to the stochastic switching behavior of a Magnetic Tunnel Junction in presence of thermal noise. We present results to illustrate the efficiency of neuromorphic systems based on such probabilistic neurons for pattern recognition tasks in presence of lateral inhibition and homeostasis. Such stochastic MTJ neurons can also potentially provide a direct mapping to the probabilistic computing elements in Belief Networks for performing regenerative tasks.

Isolation Rearing Effects on Probabilistic Learning and Cognitive Flexibility in Rats

PubMed Central

AMITAI, Nurith; YOUNG, Jared W.; HIGA, Kerin; SHARP, Richard F.; GEYER, Mark A.; POWELL, Susan B.

2013-01-01

Isolation rearing is a neurodevelopmental manipulation that produces neurochemical, structural, and behavioral alterations in rodents that have consistencies with schizophrenia. Symptoms induced by isolation rearing that mirror clinically relevant aspects of schizophrenia, such as cognitive deficits, open up the possibility of testing putative therapeutics in isolation-reared animals prior to clinical development. We investigated what effect isolation rearing would have on cognitive flexibility, a cognitive function characteristically disrupted in schizophrenia. For this purpose, we assessed cognitive flexibility using between- and within-session probabilistic reversal-learning tasks based on clinical tests. Isolation-reared rats required more sessions, though not more task trials, to acquire criterion performance in the reversal phase of the task and were slower to adjust their task strategy after reward contingencies were switched. Isolation-reared rats also completed fewer trials and exhibited lower levels of overall activity in the probabilistic reversal-learning task compared to socially reared rats. This finding contrasted with the elevated levels of unconditioned investigatory activity and reduced levels of locomotor habituation that isolation-reared rats displayed in the behavioral pattern monitor. Finally, isolation-reared rats also exhibited sensorimotor gating deficits, reflected by decreased prepulse inhibition of the startle response, consistent with previous studies. We conclude that isolation rearing constitutes a valuable, noninvasive manipulation for modeling schizophrenia-like cognitive deficits and assessing putative therapeutics. PMID:23943516

Stimulus discriminability may bias value-based probabilistic learning.

PubMed

Schutte, Iris; Slagter, Heleen A; Collins, Anne G E; Frank, Michael J; Kenemans, J Leon

2017-01-01

Reinforcement learning tasks are often used to assess participants' tendency to learn more from the positive or more from the negative consequences of one's action. However, this assessment often requires comparison in learning performance across different task conditions, which may differ in the relative salience or discriminability of the stimuli associated with more and less rewarding outcomes, respectively. To address this issue, in a first set of studies, participants were subjected to two versions of a common probabilistic learning task. The two versions differed with respect to the stimulus (Hiragana) characters associated with reward probability. The assignment of character to reward probability was fixed within version but reversed between versions. We found that performance was highly influenced by task version, which could be explained by the relative perceptual discriminability of characters assigned to high or low reward probabilities, as assessed by a separate discrimination experiment. Participants were more reliable in selecting rewarding characters that were more discriminable, leading to differences in learning curves and their sensitivity to reward probability. This difference in experienced reinforcement history was accompanied by performance biases in a test phase assessing ability to learn from positive vs. negative outcomes. In a subsequent large-scale web-based experiment, this impact of task version on learning and test measures was replicated and extended. Collectively, these findings imply a key role for perceptual factors in guiding reward learning and underscore the need to control stimulus discriminability when making inferences about individual differences in reinforcement learning.

Cognitive procedural learning among children and adolescents with or without spastic cerebral palsy: the differential effect of age.

PubMed

Gofer-Levi, M; Silberg, T; Brezner, A; Vakil, E

2014-09-01

Children learn to engage their surroundings skillfully, adopting implicit knowledge of complex regularities and associations. Probabilistic classification learning (PCL) is a type of cognitive procedural learning in which different cues are probabilistically associated with specific outcomes. Little is known about the effects of developmental disorders on cognitive skill acquisition. Twenty-four children and adolescents with cerebral palsy (CP) were compared to 24 typically developing (TD) youth in their ability to learn probabilistic associations. Performance was examined in relation to general cognitive abilities, level of motor impairment and age. Improvement in PCL was observed for all participants, with no relation to IQ. An age effect was found only among TD children. Learning curves of children with CP on a cognitive procedural learning task differ from those of TD peers and do not appear to be age sensitive. Copyright © 2014 Elsevier Ltd. All rights reserved.

Frontostriatal white matter integrity mediates adult age differences in probabilistic reward learning.

PubMed

Samanez-Larkin, Gregory R; Levens, Sara M; Perry, Lee M; Dougherty, Robert F; Knutson, Brian

2012-04-11

Frontostriatal circuits have been implicated in reward learning, and emerging findings suggest that frontal white matter structural integrity and probabilistic reward learning are reduced in older age. This cross-sectional study examined whether age differences in frontostriatal white matter integrity could account for age differences in reward learning in a community life span sample of human adults. By combining diffusion tensor imaging with a probabilistic reward learning task, we found that older age was associated with decreased reward learning and decreased white matter integrity in specific pathways running from the thalamus to the medial prefrontal cortex and from the medial prefrontal cortex to the ventral striatum. Further, white matter integrity in these thalamocorticostriatal paths could statistically account for age differences in learning. These findings suggest that the integrity of frontostriatal white matter pathways critically supports reward learning. The findings also raise the possibility that interventions that bolster frontostriatal integrity might improve reward learning and decision making.

Task-related functional connectivity of the caudate mediates the association between trait mindfulness and implicit learning in older adults.

PubMed

Stillman, Chelsea M; You, Xiaozhen; Seaman, Kendra L; Vaidya, Chandan J; Howard, James H; Howard, Darlene V

2016-08-01

Accumulating evidence shows a positive relationship between mindfulness and explicit cognitive functioning, i.e., that which occurs with conscious intent and awareness. However, recent evidence suggests that there may be a negative relationship between mindfulness and implicit types of learning, or those that occur without conscious awareness or intent. Here we examined the neural mechanisms underlying the recently reported negative relationship between dispositional mindfulness and implicit probabilistic sequence learning in both younger and older adults. We tested the hypothesis that the relationship is mediated by communication, or functional connectivity, of brain regions once traditionally considered to be central to dissociable learning systems: the caudate, medial temporal lobe (MTL), and prefrontal cortex (PFC). We first replicated the negative relationship between mindfulness and implicit learning in a sample of healthy older adults (60-90 years old) who completed three event-related runs of an implicit sequence learning task. Then, using a seed-based connectivity approach, we identified task-related connectivity associated with individual differences in both learning and mindfulness. The main finding was that caudate-MTL connectivity (bilaterally) was positively correlated with learning and negatively correlated with mindfulness. Further, the strength of task-related connectivity between these regions mediated the negative relationship between mindfulness and learning. This pattern of results was limited to the older adults. Thus, at least in healthy older adults, the functional communication between two interactive learning-relevant systems can account for the relationship between mindfulness and implicit probabilistic sequence learning.

Measuring reinforcement learning and motivation constructs in experimental animals: relevance to the negative symptoms of schizophrenia.

PubMed

Markou, Athina; Salamone, John D; Bussey, Timothy J; Mar, Adam C; Brunner, Daniela; Gilmour, Gary; Balsam, Peter

2013-11-01

The present review article summarizes and expands upon the discussions that were initiated during a meeting of the Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS; http://cntrics.ucdavis.edu) meeting. A major goal of the CNTRICS meeting was to identify experimental procedures and measures that can be used in laboratory animals to assess psychological constructs that are related to the psychopathology of schizophrenia. The issues discussed in this review reflect the deliberations of the Motivation Working Group of the CNTRICS meeting, which included most of the authors of this article as well as additional participants. After receiving task nominations from the general research community, this working group was asked to identify experimental procedures in laboratory animals that can assess aspects of reinforcement learning and motivation that may be relevant for research on the negative symptoms of schizophrenia, as well as other disorders characterized by deficits in reinforcement learning and motivation. The tasks described here that assess reinforcement learning are the Autoshaping Task, Probabilistic Reward Learning Tasks, and the Response Bias Probabilistic Reward Task. The tasks described here that assess motivation are Outcome Devaluation and Contingency Degradation Tasks and Effort-Based Tasks. In addition to describing such methods and procedures, the present article provides a working vocabulary for research and theory in this field, as well as an industry perspective about how such tasks may be used in drug discovery. It is hoped that this review can aid investigators who are conducting research in this complex area, promote translational studies by highlighting shared research goals and fostering a common vocabulary across basic and clinical fields, and facilitate the development of medications for the treatment of symptoms mediated by reinforcement learning and motivational deficits. Copyright © 2013 Elsevier Ltd. All rights reserved.

Measuring reinforcement learning and motivation constructs in experimental animals: relevance to the negative symptoms of schizophrenia

PubMed Central

Markou, Athina; Salamone, John D.; Bussey, Timothy; Mar, Adam; Brunner, Daniela; Gilmour, Gary; Balsam, Peter

2013-01-01

The present review article summarizes and expands upon the discussions that were initiated during a meeting of the Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS; http://cntrics.ucdavis.edu). A major goal of the CNTRICS meeting was to identify experimental procedures and measures that can be used in laboratory animals to assess psychological constructs that are related to the psychopathology of schizophrenia. The issues discussed in this review reflect the deliberations of the Motivation Working Group of the CNTRICS meeting, which included most of the authors of this article as well as additional participants. After receiving task nominations from the general research community, this working group was asked to identify experimental procedures in laboratory animals that can assess aspects of reinforcement learning and motivation that may be relevant for research on the negative symptoms of schizophrenia, as well as other disorders characterized by deficits in reinforcement learning and motivation. The tasks described here that assess reinforcement learning are the Autoshaping Task, Probabilistic Reward Learning Tasks, and the Response Bias Probabilistic Reward Task. The tasks described here that assess motivation are Outcome Devaluation and Contingency Degradation Tasks and Effort-Based Tasks. In addition to describing such methods and procedures, the present article provides a working vocabulary for research and theory in this field, as well as an industry perspective about how such tasks may be used in drug discovery. It is hoped that this review can aid investigators who are conducting research in this complex area, promote translational studies by highlighting shared research goals and fostering a common vocabulary across basic and clinical fields, and facilitate the development of medications for the treatment of symptoms mediated by reinforcement learning and motivational deficits. PMID:23994273

Attentional load and implicit sequence learning.

PubMed

Shanks, David R; Rowland, Lee A; Ranger, Mandeep S

2005-06-01

A widely employed conceptualization of implicit learning hypothesizes that it makes minimal demands on attentional resources. This conjecture was investigated by comparing learning under single-task and dual-task conditions in the sequential reaction time (SRT) task. Participants learned probabilistic sequences, with dual-task participants additionally having to perform a counting task using stimuli that were targets in the SRT display. Both groups were then tested for sequence knowledge under single-task (Experiments 1 and 2) or dual-task (Experiment 3) conditions. Participants also completed a free generation task (Experiments 2 and 3) under inclusion or exclusion conditions to determine if sequence knowledge was conscious or unconscious in terms of its access to intentional control. The experiments revealed that the secondary task impaired sequence learning and that sequence knowledge was consciously accessible. These findings disconfirm both the notion that implicit learning is able to proceed normally under conditions of divided attention, and that the acquired knowledge is inaccessible to consciousness. A unitary framework for conceptualizing implicit and explicit learning is proposed.

Attenuation of dopamine-modulated prefrontal value signals underlies probabilistic reward learning deficits in old age

PubMed Central

Axelsson, Jan; Riklund, Katrine; Nyberg, Lars; Dayan, Peter; Bäckman, Lars

2017-01-01

Probabilistic reward learning is characterised by individual differences that become acute in aging. This may be due to age-related dopamine (DA) decline affecting neural processing in striatum, prefrontal cortex, or both. We examined this by administering a probabilistic reward learning task to younger and older adults, and combining computational modelling of behaviour, fMRI and PET measurements of DA D1 availability. We found that anticipatory value signals in ventromedial prefrontal cortex (vmPFC) were attenuated in older adults. The strength of this signal predicted performance beyond age and was modulated by D1 availability in nucleus accumbens. These results uncover that a value-anticipation mechanism in vmPFC declines in aging, and that this mechanism is associated with DA D1 receptor availability. PMID:28870286

TASK-RELATED FUNCTIONAL CONNECTIVITY OF THE CAUDATE MEDIATES THE ASSOCIATION BETWEEN TRAIT MINDFULNESS AND IMPLICIT LEARNING IN OLDER ADULTS

PubMed Central

Stillman, Chelsea M.; You, Xiaozhen; Seaman, Kendra L.; Vaidya, Chandan J.; Howard, James H.; Howard, Darlene V.

2016-01-01

Accumulating evidence shows a positive relationship between mindfulness and explicit cognitive functioning, i.e., that which occurs with conscious intent and awareness. However, recent evidence suggests that there may be a negative relationship between mindfulness and implicit types of learning, or those that occur without conscious awareness or intent. Here we examined the neural mechanisms underlying the recently reported negative relationship between dispositional mindfulness and implicit probabilistic sequence learning in both younger and older adults. We tested the hypothesis that the relationship is mediated by communication, or functional connectivity, of brain regions once traditionally considered to be central to dissociable learning systems: the caudate, medial temporal lobe (MTL), and prefrontal cortex (PFC). We first replicated the negative relationship between mindfulness and implicit learning in a sample of healthy older adults (60–90 years old) who completed three event-related runs of an implicit sequence learning task. Then, using a seed-based connectivity approach, we identified task-related connectivity associated with individual differences in both learning and mindfulness. The main finding was that caudate-MTL connectivity (bilaterally) was positively correlated with learning and negatively correlated with mindfulness. Further, the strength of task-related connectivity between these regions mediated the negative relationship between mindfulness and learning. This pattern of results was limited to the older adults. Thus, at least in healthy older adults, the functional communication between two interactive learning-relevant systems can account for the relationship between mindfulness and implicit probabilistic sequence learning. PMID:27121302

Intact implicit learning in autism spectrum conditions.

PubMed

Brown, Jamie; Aczel, Balazs; Jiménez, Luis; Kaufman, Scott Barry; Grant, Kate Plaisted

2010-09-01

Individuals with autism spectrum condition (ASC) have diagnostic impairments in skills that are associated with an implicit acquisition; however, it is not clear whether ASC individuals show specific implicit learning deficits. We compared ASC and typically developing (TD) individuals matched for IQ on five learning tasks: four implicit learning tasks--contextual cueing, serial reaction time, artificial grammar learning, and probabilistic classification learning tasks--that used procedures expressly designed to minimize the use of explicit strategies, and one comparison explicit learning task, paired associates learning. We found implicit learning to be intact in ASC. Beyond no evidence of differences, there was evidence of statistical equivalence between the groups on all the implicit learning tasks. This was not a consequence of compensation by explicit learning ability or IQ. Furthermore, there was no evidence to relate implicit learning to ASC symptomatology. We conclude that implicit mechanisms are preserved in ASC and propose that it is disruption by other atypical processes that impact negatively on the development of skills associated with an implicit acquisition.

Differential theory of learning for efficient neural network pattern recognition

NASA Astrophysics Data System (ADS)

Hampshire, John B., II; Vijaya Kumar, Bhagavatula

1993-09-01

We describe a new theory of differential learning by which a broad family of pattern classifiers (including many well-known neural network paradigms) can learn stochastic concepts efficiently. We describe the relationship between a classifier's ability to generate well to unseen test examples and the efficiency of the strategy by which it learns. We list a series of proofs that differential learning is efficient in its information and computational resource requirements, whereas traditional probabilistic learning strategies are not. The proofs are illustrated by a simple example that lends itself to closed-form analysis. We conclude with an optical character recognition task for which three different types of differentially generated classifiers generalize significantly better than their probabilistically generated counterparts.

Differential theory of learning for efficient neural network pattern recognition

NASA Astrophysics Data System (ADS)

Hampshire, John B., II; Vijaya Kumar, Bhagavatula

1993-08-01

We describe a new theory of differential learning by which a broad family of pattern classifiers (including many well-known neural network paradigms) can learn stochastic concepts efficiently. We describe the relationship between a classifier's ability to generalize well to unseen test examples and the efficiency of the strategy by which it learns. We list a series of proofs that differential learning is efficient in its information and computational resource requirements, whereas traditional probabilistic learning strategies are not. The proofs are illustrated by a simple example that lends itself to closed-form analysis. We conclude with an optical character recognition task for which three different types of differentially generated classifiers generalize significantly better than their probabilistically generated counterparts.

Perceptual learning as improved probabilistic inference in early sensory areas.

PubMed

Bejjanki, Vikranth R; Beck, Jeffrey M; Lu, Zhong-Lin; Pouget, Alexandre

2011-05-01

Extensive training on simple tasks such as fine orientation discrimination results in large improvements in performance, a form of learning known as perceptual learning. Previous models have argued that perceptual learning is due to either sharpening and amplification of tuning curves in early visual areas or to improved probabilistic inference in later visual areas (at the decision stage). However, early theories are inconsistent with the conclusions of psychophysical experiments manipulating external noise, whereas late theories cannot explain the changes in neural responses that have been reported in cortical areas V1 and V4. Here we show that we can capture both the neurophysiological and behavioral aspects of perceptual learning by altering only the feedforward connectivity in a recurrent network of spiking neurons so as to improve probabilistic inference in early visual areas. The resulting network shows modest changes in tuning curves, in line with neurophysiological reports, along with a marked reduction in the amplitude of pairwise noise correlations.

Adult Age Differences in Learning from Positive and Negative Probabilistic Feedback

PubMed Central

Simon, Jessica R.; Howard, James H.; Howard, Darlene V.

2010-01-01

Objective Past research has investigated age differences in frontal-based decision making, but few studies have focused on the behavioral effects of striatal-based changes in healthy aging. Feedback learning has been found to vary with dopamine levels; increases in dopamine facilitate learning from positive feedback, whereas decreases facilitate learning from negative feedback. Given previous evidence of striatal dopamine depletion in healthy aging, we investigated behavioral differences between college-aged and healthy old adults using a feedback learning task that is sensitive to both frontal and striatal processes. Method Seventeen college-aged (M = 18.9 years) and 24 healthy, older adults (M = 70.3 years) completed the Probabilistic selection task, in which participants are trained on probabilistic stimulus-outcome information and then tested to determine whether they learned more from positive or negative feedback. Results As a group, the old adults learned equally well from positive and negative feedback, whereas the college-aged group learned more from positive than negative feedback, F(1, 39) = 4.10, p < .05, reffect = .3. However, these group differences were not due to the older individuals being more balanced learners. Most individuals of both ages were balanced learners, but while all of the remaining young learners had a positive bias, the remaining older learners were split between those with positive and negative learning biases (χ2(2) = 6.12, p<.047). Conclusions These behavioral results are consistent with the dopamine theory of striatal aging, and suggest there might be adult age differences in the kinds of information people use when faced with a current choice. PMID:20604627

Naïve and Robust: Class-Conditional Independence in Human Classification Learning

ERIC Educational Resources Information Center

Jarecki, Jana B.; Meder, Björn; Nelson, Jonathan D.

2018-01-01

Humans excel in categorization. Yet from a computational standpoint, learning a novel probabilistic classification task involves severe computational challenges. The present paper investigates one way to address these challenges: assuming class-conditional independence of features. This feature independence assumption simplifies the inference…

Distinct roles of dopamine and subthalamic nucleus in learning and probabilistic decision making.

PubMed

Coulthard, Elizabeth J; Bogacz, Rafal; Javed, Shazia; Mooney, Lucy K; Murphy, Gillian; Keeley, Sophie; Whone, Alan L

2012-12-01

Even simple behaviour requires us to make decisions based on combining multiple pieces of learned and new information. Making such decisions requires both learning the optimal response to each given stimulus as well as combining probabilistic information from multiple stimuli before selecting a response. Computational theories of decision making predict that learning individual stimulus-response associations and rapid combination of information from multiple stimuli are dependent on different components of basal ganglia circuitry. In particular, learning and retention of memory, required for optimal response choice, are significantly reliant on dopamine, whereas integrating information probabilistically is critically dependent upon functioning of the glutamatergic subthalamic nucleus (computing the 'normalization term' in Bayes' theorem). Here, we test these theories by investigating 22 patients with Parkinson's disease either treated with deep brain stimulation to the subthalamic nucleus and dopaminergic therapy or managed with dopaminergic therapy alone. We use computerized tasks that probe three cognitive functions-information acquisition (learning), memory over a delay and information integration when multiple pieces of sequentially presented information have to be combined. Patients performed the tasks ON or OFF deep brain stimulation and/or ON or OFF dopaminergic therapy. Consistent with the computational theories, we show that stopping dopaminergic therapy impairs memory for probabilistic information over a delay, whereas deep brain stimulation to the region of the subthalamic nucleus disrupts decision making when multiple pieces of acquired information must be combined. Furthermore, we found that when participants needed to update their decision on the basis of the last piece of information presented in the decision-making task, patients with deep brain stimulation of the subthalamic nucleus region did not slow down appropriately to revise their plan, a pattern of behaviour that mirrors the impulsivity described clinically in some patients with subthalamic nucleus deep brain stimulation. Thus, we demonstrate distinct mechanisms for two important facets of human decision making: first, a role for dopamine in memory consolidation, and second, the critical importance of the subthalamic nucleus in successful decision making when multiple pieces of information must be combined.

Integrating dynamic stopping, transfer learning and language models in an adaptive zero-training ERP speller.

PubMed

Kindermans, Pieter-Jan; Tangermann, Michael; Müller, Klaus-Robert; Schrauwen, Benjamin

2014-06-01

Most BCIs have to undergo a calibration session in which data is recorded to train decoders with machine learning. Only recently zero-training methods have become a subject of study. This work proposes a probabilistic framework for BCI applications which exploit event-related potentials (ERPs). For the example of a visual P300 speller we show how the framework harvests the structure suitable to solve the decoding task by (a) transfer learning, (b) unsupervised adaptation, (c) language model and (d) dynamic stopping. A simulation study compares the proposed probabilistic zero framework (using transfer learning and task structure) to a state-of-the-art supervised model on n = 22 subjects. The individual influence of the involved components (a)-(d) are investigated. Without any need for a calibration session, the probabilistic zero-training framework with inter-subject transfer learning shows excellent performance--competitive to a state-of-the-art supervised method using calibration. Its decoding quality is carried mainly by the effect of transfer learning in combination with continuous unsupervised adaptation. A high-performing zero-training BCI is within reach for one of the most popular BCI paradigms: ERP spelling. Recording calibration data for a supervised BCI would require valuable time which is lost for spelling. The time spent on calibration would allow a novel user to spell 29 symbols with our unsupervised approach. It could be of use for various clinical and non-clinical ERP-applications of BCI.

Integrating dynamic stopping, transfer learning and language models in an adaptive zero-training ERP speller

NASA Astrophysics Data System (ADS)

Kindermans, Pieter-Jan; Tangermann, Michael; Müller, Klaus-Robert; Schrauwen, Benjamin

2014-06-01

Objective. Most BCIs have to undergo a calibration session in which data is recorded to train decoders with machine learning. Only recently zero-training methods have become a subject of study. This work proposes a probabilistic framework for BCI applications which exploit event-related potentials (ERPs). For the example of a visual P300 speller we show how the framework harvests the structure suitable to solve the decoding task by (a) transfer learning, (b) unsupervised adaptation, (c) language model and (d) dynamic stopping. Approach. A simulation study compares the proposed probabilistic zero framework (using transfer learning and task structure) to a state-of-the-art supervised model on n = 22 subjects. The individual influence of the involved components (a)-(d) are investigated. Main results. Without any need for a calibration session, the probabilistic zero-training framework with inter-subject transfer learning shows excellent performance—competitive to a state-of-the-art supervised method using calibration. Its decoding quality is carried mainly by the effect of transfer learning in combination with continuous unsupervised adaptation. Significance. A high-performing zero-training BCI is within reach for one of the most popular BCI paradigms: ERP spelling. Recording calibration data for a supervised BCI would require valuable time which is lost for spelling. The time spent on calibration would allow a novel user to spell 29 symbols with our unsupervised approach. It could be of use for various clinical and non-clinical ERP-applications of BCI.

The Effects of Aging on the Neural Basis of Implicit Associative Learning in a Probabilistic Triplets Learning Task

ERIC Educational Resources Information Center

Simon, Jessica R.; Vaidya, Chandan J.; Howard, James H., Jr.; Howard, Darlene V.

2012-01-01

Few studies have investigated how aging influences the neural basis of implicit associative learning, and available evidence is inconclusive. One emerging behavioral pattern is that age differences increase with practice, perhaps reflecting the involvement of different brain regions with training. Many studies report hippocampal involvement early…

Probabilistic Reinforcement Learning in Adults with Autism Spectrum Disorders

PubMed Central

Solomon, Marjorie; Smith, Anne C.; Frank, Michael J.; Ly, Stanford; Carter, Cameron S.

2017-01-01

Background Autism spectrum disorders (ASDs) can be conceptualized as disorders of learning, however there have been few experimental studies taking this perspective. Methods We examined the probabilistic reinforcement learning performance of 28 adults with ASDs and 30 typically developing adults on a task requiring learning relationships between three stimulus pairs consisting of Japanese characters with feedback that was valid with different probabilities (80%, 70%, and 60%). Both univariate and Bayesian state–space data analytic methods were employed. Hypotheses were based on the extant literature as well as on neurobiological and computational models of reinforcement learning. Results Both groups learned the task after training. However, there were group differences in early learning in the first task block where individuals with ASDs acquired the most frequently accurately reinforced stimulus pair (80%) comparably to typically developing individuals; exhibited poorer acquisition of the less frequently reinforced 70% pair as assessed by state–space learning curves; and outperformed typically developing individuals on the near chance (60%) pair. Individuals with ASDs also demonstrated deficits in using positive feedback to exploit rewarded choices. Conclusions Results support the contention that individuals with ASDs are slower learners. Based on neurobiology and on the results of computational modeling, one interpretation of this pattern of findings is that impairments are related to deficits in flexible updating of reinforcement history as mediated by the orbito-frontal cortex, with spared functioning of the basal ganglia. This hypothesis about the pathophysiology of learning in ASDs can be tested using functional magnetic resonance imaging. PMID:21425243

Obtaining Accurate Probabilities Using Classifier Calibration

ERIC Educational Resources Information Center

Pakdaman Naeini, Mahdi

2016-01-01

Learning probabilistic classification and prediction models that generate accurate probabilities is essential in many prediction and decision-making tasks in machine learning and data mining. One way to achieve this goal is to post-process the output of classification models to obtain more accurate probabilities. These post-processing methods are…

A Flexible Mechanism of Rule Selection Enables Rapid Feature-Based Reinforcement Learning

PubMed Central

Balcarras, Matthew; Womelsdorf, Thilo

2016-01-01

Learning in a new environment is influenced by prior learning and experience. Correctly applying a rule that maps a context to stimuli, actions, and outcomes enables faster learning and better outcomes compared to relying on strategies for learning that are ignorant of task structure. However, it is often difficult to know when and how to apply learned rules in new contexts. In our study we explored how subjects employ different strategies for learning the relationship between stimulus features and positive outcomes in a probabilistic task context. We test the hypothesis that task naive subjects will show enhanced learning of feature specific reward associations by switching to the use of an abstract rule that associates stimuli by feature type and restricts selections to that dimension. To test this hypothesis we designed a decision making task where subjects receive probabilistic feedback following choices between pairs of stimuli. In the task, trials are grouped in two contexts by blocks, where in one type of block there is no unique relationship between a specific feature dimension (stimulus shape or color) and positive outcomes, and following an un-cued transition, alternating blocks have outcomes that are linked to either stimulus shape or color. Two-thirds of subjects (n = 22/32) exhibited behavior that was best fit by a hierarchical feature-rule model. Supporting the prediction of the model mechanism these subjects showed significantly enhanced performance in feature-reward blocks, and rapidly switched their choice strategy to using abstract feature rules when reward contingencies changed. Choice behavior of other subjects (n = 10/32) was fit by a range of alternative reinforcement learning models representing strategies that do not benefit from applying previously learned rules. In summary, these results show that untrained subjects are capable of flexibly shifting between behavioral rules by leveraging simple model-free reinforcement learning and context-specific selections to drive responses. PMID:27064794

Dopamine neurons learn relative chosen value from probabilistic rewards

PubMed Central

Lak, Armin; Stauffer, William R; Schultz, Wolfram

2016-01-01

Economic theories posit reward probability as one of the factors defining reward value. Individuals learn the value of cues that predict probabilistic rewards from experienced reward frequencies. Building on the notion that responses of dopamine neurons increase with reward probability and expected value, we asked how dopamine neurons in monkeys acquire this value signal that may represent an economic decision variable. We found in a Pavlovian learning task that reward probability-dependent value signals arose from experienced reward frequencies. We then assessed neuronal response acquisition during choices among probabilistic rewards. Here, dopamine responses became sensitive to the value of both chosen and unchosen options. Both experiments showed also the novelty responses of dopamine neurones that decreased as learning advanced. These results show that dopamine neurons acquire predictive value signals from the frequency of experienced rewards. This flexible and fast signal reflects a specific decision variable and could update neuronal decision mechanisms. DOI: http://dx.doi.org/10.7554/eLife.18044.001 PMID:27787196

Speech processing using maximum likelihood continuity mapping

DOEpatents

Hogden, John E.

2000-01-01

Speech processing is obtained that, given a probabilistic mapping between static speech sounds and pseudo-articulator positions, allows sequences of speech sounds to be mapped to smooth sequences of pseudo-articulator positions. In addition, a method for learning a probabilistic mapping between static speech sounds and pseudo-articulator position is described. The method for learning the mapping between static speech sounds and pseudo-articulator position uses a set of training data composed only of speech sounds. The said speech processing can be applied to various speech analysis tasks, including speech recognition, speaker recognition, speech coding, speech synthesis, and voice mimicry.

Speech processing using maximum likelihood continuity mapping

DOE Office of Scientific and Technical Information (OSTI.GOV)

Hogden, J.E.

Speech processing is obtained that, given a probabilistic mapping between static speech sounds and pseudo-articulator positions, allows sequences of speech sounds to be mapped to smooth sequences of pseudo-articulator positions. In addition, a method for learning a probabilistic mapping between static speech sounds and pseudo-articulator position is described. The method for learning the mapping between static speech sounds and pseudo-articulator position uses a set of training data composed only of speech sounds. The said speech processing can be applied to various speech analysis tasks, including speech recognition, speaker recognition, speech coding, speech synthesis, and voice mimicry.

Behavioral Modeling Based on Probabilistic Finite Automata: An Empirical Study.

PubMed

Tîrnăucă, Cristina; Montaña, José L; Ontañón, Santiago; González, Avelino J; Pardo, Luis M

2016-06-24

Imagine an agent that performs tasks according to different strategies. The goal of Behavioral Recognition (BR) is to identify which of the available strategies is the one being used by the agent, by simply observing the agent's actions and the environmental conditions during a certain period of time. The goal of Behavioral Cloning (BC) is more ambitious. In this last case, the learner must be able to build a model of the behavior of the agent. In both settings, the only assumption is that the learner has access to a training set that contains instances of observed behavioral traces for each available strategy. This paper studies a machine learning approach based on Probabilistic Finite Automata (PFAs), capable of achieving both the recognition and cloning tasks. We evaluate the performance of PFAs in the context of a simulated learning environment (in this case, a virtual Roomba vacuum cleaner robot), and compare it with a collection of other machine learning approaches.

Better Learning with More Error: Probabilistic Feedback Increases Sensitivity to Correlated Cues in Categorization

ERIC Educational Resources Information Center

Little, Daniel R.; Lewandowsky, Stephan

2009-01-01

Despite the fact that categories are often composed of correlated features, the evidence that people detect and use these correlations during intentional category learning has been overwhelmingly negative to date. Nonetheless, on other categorization tasks, such as feature prediction, people show evidence of correlational sensitivity. A…

Dynamic shaping of dopamine signals during probabilistic Pavlovian conditioning.

PubMed

Hart, Andrew S; Clark, Jeremy J; Phillips, Paul E M

2015-01-01

Cue- and reward-evoked phasic dopamine activity during Pavlovian and operant conditioning paradigms is well correlated with reward-prediction errors from formal reinforcement learning models, which feature teaching signals in the form of discrepancies between actual and expected reward outcomes. Additionally, in learning tasks where conditioned cues probabilistically predict rewards, dopamine neurons show sustained cue-evoked responses that are correlated with the variance of reward and are maximal to cues predicting rewards with a probability of 0.5. Therefore, it has been suggested that sustained dopamine activity after cue presentation encodes the uncertainty of impending reward delivery. In the current study we examined the acquisition and maintenance of these neural correlates using fast-scan cyclic voltammetry in rats implanted with carbon fiber electrodes in the nucleus accumbens core during probabilistic Pavlovian conditioning. The advantage of this technique is that we can sample from the same animal and recording location throughout learning with single trial resolution. We report that dopamine release in the nucleus accumbens core contains correlates of both expected value and variance. A quantitative analysis of these signals throughout learning, and during the ongoing updating process after learning in probabilistic conditions, demonstrates that these correlates are dynamically encoded during these phases. Peak CS-evoked responses are correlated with expected value and predominate during early learning while a variance-correlated sustained CS signal develops during the post-asymptotic updating phase. Copyright © 2014 Elsevier Inc. All rights reserved.

Domain-general sequence learning deficit in specific language impairment.

PubMed

Lukács, Agnes; Kemény, Ferenc

2014-05-01

Grammar-specific accounts of specific language impairment (SLI) have been challenged by recent claims that language problems are a consequence of impairments in domain-general mechanisms of learning that also play a key role in the process of language acquisition. Our studies were designed to test the generality and nature of this learning deficit by focusing on both sequential and nonsequential, and on verbal and nonverbal, domains. Twenty-nine children with SLI were compared with age-matched typically developing (TD) control children using (a) a serial reaction time task (SRT), testing the learning of motor sequences; (b) an artificial grammar learning (AGL) task, testing the extraction of regularities from auditory sequences; and (c) a weather prediction task (WP), testing probabilistic category learning in a nonsequential task. For the 2 sequence learning tasks, a significantly smaller proportion of children showed evidence of learning in the SLI than in the TD group (χ2 tests, p < .001 for the SRT task, p < .05 for the AGL task), whereas the proportion of learners on the WP task was the same in the 2 groups. The level of learning for SLI learners was comparable with that of TD children on all tasks (with great individual variation). Taken together, these findings suggest that domain-general processes of implicit sequence learning tend to be impaired in SLI. Further research is needed to clarify the relationship of deficits in implicit learning and language.

Probabilistic learning and inference in schizophrenia

PubMed Central

Averbeck, Bruno B.; Evans, Simon; Chouhan, Viraj; Bristow, Eleanor; Shergill, Sukhwinder S.

2010-01-01

Patients with schizophrenia make decisions on the basis of less evidence when required to collect information to make an inference, a behavior often called jumping to conclusions. The underlying basis for this behaviour remains controversial. We examined the cognitive processes underpinning this finding by testing subjects on the beads task, which has been used previously to elicit jumping to conclusions behaviour, and a stochastic sequence learning task, with a similar decision theoretic structure. During the sequence learning task, subjects had to learn a sequence of button presses, while receiving noisy feedback on their choices. We fit a Bayesian decision making model to the sequence task and compared model parameters to the choice behavior in the beads task in both patients and healthy subjects. We found that patients did show a jumping to conclusions style; and those who picked early in the beads task tended to learn less from positive feedback in the sequence task. This favours the likelihood of patients selecting early because they have a low threshold for making decisions, and that they make choices on the basis of relatively little evidence. PMID:20810252

Probabilistic learning and inference in schizophrenia.

PubMed

Averbeck, Bruno B; Evans, Simon; Chouhan, Viraj; Bristow, Eleanor; Shergill, Sukhwinder S

2011-04-01

Patients with schizophrenia make decisions on the basis of less evidence when required to collect information to make an inference, a behavior often called jumping to conclusions. The underlying basis for this behavior remains controversial. We examined the cognitive processes underpinning this finding by testing subjects on the beads task, which has been used previously to elicit jumping to conclusions behavior, and a stochastic sequence learning task, with a similar decision theoretic structure. During the sequence learning task, subjects had to learn a sequence of button presses, while receiving a noisy feedback on their choices. We fit a Bayesian decision making model to the sequence task and compared model parameters to the choice behavior in the beads task in both patients and healthy subjects. We found that patients did show a jumping to conclusions style; and those who picked early in the beads task tended to learn less from positive feedback in the sequence task. This favours the likelihood of patients selecting early because they have a low threshold for making decisions, and that they make choices on the basis of relatively little evidence. Published by Elsevier B.V.

Probabilistic classification learning with corrective feedback is associated with in vivo striatal dopamine release in the ventral striatum, while learning without feedback is not

PubMed Central

Wilkinson, Leonora; Tai, Yen Foung; Lin, Chia Shu; Lagnado, David Albert; Brooks, David James; Piccini, Paola; Jahanshahi, Marjan

2014-01-01

The basal ganglia (BG) mediate certain types of procedural learning, such as probabilistic classification learning on the ‘weather prediction task’ (WPT). Patients with Parkinson's disease (PD), who have BG dysfunction, are impaired at WPT-learning, but it remains unclear what component of the WPT is important for learning to occur. We tested the hypothesis that learning through processing of corrective feedback is the essential component and is associated with release of striatal dopamine. We employed two WPT paradigms, either involving learning via processing of corrective feedback (FB) or in a paired associate manner (PA). To test the prediction that learning on the FB but not PA paradigm would be associated with dopamine release in the striatum, we used serial 11C-raclopride (RAC) positron emission tomography (PET), to investigate striatal dopamine release during FB and PA WPT-learning in healthy individuals. Two groups, FB, (n = 7) and PA (n = 8), underwent RAC PET twice, once while performing the WPT and once during a control task. Based on a region-of-interest approach, striatal RAC-binding potentials reduced by 13–17% in the right ventral striatum when performing the FB compared to control task, indicating release of synaptic dopamine. In contrast, right ventral striatal RAC binding non-significantly increased by 9% during the PA task. While differences between the FB and PA versions of the WPT in effort and decision-making is also relevant, we conclude striatal dopamine is released during FB-based WPT-learning, implicating the striatum and its dopamine connections in mediating learning with FB. PMID:24777947

Learning classification with auxiliary probabilistic information

PubMed Central

Nguyen, Quang; Valizadegan, Hamed; Hauskrecht, Milos

2012-01-01

Finding ways of incorporating auxiliary information or auxiliary data into the learning process has been the topic of active data mining and machine learning research in recent years. In this work we study and develop a new framework for classification learning problem in which, in addition to class labels, the learner is provided with an auxiliary (probabilistic) information that reflects how strong the expert feels about the class label. This approach can be extremely useful for many practical classification tasks that rely on subjective label assessment and where the cost of acquiring additional auxiliary information is negligible when compared to the cost of the example analysis and labelling. We develop classification algorithms capable of using the auxiliary information to make the learning process more efficient in terms of the sample complexity. We demonstrate the benefit of the approach on a number of synthetic and real world data sets by comparing it to the learning with class labels only. PMID:25309141

Weakly Supervised Dictionary Learning

NASA Astrophysics Data System (ADS)

You, Zeyu; Raich, Raviv; Fern, Xiaoli Z.; Kim, Jinsub

2018-05-01

We present a probabilistic modeling and inference framework for discriminative analysis dictionary learning under a weak supervision setting. Dictionary learning approaches have been widely used for tasks such as low-level signal denoising and restoration as well as high-level classification tasks, which can be applied to audio and image analysis. Synthesis dictionary learning aims at jointly learning a dictionary and corresponding sparse coefficients to provide accurate data representation. This approach is useful for denoising and signal restoration, but may lead to sub-optimal classification performance. By contrast, analysis dictionary learning provides a transform that maps data to a sparse discriminative representation suitable for classification. We consider the problem of analysis dictionary learning for time-series data under a weak supervision setting in which signals are assigned with a global label instead of an instantaneous label signal. We propose a discriminative probabilistic model that incorporates both label information and sparsity constraints on the underlying latent instantaneous label signal using cardinality control. We present the expectation maximization (EM) procedure for maximum likelihood estimation (MLE) of the proposed model. To facilitate a computationally efficient E-step, we propose both a chain and a novel tree graph reformulation of the graphical model. The performance of the proposed model is demonstrated on both synthetic and real-world data.

General functioning predicts reward and punishment learning in schizophrenia.

PubMed

Somlai, Zsuzsanna; Moustafa, Ahmed A; Kéri, Szabolcs; Myers, Catherine E; Gluck, Mark A

2011-04-01

Previous studies investigating feedback-driven reinforcement learning in patients with schizophrenia have provided mixed results. In this study, we explored the clinical predictors of reward and punishment learning using a probabilistic classification learning task. Patients with schizophrenia (n=40) performed similarly to healthy controls (n=30) on the classification learning task. However, more severe negative and general symptoms were associated with lower reward-learning performance, whereas poorer general psychosocial functioning was correlated with both lower reward- and punishment-learning performances. Multiple linear regression analyses indicated that general psychosocial functioning was the only significant predictor of reinforcement learning performance when education, antipsychotic dose, and positive, negative and general symptoms were included in the analysis. These results suggest a close relationship between reinforcement learning and general psychosocial functioning in schizophrenia. Published by Elsevier B.V.

Linguistic Constraints on Statistical Word Segmentation: The Role of Consonants in Arabic and English

ERIC Educational Resources Information Center

Kastner, Itamar; Adriaans, Frans

2018-01-01

Statistical learning is often taken to lie at the heart of many cognitive tasks, including the acquisition of language. One particular task in which probabilistic models have achieved considerable success is the segmentation of speech into words. However, these models have mostly been tested against English data, and as a result little is known…

Probabilistic motor sequence learning in a virtual reality serial reaction time task.

PubMed

Sense, Florian; van Rijn, Hedderik

2018-01-01

The serial reaction time task is widely used to study learning and memory. The task is traditionally administered by showing target positions on a computer screen and collecting responses using a button box or keyboard. By comparing response times to random or sequenced items or by using different transition probabilities, various forms of learning can be studied. However, this traditional laboratory setting limits the number of possible experimental manipulations. Here, we present a virtual reality version of the serial reaction time task and show that learning effects emerge as expected despite the novel way in which responses are collected. We also show that response times are distributed as expected. The current experiment was conducted in a blank virtual reality room to verify these basic principles. For future applications, the technology can be used to modify the virtual reality environment in any conceivable way, permitting a wide range of previously impossible experimental manipulations.

How Attention Can Create Synaptic Tags for the Learning of Working Memories in Sequential Tasks

PubMed Central

Rombouts, Jaldert O.; Bohte, Sander M.; Roelfsema, Pieter R.

2015-01-01

Intelligence is our ability to learn appropriate responses to new stimuli and situations. Neurons in association cortex are thought to be essential for this ability. During learning these neurons become tuned to relevant features and start to represent them with persistent activity during memory delays. This learning process is not well understood. Here we develop a biologically plausible learning scheme that explains how trial-and-error learning induces neuronal selectivity and working memory representations for task-relevant information. We propose that the response selection stage sends attentional feedback signals to earlier processing levels, forming synaptic tags at those connections responsible for the stimulus-response mapping. Globally released neuromodulators then interact with tagged synapses to determine their plasticity. The resulting learning rule endows neural networks with the capacity to create new working memory representations of task relevant information as persistent activity. It is remarkably generic: it explains how association neurons learn to store task-relevant information for linear as well as non-linear stimulus-response mappings, how they become tuned to category boundaries or analog variables, depending on the task demands, and how they learn to integrate probabilistic evidence for perceptual decisions. PMID:25742003

A Bayesian Developmental Approach to Robotic Goal-Based Imitation Learning.

PubMed

Chung, Michael Jae-Yoon; Friesen, Abram L; Fox, Dieter; Meltzoff, Andrew N; Rao, Rajesh P N

2015-01-01

A fundamental challenge in robotics today is building robots that can learn new skills by observing humans and imitating human actions. We propose a new Bayesian approach to robotic learning by imitation inspired by the developmental hypothesis that children use self-experience to bootstrap the process of intention recognition and goal-based imitation. Our approach allows an autonomous agent to: (i) learn probabilistic models of actions through self-discovery and experience, (ii) utilize these learned models for inferring the goals of human actions, and (iii) perform goal-based imitation for robotic learning and human-robot collaboration. Such an approach allows a robot to leverage its increasing repertoire of learned behaviors to interpret increasingly complex human actions and use the inferred goals for imitation, even when the robot has very different actuators from humans. We demonstrate our approach using two different scenarios: (i) a simulated robot that learns human-like gaze following behavior, and (ii) a robot that learns to imitate human actions in a tabletop organization task. In both cases, the agent learns a probabilistic model of its own actions, and uses this model for goal inference and goal-based imitation. We also show that the robotic agent can use its probabilistic model to seek human assistance when it recognizes that its inferred actions are too uncertain, risky, or impossible to perform, thereby opening the door to human-robot collaboration.

A Bayesian Developmental Approach to Robotic Goal-Based Imitation Learning

PubMed Central

Chung, Michael Jae-Yoon; Friesen, Abram L.; Fox, Dieter; Meltzoff, Andrew N.; Rao, Rajesh P. N.

2015-01-01

A fundamental challenge in robotics today is building robots that can learn new skills by observing humans and imitating human actions. We propose a new Bayesian approach to robotic learning by imitation inspired by the developmental hypothesis that children use self-experience to bootstrap the process of intention recognition and goal-based imitation. Our approach allows an autonomous agent to: (i) learn probabilistic models of actions through self-discovery and experience, (ii) utilize these learned models for inferring the goals of human actions, and (iii) perform goal-based imitation for robotic learning and human-robot collaboration. Such an approach allows a robot to leverage its increasing repertoire of learned behaviors to interpret increasingly complex human actions and use the inferred goals for imitation, even when the robot has very different actuators from humans. We demonstrate our approach using two different scenarios: (i) a simulated robot that learns human-like gaze following behavior, and (ii) a robot that learns to imitate human actions in a tabletop organization task. In both cases, the agent learns a probabilistic model of its own actions, and uses this model for goal inference and goal-based imitation. We also show that the robotic agent can use its probabilistic model to seek human assistance when it recognizes that its inferred actions are too uncertain, risky, or impossible to perform, thereby opening the door to human-robot collaboration. PMID:26536366

Supervised Extraction of Diagnosis Codes from EMRs: Role of Feature Selection, Data Selection, and Probabilistic Thresholding.

PubMed

Rios, Anthony; Kavuluru, Ramakanth

2013-09-01

Extracting diagnosis codes from medical records is a complex task carried out by trained coders by reading all the documents associated with a patient's visit. With the popularity of electronic medical records (EMRs), computational approaches to code extraction have been proposed in the recent years. Machine learning approaches to multi-label text classification provide an important methodology in this task given each EMR can be associated with multiple codes. In this paper, we study the the role of feature selection, training data selection, and probabilistic threshold optimization in improving different multi-label classification approaches. We conduct experiments based on two different datasets: a recent gold standard dataset used for this task and a second larger and more complex EMR dataset we curated from the University of Kentucky Medical Center. While conventional approaches achieve results comparable to the state-of-the-art on the gold standard dataset, on our complex in-house dataset, we show that feature selection, training data selection, and probabilistic thresholding provide significant gains in performance.

Processing of probabilistic information in weight perception and motor prediction.

PubMed

Trampenau, Leif; van Eimeren, Thilo; Kuhtz-Buschbeck, Johann

2017-02-01

We studied the effects of probabilistic cues, i.e., of information of limited certainty, in the context of an action task (GL: grip-lift) and of a perceptual task (WP: weight perception). Normal subjects (n = 22) saw four different probabilistic visual cues, each of which announced the likely weight of an object. In the GL task, the object was grasped and lifted with a pinch grip, and the peak force rates indicated that the grip and load forces were scaled predictively according to the probabilistic information. The WP task provided the expected heaviness related to each probabilistic cue; the participants gradually adjusted the object's weight until its heaviness matched the expected weight for a given cue. Subjects were randomly assigned to two groups: one started with the GL task and the other one with the WP task. The four different probabilistic cues influenced weight adjustments in the WP task and peak force rates in the GL task in a similar manner. The interpretation and utilization of the probabilistic information was critically influenced by the initial task. Participants who started with the WP task classified the four probabilistic cues into four distinct categories and applied these categories to the subsequent GL task. On the other side, participants who started with the GL task applied three distinct categories to the four cues and retained this classification in the following WP task. The initial strategy, once established, determined the way how the probabilistic information was interpreted and implemented.

A new computational account of cognitive control over reinforcement-based decision-making: Modeling of a probabilistic learning task.

PubMed

Zendehrouh, Sareh

2015-11-01

Recent work on decision-making field offers an account of dual-system theory for decision-making process. This theory holds that this process is conducted by two main controllers: a goal-directed system and a habitual system. In the reinforcement learning (RL) domain, the habitual behaviors are connected with model-free methods, in which appropriate actions are learned through trial-and-error experiences. However, goal-directed behaviors are associated with model-based methods of RL, in which actions are selected using a model of the environment. Studies on cognitive control also suggest that during processes like decision-making, some cortical and subcortical structures work in concert to monitor the consequences of decisions and to adjust control according to current task demands. Here a computational model is presented based on dual system theory and cognitive control perspective of decision-making. The proposed model is used to simulate human performance on a variant of probabilistic learning task. The basic proposal is that the brain implements a dual controller, while an accompanying monitoring system detects some kinds of conflict including a hypothetical cost-conflict one. The simulation results address existing theories about two event-related potentials, namely error related negativity (ERN) and feedback related negativity (FRN), and explore the best account of them. Based on the results, some testable predictions are also presented. Copyright © 2015 Elsevier Ltd. All rights reserved.

Constrained dictionary learning and probabilistic hypergraph ranking for person re-identification

NASA Astrophysics Data System (ADS)

He, You; Wu, Song; Pu, Nan; Qian, Li; Xiao, Guoqiang

2018-04-01

Person re-identification is a fundamental and inevitable task in public security. In this paper, we propose a novel framework to improve the performance of this task. First, two different types of descriptors are extracted to represent a pedestrian: (1) appearance-based superpixel features, which are constituted mainly by conventional color features and extracted from the supepixel rather than a whole picture and (2) due to the limitation of discrimination of appearance features, the deep features extracted by feature fusion Network are also used. Second, a view invariant subspace is learned by dictionary learning constrained by the minimum negative sample (termed as DL-cMN) to reduce the noise in appearance-based superpixel feature domain. Then, we use deep features and sparse codes transformed by appearancebased features to establish the hyperedges respectively by k-nearest neighbor, rather than jointing different features simply. Finally, a final ranking is performed by probabilistic hypergraph ranking algorithm. Extensive experiments on three challenging datasets (VIPeR, PRID450S and CUHK01) demonstrate the advantages and effectiveness of our proposed algorithm.

Habit learning and the genetics of the dopamine D3 receptor: evidence from patients with schizophrenia and healthy controls.

PubMed

Kéri, Szabolcs; Juhász, Anna; Rimanóczy, Agnes; Szekeres, György; Kelemen, Oguz; Cimmer, Csongor; Szendi, István; Benedek, György; Janka, Zoltán

2005-06-01

In this study, the authors investigated the relationship between the Ser9Gly (SG) polymorphism of the dopamine D3 receptor (DRD3) and striatal habit learning in healthy controls and patients with schizophrenia. Participants were given the weather prediction task, during which probabilistic cue-response associations were learned for tarot cards and weather outcomes (rain or sunshine). In both healthy controls and patients with schizophrenia, participants with Ser9Ser (SS) genotype did not learn during the early phase of the task (1-50 trials), whereas participants with SG genotype did so. During the late phase of the task (51-100 trials), both participants with SS and SG genotype exhibited significant learning. Learning rate was normal in patients with schizophrenia. These results suggest that the DRD3 variant containing glycine is associated with more efficient striatal habit learning in healthy controls and patients with schizophrenia. (c) 2005 APA, all rights reserved.

Dynamics of EEG functional connectivity during statistical learning.

PubMed

Tóth, Brigitta; Janacsek, Karolina; Takács, Ádám; Kóbor, Andrea; Zavecz, Zsófia; Nemeth, Dezso

2017-10-01

Statistical learning is a fundamental mechanism of the brain, which extracts and represents regularities of our environment. Statistical learning is crucial in predictive processing, and in the acquisition of perceptual, motor, cognitive, and social skills. Although previous studies have revealed competitive neurocognitive processes underlying statistical learning, the neural communication of the related brain regions (functional connectivity, FC) has not yet been investigated. The present study aimed to fill this gap by investigating FC networks that promote statistical learning in humans. Young adults (N=28) performed a statistical learning task while 128-channels EEG was acquired. The task involved probabilistic sequences, which enabled to measure incidental/implicit learning of conditional probabilities. Phase synchronization in seven frequency bands was used to quantify FC between cortical regions during the first, second, and third periods of the learning task, respectively. Here we show that statistical learning is negatively correlated with FC of the anterior brain regions in slow (theta) and fast (beta) oscillations. These negative correlations increased as the learning progressed. Our findings provide evidence that dynamic antagonist brain networks serve a hallmark of statistical learning. Copyright © 2017 Elsevier Inc. All rights reserved.

Impairment of probabilistic reward-based learning in schizophrenia.

PubMed

Weiler, Julia A; Bellebaum, Christian; Brüne, Martin; Juckel, Georg; Daum, Irene

2009-09-01

Recent models assume that some symptoms of schizophrenia originate from defective reward processing mechanisms. Understanding the precise nature of reward-based learning impairments might thus make an important contribution to the understanding of schizophrenia and the development of treatment strategies. The present study investigated several features of probabilistic reward-based stimulus association learning, namely the acquisition of initial contingencies, reversal learning, generalization abilities, and the effects of reward magnitude. Compared to healthy controls, individuals with schizophrenia exhibited attenuated overall performance during acquisition, whereas learning rates across blocks were similar to the rates of controls. On the group level, persons with schizophrenia were, however, unable to learn the reversal of the initial reward contingencies. Exploratory analysis of only the subgroup of individuals with schizophrenia who showed significant learning during acquisition yielded deficits in reversal learning with low reward magnitudes only. There was further evidence of a mild generalization impairment of the persons with schizophrenia in an acquired equivalence task. In summary, although there was evidence of intact basic processing of reward magnitudes, individuals with schizophrenia were impaired at using this feedback for the adaptive guidance of behavior.

Rapidly Measuring the Speed of Unconscious Learning: Amnesics Learn Quickly and Happy People Slowly

PubMed Central

Dienes, Zoltan; Baddeley, Roland J.; Jansari, Ashok

2012-01-01

Background We introduce a method for quickly determining the rate of implicit learning. Methodology/Principal Findings The task involves making a binary prediction for a probabilistic sequence over 10 minutes; from this it is possible to determine the influence of events of a different number of trials in the past on the current decision. This profile directly reflects the learning rate parameter of a large class of learning algorithms including the delta and Rescorla-Wagner rules. To illustrate the use of the method, we compare a person with amnesia with normal controls and we compare people with induced happy and sad moods. Conclusions/Significance Learning on the task is likely both associative and implicit. We argue theoretically and demonstrate empirically that both amnesia and also transient negative moods can be associated with an especially large learning rate: People with amnesia can learn quickly and happy people slowly. PMID:22457759

Sleep-Dependent Consolidation of Rewarded Behavior Is Diminished in Children with Attention Deficit Hyperactivity Disorder and a Comorbid Disorder of Social Behavior

PubMed Central

Wiesner, Christian D.; Molzow, Ina; Prehn-Kristensen, Alexander; Baving, Lioba

2017-01-01

Children suffering from attention-deficit hyperactivity disorder (ADHD) often also display impaired learning and memory. Previous research has documented aberrant reward processing in ADHD as well as impaired sleep-dependent consolidation of declarative memory. We investigated whether sleep also fosters the consolidation of behavior learned by probabilistic reward and whether ADHD patients with a comorbid disorder of social behavior show deficits in this memory domain, too. A group of 17 ADHD patients with comorbid disorders of social behavior aged 8–12 years and healthy controls matched for age, IQ, and handedness took part in the experiment. During the encoding task, children worked on a probabilistic learning task acquiring behavioral preferences for stimuli rewarded most often. After a 12-hr retention interval of either sleep at night or wakefulness during the day, a reversal task was presented where the contingencies were reversed. Consolidation of rewarded behavior is indicated by greater resistance to reversal learning. We found that healthy children consolidate rewarded behavior better during a night of sleep than during a day awake and that the sleep-dependent consolidation of rewarded behavior by trend correlates with non-REM sleep but not with REM sleep. In contrast, children with ADHD and comorbid disorders of social behavior do not show sleep-dependent consolidation of rewarded behavior. Moreover, their consolidation of rewarded behavior does not correlate with sleep. The results indicate that dysfunctional sleep in children suffering from ADHD and disorders of social behavior might be a crucial factor in the consolidation of behavior learned by reward. PMID:28228742

PROBABILISTIC PROGRAMMING FOR ADVANCED MACHINE LEARNING (PPAML) DISCRIMINATIVE LEARNING FOR GENERATIVE TASKS (DILIGENT)

DTIC Science & Technology

2017-11-29

Structural connections of the frames (fragments) in the knowledge. We call the fundamental elements of the knowledge a limited number of elements...the result of contracted fundamental research deemed exempt from public affairs security and policy review in accordance with SAF/AQR memorandum dated...AVAILABILITY STATEMENT Approved for Public Release; Distribution Unlimited. This report is the result of contracted fundamental research deemed exempt from

The penumbra of learning: a statistical theory of synaptic tagging and capture.

PubMed

Gershman, Samuel J

2014-01-01

Learning in humans and animals is accompanied by a penumbra: Learning one task benefits from learning an unrelated task shortly before or after. At the cellular level, the penumbra of learning appears when weak potentiation of one synapse is amplified by strong potentiation of another synapse on the same neuron during a critical time window. Weak potentiation sets a molecular tag that enables the synapse to capture plasticity-related proteins synthesized in response to strong potentiation at another synapse. This paper describes a computational model which formalizes synaptic tagging and capture in terms of statistical learning mechanisms. According to this model, synaptic strength encodes a probabilistic inference about the dynamically changing association between pre- and post-synaptic firing rates. The rate of change is itself inferred, coupling together different synapses on the same neuron. When the inputs to one synapse change rapidly, the inferred rate of change increases, amplifying learning at other synapses.

Selective effects of 5-HT2C receptor modulation on performance of a novel valence-probe visual discrimination task and probabilistic reversal learning in mice.

PubMed

Phillips, Benjamin U; Dewan, Sigma; Nilsson, Simon R O; Robbins, Trevor W; Heath, Christopher J; Saksida, Lisa M; Bussey, Timothy J; Alsiö, Johan

2018-04-22

Dysregulation of the serotonin (5-HT) system is a pathophysiological component in major depressive disorder (MDD), a condition closely associated with abnormal emotional responsivity to positive and negative feedback. However, the precise mechanism through which 5-HT tone biases feedback responsivity remains unclear. 5-HT2C receptors (5-HT2CRs) are closely linked with aspects of depressive symptomatology, including abnormalities in reinforcement processes and response to stress. Thus, we aimed to determine the impact of 5-HT2CR function on response to feedback in biased reinforcement learning. We used two touchscreen assays designed to assess the impact of positive and negative feedback on probabilistic reinforcement in mice, including a novel valence-probe visual discrimination (VPVD) and a probabilistic reversal learning procedure (PRL). Systemic administration of a 5-HT2CR agonist and antagonist resulted in selective changes in the balance of feedback sensitivity bias on these tasks. Specifically, on VPVD, SB 242084, the 5-HT2CR antagonist, impaired acquisition of a discrimination dependent on appropriate integration of positive and negative feedback. On PRL, SB 242084 at 1 mg/kg resulted in changes in behaviour consistent with reduced sensitivity to positive feedback. In contrast, WAY 163909, the 5-HT2CR agonist, resulted in changes associated with increased sensitivity to positive feedback and decreased sensitivity to negative feedback. These results suggest that 5-HT2CRs tightly regulate feedback sensitivity bias in mice with consequent effects on learning and cognitive flexibility and specify a framework for the influence of 5-HT2CRs on sensitivity to reinforcement.

bnstruct: an R package for Bayesian Network structure learning in the presence of missing data.

PubMed

Franzin, Alberto; Sambo, Francesco; Di Camillo, Barbara

2017-04-15

A Bayesian Network is a probabilistic graphical model that encodes probabilistic dependencies between a set of random variables. We introduce bnstruct, an open source R package to (i) learn the structure and the parameters of a Bayesian Network from data in the presence of missing values and (ii) perform reasoning and inference on the learned Bayesian Networks. To the best of our knowledge, there is no other open source software that provides methods for all of these tasks, particularly the manipulation of missing data, which is a common situation in practice. The software is implemented in R and C and is available on CRAN under a GPL licence. francesco.sambo@unipd.it. Supplementary data are available at Bioinformatics online. © The Author 2016. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com

A probabilistic framework to infer brain functional connectivity from anatomical connections.

PubMed

Deligianni, Fani; Varoquaux, Gael; Thirion, Bertrand; Robinson, Emma; Sharp, David J; Edwards, A David; Rueckert, Daniel

2011-01-01

We present a novel probabilistic framework to learn across several subjects a mapping from brain anatomical connectivity to functional connectivity, i.e. the covariance structure of brain activity. This prediction problem must be formulated as a structured-output learning task, as the predicted parameters are strongly correlated. We introduce a model selection framework based on cross-validation with a parametrization-independent loss function suitable to the manifold of covariance matrices. Our model is based on constraining the conditional independence structure of functional activity by the anatomical connectivity. Subsequently, we learn a linear predictor of a stationary multivariate autoregressive model. This natural parameterization of functional connectivity also enforces the positive-definiteness of the predicted covariance and thus matches the structure of the output space. Our results show that functional connectivity can be explained by anatomical connectivity on a rigorous statistical basis, and that a proper model of functional connectivity is essential to assess this link.

The influence of trial order on learning from reward vs. punishment in a probabilistic categorization task: experimental and computational analyses.

PubMed

Moustafa, Ahmed A; Gluck, Mark A; Herzallah, Mohammad M; Myers, Catherine E

2015-01-01

Previous research has shown that trial ordering affects cognitive performance, but this has not been tested using category-learning tasks that differentiate learning from reward and punishment. Here, we tested two groups of healthy young adults using a probabilistic category learning task of reward and punishment in which there are two types of trials (reward, punishment) and three possible outcomes: (1) positive feedback for correct responses in reward trials; (2) negative feedback for incorrect responses in punishment trials; and (3) no feedback for incorrect answers in reward trials and correct answers in punishment trials. Hence, trials without feedback are ambiguous, and may represent either successful avoidance of punishment or failure to obtain reward. In Experiment 1, the first group of subjects received an intermixed task in which reward and punishment trials were presented in the same block, as a standard baseline task. In Experiment 2, a second group completed the separated task, in which reward and punishment trials were presented in separate blocks. Additionally, in order to understand the mechanisms underlying performance in the experimental conditions, we fit individual data using a Q-learning model. Results from Experiment 1 show that subjects who completed the intermixed task paradoxically valued the no-feedback outcome as a reinforcer when it occurred on reinforcement-based trials, and as a punisher when it occurred on punishment-based trials. This is supported by patterns of empirical responding, where subjects showed more win-stay behavior following an explicit reward than following an omission of punishment, and more lose-shift behavior following an explicit punisher than following an omission of reward. In Experiment 2, results showed similar performance whether subjects received reward-based or punishment-based trials first. However, when the Q-learning model was applied to these data, there were differences between subjects in the reward-first and punishment-first conditions on the relative weighting of neutral feedback. Specifically, early training on reward-based trials led to omission of reward being treated as similar to punishment, but prior training on punishment-based trials led to omission of reward being treated more neutrally. This suggests that early training on one type of trials, specifically reward-based trials, can create a bias in how neutral feedback is processed, relative to those receiving early punishment-based training or training that mixes positive and negative outcomes.

Quantum-Assisted Learning of Hardware-Embedded Probabilistic Graphical Models

NASA Astrophysics Data System (ADS)

Benedetti, Marcello; Realpe-Gómez, John; Biswas, Rupak; Perdomo-Ortiz, Alejandro

2017-10-01

Mainstream machine-learning techniques such as deep learning and probabilistic programming rely heavily on sampling from generally intractable probability distributions. There is increasing interest in the potential advantages of using quantum computing technologies as sampling engines to speed up these tasks or to make them more effective. However, some pressing challenges in state-of-the-art quantum annealers have to be overcome before we can assess their actual performance. The sparse connectivity, resulting from the local interaction between quantum bits in physical hardware implementations, is considered the most severe limitation to the quality of constructing powerful generative unsupervised machine-learning models. Here, we use embedding techniques to add redundancy to data sets, allowing us to increase the modeling capacity of quantum annealers. We illustrate our findings by training hardware-embedded graphical models on a binarized data set of handwritten digits and two synthetic data sets in experiments with up to 940 quantum bits. Our model can be trained in quantum hardware without full knowledge of the effective parameters specifying the corresponding quantum Gibbs-like distribution; therefore, this approach avoids the need to infer the effective temperature at each iteration, speeding up learning; it also mitigates the effect of noise in the control parameters, making it robust to deviations from the reference Gibbs distribution. Our approach demonstrates the feasibility of using quantum annealers for implementing generative models, and it provides a suitable framework for benchmarking these quantum technologies on machine-learning-related tasks.

REM Sleep Enhancement of Probabilistic Classification Learning is Sensitive to Subsequent Interference

PubMed Central

Barsky, Murray M.; Tucker, Matthew A.; Stickgold, Robert

2015-01-01

During wakefulness the brain creates meaningful relationships between disparate stimuli in ways that escape conscious awareness. Processes active during sleep can strengthen these relationships, leading to more adaptive use of those stimuli when encountered during subsequent wake. Performance on the weather prediction task (WPT), a well-studied measure of implicit probabilistic learning, has been shown to improve significantly following a night of sleep, with stronger initial learning predicting more nocturnal REM sleep. We investigated this relationship further, studying the effect on WPT performance of a daytime nap containing REM sleep. We also added an interference condition after the nap/wake period as an additional probe of memory strength. Our results show that a nap significantly boosts WPT performance, and that this improvement is correlated with the amount of REM sleep obtained during the nap. When interference training is introduced following the nap, however, this REM-sleep benefit vanishes. In contrast, following an equal period of wake, performance is both unchanged from training and unaffected by interference training. Thus, while the true probabilistic relationships between WPT stimuli are strengthened by sleep, these changes are selectively susceptible to the destructive effects of retroactive interference, at least in the short term. PMID:25769506

Functional specialization within the striatum along both the dorsal/ventral and anterior/posterior axes during associative learning via reward and punishment

PubMed Central

Mattfeld, Aaron T.; Gluck, Mark A.; Stark, Craig E.L.

2011-01-01

The goal of the present study was to elucidate the role of the human striatum in learning via reward and punishment during an associative learning task. Previous studies have identified the striatum as a critical component in the neural circuitry of reward-related learning. It remains unclear, however, under what task conditions, and to what extent, the striatum is modulated by punishment during an instrumental learning task. Using high-resolution functional magnetic resonance imaging (fMRI) during a reward- and punishment-based probabilistic associative learning task, we observed activity in the ventral putamen for stimuli learned via reward regardless of whether participants were correct or incorrect (i.e., outcome). In contrast, activity in the dorsal caudate was modulated by trials that received feedback—either correct reward or incorrect punishment trials. We also identified an anterior/posterior dissociation reflecting reward and punishment prediction error estimates. Additionally, differences in patterns of activity that correlated with the amount of training were identified along the anterior/posterior axis of the striatum. We suggest that unique subregions of the striatum—separated along both a dorsal/ventral and anterior/posterior axis— differentially participate in the learning of associations through reward and punishment. PMID:22021252

Paired-Associate and Feedback-Based Weather Prediction Tasks Support Multiple Category Learning Systems.

PubMed

Li, Kaiyun; Fu, Qiufang; Sun, Xunwei; Zhou, Xiaoyan; Fu, Xiaolan

2016-01-01

It remains unclear whether probabilistic category learning in the feedback-based weather prediction task (FB-WPT) can be mediated by a non-declarative or procedural learning system. To address this issue, we compared the effects of training time and verbal working memory, which influence the declarative learning system but not the non-declarative learning system, in the FB and paired-associate (PA) WPTs, as the PA task recruits a declarative learning system. The results of Experiment 1 showed that the optimal accuracy in the PA condition was significantly decreased when the training time was reduced from 7 to 3 s, but this did not occur in the FB condition, although shortened training time impaired the acquisition of explicit knowledge in both conditions. The results of Experiment 2 showed that the concurrent working memory task impaired the optimal accuracy and the acquisition of explicit knowledge in the PA condition but did not influence the optimal accuracy or the acquisition of self-insight knowledge in the FB condition. The apparent dissociation results between the FB and PA conditions suggested that a non-declarative or procedural learning system is involved in the FB-WPT and provided new evidence for the multiple-systems theory of human category learning.

Paired-Associate and Feedback-Based Weather Prediction Tasks Support Multiple Category Learning Systems

PubMed Central

Li, Kaiyun; Fu, Qiufang; Sun, Xunwei; Zhou, Xiaoyan; Fu, Xiaolan

2016-01-01

It remains unclear whether probabilistic category learning in the feedback-based weather prediction task (FB-WPT) can be mediated by a non-declarative or procedural learning system. To address this issue, we compared the effects of training time and verbal working memory, which influence the declarative learning system but not the non-declarative learning system, in the FB and paired-associate (PA) WPTs, as the PA task recruits a declarative learning system. The results of Experiment 1 showed that the optimal accuracy in the PA condition was significantly decreased when the training time was reduced from 7 to 3 s, but this did not occur in the FB condition, although shortened training time impaired the acquisition of explicit knowledge in both conditions. The results of Experiment 2 showed that the concurrent working memory task impaired the optimal accuracy and the acquisition of explicit knowledge in the PA condition but did not influence the optimal accuracy or the acquisition of self-insight knowledge in the FB condition. The apparent dissociation results between the FB and PA conditions suggested that a non-declarative or procedural learning system is involved in the FB-WPT and provided new evidence for the multiple-systems theory of human category learning. PMID:27445958

HIV+ Men and Women Show Different Performance Patterns on Procedural Learning Tasks

PubMed Central

Martin, Eileen; Gonzalez, Raul; Vassileva, Jasmin; Maki, Pauline

2010-01-01

The literature suggests that nondeclarative, or nonconscious, learning might be impaired among HIV+ individuals compared with HIV− matched control groups, but these studies have included relatively few women. We administered measures of motor skill and probabilistic learning, tasks with a nondeclarative or procedural learning component that are dependent on integrity of prefrontal-striatal systems, to well-matched groups of 148 men and 65 women with a history of substance dependence that included 45 men and 30 women seropositive for HIV. All participants were abstinent at testing. Compared to HIV− women, HIV+ women performed significantly more poorly on both tasks, but HIV+ men’s performance did not differ significantly compared to HIV− men on either task. These different patterns of performance indicate that features of HIV-associated neurocognitive disorder (HAND) can not always be generalized from men to women. Additional studies are needed to address directly the possibility of sex differences in HIV-associated neurocognitive disorder (HAND) and the possibility that women might be more vulnerable to the effects of HIV and substance dependence on some neurocognitive functions. PMID:20694870

Using Deep Learning for Compound Selectivity Prediction.

PubMed

Zhang, Ruisheng; Li, Juan; Lu, Jingjing; Hu, Rongjing; Yuan, Yongna; Zhao, Zhili

2016-01-01

Compound selectivity prediction plays an important role in identifying potential compounds that bind to the target of interest with high affinity. However, there is still short of efficient and accurate computational approaches to analyze and predict compound selectivity. In this paper, we propose two methods to improve the compound selectivity prediction. We employ an improved multitask learning method in Neural Networks (NNs), which not only incorporates both activity and selectivity for other targets, but also uses a probabilistic classifier with a logistic regression. We further improve the compound selectivity prediction by using the multitask learning method in Deep Belief Networks (DBNs) which can build a distributed representation model and improve the generalization of the shared tasks. In addition, we assign different weights to the auxiliary tasks that are related to the primary selectivity prediction task. In contrast to other related work, our methods greatly improve the accuracy of the compound selectivity prediction, in particular, using the multitask learning in DBNs with modified weights obtains the best performance.

Probabilistic Reward- and Punishment-based Learning in Opioid Addiction: Experimental and Computational Data

PubMed Central

Myers, Catherine E.; Sheynin, Jony; Baldson, Tarryn; Luzardo, Andre; Beck, Kevin D.; Hogarth, Lee; Haber, Paul; Moustafa, Ahmed A.

2016-01-01

Addiction is the continuation of a habit in spite of negative consequences. A vast literature gives evidence that this poor decision-making behavior in individuals addicted to drugs also generalizes to laboratory decision making tasks, suggesting that the impairment in decision-making is not limited to decisions about taking drugs. In the current experiment, opioid-addicted individuals and matched controls with no history of illicit drug use were administered a probabilistic classification task that embeds both reward-based and punishment-based learning trials, and a computational model of decision making was applied to understand the mechanisms describing individuals’ performance on the task. Although behavioral results showed thatopioid-addicted individuals performed as well as controls on both reward- and punishment-based learning, the modeling results suggested subtle differences in how decisions were made between the two groups. Specifically, the opioid-addicted group showed decreased tendency to repeat prior responses, meaning that they were more likely to “chase reward” when expectancies were violated, whereas controls were more likely to stick with a previously-successful response rule, despite occasional expectancy violations. This tendency to chase short-term reward, potentially at the expense of developing rules that maximize reward over the long term, may be a contributing factor to opioid addiction. Further work is indicated to better understand whether this tendency arises as a result of brain changes in the wake of continued opioid use/abuse, or might be a pre-existing factor that may contribute to risk for addiction. PMID:26381438

Intact implicit statistical learning in borderline personality disorder.

PubMed

Unoka, Zsolt; Vizin, Gabriella; Bjelik, Anna; Radics, Dóra; Nemeth, Dezso; Janacsek, Karolina

2017-09-01

Wide-spread neuropsychological deficits have been identified in borderline personality disorder (BPD). Previous research found impairments in decision making, declarative memory, working memory and executive functions; however, no studies have focused on implicit learning in BPD yet. The aim of our study was to investigate implicit statistical learning by comparing learning performance of 19 BPD patients and 19 healthy, age-, education- and gender-matched controls on a probabilistic sequence learning task. Moreover, we also tested whether participants retain the acquired knowledge after a delay period. To this end, participants were retested on a shorter version of the same task 24h after the learning phase. We found intact implicit statistical learning as well as retention of the acquired knowledge in this personality disorder. BPD patients seem to be able to extract and represent regularities implicitly, which is in line with the notion that implicit learning is less susceptible to illness compared to the more explicit processes. Copyright © 2017 Elsevier Ireland Ltd. All rights reserved.

Visuo-motor and cognitive procedural learning in children with basal ganglia pathology.

PubMed

Mayor-Dubois, C; Maeder, P; Zesiger, P; Roulet-Perez, E

2010-06-01

We investigated procedural learning in 18 children with basal ganglia (BG) lesions or dysfunctions of various aetiologies, using a visuo-motor learning test, the Serial Reaction Time (SRT) task, and a cognitive learning test, the Probabilistic Classification Learning (PCL) task. We compared patients with early (<1 year old, n=9), later onset (>6 years old, n=7) or progressive disorder (idiopathic dystonia, n=2). All patients showed deficits in both visuo-motor and cognitive domains, except those with idiopathic dystonia, who displayed preserved classification learning skills. Impairments seem to be independent from the age of onset of pathology. As far as we know, this study is the first to investigate motor and cognitive procedural learning in children with BG damage. Procedural impairments were documented whatever the aetiology of the BG damage/dysfunction and time of pathology onset, thus supporting the claim of very early skill learning development and lack of plasticity in case of damage. Copyright 2010 Elsevier Ltd. All rights reserved.

Learning to Obtain Reward, but Not Avoid Punishment, Is Affected by Presence of PTSD Symptoms in Male Veterans: Empirical Data and Computational Model

DTIC Science & Technology

2013-08-27

University of New Jersey, Newark, New Jersey, United States of America, 3 Department of Psychology , Rutgers, The State University of New Jersey...United States of America, 5 Marcs Institute for Brain and Behaviour & School of Social Sciences and Psychology , University of Western Sydney, Sydney...for current, severe PTSD symptoms (PTSS) were tested on a probabilistic classification task [19] that interleaves reward learning and punishment

Learning to Estimate Dynamical State with Probabilistic Population Codes.

PubMed

Makin, Joseph G; Dichter, Benjamin K; Sabes, Philip N

2015-11-01

Tracking moving objects, including one's own body, is a fundamental ability of higher organisms, playing a central role in many perceptual and motor tasks. While it is unknown how the brain learns to follow and predict the dynamics of objects, it is known that this process of state estimation can be learned purely from the statistics of noisy observations. When the dynamics are simply linear with additive Gaussian noise, the optimal solution is the well known Kalman filter (KF), the parameters of which can be learned via latent-variable density estimation (the EM algorithm). The brain does not, however, directly manipulate matrices and vectors, but instead appears to represent probability distributions with the firing rates of population of neurons, "probabilistic population codes." We show that a recurrent neural network-a modified form of an exponential family harmonium (EFH)-that takes a linear probabilistic population code as input can learn, without supervision, to estimate the state of a linear dynamical system. After observing a series of population responses (spike counts) to the position of a moving object, the network learns to represent the velocity of the object and forms nearly optimal predictions about the position at the next time-step. This result builds on our previous work showing that a similar network can learn to perform multisensory integration and coordinate transformations for static stimuli. The receptive fields of the trained network also make qualitative predictions about the developing and learning brain: tuning gradually emerges for higher-order dynamical states not explicitly present in the inputs, appearing as delayed tuning for the lower-order states.

Learning to Estimate Dynamical State with Probabilistic Population Codes

PubMed Central

Sabes, Philip N.

2015-01-01

Tracking moving objects, including one’s own body, is a fundamental ability of higher organisms, playing a central role in many perceptual and motor tasks. While it is unknown how the brain learns to follow and predict the dynamics of objects, it is known that this process of state estimation can be learned purely from the statistics of noisy observations. When the dynamics are simply linear with additive Gaussian noise, the optimal solution is the well known Kalman filter (KF), the parameters of which can be learned via latent-variable density estimation (the EM algorithm). The brain does not, however, directly manipulate matrices and vectors, but instead appears to represent probability distributions with the firing rates of population of neurons, “probabilistic population codes.” We show that a recurrent neural network—a modified form of an exponential family harmonium (EFH)—that takes a linear probabilistic population code as input can learn, without supervision, to estimate the state of a linear dynamical system. After observing a series of population responses (spike counts) to the position of a moving object, the network learns to represent the velocity of the object and forms nearly optimal predictions about the position at the next time-step. This result builds on our previous work showing that a similar network can learn to perform multisensory integration and coordinate transformations for static stimuli. The receptive fields of the trained network also make qualitative predictions about the developing and learning brain: tuning gradually emerges for higher-order dynamical states not explicitly present in the inputs, appearing as delayed tuning for the lower-order states. PMID:26540152

Games people play: How video games improve probabilistic learning.

PubMed

Schenk, Sabrina; Lech, Robert K; Suchan, Boris

2017-09-29

Recent research suggests that video game playing is associated with many cognitive benefits. However, little is known about the neural mechanisms mediating such effects, especially with regard to probabilistic categorization learning, which is a widely unexplored area in gaming research. Therefore, the present study aimed to investigate the neural correlates of probabilistic classification learning in video gamers in comparison to non-gamers. Subjects were scanned in a 3T magnetic resonance imaging (MRI) scanner while performing a modified version of the weather prediction task. Behavioral data yielded evidence for better categorization performance of video gamers, particularly under conditions characterized by stronger uncertainty. Furthermore, a post-experimental questionnaire showed that video gamers had acquired higher declarative knowledge about the card combinations and the related weather outcomes. Functional imaging data revealed for video gamers stronger activation clusters in the hippocampus, the precuneus, the cingulate gyrus and the middle temporal gyrus as well as in occipital visual areas and in areas related to attentional processes. All these areas are connected with each other and represent critical nodes for semantic memory, visual imagery and cognitive control. Apart from this, and in line with previous studies, both groups showed activation in brain areas that are related to attention and executive functions as well as in the basal ganglia and in memory-associated regions of the medial temporal lobe. These results suggest that playing video games might enhance the usage of declarative knowledge as well as hippocampal involvement and enhances overall learning performance during probabilistic learning. In contrast to non-gamers, video gamers showed better categorization performance, independently of the uncertainty of the condition. Copyright © 2017 Elsevier B.V. All rights reserved.

Behavioral and Electrophysiological Alterations for Reinforcement Learning in Manic and Euthymic Patients with Bipolar Disorder.

PubMed

Ryu, Vin; Ha, Ra Yeon; Lee, Su Jin; Ha, Kyooseob; Cho, Hyun-Sang

2017-03-01

Bipolar disorder is characterized by behavioral changes such as risk-taking and increasing goal-directed activities, which may result from altered reward processing. Patients with bipolar disorder show impaired reward learning in situations that require the integration of reinforced feedback over time. In this study, we examined the behavioral and electrophysiological characteristics of reward learning in manic and euthymic patients with bipolar disorder using a probabilistic reward task. Twenty-four manic and 20 euthymic patients with bipolar I disorder and 24 healthy control subjects performed the probabilistic reward task. We assessed response bias (RB) as a preference for the stimulus paired with the more frequent reward and feedback-related negativity (FRN) to correct identification of the rich stimulus. Both manic and euthymic patients showed significantly lower RB scores in the early learning stage (block 1) in comparison with the late learning stage (block 2 or block 3) of the task, as well as significantly lower RB scores in the early stage compared to healthy subjects. Relatively more negative FRN amplitude is elicited by no presentation of an expected reward, compared to that elicited by presentation of expected feedback. The FRN became significantly more negative from the early (block 1) to the later stages (blocks 2 and 3) in both manic and euthymic patients, but not in healthy subjects. Changes in RB scores and FRN amplitudes between blocks 2 and 3 and block 1 correlated positively in healthy controls, but correlated negatively in manic and euthymic patients. The severity of manic symptoms correlated positively with reward learning scores and negatively with the FRN. These findings suggest that patients with bipolar disorder during euthymic or manic states have behavioral and electrophysiological alterations in reward learning compared to healthy subjects. This dysfunctional reward processing may be related to the abnormal decision-making or altered goal-directed activities frequently seen in patients with bipolar disorder. © 2017 John Wiley & Sons Ltd.

An fMRI investigation of the fronto-striatal learning system in women who exhibit eating disorder behaviors

PubMed Central

Celone, Kim A.; Thompson-Brenner, Heather; Ross, Robert S.; Pratt, Elizabeth M.; Stern, Chantal E.

2013-01-01

In the present study, we sought to examine whether the fronto-striatal learning system, which has been implicated in bulimia nervosa, would demonstrate altered BOLD activity during probabilistic category learning in women who met subthreshold criteria for bulimia nervosa (Sub-BN). Sub-BN, which falls within the clinical category of Eating Disorder Not Otherwise Specified (EDNOS), is comprised of individuals who demonstrate recurrent binge eating, efforts to minimize their caloric intake and caloric retention, and elevated levels of concern about shape, weight, and/or eating, but just fail to meet the diagnostic threshold for bulimia nervosa (BN). fMRI data were collected from eighteen women with subthreshold-BN (Sub-BN) and nineteen healthy control women group-matched for age, education and body mass index (MC) during the weather prediction task. Sub-BN participants demonstrated increased caudate nucleus and dorsolateral prefrontal cortex (DLPFC) activation during the learning of probabilistic categories. Though the two subject groups did not differ in behavioral performance, over the course of learning, Sub-BN participants showed a dynamic pattern of brain activity differences when compared to matched control participants. Regions implicated in episodic memory, including the medial temporal lobe (MTL), retrosplenial cortex, middle frontal gyrus, and anterior and posterior cingulate cortex showed decreased activity in the Sub-BN participants compared to MCs during early learning which was followed by increased involvement of the DLPFC during later learning. These findings demonstrate that women with Sub-BN demonstrate differences in fronto-striatal learning system activity, as well as a distinct functional pattern between fronto-striatal and MTL learning systems during the course of implicit probabilistic category learning. PMID:21419229

Cognitive Inflexibility in Gamblers is Primarily Present in Reward-Related Decision Making

PubMed Central

Boog, Michiel; Höppener, Paul; v. d. Wetering, Ben J. M.; Goudriaan, Anna E.; Boog, Matthijs C.; Franken, Ingmar H. A.

2014-01-01

One hallmark of gambling disorder (GD) is the observation that gamblers have problems stopping their gambling behavior once it is initiated. On a neuropsychological level, it has been hypothesized that this is the result of a cognitive inflexibility. The present study investigated cognitive inflexibility in patients with GD using a task involving cognitive inflexibility with a reward element (i.e., reversal learning) and a task measuring general cognitive inflexibility without such a component (i.e., response perseveration). For this purpose, scores of a reward-based reversal learning task (probabilistic reversal learning task) and the Wisconsin card sorting task were compared between a group of treatment seeking patients with GD and a gender and age matched control group. The results show that pathological gamblers have impaired performance on the neurocognitive task measuring reward-based cognitive inflexibility. However, no difference between the groups is observed regarding non-reward-based cognitive inflexibility. This suggests that cognitive inflexibility in GD is the result of an aberrant reward-based learning, and not based on a more general problem with cognitive flexibility. The pattern of observed problems is suggestive of a dysfunction of the orbitofrontal cortex, the ventrolateral prefrontal cortex, and the ventral regions of the striatum in gamblers. Relevance for the neurocognition of problematic gambling is discussed. PMID:25165438

What Can Student Work Show? From Playing a Game to Exploring Probability Theory

ERIC Educational Resources Information Center

Taylor, Merilyn; Hawera, Ngarewa

2016-01-01

Rich learning tasks embedded within a familiar context allow students to work like mathematicians while making sense of the mathematics. This article demonstrates how 11-12 year-old students were able to employ all of the proficiency strands while demonstrating a deep understanding of some of the "big ideas" of probabilistic thinking.

Gaussian Processes for Data-Efficient Learning in Robotics and Control.

PubMed

Deisenroth, Marc Peter; Fox, Dieter; Rasmussen, Carl Edward

2015-02-01

Autonomous learning has been a promising direction in control and robotics for more than a decade since data-driven learning allows to reduce the amount of engineering knowledge, which is otherwise required. However, autonomous reinforcement learning (RL) approaches typically require many interactions with the system to learn controllers, which is a practical limitation in real systems, such as robots, where many interactions can be impractical and time consuming. To address this problem, current learning approaches typically require task-specific knowledge in form of expert demonstrations, realistic simulators, pre-shaped policies, or specific knowledge about the underlying dynamics. In this paper, we follow a different approach and speed up learning by extracting more information from data. In particular, we learn a probabilistic, non-parametric Gaussian process transition model of the system. By explicitly incorporating model uncertainty into long-term planning and controller learning our approach reduces the effects of model errors, a key problem in model-based learning. Compared to state-of-the art RL our model-based policy search method achieves an unprecedented speed of learning. We demonstrate its applicability to autonomous learning in real robot and control tasks.

Diagnosis of students' ability in a statistical course based on Rasch probabilistic outcome

NASA Astrophysics Data System (ADS)

Mahmud, Zamalia; Ramli, Wan Syahira Wan; Sapri, Shamsiah; Ahmad, Sanizah

2017-06-01

Measuring students' ability and performance are important in assessing how well students have learned and mastered the statistical courses. Any improvement in learning will depend on the student's approaches to learning, which are relevant to some factors of learning, namely assessment methods carrying out tasks consisting of quizzes, tests, assignment and final examination. This study has attempted an alternative approach to measure students' ability in an undergraduate statistical course based on the Rasch probabilistic model. Firstly, this study aims to explore the learning outcome patterns of students in a statistics course (Applied Probability and Statistics) based on an Entrance-Exit survey. This is followed by investigating students' perceived learning ability based on four Course Learning Outcomes (CLOs) and students' actual learning ability based on their final examination scores. Rasch analysis revealed that students perceived themselves as lacking the ability to understand about 95% of the statistics concepts at the beginning of the class but eventually they had a good understanding at the end of the 14 weeks class. In terms of students' performance in their final examination, their ability in understanding the topics varies at different probability values given the ability of the students and difficulty of the questions. Majority found the probability and counting rules topic to be the most difficult to learn.

The influence of trial order on learning from reward vs. punishment in a probabilistic categorization task: experimental and computational analyses

PubMed Central

Moustafa, Ahmed A.; Gluck, Mark A.; Herzallah, Mohammad M.; Myers, Catherine E.

2015-01-01

Previous research has shown that trial ordering affects cognitive performance, but this has not been tested using category-learning tasks that differentiate learning from reward and punishment. Here, we tested two groups of healthy young adults using a probabilistic category learning task of reward and punishment in which there are two types of trials (reward, punishment) and three possible outcomes: (1) positive feedback for correct responses in reward trials; (2) negative feedback for incorrect responses in punishment trials; and (3) no feedback for incorrect answers in reward trials and correct answers in punishment trials. Hence, trials without feedback are ambiguous, and may represent either successful avoidance of punishment or failure to obtain reward. In Experiment 1, the first group of subjects received an intermixed task in which reward and punishment trials were presented in the same block, as a standard baseline task. In Experiment 2, a second group completed the separated task, in which reward and punishment trials were presented in separate blocks. Additionally, in order to understand the mechanisms underlying performance in the experimental conditions, we fit individual data using a Q-learning model. Results from Experiment 1 show that subjects who completed the intermixed task paradoxically valued the no-feedback outcome as a reinforcer when it occurred on reinforcement-based trials, and as a punisher when it occurred on punishment-based trials. This is supported by patterns of empirical responding, where subjects showed more win-stay behavior following an explicit reward than following an omission of punishment, and more lose-shift behavior following an explicit punisher than following an omission of reward. In Experiment 2, results showed similar performance whether subjects received reward-based or punishment-based trials first. However, when the Q-learning model was applied to these data, there were differences between subjects in the reward-first and punishment-first conditions on the relative weighting of neutral feedback. Specifically, early training on reward-based trials led to omission of reward being treated as similar to punishment, but prior training on punishment-based trials led to omission of reward being treated more neutrally. This suggests that early training on one type of trials, specifically reward-based trials, can create a bias in how neutral feedback is processed, relative to those receiving early punishment-based training or training that mixes positive and negative outcomes. PMID:26257616

Relative risk of probabilistic category learning deficits in patients with schizophrenia and their siblings

PubMed Central

Weickert, Thomas W.; Goldberg, Terry E.; Egan, Michael F.; Apud, Jose A.; Meeter, Martijn; Myers, Catherine E.; Gluck, Mark A; Weinberger, Daniel R.

2010-01-01

Background While patients with schizophrenia display an overall probabilistic category learning performance deficit, the extent to which this deficit occurs in unaffected siblings of patients with schizophrenia is unknown. There are also discrepant findings regarding probabilistic category learning acquisition rate and performance in patients with schizophrenia. Methods A probabilistic category learning test was administered to 108 patients with schizophrenia, 82 unaffected siblings, and 121 healthy participants. Results Patients with schizophrenia displayed significant differences from their unaffected siblings and healthy participants with respect to probabilistic category learning acquisition rates. Although siblings on the whole failed to differ from healthy participants on strategy and quantitative indices of overall performance and learning acquisition, application of a revised learning criterion enabling classification into good and poor learners based on individual learning curves revealed significant differences between percentages of sibling and healthy poor learners: healthy (13.2%), siblings (34.1%), patients (48.1%), yielding a moderate relative risk. Conclusions These results clarify previous discrepant findings pertaining to probabilistic category learning acquisition rate in schizophrenia and provide the first evidence for the relative risk of probabilistic category learning abnormalities in unaffected siblings of patients with schizophrenia, supporting genetic underpinnings of probabilistic category learning deficits in schizophrenia. These findings also raise questions regarding the contribution of antipsychotic medication to the probabilistic category learning deficit in schizophrenia. The distinction between good and poor learning may be used to inform genetic studies designed to detect schizophrenia risk alleles. PMID:20172502

Motivation and value influences in the relative balance of goal-directed and habitual behaviours in obsessive-compulsive disorder.

PubMed

Voon, V; Baek, K; Enander, J; Worbe, Y; Morris, L S; Harrison, N A; Robbins, T W; Rück, C; Daw, N

2015-11-03

Our decisions are based on parallel and competing systems of goal-directed and habitual learning, systems which can be impaired in pathological behaviours. Here we focus on the influence of motivation and compare reward and loss outcomes in subjects with obsessive-compulsive disorder (OCD) on model-based goal-directed and model-free habitual behaviours using the two-step task. We further investigate the relationship with acquisition learning using a one-step probabilistic learning task. Forty-eight OCD subjects and 96 healthy volunteers were tested on a reward and 30 OCD subjects and 53 healthy volunteers on the loss version of the two-step task. Thirty-six OCD subjects and 72 healthy volunteers were also tested on a one-step reversal task. OCD subjects compared with healthy volunteers were less goal oriented (model-based) and more habitual (model-free) to reward outcomes with a shift towards greater model-based and lower habitual choices to loss outcomes. OCD subjects also had enhanced acquisition learning to loss outcomes on the one-step task, which correlated with goal-directed learning in the two-step task. OCD subjects had greater stay behaviours or perseveration in the one-step task irrespective of outcome. Compulsion severity was correlated with habitual learning in the reward condition. Obsession severity was correlated with greater switching after loss outcomes. In healthy volunteers, we further show that greater reward magnitudes are associated with a shift towards greater goal-directed learning further emphasizing the role of outcome salience. Our results highlight an important influence of motivation on learning processes in OCD and suggest that distinct clinical strategies based on valence may be warranted.

Learning Sparse Feature Representations using Probabilistic Quadtrees and Deep Belief Nets

DTIC Science & Technology

2015-04-24

Feature Representations usingProbabilistic Quadtrees and Deep Belief Nets Learning sparse feature representations is a useful instru- ment for solving an...novel framework for the classifi cation of handwritten digits that learns sparse representations using probabilistic quadtrees and Deep Belief Nets... Learning Sparse Feature Representations usingProbabilistic Quadtrees and Deep Belief Nets Report Title Learning sparse feature representations is a useful

What is in the feedback? Effect of induced happiness vs. sadness on probabilistic learning with vs. without exploration.

PubMed

Bakic, Jasmina; De Raedt, Rudi; Jepma, Marieke; Pourtois, Gilles

2015-01-01

According to dominant neuropsychological theories of affect, emotions signal salience of events and in turn facilitate a wide spectrum of response options or action tendencies. Valence of an emotional experience is pivotal here, as it alters reward and punishment processing, as well as the balance between safety and risk taking, which can be translated into changes in the exploration-exploitation trade-off during reinforcement learning (RL). To test this idea, we compared the behavioral performance of three groups of participants that all completed a variant of a standard probabilistic learning task, but who differed regarding which mood state was actually induced and maintained (happy, sad or neutral). To foster a change from an exploration to an exploitation-based mode, we removed feedback information once learning was reliably established. Although changes in mood were successful, learning performance was balanced between the three groups. Critically, when focusing on exploitation-driven learning only, they did not differ either. Moreover, mood valence did not alter the learning rate or exploration per se, when titrated using complementing computational modeling. By comparing systematically these results to our previous study (Bakic et al., 2014), we found that arousal levels did differ between studies, which might account for limited modulatory effects of (positive) mood on RL in the present case. These results challenge the assumption that mood valence alone is enough to create strong shifts in the way exploitation or exploration is eventually carried out during (probabilistic) learning. In this context, we discuss the possibility that both valence and arousal are actually necessary components of the emotional mood state to yield changes in the use and exploration of incentives cues during RL.

Developmental Change in Feedback Processing as Reflected by Phasic Heart Rate Changes

ERIC Educational Resources Information Center

Crone, Eveline A.; Jennings, J. Richard; Van der Molen, Maurits W.

2004-01-01

Heart rate was recorded from 3 age groups (8-10, 12, and 20-26 years) while they performed a probabilistic learning task. Stimuli had to be sorted by pressing a left versus right key, followed by positive or negative feedback. Adult heart rate slowed following negative feedback when stimuli were consistently mapped onto the left or right key…

Think twice: Impulsivity and decision making in obsessive-compulsive disorder.

PubMed

Grassi, Giacomo; Pallanti, Stefano; Righi, Lorenzo; Figee, Martijn; Mantione, Mariska; Denys, Damiaan; Piccagliani, Daniele; Rossi, Alessandro; Stratta, Paolo

2015-12-01

Recent studies have challenged the anxiety-avoidance model of obsessive-compulsive disorder (OCD), linking OCD to impulsivity, risky-decision-making and reward-system dysfunction, which can also be found in addiction and might support the conceptualization of OCD as a behavioral addiction. Here, we conducted an exploratory investigation of the behavioral addiction model of OCD by assessing whether OCD patients are more impulsive, have impaired decision-making, and biased probabilistic reasoning, three core dimensions of addiction, in a sample of OCD patients and healthy controls. We assessed these dimensions on 38 OCD patients and 39 healthy controls with the Barratt Impulsiveness Scale (BIS-11), the Iowa Gambling Task (IGT) and the Beads Task. OCD patients had significantly higher BIS-11 scores than controls, in particular on the cognitive subscales. They performed significantly worse than controls on the IGT preferring immediate reward despite negative future consequences, and did not learn from losses. Finally, OCD patients demonstrated biased probabilistic reasoning as reflected by significantly fewer draws to decision than controls on the Beads Task. OCD patients are more impulsive than controls and demonstrate risky decision-making and biased probabilistic reasoning. These results might suggest that other conceptualizations of OCD, such as the behavioral addiction model, may be more suitable than the anxiety-avoidance one. However, further studies directly comparing OCD and behavioral addiction patients are needed in order to scrutinize this model.

Probabilistic reward- and punishment-based learning in opioid addiction: Experimental and computational data.

PubMed

Myers, Catherine E; Sheynin, Jony; Balsdon, Tarryn; Luzardo, Andre; Beck, Kevin D; Hogarth, Lee; Haber, Paul; Moustafa, Ahmed A

2016-01-01

Addiction is the continuation of a habit in spite of negative consequences. A vast literature gives evidence that this poor decision-making behavior in individuals addicted to drugs also generalizes to laboratory decision making tasks, suggesting that the impairment in decision-making is not limited to decisions about taking drugs. In the current experiment, opioid-addicted individuals and matched controls with no history of illicit drug use were administered a probabilistic classification task that embeds both reward-based and punishment-based learning trials, and a computational model of decision making was applied to understand the mechanisms describing individuals' performance on the task. Although behavioral results showed that opioid-addicted individuals performed as well as controls on both reward- and punishment-based learning, the modeling results suggested subtle differences in how decisions were made between the two groups. Specifically, the opioid-addicted group showed decreased tendency to repeat prior responses, meaning that they were more likely to "chase reward" when expectancies were violated, whereas controls were more likely to stick with a previously-successful response rule, despite occasional expectancy violations. This tendency to chase short-term reward, potentially at the expense of developing rules that maximize reward over the long term, may be a contributing factor to opioid addiction. Further work is indicated to better understand whether this tendency arises as a result of brain changes in the wake of continued opioid use/abuse, or might be a pre-existing factor that may contribute to risk for addiction. Copyright © 2015 Elsevier B.V. All rights reserved.

Learning Semantics of Gestural Instructions for Human-Robot Collaboration

PubMed Central

Shukla, Dadhichi; Erkent, Özgür; Piater, Justus

2018-01-01

Designed to work safely alongside humans, collaborative robots need to be capable partners in human-robot teams. Besides having key capabilities like detecting gestures, recognizing objects, grasping them, and handing them over, these robots need to seamlessly adapt their behavior for efficient human-robot collaboration. In this context we present the fast, supervised Proactive Incremental Learning (PIL) framework for learning associations between human hand gestures and the intended robotic manipulation actions. With the proactive aspect, the robot is competent to predict the human's intent and perform an action without waiting for an instruction. The incremental aspect enables the robot to learn associations on the fly while performing a task. It is a probabilistic, statistically-driven approach. As a proof of concept, we focus on a table assembly task where the robot assists its human partner. We investigate how the accuracy of gesture detection affects the number of interactions required to complete the task. We also conducted a human-robot interaction study with non-roboticist users comparing a proactive with a reactive robot that waits for instructions. PMID:29615888

Learning Semantics of Gestural Instructions for Human-Robot Collaboration.

PubMed

Shukla, Dadhichi; Erkent, Özgür; Piater, Justus

2018-01-01

Designed to work safely alongside humans, collaborative robots need to be capable partners in human-robot teams. Besides having key capabilities like detecting gestures, recognizing objects, grasping them, and handing them over, these robots need to seamlessly adapt their behavior for efficient human-robot collaboration. In this context we present the fast, supervised Proactive Incremental Learning (PIL) framework for learning associations between human hand gestures and the intended robotic manipulation actions. With the proactive aspect, the robot is competent to predict the human's intent and perform an action without waiting for an instruction. The incremental aspect enables the robot to learn associations on the fly while performing a task. It is a probabilistic, statistically-driven approach. As a proof of concept, we focus on a table assembly task where the robot assists its human partner. We investigate how the accuracy of gesture detection affects the number of interactions required to complete the task. We also conducted a human-robot interaction study with non-roboticist users comparing a proactive with a reactive robot that waits for instructions.

Learning to choose: Cognitive aging and strategy selection learning in decision making.

PubMed

Mata, Rui; von Helversen, Bettina; Rieskamp, Jörg

2010-06-01

Decision makers often have to learn from experience. In these situations, people must use the available feedback to select the appropriate decision strategy. How does the ability to select decision strategies on the basis of experience change with age? We examined younger and older adults' strategy selection learning in a probabilistic inference task using a computational model of strategy selection learning. Older adults showed poorer decision performance compared with younger adults. In particular, older adults performed poorly in an environment favoring the use of a more cognitively demanding strategy. The results suggest that the impact of cognitive aging on strategy selection learning depends on the structure of the decision environment. (c) 2010 APA, all rights reserved

Feedback-Driven Trial-by-Trial Learning in Autism Spectrum Disorders

PubMed Central

Solomon, Marjorie; Frank, Michael J.; Ragland, J. Daniel; Smith, Anne C.; Niendam, Tara A.; Lesh, Tyler A.; Grayson, David S.; Beck, Jonathan S.; Matter, John C.; Carter, Cameron S.

2017-01-01

Objective Impairments in learning are central to autism spectrum disorders. The authors investigated the cognitive and neural basis of these deficits in young adults with autism spectrum disorders using a well-characterized probabilistic reinforcement learning paradigm. Method The probabilistic selection task was implemented among matched participants with autism spectrum disorders (N=22) and with typical development (N=25), aged 18–40 years, using rapid event-related functional MRI. Participants were trained to choose the correct stimulus in high-probability (AB), medium-probability (CD), and low-probability (EF) pairs, presented with valid feedback 80%, 70%, and 60% of the time, respectively. Whole-brain voxel-wise and parametric modulator analyses examined early and late learning during the stimulus and feedback epochs of the task. Results The groups exhibited comparable performance on medium- and low-probability pairs. Typically developing persons showed higher accuracy on the high-probability pair, better win-stay performance (selection of the previously rewarded stimulus on the next trial of that type), and more robust recruitment of the anterior and medial prefrontal cortex during the stimulus epoch, suggesting development of an intact reward-based working memory for recent stimulus values. Throughout the feedback epoch, individuals with autism spectrum disorders exhibited greater recruitment of the anterior cingulate and orbito-frontal cortices compared with individuals with typical development, indicating continuing trial-by-trial activity related to feedback processing. Conclusions Individuals with autism spectrum disorders exhibit learning deficits reflecting impaired ability to develop an effective reward-based working memory to guide stimulus selection. Instead, they continue to rely on trial-by-trial feedback processing to support learning dependent upon engagement of the anterior cingulate and orbito-frontal cortices. PMID:25158242

Probabilistic Learning in Junior High School: Investigation of Student Probabilistic Thinking Levels

NASA Astrophysics Data System (ADS)

Kurniasih, R.; Sujadi, I.

2017-09-01

This paper was to investigate level on students’ probabilistic thinking. Probabilistic thinking level is level of probabilistic thinking. Probabilistic thinking is thinking about probabilistic or uncertainty matter in probability material. The research’s subject was students in grade 8th Junior High School students. The main instrument is a researcher and a supporting instrument is probabilistic thinking skills test and interview guidelines. Data was analyzed using triangulation method. The results showed that the level of students probabilistic thinking before obtaining a teaching opportunity at the level of subjective and transitional. After the students’ learning level probabilistic thinking is changing. Based on the results of research there are some students who have in 8th grade level probabilistic thinking numerically highest of levels. Level of students’ probabilistic thinking can be used as a reference to make a learning material and strategy.

Learning Probabilistic Logic Models from Probabilistic Examples

PubMed Central

Chen, Jianzhong; Muggleton, Stephen; Santos, José

2009-01-01

Abstract We revisit an application developed originally using abductive Inductive Logic Programming (ILP) for modeling inhibition in metabolic networks. The example data was derived from studies of the effects of toxins on rats using Nuclear Magnetic Resonance (NMR) time-trace analysis of their biofluids together with background knowledge representing a subset of the Kyoto Encyclopedia of Genes and Genomes (KEGG). We now apply two Probabilistic ILP (PILP) approaches - abductive Stochastic Logic Programs (SLPs) and PRogramming In Statistical modeling (PRISM) to the application. Both approaches support abductive learning and probability predictions. Abductive SLPs are a PILP framework that provides possible worlds semantics to SLPs through abduction. Instead of learning logic models from non-probabilistic examples as done in ILP, the PILP approach applied in this paper is based on a general technique for introducing probability labels within a standard scientific experimental setting involving control and treated data. Our results demonstrate that the PILP approach provides a way of learning probabilistic logic models from probabilistic examples, and the PILP models learned from probabilistic examples lead to a significant decrease in error accompanied by improved insight from the learned results compared with the PILP models learned from non-probabilistic examples. PMID:19888348

Learning Probabilistic Logic Models from Probabilistic Examples.

PubMed

Chen, Jianzhong; Muggleton, Stephen; Santos, José

2008-10-01

We revisit an application developed originally using abductive Inductive Logic Programming (ILP) for modeling inhibition in metabolic networks. The example data was derived from studies of the effects of toxins on rats using Nuclear Magnetic Resonance (NMR) time-trace analysis of their biofluids together with background knowledge representing a subset of the Kyoto Encyclopedia of Genes and Genomes (KEGG). We now apply two Probabilistic ILP (PILP) approaches - abductive Stochastic Logic Programs (SLPs) and PRogramming In Statistical modeling (PRISM) to the application. Both approaches support abductive learning and probability predictions. Abductive SLPs are a PILP framework that provides possible worlds semantics to SLPs through abduction. Instead of learning logic models from non-probabilistic examples as done in ILP, the PILP approach applied in this paper is based on a general technique for introducing probability labels within a standard scientific experimental setting involving control and treated data. Our results demonstrate that the PILP approach provides a way of learning probabilistic logic models from probabilistic examples, and the PILP models learned from probabilistic examples lead to a significant decrease in error accompanied by improved insight from the learned results compared with the PILP models learned from non-probabilistic examples.

Exploration of Advanced Probabilistic and Stochastic Design Methods

NASA Technical Reports Server (NTRS)

Mavris, Dimitri N.

2003-01-01

The primary objective of the three year research effort was to explore advanced, non-deterministic aerospace system design methods that may have relevance to designers and analysts. The research pursued emerging areas in design methodology and leverage current fundamental research in the area of design decision-making, probabilistic modeling, and optimization. The specific focus of the three year investigation was oriented toward methods to identify and analyze emerging aircraft technologies in a consistent and complete manner, and to explore means to make optimal decisions based on this knowledge in a probabilistic environment. The research efforts were classified into two main areas. First, Task A of the grant has had the objective of conducting research into the relative merits of possible approaches that account for both multiple criteria and uncertainty in design decision-making. In particular, in the final year of research, the focus was on the comparison and contrasting between three methods researched. Specifically, these three are the Joint Probabilistic Decision-Making (JPDM) technique, Physical Programming, and Dempster-Shafer (D-S) theory. The next element of the research, as contained in Task B, was focused upon exploration of the Technology Identification, Evaluation, and Selection (TIES) methodology developed at ASDL, especially with regards to identification of research needs in the baseline method through implementation exercises. The end result of Task B was the documentation of the evolution of the method with time and a technology transfer to the sponsor regarding the method, such that an initial capability for execution could be obtained by the sponsor. Specifically, the results of year 3 efforts were the creation of a detailed tutorial for implementing the TIES method. Within the tutorial package, templates and detailed examples were created for learning and understanding the details of each step. For both research tasks, sample files and tutorials are attached in electronic form with the enclosed CD.

Drift diffusion model of reward and punishment learning in rare alpha-synuclein gene carriers.

PubMed

Moustafa, Ahmed A; Kéri, Szabolcs; Polner, Bertalan; White, Corey

To understand the cognitive effects of alpha-synuclein polymorphism, we employed a drift diffusion model (DDM) to analyze reward- and punishment-guided probabilistic learning task data of participants with the rare alpha-synuclein gene duplication and age- and education-matched controls. Overall, the DDM analysis showed that, relative to controls, asymptomatic alpha-synuclein gene duplication carriers had significantly increased learning from negative feedback, while they tended to show impaired learning from positive feedback. No significant differences were found in response caution, response bias, or motor/encoding time. We here discuss the implications of these computational findings to the understanding of the neural mechanism of alpha-synuclein gene duplication.

Optimism as a Prior Belief about the Probability of Future Reward

PubMed Central

Kalra, Aditi; Seriès, Peggy

2014-01-01

Optimists hold positive a priori beliefs about the future. In Bayesian statistical theory, a priori beliefs can be overcome by experience. However, optimistic beliefs can at times appear surprisingly resistant to evidence, suggesting that optimism might also influence how new information is selected and learned. Here, we use a novel Pavlovian conditioning task, embedded in a normative framework, to directly assess how trait optimism, as classically measured using self-report questionnaires, influences choices between visual targets, by learning about their association with reward progresses. We find that trait optimism relates to an a priori belief about the likelihood of rewards, but not losses, in our task. Critically, this positive belief behaves like a probabilistic prior, i.e. its influence reduces with increasing experience. Contrary to findings in the literature related to unrealistic optimism and self-beliefs, it does not appear to influence the iterative learning process directly. PMID:24853098

Linking Individual Learning Styles to Approach-Avoidance Motivational Traits and Computational Aspects of Reinforcement Learning

PubMed Central

Carl Aberg, Kristoffer; Doell, Kimberly C.; Schwartz, Sophie

2016-01-01

Learning how to gain rewards (approach learning) and avoid punishments (avoidance learning) is fundamental for everyday life. While individual differences in approach and avoidance learning styles have been related to genetics and aging, the contribution of personality factors, such as traits, remains undetermined. Moreover, little is known about the computational mechanisms mediating differences in learning styles. Here, we used a probabilistic selection task with positive and negative feedbacks, in combination with computational modelling, to show that individuals displaying better approach (vs. avoidance) learning scored higher on measures of approach (vs. avoidance) trait motivation, but, paradoxically, also displayed reduced learning speed following positive (vs. negative) outcomes. These data suggest that learning different types of information depend on associated reward values and internal motivational drives, possibly determined by personality traits. PMID:27851807

IEA Wind Task 36 Forecasting

NASA Astrophysics Data System (ADS)

Giebel, Gregor; Cline, Joel; Frank, Helmut; Shaw, Will; Pinson, Pierre; Hodge, Bri-Mathias; Kariniotakis, Georges; Sempreviva, Anna Maria; Draxl, Caroline

2017-04-01

Wind power forecasts have been used operatively for over 20 years. Despite this fact, there are still several possibilities to improve the forecasts, both from the weather prediction side and from the usage of the forecasts. The new International Energy Agency (IEA) Task on Wind Power Forecasting tries to organise international collaboration, among national weather centres with an interest and/or large projects on wind forecast improvements (NOAA, DWD, UK MetOffice, …) and operational forecaster and forecast users. The Task is divided in three work packages: Firstly, a collaboration on the improvement of the scientific basis for the wind predictions themselves. This includes numerical weather prediction model physics, but also widely distributed information on accessible datasets for verification. Secondly, we will be aiming at an international pre-standard (an IEA Recommended Practice) on benchmarking and comparing wind power forecasts, including probabilistic forecasts aiming at industry and forecasters alike. This WP will also organise benchmarks, in cooperation with the IEA Task WakeBench. Thirdly, we will be engaging end users aiming at dissemination of the best practice in the usage of wind power predictions, especially probabilistic ones. The Operating Agent is Gregor Giebel of DTU, Co-Operating Agent is Joel Cline of the US Department of Energy. Collaboration in the task is solicited from everyone interested in the forecasting business. We will collaborate with IEA Task 31 Wakebench, which developed the Windbench benchmarking platform, which this task will use for forecasting benchmarks. The task runs for three years, 2016-2018. Main deliverables are an up-to-date list of current projects and main project results, including datasets which can be used by researchers around the world to improve their own models, an IEA Recommended Practice on performance evaluation of probabilistic forecasts, a position paper regarding the use of probabilistic forecasts, and one or more benchmark studies implemented on the Windbench platform hosted at CENER. Additionally, spreading of relevant information in both the forecasters and the users community is paramount. The poster also shows the work done in the first half of the Task, e.g. the collection of available datasets and the learnings from a public workshop on 9 June in Barcelona on Experiences with the Use of Forecasts and Gaps in Research. Participation is open for all interested parties in member states of the IEA Annex on Wind Power, see ieawind.org for the up-to-date list. For collaboration, please contact the author grgi@dtu.dk).

Think twice: Impulsivity and decision making in obsessive–compulsive disorder

PubMed Central

Grassi, Giacomo; Pallanti, Stefano; Righi, Lorenzo; Figee, Martijn; Mantione, Mariska; Denys, Damiaan; Piccagliani, Daniele; Rossi, Alessandro; Stratta, Paolo

2015-01-01

Background and Aims Recent studies have challenged the anxiety-avoidance model of obsessive–compulsive disorder (OCD), linking OCD to impulsivity, risky-decision-making and reward-system dysfunction, which can also be found in addiction and might support the conceptualization of OCD as a behavioral addiction. Here, we conducted an exploratory investigation of the behavioral addiction model of OCD by assessing whether OCD patients are more impulsive, have impaired decision-making, and biased probabilistic reasoning, three core dimensions of addiction, in a sample of OCD patients and healthy controls. Methods We assessed these dimensions on 38 OCD patients and 39 healthy controls with the Barratt Impulsiveness Scale (BIS-11), the Iowa Gambling Task (IGT) and the Beads Task. Results OCD patients had significantly higher BIS-11 scores than controls, in particular on the cognitive subscales. They performed significantly worse than controls on the IGT preferring immediate reward despite negative future consequences, and did not learn from losses. Finally, OCD patients demonstrated biased probabilistic reasoning as reflected by significantly fewer draws to decision than controls on the Beads Task. Conclusions OCD patients are more impulsive than controls and demonstrate risky decision-making and biased probabilistic reasoning. These results might suggest that other conceptualizations of OCD, such as the behavioral addiction model, may be more suitable than the anxiety-avoidance one. However, further studies directly comparing OCD and behavioral addiction patients are needed in order to scrutinize this model. PMID:26690621

Learning Probabilistic Features for Robotic Navigation Using Laser Sensors

PubMed Central

Aznar, Fidel; Pujol, Francisco A.; Pujol, Mar; Rizo, Ramón; Pujol, María-José

2014-01-01

SLAM is a popular task used by robots and autonomous vehicles to build a map of an unknown environment and, at the same time, to determine their location within the map. This paper describes a SLAM-based, probabilistic robotic system able to learn the essential features of different parts of its environment. Some previous SLAM implementations had computational complexities ranging from O(Nlog(N)) to O(N 2), where N is the number of map features. Unlike these methods, our approach reduces the computational complexity to O(N) by using a model to fuse the information from the sensors after applying the Bayesian paradigm. Once the training process is completed, the robot identifies and locates those areas that potentially match the sections that have been previously learned. After the training, the robot navigates and extracts a three-dimensional map of the environment using a single laser sensor. Thus, it perceives different sections of its world. In addition, in order to make our system able to be used in a low-cost robot, low-complexity algorithms that can be easily implemented on embedded processors or microcontrollers are used. PMID:25415377

Learning probabilistic features for robotic navigation using laser sensors.

PubMed

Aznar, Fidel; Pujol, Francisco A; Pujol, Mar; Rizo, Ramón; Pujol, María-José

2014-01-01

SLAM is a popular task used by robots and autonomous vehicles to build a map of an unknown environment and, at the same time, to determine their location within the map. This paper describes a SLAM-based, probabilistic robotic system able to learn the essential features of different parts of its environment. Some previous SLAM implementations had computational complexities ranging from O(Nlog(N)) to O(N(2)), where N is the number of map features. Unlike these methods, our approach reduces the computational complexity to O(N) by using a model to fuse the information from the sensors after applying the Bayesian paradigm. Once the training process is completed, the robot identifies and locates those areas that potentially match the sections that have been previously learned. After the training, the robot navigates and extracts a three-dimensional map of the environment using a single laser sensor. Thus, it perceives different sections of its world. In addition, in order to make our system able to be used in a low-cost robot, low-complexity algorithms that can be easily implemented on embedded processors or microcontrollers are used.

Generalized multiple kernel learning with data-dependent priors.

PubMed

Mao, Qi; Tsang, Ivor W; Gao, Shenghua; Wang, Li

2015-06-01

Multiple kernel learning (MKL) and classifier ensemble are two mainstream methods for solving learning problems in which some sets of features/views are more informative than others, or the features/views within a given set are inconsistent. In this paper, we first present a novel probabilistic interpretation of MKL such that maximum entropy discrimination with a noninformative prior over multiple views is equivalent to the formulation of MKL. Instead of using the noninformative prior, we introduce a novel data-dependent prior based on an ensemble of kernel predictors, which enhances the prediction performance of MKL by leveraging the merits of the classifier ensemble. With the proposed probabilistic framework of MKL, we propose a hierarchical Bayesian model to learn the proposed data-dependent prior and classification model simultaneously. The resultant problem is convex and other information (e.g., instances with either missing views or missing labels) can be seamlessly incorporated into the data-dependent priors. Furthermore, a variety of existing MKL models can be recovered under the proposed MKL framework and can be readily extended to incorporate these priors. Extensive experiments demonstrate the benefits of our proposed framework in supervised and semisupervised settings, as well as in tasks with partial correspondence among multiple views.

The effects of reduced dopamine transporter function and chronic lithium on motivation, probabilistic learning, and neurochemistry in mice: Modeling bipolar mania.

PubMed

Milienne-Petiot, Morgane; Kesby, James P; Graves, Mary; van Enkhuizen, Jordy; Semenova, Svetlana; Minassian, Arpi; Markou, Athina; Geyer, Mark A; Young, Jared W

2017-02-01

Bipolar disorder (BD) mania patients exhibit poor cognition and reward-seeking/hypermotivation, negatively impacting a patient's quality of life. Current treatments (e.g., lithium), do not treat such deficits. Treatment development has been limited due to a poor understanding of the neural mechanisms underlying these behaviors. Here, we investigated putative mechanisms underlying cognition and reward-seeking/motivational changes relevant to BD mania patients using two validated mouse models and neurochemical analyses. The effects of reducing dopamine transporter (DAT) functioning via genetic (knockdown vs. wild-type littermates), or pharmacological (GBR12909- vs. vehicle-treated C57BL/6J mice) means were assessed in the probabilistic reversal learning task (PRLT), and progressive ratio breakpoint (PRB) test, during either water or chronic lithium treatment. These tasks quantify reward learning and effortful motivation, respectively. Neurochemistry was performed on brain samples of DAT mutants ± chronic lithium using high performance liquid chromatography. Reduced DAT functioning increased reversals in the PRLT, an effect partially attenuated by chronic lithium. Chronic lithium alone slowed PRLT acquisition. Reduced DAT functioning increased motivation (PRB), an effect attenuated by lithium in GBR12909-treated mice. Neurochemical analyses revealed that DAT knockdown mice exhibited elevated homovanillic acid levels, but that lithium had no effect on these elevated levels. Reducing DAT functioning recreates many aspects of BD mania including hypermotivation and improved reversal learning (switching), as well as elevated homovanillic acid levels. Chronic lithium only exerted main effects, impairing learning and elevating norepinephrine and serotonin levels of mice, not specifically treating the underlying mechanisms identified in these models. Copyright © 2016 Elsevier Ltd. All rights reserved.

Reasoning about Probabilistic Security Using Task-PIOAs

NASA Astrophysics Data System (ADS)

Jaggard, Aaron D.; Meadows, Catherine; Mislove, Michael; Segala, Roberto

Task-structured probabilistic input/output automata (Task-PIOAs) are concurrent probabilistic automata that, among other things, have been used to provide a formal framework for the universal composability paradigms of protocol security. One of their advantages is that that they allow one to distinguish high-level nondeterminism that can affect the outcome of the protocol, from low-level choices, which can't. We present an alternative approach to analyzing the structure of Task-PIOAs that relies on ordered sets. We focus on two of the components that are required to define and apply Task-PIOAs: discrete probability theory and automata theory. We believe our development gives insight into the structure of Task-PIOAs and how they can be utilized to model crypto-protocols. We illustrate our approach with an example from anonymity, an area that has not previously been addressed using Task-PIOAs. We model Chaum's Dining Cryptographers Protocol at a level that does not require cryptographic primitives in the analysis. We show via this example how our approach can leverage a proof of security in the case a principal behaves deterministically to prove security when that principal behaves probabilistically.

Dual learning processes underlying human decision-making in reversal learning tasks: functional significance and evidence from the model fit to human behavior

PubMed Central

Bai, Yu; Katahira, Kentaro; Ohira, Hideki

2014-01-01

Humans are capable of correcting their actions based on actions performed in the past, and this ability enables them to adapt to a changing environment. The computational field of reinforcement learning (RL) has provided a powerful explanation for understanding such processes. Recently, the dual learning system, modeled as a hybrid model that incorporates value update based on reward-prediction error and learning rate modulation based on the surprise signal, has gained attention as a model for explaining various neural signals. However, the functional significance of the hybrid model has not been established. In the present study, we used computer simulation in a reversal learning task to address functional significance in a probabilistic reversal learning task. The hybrid model was found to perform better than the standard RL model in a large parameter setting. These results suggest that the hybrid model is more robust against the mistuning of parameters compared with the standard RL model when decision-makers continue to learn stimulus-reward contingencies, which can create abrupt changes. The parameter fitting results also indicated that the hybrid model fit better than the standard RL model for more than 50% of the participants, which suggests that the hybrid model has more explanatory power for the behavioral data than the standard RL model. PMID:25161635

Strategies in probabilistic feedback learning in Parkinson patients OFF medication.

PubMed

Bellebaum, C; Kobza, S; Ferrea, S; Schnitzler, A; Pollok, B; Südmeyer, M

2016-04-21

Studies on classification learning suggested that altered dopamine function in Parkinson's Disease (PD) specifically affects learning from feedback. In patients OFF medication, enhanced learning from negative feedback has been described. This learning bias was not seen in observational learning from feedback, indicating different neural mechanisms for this type of learning. The present study aimed to compare the acquisition of stimulus-response-outcome associations in PD patients OFF medication and healthy control subjects in active and observational learning. 16 PD patients OFF medication and 16 controls were examined with three parallel learning tasks each, two feedback-based (active and observational) and one non-feedback-based paired associates task. No acquisition deficit was seen in the patients for any of the tasks. More detailed analyses on the learning strategies did, however, reveal that the patients showed more lose-shift responses during active feedback learning than controls, and that lose-shift and win-stay responses more strongly determined performance accuracy in patients than controls. For observational feedback learning, the performance of both groups correlated similarly with the performance in non-feedback-based paired associates learning and with the accuracy of observed performance. Also, patients and controls showed comparable evidence of feedback processing in observational learning. In active feedback learning, PD patients use alternative learning strategies than healthy controls. Analyses on observational learning did not yield differences between patients and controls, adding to recent evidence of a differential role of the human striatum in active and observational learning from feedback. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

Brain function during probabilistic learning in relation to IQ and level of education.

PubMed

van den Bos, Wouter; Crone, Eveline A; Güroğlu, Berna

2012-02-15

Knowing how to adapt your behavior based on feedback lies at the core of successful learning. We investigated the relation between brain function, grey matter volume, educational level and IQ in a Dutch adolescent sample. In total 45 healthy volunteers between ages 13 and 16 were recruited from schools for pre-vocational and pre-university education. For each individual, IQ was estimated using two subtests from the WISC-III-R (similarities and block design). While in the magnetic resonance imaging (MRI) scanner, participants performed a probabilistic learning task. Behavioral comparisons showed that participants with higher IQ used a more adaptive learning strategy after receiving positive feedback. Analysis of neural activation revealed that higher IQ was associated with increased activation in DLPFC and dACC when receiving positive feedback, specifically for rules with low reward probability (i.e., unexpected positive feedback). Furthermore, VBM analyses revealed that IQ correlated positively with grey matter volume within these regions. These results provide support for IQ-related individual differences in the developmental time courses of neural circuitry supporting feedback-based learning. Current findings are interpreted in terms of a prolonged window of flexibility and opportunity for adolescents with higher IQ scores. Copyright © 2011 Elsevier Ltd. All rights reserved.

Probabilistic machine learning and artificial intelligence.

PubMed

Ghahramani, Zoubin

2015-05-28

How can a machine learn from experience? Probabilistic modelling provides a framework for understanding what learning is, and has therefore emerged as one of the principal theoretical and practical approaches for designing machines that learn from data acquired through experience. The probabilistic framework, which describes how to represent and manipulate uncertainty about models and predictions, has a central role in scientific data analysis, machine learning, robotics, cognitive science and artificial intelligence. This Review provides an introduction to this framework, and discusses some of the state-of-the-art advances in the field, namely, probabilistic programming, Bayesian optimization, data compression and automatic model discovery.

Probabilistic machine learning and artificial intelligence

NASA Astrophysics Data System (ADS)

Ghahramani, Zoubin

2015-05-01

How can a machine learn from experience? Probabilistic modelling provides a framework for understanding what learning is, and has therefore emerged as one of the principal theoretical and practical approaches for designing machines that learn from data acquired through experience. The probabilistic framework, which describes how to represent and manipulate uncertainty about models and predictions, has a central role in scientific data analysis, machine learning, robotics, cognitive science and artificial intelligence. This Review provides an introduction to this framework, and discusses some of the state-of-the-art advances in the field, namely, probabilistic programming, Bayesian optimization, data compression and automatic model discovery.

Cognitive deficits are a matter of emotional context: inflexible strategy use mediates context-specific learning impairments in OCD.

PubMed

Zetsche, Ulrike; Rief, Winfried; Westermann, Stefan; Exner, Cornelia

2015-01-01

The present study examines the interplay between cognitive deficits and emotional context in obsessive-compulsive disorder (OCD) and social phobia (SP). Specifically, this study examines whether the inflexible use of efficient learning strategies in an emotional context underlies impairments in probabilistic classification learning (PCL) in OCD, and whether PCL impairments are specific to OCD. Twenty-three participants with OCD, 30 participants with SP and 30 healthy controls completed a neutral and an OCD-specific PCL task. OCD participants failed to adopt efficient learning strategies and showed fewer beneficial strategy switches than controls only in an OCD-specific context, but not in a neutral context. Additionally, OCD participants did not show any explicit memory impairments. Number of beneficial strategy switches in the OCD-specific task mediated the difference in PCL performance between OCD and control participants. Individuals with SP were impaired in both PCL tasks. In contrast to neuropsychological models postulating general cognitive impairments in OCD, the present findings suggest that it is the interaction between cognition and emotion that is impaired in OCD. Specifically, activated disorder-specific fears may impair the flexible adoption of efficient learning strategies and compromise otherwise unimpaired PCL. Impairments in PCL are not specific to OCD.

Spiking neuron network Helmholtz machine.

PubMed

Sountsov, Pavel; Miller, Paul

2015-01-01

An increasing amount of behavioral and neurophysiological data suggests that the brain performs optimal (or near-optimal) probabilistic inference and learning during perception and other tasks. Although many machine learning algorithms exist that perform inference and learning in an optimal way, the complete description of how one of those algorithms (or a novel algorithm) can be implemented in the brain is currently incomplete. There have been many proposed solutions that address how neurons can perform optimal inference but the question of how synaptic plasticity can implement optimal learning is rarely addressed. This paper aims to unify the two fields of probabilistic inference and synaptic plasticity by using a neuronal network of realistic model spiking neurons to implement a well-studied computational model called the Helmholtz Machine. The Helmholtz Machine is amenable to neural implementation as the algorithm it uses to learn its parameters, called the wake-sleep algorithm, uses a local delta learning rule. Our spiking-neuron network implements both the delta rule and a small example of a Helmholtz machine. This neuronal network can learn an internal model of continuous-valued training data sets without supervision. The network can also perform inference on the learned internal models. We show how various biophysical features of the neural implementation constrain the parameters of the wake-sleep algorithm, such as the duration of the wake and sleep phases of learning and the minimal sample duration. We examine the deviations from optimal performance and tie them to the properties of the synaptic plasticity rule.

Spiking neuron network Helmholtz machine

PubMed Central

Sountsov, Pavel; Miller, Paul

2015-01-01

An increasing amount of behavioral and neurophysiological data suggests that the brain performs optimal (or near-optimal) probabilistic inference and learning during perception and other tasks. Although many machine learning algorithms exist that perform inference and learning in an optimal way, the complete description of how one of those algorithms (or a novel algorithm) can be implemented in the brain is currently incomplete. There have been many proposed solutions that address how neurons can perform optimal inference but the question of how synaptic plasticity can implement optimal learning is rarely addressed. This paper aims to unify the two fields of probabilistic inference and synaptic plasticity by using a neuronal network of realistic model spiking neurons to implement a well-studied computational model called the Helmholtz Machine. The Helmholtz Machine is amenable to neural implementation as the algorithm it uses to learn its parameters, called the wake-sleep algorithm, uses a local delta learning rule. Our spiking-neuron network implements both the delta rule and a small example of a Helmholtz machine. This neuronal network can learn an internal model of continuous-valued training data sets without supervision. The network can also perform inference on the learned internal models. We show how various biophysical features of the neural implementation constrain the parameters of the wake-sleep algorithm, such as the duration of the wake and sleep phases of learning and the minimal sample duration. We examine the deviations from optimal performance and tie them to the properties of the synaptic plasticity rule. PMID:25954191

Dissociation between active and observational learning from positive and negative feedback in Parkinsonism.

PubMed

Kobza, Stefan; Ferrea, Stefano; Schnitzler, Alfons; Pollok, Bettina; Südmeyer, Martin; Bellebaum, Christian

2012-01-01

Feedback to both actively performed and observed behaviour allows adaptation of future actions. Positive feedback leads to increased activity of dopamine neurons in the substantia nigra, whereas dopamine neuron activity is decreased following negative feedback. Dopamine level reduction in unmedicated Parkinson's Disease patients has been shown to lead to a negative learning bias, i.e. enhanced learning from negative feedback. Recent findings suggest that the neural mechanisms of active and observational learning from feedback might differ, with the striatum playing a less prominent role in observational learning. Therefore, it was hypothesized that unmedicated Parkinson's Disease patients would show a negative learning bias only in active but not in observational learning. In a between-group design, 19 Parkinson's Disease patients and 40 healthy controls engaged in either an active or an observational probabilistic feedback-learning task. For both tasks, transfer phases aimed to assess the bias to learn better from positive or negative feedback. As expected, actively learning patients showed a negative learning bias, whereas controls learned better from positive feedback. In contrast, no difference between patients and controls emerged for observational learning, with both groups showing better learning from positive feedback. These findings add to neural models of reinforcement-learning by suggesting that dopamine-modulated input to the striatum plays a minor role in observational learning from feedback. Future research will have to elucidate the specific neural underpinnings of observational learning.

Chronic cocaine but not chronic amphetamine use is associated with perseverative responding in humans

PubMed Central

Roiser, Jonathan P.; Robbins, Trevor W.; Sahakian, Barbara J.

2013-01-01

Rationale Chronic drug use has been associated with increased impulsivity and maladaptive behaviour, but the underlying mechanisms of this impairment remain unclear. We investigated the ability to adapt behaviour according to changes in reward contingencies, using a probabilistic reversal-learning task, in chronic drug users and controls. Materials and methods Five groups were compared: chronic amphetamine users (n = 30); chronic cocaine users (n = 27); chronic opiate users (n = 42); former drug users of psychostimulants and opiates (n = 26); and healthy non-drug-taking control volunteers (n = 25). Participants had to make a forced choice between two alternative stimuli on each trial to acquire a stimulus–reward association on the basis of degraded feedback and subsequently to reverse their responses when the reward contingencies changed. Results Chronic cocaine users demonstrated little behavioural change in response to the change in reward contingencies, as reflected by perseverative responding to the previously rewarded stimulus. Perseverative responding was observed in cocaine users regardless of whether they completed the reversal stage successfully. Task performance in chronic users of amphetamines and opiates, as well as in former drug users, was not measurably impaired. Conclusion Our findings provide convincing evidence for response perseveration in cocaine users during probabilistic reversal-learning. Pharmacological differences between amphetamine and cocaine, in particular their respective effects on the 5-HT system, may account for the divergent task performance between the two psychostimulant user groups. The inability to reverse responses according to changes in reinforcement contingencies may underlie the maladaptive behaviour patterns observed in chronic cocaine users but not in chronic users of amphetamines or opiates. PMID:18214445

Accuracy of latent-variable estimation in Bayesian semi-supervised learning.

PubMed

Yamazaki, Keisuke

2015-09-01

Hierarchical probabilistic models, such as Gaussian mixture models, are widely used for unsupervised learning tasks. These models consist of observable and latent variables, which represent the observable data and the underlying data-generation process, respectively. Unsupervised learning tasks, such as cluster analysis, are regarded as estimations of latent variables based on the observable ones. The estimation of latent variables in semi-supervised learning, where some labels are observed, will be more precise than that in unsupervised, and one of the concerns is to clarify the effect of the labeled data. However, there has not been sufficient theoretical analysis of the accuracy of the estimation of latent variables. In a previous study, a distribution-based error function was formulated, and its asymptotic form was calculated for unsupervised learning with generative models. It has been shown that, for the estimation of latent variables, the Bayes method is more accurate than the maximum-likelihood method. The present paper reveals the asymptotic forms of the error function in Bayesian semi-supervised learning for both discriminative and generative models. The results show that the generative model, which uses all of the given data, performs better when the model is well specified. Copyright © 2015 Elsevier Ltd. All rights reserved.

Reinforcement Learning Deficits in People with Schizophrenia Persist after Extended Trials

PubMed Central

Cicero, David C.; Martin, Elizabeth A.; Becker, Theresa M.; Kerns, John G.

2014-01-01

Previous research suggests that people with schizophrenia have difficulty learning from positive feedback and when learning needs to occur rapidly. However, they seem to have relatively intact learning from negative feedback when learning occurs gradually. Participants are typically given a limited amount of acquisition trials to learn the reward contingencies and then tested about what they learned. The current study examined whether participants with schizophrenia continue to display these deficits when given extra time to learn the contingences. Participants with schizophrenia and matched healthy controls completed the Probabilistic Selection Task, which measures positive and negative feedback learning separately. Participants with schizophrenia showed a deficit in learning from both positive and negative feedback. These reward learning deficits persisted even if people with schizophrenia are given extra time (up to 10 blocks of 60 trials) to learn the reward contingencies. These results suggest that the observed deficits cannot be attributed solely to slower learning and instead reflect a specific deficit in reinforcement learning. PMID:25172610

Reinforcement learning deficits in people with schizophrenia persist after extended trials.

PubMed

Cicero, David C; Martin, Elizabeth A; Becker, Theresa M; Kerns, John G

2014-12-30

Previous research suggests that people with schizophrenia have difficulty learning from positive feedback and when learning needs to occur rapidly. However, they seem to have relatively intact learning from negative feedback when learning occurs gradually. Participants are typically given a limited amount of acquisition trials to learn the reward contingencies and then tested about what they learned. The current study examined whether participants with schizophrenia continue to display these deficits when given extra time to learn the contingences. Participants with schizophrenia and matched healthy controls completed the Probabilistic Selection Task, which measures positive and negative feedback learning separately. Participants with schizophrenia showed a deficit in learning from both positive feedback and negative feedback. These reward learning deficits persisted even if people with schizophrenia are given extra time (up to 10 blocks of 60 trials) to learn the reward contingencies. These results suggest that the observed deficits cannot be attributed solely to slower learning and instead reflect a specific deficit in reinforcement learning. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.

An Upside to Reward Sensitivity: The Hippocampus Supports Enhanced Reinforcement Learning in Adolescence.

PubMed

Davidow, Juliet Y; Foerde, Karin; Galván, Adriana; Shohamy, Daphna

2016-10-05

Adolescents are notorious for engaging in reward-seeking behaviors, a tendency attributed to heightened activity in the brain's reward systems during adolescence. It has been suggested that reward sensitivity in adolescence might be adaptive, but evidence of an adaptive role has been scarce. Using a probabilistic reinforcement learning task combined with reinforcement learning models and fMRI, we found that adolescents showed better reinforcement learning and a stronger link between reinforcement learning and episodic memory for rewarding outcomes. This behavioral benefit was related to heightened prediction error-related BOLD activity in the hippocampus and to stronger functional connectivity between the hippocampus and the striatum at the time of reinforcement. These findings reveal an important role for the hippocampus in reinforcement learning in adolescence and suggest that reward sensitivity in adolescence is related to adaptive differences in how adolescents learn from experience. Copyright © 2016 Elsevier Inc. All rights reserved.

Altered brain activation in a reversal learning task unmasks adaptive changes in cognitive control in writer's cramp.

PubMed

Zeuner, Kirsten E; Knutzen, Arne; Granert, Oliver; Sablowsky, Simone; Götz, Julia; Wolff, Stephan; Jansen, Olav; Dressler, Dirk; Schneider, Susanne A; Klein, Christine; Deuschl, Günther; van Eimeren, Thilo; Witt, Karsten

2016-01-01

Previous receptor binding studies suggest dopamine function is altered in the basal ganglia circuitry in task-specific dystonia, a condition characterized by contraction of agonist and antagonist muscles while performing specific tasks. Dopamine plays a role in reward-based learning. Using fMRI, this study compared 31 right-handed writer's cramp patients to 35 controls in reward-based learning of a probabilistic reversal-learning task. All subjects chose between two stimuli and indicated their response with their left or right index finger. One stimulus response was rewarded 80%, the other 20%. After contingencies reversal, the second stimulus response was rewarded in 80%. We further linked the DRD2/ANKK1-TaqIa polymorphism, which is associated with 30% reduction of the striatal dopamine receptor density with reward-based learning and assumed impaired reversal learning in A + subjects. Feedback learning in patients was normal. Blood-oxygen level dependent (BOLD) signal in controls increased with negative feedback in the insula, rostral cingulate cortex, middle frontal gyrus and parietal cortex (pFWE < 0.05). In comparison to controls, patients showed greater increase in BOLD activity following negative feedback in the dorsal anterior cingulate cortex (BA32). The genetic status was not correlated with the BOLD activity. The Brodmann area 32 (BA32) is part of the dorsal anterior cingulate cortex (dACC) that plays an important role in coordinating and integrating information to guide behavior and in reward-based learning. The dACC is connected with the basal ganglia-thalamo-loop modulated by dopaminergic signaling. This finding suggests disturbed integration of reinforcement history in decision making and implicate that the reward system might contribute to the pathogenesis in writer's cramp.

DYT1 dystonia increases risk taking in humans.

PubMed

Arkadir, David; Radulescu, Angela; Raymond, Deborah; Lubarr, Naomi; Bressman, Susan B; Mazzoni, Pietro; Niv, Yael

2016-06-01

It has been difficult to link synaptic modification to overt behavioral changes. Rodent models of DYT1 dystonia, a motor disorder caused by a single gene mutation, demonstrate increased long-term potentiation and decreased long-term depression in corticostriatal synapses. Computationally, such asymmetric learning predicts risk taking in probabilistic tasks. Here we demonstrate abnormal risk taking in DYT1 dystonia patients, which is correlated with disease severity, thereby supporting striatal plasticity in shaping choice behavior in humans.

Neurodynamical model of collective brain

NASA Technical Reports Server (NTRS)

Zak, Michail

1992-01-01

A dynamical system which mimics collective purposeful activities of a set of units of intelligence is introduced and discussed. A global control of the unit activities is replaced by the probabilistic correlations between them. These correlations are learned during a long term period of performing collective tasks, and are stored in the synaptic interconnections. The model is represented by a system of ordinary differential equations with terminal attractors and repellers, and does not contain any man-made digital devices.

STDP Installs in Winner-Take-All Circuits an Online Approximation to Hidden Markov Model Learning

PubMed Central

Kappel, David; Nessler, Bernhard; Maass, Wolfgang

2014-01-01

In order to cross a street without being run over, we need to be able to extract very fast hidden causes of dynamically changing multi-modal sensory stimuli, and to predict their future evolution. We show here that a generic cortical microcircuit motif, pyramidal cells with lateral excitation and inhibition, provides the basis for this difficult but all-important information processing capability. This capability emerges in the presence of noise automatically through effects of STDP on connections between pyramidal cells in Winner-Take-All circuits with lateral excitation. In fact, one can show that these motifs endow cortical microcircuits with functional properties of a hidden Markov model, a generic model for solving such tasks through probabilistic inference. Whereas in engineering applications this model is adapted to specific tasks through offline learning, we show here that a major portion of the functionality of hidden Markov models arises already from online applications of STDP, without any supervision or rewards. We demonstrate the emergent computing capabilities of the model through several computer simulations. The full power of hidden Markov model learning can be attained through reward-gated STDP. This is due to the fact that these mechanisms enable a rejection sampling approximation to theoretically optimal learning. We investigate the possible performance gain that can be achieved with this more accurate learning method for an artificial grammar task. PMID:24675787

Error Discounting in Probabilistic Category Learning

ERIC Educational Resources Information Center

Craig, Stewart; Lewandowsky, Stephan; Little, Daniel R.

2011-01-01

The assumption in some current theories of probabilistic categorization is that people gradually attenuate their learning in response to unavoidable error. However, existing evidence for this error discounting is sparse and open to alternative interpretations. We report 2 probabilistic-categorization experiments in which we investigated error…

a Probabilistic Embedding Clustering Method for Urban Structure Detection

NASA Astrophysics Data System (ADS)

Lin, X.; Li, H.; Zhang, Y.; Gao, L.; Zhao, L.; Deng, M.

2017-09-01

Urban structure detection is a basic task in urban geography. Clustering is a core technology to detect the patterns of urban spatial structure, urban functional region, and so on. In big data era, diverse urban sensing datasets recording information like human behaviour and human social activity, suffer from complexity in high dimension and high noise. And unfortunately, the state-of-the-art clustering methods does not handle the problem with high dimension and high noise issues concurrently. In this paper, a probabilistic embedding clustering method is proposed. Firstly, we come up with a Probabilistic Embedding Model (PEM) to find latent features from high dimensional urban sensing data by "learning" via probabilistic model. By latent features, we could catch essential features hidden in high dimensional data known as patterns; with the probabilistic model, we can also reduce uncertainty caused by high noise. Secondly, through tuning the parameters, our model could discover two kinds of urban structure, the homophily and structural equivalence, which means communities with intensive interaction or in the same roles in urban structure. We evaluated the performance of our model by conducting experiments on real-world data and experiments with real data in Shanghai (China) proved that our method could discover two kinds of urban structure, the homophily and structural equivalence, which means clustering community with intensive interaction or under the same roles in urban space.

Development of probabilistic thinking-oriented learning tools for probability materials at junior high school students

NASA Astrophysics Data System (ADS)

Sari, Dwi Ivayana; Hermanto, Didik

2017-08-01

This research is a developmental research of probabilistic thinking-oriented learning tools for probability materials at ninth grade students. This study is aimed to produce a good probabilistic thinking-oriented learning tools. The subjects were IX-A students of MTs Model Bangkalan. The stages of this development research used 4-D development model which has been modified into define, design and develop. Teaching learning tools consist of lesson plan, students' worksheet, learning teaching media and students' achievement test. The research instrument used was a sheet of learning tools validation, a sheet of teachers' activities, a sheet of students' activities, students' response questionnaire and students' achievement test. The result of those instruments were analyzed descriptively to answer research objectives. The result was teaching learning tools in which oriented to probabilistic thinking of probability at ninth grade students which has been valid. Since teaching and learning tools have been revised based on validation, and after experiment in class produced that teachers' ability in managing class was effective, students' activities were good, students' responses to the learning tools were positive and the validity, sensitivity and reliability category toward achievement test. In summary, this teaching learning tools can be used by teacher to teach probability for develop students' probabilistic thinking.

Dissociation between Active and Observational Learning from Positive and Negative Feedback in Parkinsonism

PubMed Central

Kobza, Stefan; Ferrea, Stefano; Schnitzler, Alfons; Pollok, Bettina

2012-01-01

Feedback to both actively performed and observed behaviour allows adaptation of future actions. Positive feedback leads to increased activity of dopamine neurons in the substantia nigra, whereas dopamine neuron activity is decreased following negative feedback. Dopamine level reduction in unmedicated Parkinson’s Disease patients has been shown to lead to a negative learning bias, i.e. enhanced learning from negative feedback. Recent findings suggest that the neural mechanisms of active and observational learning from feedback might differ, with the striatum playing a less prominent role in observational learning. Therefore, it was hypothesized that unmedicated Parkinson’s Disease patients would show a negative learning bias only in active but not in observational learning. In a between-group design, 19 Parkinson’s Disease patients and 40 healthy controls engaged in either an active or an observational probabilistic feedback-learning task. For both tasks, transfer phases aimed to assess the bias to learn better from positive or negative feedback. As expected, actively learning patients showed a negative learning bias, whereas controls learned better from positive feedback. In contrast, no difference between patients and controls emerged for observational learning, with both groups showing better learning from positive feedback. These findings add to neural models of reinforcement-learning by suggesting that dopamine-modulated input to the striatum plays a minor role in observational learning from feedback. Future research will have to elucidate the specific neural underpinnings of observational learning. PMID:23185586

Working memory and reward association learning impairments in obesity.

PubMed

Coppin, Géraldine; Nolan-Poupart, Sarah; Jones-Gotman, Marilyn; Small, Dana M

2014-12-01

Obesity has been associated with impaired executive functions including working memory. Less explored is the influence of obesity on learning and memory. In the current study we assessed stimulus reward association learning, explicit learning and memory and working memory in healthy weight, overweight and obese individuals. Explicit learning and memory did not differ as a function of group. In contrast, working memory was significantly and similarly impaired in both overweight and obese individuals compared to the healthy weight group. In the first reward association learning task the obese, but not healthy weight or overweight participants consistently formed paradoxical preferences for a pattern associated with a negative outcome (fewer food rewards). To determine if the deficit was specific to food reward a second experiment was conducted using money. Consistent with Experiment 1, obese individuals selected the pattern associated with a negative outcome (fewer monetary rewards) more frequently than healthy weight individuals and thus failed to develop a significant preference for the most rewarded patterns as was observed in the healthy weight group. Finally, on a probabilistic learning task, obese compared to healthy weight individuals showed deficits in negative, but not positive outcome learning. Taken together, our results demonstrate deficits in working memory and stimulus reward learning in obesity and suggest that obese individuals are impaired in learning to avoid negative outcomes. Copyright © 2014 Elsevier Ltd. All rights reserved.

Error Discounting in Probabilistic Category Learning

PubMed Central

Craig, Stewart; Lewandowsky, Stephan; Little, Daniel R.

2011-01-01

Some current theories of probabilistic categorization assume that people gradually attenuate their learning in response to unavoidable error. However, existing evidence for this error discounting is sparse and open to alternative interpretations. We report two probabilistic-categorization experiments that investigated error discounting by shifting feedback probabilities to new values after different amounts of training. In both experiments, responding gradually became less responsive to errors, and learning was slowed for some time after the feedback shift. Both results are indicative of error discounting. Quantitative modeling of the data revealed that adding a mechanism for error discounting significantly improved the fits of an exemplar-based and a rule-based associative learning model, as well as of a recency-based model of categorization. We conclude that error discounting is an important component of probabilistic learning. PMID:21355666

Dopamine D3 Receptor Availability Is Associated with Inflexible Decision Making.

PubMed

Groman, Stephanie M; Smith, Nathaniel J; Petrullli, J Ryan; Massi, Bart; Chen, Lihui; Ropchan, Jim; Huang, Yiyun; Lee, Daeyeol; Morris, Evan D; Taylor, Jane R

2016-06-22

Dopamine D2/3 receptor signaling is critical for flexible adaptive behavior; however, it is unclear whether D2, D3, or both receptor subtypes modulate precise signals of feedback and reward history that underlie optimal decision making. Here, PET with the radioligand [(11)C]-(+)-PHNO was used to quantify individual differences in putative D3 receptor availability in rodents trained on a novel three-choice spatial acquisition and reversal-learning task with probabilistic reinforcement. Binding of [(11)C]-(+)-PHNO in the midbrain was negatively related to the ability of rats to adapt to changes in rewarded locations, but not to the initial learning. Computational modeling of choice behavior in the reversal phase indicated that [(11)C]-(+)-PHNO binding in the midbrain was related to the learning rate and sensitivity to positive, but not negative, feedback. Administration of a D3-preferring agonist likewise impaired reversal performance by reducing the learning rate and sensitivity to positive feedback. These results demonstrate a previously unrecognized role for D3 receptors in select aspects of reinforcement learning and suggest that individual variation in midbrain D3 receptors influences flexible behavior. Our combined neuroimaging, behavioral, pharmacological, and computational approach implicates the dopamine D3 receptor in decision-making processes that are altered in psychiatric disorders. Flexible decision-making behavior is dependent upon dopamine D2/3 signaling in corticostriatal brain regions. However, the role of D3 receptors in adaptive, goal-directed behavior has not been thoroughly investigated. By combining PET imaging with the D3-preferring radioligand [(11)C]-(+)-PHNO, pharmacology, a novel three-choice probabilistic discrimination and reversal task and computational modeling of behavior in rats, we report that naturally occurring variation in [(11)C]-(+)-PHNO receptor availability relates to specific aspects of flexible decision making. We confirm these relationships using a D3-preferring agonist, thus identifying a unique role of midbrain D3 receptors in decision-making processes. Copyright © 2016 the authors 0270-6474/16/366732-10$15.00/0.

A Joint Gaussian Process Model for Active Visual Recognition with Expertise Estimation in Crowdsourcing

PubMed Central

Long, Chengjiang; Hua, Gang; Kapoor, Ashish

2015-01-01

We present a noise resilient probabilistic model for active learning of a Gaussian process classifier from crowds, i.e., a set of noisy labelers. It explicitly models both the overall label noise and the expertise level of each individual labeler with two levels of flip models. Expectation propagation is adopted for efficient approximate Bayesian inference of our probabilistic model for classification, based on which, a generalized EM algorithm is derived to estimate both the global label noise and the expertise of each individual labeler. The probabilistic nature of our model immediately allows the adoption of the prediction entropy for active selection of data samples to be labeled, and active selection of high quality labelers based on their estimated expertise to label the data. We apply the proposed model for four visual recognition tasks, i.e., object category recognition, multi-modal activity recognition, gender recognition, and fine-grained classification, on four datasets with real crowd-sourced labels from the Amazon Mechanical Turk. The experiments clearly demonstrate the efficacy of the proposed model. In addition, we extend the proposed model with the Predictive Active Set Selection Method to speed up the active learning system, whose efficacy is verified by conducting experiments on the first three datasets. The results show our extended model can not only preserve a higher accuracy, but also achieve a higher efficiency. PMID:26924892

A neurocomputational account of cognitive deficits in Parkinson’s disease

PubMed Central

Hélie, Sébastien; Paul, Erick J.; Ashby, F. Gregory

2014-01-01

Parkinson’s disease (PD) is caused by the accelerated death of dopamine (DA) producing neurons. Numerous studies documenting cognitive deficits of PD patients have revealed impairments in a variety of tasks related to memory, learning, visuospatial skills, and attention. While there have been several studies documenting cognitive deficits of PD patients, very few computational models have been proposed. In this article, we use the COVIS model of category learning to simulate DA depletion and show that the model suffers from cognitive symptoms similar to those of human participants affected by PD. Specifically, DA depletion in COVIS produced deficits in rule-based categorization, non-linear information-integration categorization, probabilistic classification, rule maintenance, and rule switching. These were observed by simulating results from younger controls, older controls, PD patients, and severe PD patients in five well-known tasks. Differential performance among the different age groups and clinical populations was modeled simply by changing the amount of DA available in the model. This suggests that COVIS may not only be an adequate model of the simulated tasks and phenomena but also more generally of the role of DA in these tasks and phenomena. PMID:22683450

Probabilistic Common Spatial Patterns for Multichannel EEG Analysis

PubMed Central

Chen, Zhe; Gao, Xiaorong; Li, Yuanqing; Brown, Emery N.; Gao, Shangkai

2015-01-01

Common spatial patterns (CSP) is a well-known spatial filtering algorithm for multichannel electroencephalogram (EEG) analysis. In this paper, we cast the CSP algorithm in a probabilistic modeling setting. Specifically, probabilistic CSP (P-CSP) is proposed as a generic EEG spatio-temporal modeling framework that subsumes the CSP and regularized CSP algorithms. The proposed framework enables us to resolve the overfitting issue of CSP in a principled manner. We derive statistical inference algorithms that can alleviate the issue of local optima. In particular, an efficient algorithm based on eigendecomposition is developed for maximum a posteriori (MAP) estimation in the case of isotropic noise. For more general cases, a variational algorithm is developed for group-wise sparse Bayesian learning for the P-CSP model and for automatically determining the model size. The two proposed algorithms are validated on a simulated data set. Their practical efficacy is also demonstrated by successful applications to single-trial classifications of three motor imagery EEG data sets and by the spatio-temporal pattern analysis of one EEG data set recorded in a Stroop color naming task. PMID:26005228

Adolescents' Heightened Risk-Seeking in a Probabilistic Gambling Task

ERIC Educational Resources Information Center

Burnett, Stephanie; Bault, Nadege; Coricelli, Giorgio; Blakemore, Sarah-Jayne

2010-01-01

This study investigated adolescent males' decision-making under risk, and the emotional response to decision outcomes, using a probabilistic gambling task designed to evoke counterfactually mediated emotions (relief and regret). Participants were 20 adolescents (aged 9-11), 26 young adolescents (aged 12-15), 20 mid-adolescents (aged 15-18) and 17…

Attenuating GABAA Receptor Signaling in Dopamine Neurons Selectively Enhances Reward Learning and Alters Risk Preference in Mice

PubMed Central

Parker, Jones G.; Wanat, Matthew J.; Soden, Marta E.; Ahmad, Kinza; Zweifel, Larry S.; Bamford, Nigel S.; Palmiter, Richard D.

2011-01-01

Phasic dopamine transmission encodes the value of reward-predictive stimuli and influences both learning and decision-making. Altered dopamine signaling is associated with psychiatric conditions characterized by risky choices such as pathological gambling. These observations highlight the importance of understanding how dopamine neuron activity is modulated. While excitatory drive onto dopamine neurons is critical for generating phasic dopamine responses, emerging evidence suggests that inhibitory signaling also modulates these responses. To address the functional importance of inhibitory signaling in dopamine neurons, we generated mice lacking the β3 subunit of the GABAA receptor specifically in dopamine neurons (β3-KO mice) and examined their behavior in tasks that assessed appetitive learning, aversive learning, and risk preference. Dopamine neurons in midbrain slices from β3-KO mice exhibited attenuated GABA-evoked inhibitory post-synaptic currents. Furthermore, electrical stimulation of excitatory afferents to dopamine neurons elicited more dopamine release in the nucleus accumbens of β3-KO mice as measured by fast-scan cyclic voltammetry. β3-KO mice were more active than controls when given morphine, which correlated with potential compensatory upregulation of GABAergic tone onto dopamine neurons. β3-KO mice learned faster in two food-reinforced learning paradigms, but extinguished their learned behavior normally. Enhanced learning was specific for appetitive tasks, as aversive learning was unaffected in β3-KO mice. Finally, we found that β3-KO mice had enhanced risk preference in a probabilistic selection task that required mice to choose between a small certain reward and a larger uncertain reward. Collectively, these findings identify a selective role for GABAA signaling in dopamine neurons in appetitive learning and decision-making. PMID:22114279

Selective Estrogen Receptor Modulation Increases Hippocampal Activity during Probabilistic Association Learning in Schizophrenia

PubMed Central

Kindler, Jochen; Weickert, Cynthia Shannon; Skilleter, Ashley J; Catts, Stanley V; Lenroot, Rhoshel; Weickert, Thomas W

2015-01-01

People with schizophrenia show probabilistic association learning impairment in conjunction with abnormal neural activity. The selective estrogen receptor modulator (SERM) raloxifene preserves neural activity during memory in healthy older men and improves memory in schizophrenia. Here, we tested the extent to which raloxifene modifies neural activity during learning in schizophrenia. Nineteen people with schizophrenia participated in a twelve-week randomized, double-blind, placebo-controlled, cross-over adjunctive treatment trial of the SERM raloxifene administered orally at 120 mg daily to assess brain activity during probabilistic association learning using functional magnetic resonance imaging (fMRI). Raloxifene improved probabilistic association learning and significantly increased fMRI BOLD activity in the hippocampus and parahippocampal gyrus relative to placebo. A separate region of interest confirmatory analysis in 21 patients vs 36 healthy controls showed a positive association between parahippocampal neural activity and learning in patients, but no such relationship in the parahippocampal gyrus of healthy controls. Thus, selective estrogen receptor modulation by raloxifene concurrently increases activity in the parahippocampal gyrus and improves probabilistic association learning in schizophrenia. These results support a role for estrogen receptor modulation of mesial temporal lobe neural activity in the remediation of learning disabilities in both men and women with schizophrenia. PMID:25829142

REWARD/PUNISHMENT REVERSAL LEARNING IN OLDER SUICIDE ATTEMPTERS

PubMed Central

Dombrovski, Alexandre Y.; Clark, Luke; Siegle, Greg J.; Butters, Meryl A.; Ichikawa, Naho; Sahakian, Barbara; Szanto, Katalin

2011-01-01

Objective Suicide rates are very high in old age, and the contribution of cognitive risk factors remains poorly understood. Suicide may be viewed as an outcome of an altered decision process. We hypothesized that impairment in a component of affective decision-making – reward/punishment-based learning – is associated with attempted suicide in late-life depression. We expected that suicide attempters would discount past reward/punishment history, focusing excessively on the most recent rewards and punishments. Further, we hypothesized that this impairment could be dissociated from executive abilities such as forward planning. Method We assessed reward/punishment-based learning using the Probabilistic Reversal Learning task in 65 individuals aged 60 and older: suicide attempters, suicide ideators, non-suicidal depressed elderly, and non-depressed controls. We used a reinforcement learning computational model to decompose reward/punishment processing over time. The Stockings of Cambridge test served as a control measure of executive function. Results Suicide attempters but not suicide ideators showed impaired probabilistic reversal learning compared to both non-suicidal depressed elderly and to non-depressed controls, after controlling for effects of education, global cognitive function, and substance use. Model-based analyses revealed that suicide attempters discounted previous history to a higher degree, compared to controls, basing their choice largely on reward/punishment received on the last trial. Groups did not differ in their performance on the Stockings of Cambridge. Conclusions Older suicide attempters display impaired reward/punishment-based learning. We propose a hypothesis that older suicide attempters make overly present-focused decisions, ignoring past experiences. Modification of this ‘myopia for the past’ may have therapeutic potential. PMID:20231320

Young children do not succeed in choice tasks that imply evaluating chances.

PubMed

Girotto, Vittorio; Fontanari, Laura; Gonzalez, Michel; Vallortigara, Giorgio; Blaye, Agnès

2016-07-01

Preverbal infants manifest probabilistic intuitions in their reactions to the outcomes of simple physical processes and in their choices. Their ability conflicts with the evidence that, before the age of about 5years, children's verbal judgments do not reveal probability understanding. To assess these conflicting results, three studies tested 3-5-year-olds on choice tasks on which infants perform successfully. The results showed that children of all age groups made optimal choices in tasks that did not require forming probabilistic expectations. In probabilistic tasks, however, only 5-year-olds made optimal choices. Younger children performed at random and/or were guided by superficial heuristics. These results suggest caution in interpreting infants' ability to evaluate chance, and indicate that the development of this ability may not follow a linear trajectory. Copyright © 2016 Elsevier B.V. All rights reserved.

Spared internal but impaired external reward prediction error signals in major depressive disorder during reinforcement learning.

PubMed

Bakic, Jasmina; Pourtois, Gilles; Jepma, Marieke; Duprat, Romain; De Raedt, Rudi; Baeken, Chris

2017-01-01

Major depressive disorder (MDD) creates debilitating effects on a wide range of cognitive functions, including reinforcement learning (RL). In this study, we sought to assess whether reward processing as such, or alternatively the complex interplay between motivation and reward might potentially account for the abnormal reward-based learning in MDD. A total of 35 treatment resistant MDD patients and 44 age matched healthy controls (HCs) performed a standard probabilistic learning task. RL was titrated using behavioral, computational modeling and event-related brain potentials (ERPs) data. MDD patients showed comparable learning rate compared to HCs. However, they showed decreased lose-shift responses as well as blunted subjective evaluations of the reinforcers used during the task, relative to HCs. Moreover, MDD patients showed normal internal (at the level of error-related negativity, ERN) but abnormal external (at the level of feedback-related negativity, FRN) reward prediction error (RPE) signals during RL, selectively when additional efforts had to be made to establish learning. Collectively, these results lend support to the assumption that MDD does not impair reward processing per se during RL. Instead, it seems to alter the processing of the emotional value of (external) reinforcers during RL, when additional intrinsic motivational processes have to be engaged. © 2016 Wiley Periodicals, Inc.

Cognitive flexibility impairment and reduced frontal cortex BDNF expression in the ouabain model of mania

PubMed Central

Amodeo, Dionisio A.; Grospe, Gena; Zang, Hui; Dwivedi, Yogesh; Ragozzino, Michael E.

2016-01-01

Central infusion of the Na+/K+-ATPase inhibitor, ouabain in rats serves as an animal model of mania because it leads to hyperactivity, as well as reproduces ion dysregulation and reduced BDNF levels similar to that observed in bipolar disorder. Bipolar disorder is also associated with cognitive inflexibility and working memory deficits. It is unknown whether ouabain treatment in rats leads to similar cognitive flexibility and working memory deficits. The present study examined the effects of an intracerebral ventricular infusion of ouabain in rats on spontaneous alternation, probabilistic reversal learning and BDNF expression levels in the frontal cortex. Ouabain treatment significantly increased locomotor activity, but did not affect alternation performance in a Y-maze. Ouabain treatment selectively impaired reversal learning in a spatial discrimination task using an 80/20 probabilistic reinforcement procedure. The reversal learning deficit in ouabain-treated rats resulted from an impaired ability to maintain a new choice pattern (increased regressive errors). Ouabain treatment also decreased sensitivity to negative feedback during the initial phase of reversal learning. Expression of BDNF mRNA and protein levels was downregulated in the frontal cortex which also negatively correlated with regressive errors. These findings suggest that the ouabain model of mania may be useful in understanding the neuropathophysiology that contributes to cognitive flexibility deficits and test potential treatments to alleviate cognitive deficits in bipolar disorder. PMID:27267245

Frontostriatal anatomical connections predict age- and difficulty-related differences in reinforcement learning.

PubMed

van de Vijver, Irene; Ridderinkhof, K Richard; Harsay, Helga; Reneman, Liesbeth; Cavanagh, James F; Buitenweg, Jessika I V; Cohen, Michael X

2016-10-01

Reinforcement learning (RL) is supported by a network of striatal and frontal cortical structures that are connected through white-matter fiber bundles. With age, the integrity of these white-matter connections declines. The role of structural frontostriatal connectivity in individual and age-related differences in RL is unclear, although local white-matter density and diffusivity have been linked to individual differences in RL. Here we show that frontostriatal tract counts in young human adults (aged 18-28), as assessed noninvasively with diffusion-weighted magnetic resonance imaging and probabilistic tractography, positively predicted individual differences in RL when learning was difficult (70% valid feedback). In older adults (aged 63-87), in contrast, learning under both easy (90% valid feedback) and difficult conditions was predicted by tract counts in the same frontostriatal network. Furthermore, network-level analyses showed a double dissociation between the task-relevant networks in young and older adults, suggesting that older adults relied on different frontostriatal networks than young adults to obtain the same task performance. These results highlight the importance of successful information integration across striatal and frontal regions during RL, especially with variable outcomes. Copyright © 2016 Elsevier Inc. All rights reserved.

Habitual attention in older and young adults.

PubMed

Jiang, Yuhong V; Koutstaal, Wilma; Twedell, Emily L

2016-12-01

Age-related decline is pervasive in tasks that require explicit learning and memory, but such reduced function is not universally observed in tasks involving incidental learning. It is unknown if habitual attention, involving incidental probabilistic learning, is preserved in older adults. Previous research on habitual attention investigated contextual cuing in young and older adults, yet contextual cuing relies not only on spatial attention but also on context processing. Here we isolated habitual attention from context processing in young and older adults. Using a challenging visual search task in which the probability of finding targets was greater in 1 of 4 visual quadrants in all contexts, we examined the acquisition, persistence, and spatial-reference frame of habitual attention. Although older adults showed slower visual search times and steeper search slopes (more time per additional item in the search display), like young adults they rapidly acquired a strong, persistent search habit toward the high-probability quadrant. In addition, habitual attention was strongly viewer-centered in both young and older adults. The demonstration of preserved viewer-centered habitual attention in older adults suggests that it may be used to counter declines in controlled attention. This, in turn, suggests the importance, for older adults, of maintaining habit-related spatial arrangements. (PsycINFO Database Record (c) 2016 APA, all rights reserved).

Misfortune may be a blessing in disguise: Fairness perception and emotion modulate decision making.

PubMed

Liu, Hong-Hsiang; Hwang, Yin-Dir; Hsieh, Ming H; Hsu, Yung-Fong; Lai, Wen-Sung

2017-08-01

Fairness perception and equality during social interactions frequently elicit affective arousal and affect decision making. By integrating the dictator game and a probabilistic gambling task, this study aimed to investigate the effects of a negative experience induced by perceived unfairness on decision making using behavioral, model fitting, and electrophysiological approaches. Participants were randomly assigned to the neutral, harsh, or kind groups, which consisted of various asset allocation scenarios to induce different levels of perceived unfairness. The monetary gain was subsequently considered the initial asset in a negatively rewarded, probabilistic gambling task in which the participants were instructed to maintain as much asset as possible. Our behavioral results indicated that the participants in the harsh group exhibited increased levels of negative emotions but retained greater total game scores than the participants in the other two groups. Parameter estimation of a reinforcement learning model using a Bayesian approach indicated that these participants were more loss aversive and consistent in decision making. Data from simultaneous ERP recordings further demonstrated that these participants exhibited larger feedback-related negativity to unexpected outcomes in the gambling task, which suggests enhanced reward sensitivity and signaling of reward prediction error. Collectively, our study suggests that a negative experience may be an advantage in the modulation of reward-based decision making. © 2017 Society for Psychophysiological Research.

The Sense of Confidence during Probabilistic Learning: A Normative Account.

PubMed

Meyniel, Florent; Schlunegger, Daniel; Dehaene, Stanislas

2015-06-01

Learning in a stochastic environment consists of estimating a model from a limited amount of noisy data, and is therefore inherently uncertain. However, many classical models reduce the learning process to the updating of parameter estimates and neglect the fact that learning is also frequently accompanied by a variable "feeling of knowing" or confidence. The characteristics and the origin of these subjective confidence estimates thus remain largely unknown. Here we investigate whether, during learning, humans not only infer a model of their environment, but also derive an accurate sense of confidence from their inferences. In our experiment, humans estimated the transition probabilities between two visual or auditory stimuli in a changing environment, and reported their mean estimate and their confidence in this report. To formalize the link between both kinds of estimate and assess their accuracy in comparison to a normative reference, we derive the optimal inference strategy for our task. Our results indicate that subjects accurately track the likelihood that their inferences are correct. Learning and estimating confidence in what has been learned appear to be two intimately related abilities, suggesting that they arise from a single inference process. We show that human performance matches several properties of the optimal probabilistic inference. In particular, subjective confidence is impacted by environmental uncertainty, both at the first level (uncertainty in stimulus occurrence given the inferred stochastic characteristics) and at the second level (uncertainty due to unexpected changes in these stochastic characteristics). Confidence also increases appropriately with the number of observations within stable periods. Our results support the idea that humans possess a quantitative sense of confidence in their inferences about abstract non-sensory parameters of the environment. This ability cannot be reduced to simple heuristics, it seems instead a core property of the learning process.

The Sense of Confidence during Probabilistic Learning: A Normative Account

PubMed Central

Meyniel, Florent; Schlunegger, Daniel; Dehaene, Stanislas

2015-01-01

Learning in a stochastic environment consists of estimating a model from a limited amount of noisy data, and is therefore inherently uncertain. However, many classical models reduce the learning process to the updating of parameter estimates and neglect the fact that learning is also frequently accompanied by a variable “feeling of knowing” or confidence. The characteristics and the origin of these subjective confidence estimates thus remain largely unknown. Here we investigate whether, during learning, humans not only infer a model of their environment, but also derive an accurate sense of confidence from their inferences. In our experiment, humans estimated the transition probabilities between two visual or auditory stimuli in a changing environment, and reported their mean estimate and their confidence in this report. To formalize the link between both kinds of estimate and assess their accuracy in comparison to a normative reference, we derive the optimal inference strategy for our task. Our results indicate that subjects accurately track the likelihood that their inferences are correct. Learning and estimating confidence in what has been learned appear to be two intimately related abilities, suggesting that they arise from a single inference process. We show that human performance matches several properties of the optimal probabilistic inference. In particular, subjective confidence is impacted by environmental uncertainty, both at the first level (uncertainty in stimulus occurrence given the inferred stochastic characteristics) and at the second level (uncertainty due to unexpected changes in these stochastic characteristics). Confidence also increases appropriately with the number of observations within stable periods. Our results support the idea that humans possess a quantitative sense of confidence in their inferences about abstract non-sensory parameters of the environment. This ability cannot be reduced to simple heuristics, it seems instead a core property of the learning process. PMID:26076466

Statistical Learning of Probabilistic Nonadjacent Dependencies by Multiple-Cue Integration

ERIC Educational Resources Information Center

van den Bos, Esther; Christiansen, Morten H.; Misyak, Jennifer B.

2012-01-01

Previous studies have indicated that dependencies between nonadjacent elements can be acquired by statistical learning when each element predicts only one other element (deterministic dependencies). The present study investigates statistical learning of probabilistic nonadjacent dependencies, in which each element predicts several other elements…

The cerebellum and decision making under uncertainty.

PubMed

Blackwood, Nigel; Ffytche, Dominic; Simmons, Andrew; Bentall, Richard; Murray, Robin; Howard, Robert

2004-06-01

This study aimed to identify the neural basis of probabilistic reasoning, a type of inductive inference that aids decision making under conditions of uncertainty. Eight normal subjects performed two separate two-alternative-choice tasks (the balls in a bottle and personality survey tasks) while undergoing functional magnetic resonance imaging (fMRI). The experimental conditions within each task were chosen so that they differed only in their requirement to make a decision under conditions of uncertainty (probabilistic reasoning and frequency determination required) or under conditions of certainty (frequency determination required). The same visual stimuli and motor responses were used in the experimental conditions. We provide evidence that the neo-cerebellum, in conjunction with the premotor cortex, inferior parietal lobule and medial occipital cortex, mediates the probabilistic inferences that guide decision making under uncertainty. We hypothesise that the neo-cerebellum constructs internal working models of uncertain events in the external world, and that such probabilistic models subserve the predictive capacity central to induction. Copyright 2004 Elsevier B.V.

Modeling development of natural multi-sensory integration using neural self-organisation and probabilistic population codes

NASA Astrophysics Data System (ADS)

Bauer, Johannes; Dávila-Chacón, Jorge; Wermter, Stefan

2015-10-01

Humans and other animals have been shown to perform near-optimally in multi-sensory integration tasks. Probabilistic population codes (PPCs) have been proposed as a mechanism by which optimal integration can be accomplished. Previous approaches have focussed on how neural networks might produce PPCs from sensory input or perform calculations using them, like combining multiple PPCs. Less attention has been given to the question of how the necessary organisation of neurons can arise and how the required knowledge about the input statistics can be learned. In this paper, we propose a model of learning multi-sensory integration based on an unsupervised learning algorithm in which an artificial neural network learns the noise characteristics of each of its sources of input. Our algorithm borrows from the self-organising map the ability to learn latent-variable models of the input and extends it to learning to produce a PPC approximating a probability density function over the latent variable behind its (noisy) input. The neurons in our network are only required to perform simple calculations and we make few assumptions about input noise properties and tuning functions. We report on a neurorobotic experiment in which we apply our algorithm to multi-sensory integration in a humanoid robot to demonstrate its effectiveness and compare it to human multi-sensory integration on the behavioural level. We also show in simulations that our algorithm performs near-optimally under certain plausible conditions, and that it reproduces important aspects of natural multi-sensory integration on the neural level.

Overconfidence among beginners: Is a little learning a dangerous thing?

PubMed

Sanchez, Carmen; Dunning, David

2018-01-01

Across 6 studies we investigated the development of overconfidence among beginners. In 4 of the studies, participants completed multicue probabilistic learning tasks (e.g., learning to diagnose "zombie diseases" from physical symptoms). Although beginners did not start out overconfident in their judgments, they rapidly surged to a "beginner's bubble" of overconfidence. This bubble was traced to exuberant and error-filled theorizing about how to approach the task formed after just a few learning experiences. Later trials challenged and refined those theories, leading to a temporary leveling off of confidence while performance incrementally improved, although confidence began to rise again after this pause. In 2 additional studies we found a real-world echo of this pattern of overconfidence across the life course. Self-ratings of financial literacy surged among young adults, then leveled off among older respondents until late adulthood, where it begins to rise again, with actual financial knowledge all the while rising more slowly, consistently, and incrementally throughout adulthood. Hence, when it comes to overconfident judgment, a little learning does appear to be a dangerous thing. Although beginners start with humble self-perceptions, with just a little experience their confidence races ahead of their actual performance. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

A Learning Framework for Winner-Take-All Networks with Stochastic Synapses.

PubMed

Mostafa, Hesham; Cauwenberghs, Gert

2018-06-01

Many recent generative models make use of neural networks to transform the probability distribution of a simple low-dimensional noise process into the complex distribution of the data. This raises the question of whether biological networks operate along similar principles to implement a probabilistic model of the environment through transformations of intrinsic noise processes. The intrinsic neural and synaptic noise processes in biological networks, however, are quite different from the noise processes used in current abstract generative networks. This, together with the discrete nature of spikes and local circuit interactions among the neurons, raises several difficulties when using recent generative modeling frameworks to train biologically motivated models. In this letter, we show that a biologically motivated model based on multilayer winner-take-all circuits and stochastic synapses admits an approximate analytical description. This allows us to use the proposed networks in a variational learning setting where stochastic backpropagation is used to optimize a lower bound on the data log likelihood, thereby learning a generative model of the data. We illustrate the generality of the proposed networks and learning technique by using them in a structured output prediction task and a semisupervised learning task. Our results extend the domain of application of modern stochastic network architectures to networks where synaptic transmission failure is the principal noise mechanism.

DYT1 dystonia increases risk taking in humans

PubMed Central

Arkadir, David; Radulescu, Angela; Raymond, Deborah; Lubarr, Naomi; Bressman, Susan B; Mazzoni, Pietro; Niv, Yael

2016-01-01

It has been difficult to link synaptic modification to overt behavioral changes. Rodent models of DYT1 dystonia, a motor disorder caused by a single gene mutation, demonstrate increased long-term potentiation and decreased long-term depression in corticostriatal synapses. Computationally, such asymmetric learning predicts risk taking in probabilistic tasks. Here we demonstrate abnormal risk taking in DYT1 dystonia patients, which is correlated with disease severity, thereby supporting striatal plasticity in shaping choice behavior in humans. DOI: http://dx.doi.org/10.7554/eLife.14155.001 PMID:27249418

Rethinking the learning of belief network probabilities

DOE Office of Scientific and Technical Information (OSTI.GOV)

Musick, R.

Belief networks are a powerful tool for knowledge discovery that provide concise, understandable probabilistic models of data. There are methods grounded in probability theory to incrementally update the relationships described by the belief network when new information is seen, to perform complex inferences over any set of variables in the data, to incorporate domain expertise and prior knowledge into the model, and to automatically learn the model from data. This paper concentrates on part of the belief network induction problem, that of learning the quantitative structure (the conditional probabilities), given the qualitative structure. In particular, the current practice of rotemore » learning the probabilities in belief networks can be significantly improved upon. We advance the idea of applying any learning algorithm to the task of conditional probability learning in belief networks, discuss potential benefits, and show results of applying neutral networks and other algorithms to a medium sized car insurance belief network. The results demonstrate from 10 to 100% improvements in model error rates over the current approaches.« less

Beyond Nonutilization: Irrelevant Cues Can Gate Learning in Probabilistic Categorization

ERIC Educational Resources Information Center

Little, Daniel R.; Lewandowsky, Stephan

2009-01-01

In probabilistic categorization, also known as multiple cue probability learning (MCPL), people learn to predict a discrete outcome on the basis of imperfectly valid cues. In MCPL, normatively irrelevant cues are usually ignored, which stands in apparent conflict with recent research in deterministic categorization that has shown that people…

One-year retention of general and sequence-specific skills in a probabilistic, serial reaction time task.

PubMed

Romano, Jennifer C; Howard, James H; Howard, Darlene V

2010-05-01

Procedural skills such as riding a bicycle and playing a musical instrument play a central role in daily life. Such skills are learned gradually and are retained throughout life. The present study investigated 1-year retention of procedural skill in a version of the widely used serial reaction time task (SRTT) in young and older motor-skill experts and older controls in two experiments. The young experts were college-age piano and action video-game players, and the older experts were piano players. Previous studies have reported sequence-specific skill retention in the SRTT as long as 2 weeks but not at 1 year. Results indicated that both young and older experts and older non-experts revealed sequence-specific skill retention after 1 year with some evidence that general motor skill was retained as well. These findings are consistent with theoretical accounts of procedural skill learning such as the procedural reinstatement theory as well as with previous studies of retention of other motor skills.

Synaptic and nonsynaptic plasticity approximating probabilistic inference

PubMed Central

Tully, Philip J.; Hennig, Matthias H.; Lansner, Anders

2014-01-01

Learning and memory operations in neural circuits are believed to involve molecular cascades of synaptic and nonsynaptic changes that lead to a diverse repertoire of dynamical phenomena at higher levels of processing. Hebbian and homeostatic plasticity, neuromodulation, and intrinsic excitability all conspire to form and maintain memories. But it is still unclear how these seemingly redundant mechanisms could jointly orchestrate learning in a more unified system. To this end, a Hebbian learning rule for spiking neurons inspired by Bayesian statistics is proposed. In this model, synaptic weights and intrinsic currents are adapted on-line upon arrival of single spikes, which initiate a cascade of temporally interacting memory traces that locally estimate probabilities associated with relative neuronal activation levels. Trace dynamics enable synaptic learning to readily demonstrate a spike-timing dependence, stably return to a set-point over long time scales, and remain competitive despite this stability. Beyond unsupervised learning, linking the traces with an external plasticity-modulating signal enables spike-based reinforcement learning. At the postsynaptic neuron, the traces are represented by an activity-dependent ion channel that is shown to regulate the input received by a postsynaptic cell and generate intrinsic graded persistent firing levels. We show how spike-based Hebbian-Bayesian learning can be performed in a simulated inference task using integrate-and-fire (IAF) neurons that are Poisson-firing and background-driven, similar to the preferred regime of cortical neurons. Our results support the view that neurons can represent information in the form of probability distributions, and that probabilistic inference could be a functional by-product of coupled synaptic and nonsynaptic mechanisms operating over several timescales. The model provides a biophysical realization of Bayesian computation by reconciling several observed neural phenomena whose functional effects are only partially understood in concert. PMID:24782758

Striatum-medial prefrontal cortex connectivity predicts developmental changes in reinforcement learning.

PubMed

van den Bos, Wouter; Cohen, Michael X; Kahnt, Thorsten; Crone, Eveline A

2012-06-01

During development, children improve in learning from feedback to adapt their behavior. However, it is still unclear which neural mechanisms might underlie these developmental changes. In the current study, we used a reinforcement learning model to investigate neurodevelopmental changes in the representation and processing of learning signals. Sixty-seven healthy volunteers between ages 8 and 22 (children: 8-11 years, adolescents: 13-16 years, and adults: 18-22 years) performed a probabilistic learning task while in a magnetic resonance imaging scanner. The behavioral data demonstrated age differences in learning parameters with a stronger impact of negative feedback on expected value in children. Imaging data revealed that the neural representation of prediction errors was similar across age groups, but functional connectivity between the ventral striatum and the medial prefrontal cortex changed as a function of age. Furthermore, the connectivity strength predicted the tendency to alter expectations after receiving negative feedback. These findings suggest that the underlying mechanisms of developmental changes in learning are not related to differences in the neural representation of learning signals per se but rather in how learning signals are used to guide behavior and expectations.

Emergence of β-Band Oscillations in the Aged Rat Amygdala during Discrimination Learning and Decision Making Tasks

PubMed Central

Samson, Rachel D.; Duarte, Leroy; Venkatesh, Anu

2017-01-01

Abstract Older adults tend to use strategies that differ from those used by young adults to solve decision-making tasks. MRI experiments suggest that altered strategy use during aging can be accompanied by a change in extent of activation of a given brain region, inter-hemispheric bilateralization or added brain structures. It has been suggested that these changes reflect compensation for less effective networks to enable optimal performance. One way that communication can be influenced within and between brain networks is through oscillatory events that help structure and synchronize incoming and outgoing information. It is unknown how aging impacts local oscillatory activity within the basolateral complex of the amygdala (BLA). The present study recorded local field potentials (LFPs) and single units in old and young rats during the performance of tasks that involve discrimination learning and probabilistic decision making. We found task- and age-specific increases in power selectively within the β range (15–30 Hz). The increased β power occurred after lever presses, as old animals reached the goal location. Periods of high-power β developed over training days in the aged rats, and was greatest in early trials of a session. β Power was also greater after pressing for the large reward option. These data suggest that aging of BLA networks results in strengthened synchrony of β oscillations when older animals are learning or deciding between rewards of different size. Whether this increased synchrony reflects the neural basis of a compensatory strategy change of old animals in reward-based decision-making tasks, remains to be verified. PMID:29034315

Pharmacological Fingerprints of Contextual Uncertainty

PubMed Central

Ruge, Diane; Stephan, Klaas E.

2016-01-01

Successful interaction with the environment requires flexible updating of our beliefs about the world. By estimating the likelihood of future events, it is possible to prepare appropriate actions in advance and execute fast, accurate motor responses. According to theoretical proposals, agents track the variability arising from changing environments by computing various forms of uncertainty. Several neuromodulators have been linked to uncertainty signalling, but comprehensive empirical characterisation of their relative contributions to perceptual belief updating, and to the selection of motor responses, is lacking. Here we assess the roles of noradrenaline, acetylcholine, and dopamine within a single, unified computational framework of uncertainty. Using pharmacological interventions in a sample of 128 healthy human volunteers and a hierarchical Bayesian learning model, we characterise the influences of noradrenergic, cholinergic, and dopaminergic receptor antagonism on individual computations of uncertainty during a probabilistic serial reaction time task. We propose that noradrenaline influences learning of uncertain events arising from unexpected changes in the environment. In contrast, acetylcholine balances attribution of uncertainty to chance fluctuations within an environmental context, defined by a stable set of probabilistic associations, or to gross environmental violations following a contextual switch. Dopamine supports the use of uncertainty representations to engender fast, adaptive responses. PMID:27846219

The profile of executive function in OCD hoarders and hoarding disorder☆

PubMed Central

Morein-Zamir, Sharon; Papmeyer, Martina; Pertusa, Alberto; Chamberlain, Samuel R.; Fineberg, Naomi A.; Sahakian, Barbara J.; Mataix-Cols, David; Robbins, Trevor W.

2014-01-01

Hoarding disorder is a new mental disorder in DSM-5. It is classified alongside OCD and other presumably related disorders in the Obsessive-Compulsive and Related Disorders chapter. We examined cognitive performance in two distinct groups comprising individuals with both OCD and severe hoarding, and individuals with hoarding disorder without comorbid OCD. Participants completed executive function tasks assessing inhibitory control, cognitive flexibility, spatial planning, probabilistic learning and reversal and decision making. Compared to a matched healthy control group, OCD hoarders showed significantly worse performance on measures of response inhibition, set shifting, spatial planning, probabilistic learning and reversal, with intact decision making. Despite having a strikingly different clinical presentation, individuals with only hoarding disorder did not differ significantly from OCD hoarders on any cognitive measure suggesting the two hoarding groups have a similar pattern of cognitive difficulties. Tests of cognitive flexibility were least similar across the groups, but differences were small and potentially reflected subtle variation in underlying brain pathology together with psychometric limitations. These results highlight both commonalities and potential differences between OCD and hoarding disorder, and together with other lines of evidence, support the inclusion of the new disorder within the new Obsessive-Compulsive and Related Disorders chapter in DSM-5. PMID:24467873

Learning Additional Languages as Hierarchical Probabilistic Inference: Insights From First Language Processing.

PubMed

Pajak, Bozena; Fine, Alex B; Kleinschmidt, Dave F; Jaeger, T Florian

2016-12-01

We present a framework of second and additional language (L2/L n ) acquisition motivated by recent work on socio-indexical knowledge in first language (L1) processing. The distribution of linguistic categories covaries with socio-indexical variables (e.g., talker identity, gender, dialects). We summarize evidence that implicit probabilistic knowledge of this covariance is critical to L1 processing, and propose that L2/L n learning uses the same type of socio-indexical information to probabilistically infer latent hierarchical structure over previously learned and new languages. This structure guides the acquisition of new languages based on their inferred place within that hierarchy, and is itself continuously revised based on new input from any language. This proposal unifies L1 processing and L2/L n acquisition as probabilistic inference under uncertainty over socio-indexical structure. It also offers a new perspective on crosslinguistic influences during L2/L n learning, accommodating gradient and continued transfer (both negative and positive) from previously learned to novel languages, and vice versa.

Learning Additional Languages as Hierarchical Probabilistic Inference: Insights From First Language Processing

PubMed Central

Pajak, Bozena; Fine, Alex B.; Kleinschmidt, Dave F.; Jaeger, T. Florian

2015-01-01

We present a framework of second and additional language (L2/Ln) acquisition motivated by recent work on socio-indexical knowledge in first language (L1) processing. The distribution of linguistic categories covaries with socio-indexical variables (e.g., talker identity, gender, dialects). We summarize evidence that implicit probabilistic knowledge of this covariance is critical to L1 processing, and propose that L2/Ln learning uses the same type of socio-indexical information to probabilistically infer latent hierarchical structure over previously learned and new languages. This structure guides the acquisition of new languages based on their inferred place within that hierarchy, and is itself continuously revised based on new input from any language. This proposal unifies L1 processing and L2/Ln acquisition as probabilistic inference under uncertainty over socio-indexical structure. It also offers a new perspective on crosslinguistic influences during L2/Ln learning, accommodating gradient and continued transfer (both negative and positive) from previously learned to novel languages, and vice versa. PMID:28348442

The composite load spectra project

NASA Technical Reports Server (NTRS)

Newell, J. F.; Ho, H.; Kurth, R. E.

1990-01-01

Probabilistic methods and generic load models capable of simulating the load spectra that are induced in space propulsion system components are being developed. Four engine component types (the transfer ducts, the turbine blades, the liquid oxygen posts and the turbopump oxidizer discharge duct) were selected as representative hardware examples. The composite load spectra that simulate the probabilistic loads for these components are typically used as the input loads for a probabilistic structural analysis. The knowledge-based system approach used for the composite load spectra project provides an ideal environment for incremental development. The intelligent database paradigm employed in developing the expert system provides a smooth coupling between the numerical processing and the symbolic (information) processing. Large volumes of engine load information and engineering data are stored in database format and managed by a database management system. Numerical procedures for probabilistic load simulation and database management functions are controlled by rule modules. Rules were hard-wired as decision trees into rule modules to perform process control tasks. There are modules to retrieve load information and models. There are modules to select loads and models to carry out quick load calculations or make an input file for full duty-cycle time dependent load simulation. The composite load spectra load expert system implemented today is capable of performing intelligent rocket engine load spectra simulation. Further development of the expert system will provide tutorial capability for users to learn from it.

Big data learning and suggestions in modern apps

NASA Astrophysics Data System (ADS)

Sharma, G.; Nadesh, R. K.; ArivuSelvan, K.

2017-11-01

Among many other tasks involved for emergent location-based applications such as those involved in prescribing touring places and those focused on publicizing based on destination, destination prediction is vital. Dealing with destination prediction involves determining the probability of a location (destination) depending on historical trajectories. In this paper, a destination prediction based on probabilistic model (Machine Learning Model) feed-forward neural networks will be presented, which will work by making the observation of driver’s habits. Some individuals drive to same locations such as work involving same route every day of the working week. Here, streaming of real-time driving data will be sent through Kafka queue in apache storm for real-time processing and finally storing the data in MongoDB.

Translational Rodent Paradigms to Investigate Neuromechanisms Underlying Behaviors Relevant to Amotivation and Altered Reward Processing in Schizophrenia

PubMed Central

Young, Jared W.; Markou, Athina

2015-01-01

Amotivation and reward-processing deficits have long been described in patients with schizophrenia and considered large contributors to patients’ inability to integrate well in society. No effective treatments exist for these symptoms, partly because the neuromechanisms mediating such symptoms are poorly understood. Here, we propose a translational neuroscientific approach that can be used to assess reward/motivational deficits related to the negative symptoms of schizophrenia using behavioral paradigms that can also be conducted in experimental animals. By designing and using objective laboratory behavioral tools that are parallel in their parameters in rodents and humans, the neuromechanisms underlying behaviors with relevance to these symptoms of schizophrenia can be investigated. We describe tasks that measure the motivation of rodents to expend physical and cognitive effort to gain rewards, as well as probabilistic learning tasks that assess both reward learning and feedback-based decision making. The latter tasks are relevant because of demonstrated links of performance deficits correlating with negative symptoms in patients with schizophrenia. These tasks utilize operant techniques in order to investigate neural circuits targeting a specific domain across species. These tasks therefore enable the development of insights into altered mechanisms leading to negative symptom-relevant behaviors in patients with schizophrenia. Such findings will then enable the development of targeted treatments for these altered neuromechanisms and behaviors seen in schizophrenia. PMID:26194891

Reconstructing Constructivism: Causal Models, Bayesian Learning Mechanisms, and the Theory Theory

ERIC Educational Resources Information Center

Gopnik, Alison; Wellman, Henry M.

2012-01-01

We propose a new version of the "theory theory" grounded in the computational framework of probabilistic causal models and Bayesian learning. Probabilistic models allow a constructivist but rigorous and detailed approach to cognitive development. They also explain the learning of both more specific causal hypotheses and more abstract framework…

Distinct Roles of Dopamine and Subthalamic Nucleus in Learning and Probabilistic Decision Making

ERIC Educational Resources Information Center

Coulthard, Elizabeth J.; Bogacz, Rafal; Javed, Shazia; Mooney, Lucy K.; Murphy, Gillian; Keeley, Sophie; Whone, Alan L.

2012-01-01

Even simple behaviour requires us to make decisions based on combining multiple pieces of learned and new information. Making such decisions requires both learning the optimal response to each given stimulus as well as combining probabilistic information from multiple stimuli before selecting a response. Computational theories of decision making…

The Neural Correlates of Implicit Sequence Learning in Schizophrenia

PubMed Central

Marvel, Cherie L.; Turner, Beth M.; O’Leary, Daniel S.; Johnson, Hans J.; Pierson, Ronald K.; Boles Ponto, Laura L.; Andreasen, Nancy C.

2009-01-01

Twenty-seven schizophrenia spectrum patients and 25 healthy controls performed a probabilistic version of the serial reaction time task (SRT) that included sequence trials embedded within random trials. Patients showed diminished, yet measurable, sequence learning. Postexperimental analyses revealed that a group of patients performed above chance when generating short spans of the sequence. This high-generation group showed SRT learning that was similar in magnitude to that of controls. Their learning was evident from the very 1st block; however, unlike controls, learning did not develop further with continued testing. A subset of 12 patients and 11 controls performed the SRT in conjunction with positron emission tomography. High-generation performance, which corresponded to SRT learning in patients, correlated to activity in the premotor cortex and parahippocampus. These areas have been associated with stimulus-driven visuospatial processing. Taken together, these results suggest that a subset of patients who showed moderate success on the SRT used an explicit stimulus-driven strategy to process the sequential stimuli. This adaptive strategy facilitated sequence learning but may have interfered with conventional implicit learning of the overall stimulus pattern. PMID:17983290

A Guide to the Literature on Learning Graphical Models

NASA Technical Reports Server (NTRS)

Buntine, Wray L.; Friedland, Peter (Technical Monitor)

1994-01-01

This literature review discusses different methods under the general rubric of learning Bayesian networks from data, and more generally, learning probabilistic graphical models. Because many problems in artificial intelligence, statistics and neural networks can be represented as a probabilistic graphical model, this area provides a unifying perspective on learning. This paper organizes the research in this area along methodological lines of increasing complexity.

Confidence and psychosis: a neuro-computational account of contingency learning disruption by NMDA blockade

PubMed Central

Vinckier, F; Gaillard, R; Palminteri, S; Rigoux, L; Salvador, A; Fornito, A; Adapa, R; Krebs, M O; Pessiglione, M; Fletcher, P C

2016-01-01

A state of pathological uncertainty about environmental regularities might represent a key step in the pathway to psychotic illness. Early psychosis can be investigated in healthy volunteers under ketamine, an NMDA receptor antagonist. Here, we explored the effects of ketamine on contingency learning using a placebo-controlled, double-blind, crossover design. During functional magnetic resonance imaging, participants performed an instrumental learning task, in which cue-outcome contingencies were probabilistic and reversed between blocks. Bayesian model comparison indicated that in such an unstable environment, reinforcement learning parameters are downregulated depending on confidence level, an adaptive mechanism that was specifically disrupted by ketamine administration. Drug effects were underpinned by altered neural activity in a fronto-parietal network, which reflected the confidence-based shift to exploitation of learned contingencies. Our findings suggest that an early characteristic of psychosis lies in a persistent doubt that undermines the stabilization of behavioral policy resulting in a failure to exploit regularities in the environment. PMID:26055423

Confidence and psychosis: a neuro-computational account of contingency learning disruption by NMDA blockade.

PubMed

Vinckier, F; Gaillard, R; Palminteri, S; Rigoux, L; Salvador, A; Fornito, A; Adapa, R; Krebs, M O; Pessiglione, M; Fletcher, P C

2016-07-01

A state of pathological uncertainty about environmental regularities might represent a key step in the pathway to psychotic illness. Early psychosis can be investigated in healthy volunteers under ketamine, an NMDA receptor antagonist. Here, we explored the effects of ketamine on contingency learning using a placebo-controlled, double-blind, crossover design. During functional magnetic resonance imaging, participants performed an instrumental learning task, in which cue-outcome contingencies were probabilistic and reversed between blocks. Bayesian model comparison indicated that in such an unstable environment, reinforcement learning parameters are downregulated depending on confidence level, an adaptive mechanism that was specifically disrupted by ketamine administration. Drug effects were underpinned by altered neural activity in a fronto-parietal network, which reflected the confidence-based shift to exploitation of learned contingencies. Our findings suggest that an early characteristic of psychosis lies in a persistent doubt that undermines the stabilization of behavioral policy resulting in a failure to exploit regularities in the environment.

Constraints on decision making: implications from genetics, personality, and addiction.

PubMed

Baker, Travis E; Stockwell, Tim; Holroyd, Clay B

2013-09-01

An influential neurocomputational theory of the biological mechanisms of decision making, the "basal ganglia go/no-go model," holds that individual variability in decision making is determined by differences in the makeup of a striatal system for approach and avoidance learning. The model has been tested empirically with the probabilistic selection task (PST), which determines whether individuals learn better from positive or negative feedback. In accordance with the model, in the present study we examined whether an individual's ability to learn from positive and negative reinforcement can be predicted by genetic factors related to the midbrain dopamine system. We also asked whether psychiatric and personality factors related to substance dependence and dopamine affect PST performance. Although we found characteristics that predicted individual differences in approach versus avoidance learning, these observations were qualified by additional findings that appear inconsistent with the predictions of the go/no-go model. These results highlight a need for future research to validate the PST as a measure of basal ganglia reward learning.

Drift diffusion model of reward and punishment learning in schizophrenia: Modeling and experimental data.

PubMed

Moustafa, Ahmed A; Kéri, Szabolcs; Somlai, Zsuzsanna; Balsdon, Tarryn; Frydecka, Dorota; Misiak, Blazej; White, Corey

2015-09-15

In this study, we tested reward- and punishment learning performance using a probabilistic classification learning task in patients with schizophrenia (n=37) and healthy controls (n=48). We also fit subjects' data using a Drift Diffusion Model (DDM) of simple decisions to investigate which components of the decision process differ between patients and controls. Modeling results show between-group differences in multiple components of the decision process. Specifically, patients had slower motor/encoding time, higher response caution (favoring accuracy over speed), and a deficit in classification learning for punishment, but not reward, trials. The results suggest that patients with schizophrenia adopt a compensatory strategy of favoring accuracy over speed to improve performance, yet still show signs of a deficit in learning based on negative feedback. Our data highlights the importance of applying fitting models (particularly drift diffusion models) to behavioral data. The implications of these findings are discussed relative to theories of schizophrenia and cognitive processing. Copyright © 2015 Elsevier B.V. All rights reserved.

Classifying Cognitive Profiles Using Machine Learning with Privileged Information in Mild Cognitive Impairment.

PubMed

Alahmadi, Hanin H; Shen, Yuan; Fouad, Shereen; Luft, Caroline Di B; Bentham, Peter; Kourtzi, Zoe; Tino, Peter

2016-01-01

Early diagnosis of dementia is critical for assessing disease progression and potential treatment. State-or-the-art machine learning techniques have been increasingly employed to take on this diagnostic task. In this study, we employed Generalized Matrix Learning Vector Quantization (GMLVQ) classifiers to discriminate patients with Mild Cognitive Impairment (MCI) from healthy controls based on their cognitive skills. Further, we adopted a "Learning with privileged information" approach to combine cognitive and fMRI data for the classification task. The resulting classifier operates solely on the cognitive data while it incorporates the fMRI data as privileged information (PI) during training. This novel classifier is of practical use as the collection of brain imaging data is not always possible with patients and older participants. MCI patients and healthy age-matched controls were trained to extract structure from temporal sequences. We ask whether machine learning classifiers can be used to discriminate patients from controls and whether differences between these groups relate to individual cognitive profiles. To this end, we tested participants in four cognitive tasks: working memory, cognitive inhibition, divided attention, and selective attention. We also collected fMRI data before and after training on a probabilistic sequence learning task and extracted fMRI responses and connectivity as features for machine learning classifiers. Our results show that the PI guided GMLVQ classifiers outperform the baseline classifier that only used the cognitive data. In addition, we found that for the baseline classifier, divided attention is the only relevant cognitive feature. When PI was incorporated, divided attention remained the most relevant feature while cognitive inhibition became also relevant for the task. Interestingly, this analysis for the fMRI GMLVQ classifier suggests that (1) when overall fMRI signal is used as inputs to the classifier, the post-training session is most relevant; and (2) when the graph feature reflecting underlying spatiotemporal fMRI pattern is used, the pre-training session is most relevant. Taken together these results suggest that brain connectivity before training and overall fMRI signal after training are both diagnostic of cognitive skills in MCI.

Relationship between perceptual learning in speech and statistical learning in younger and older adults

PubMed Central

Neger, Thordis M.; Rietveld, Toni; Janse, Esther

2014-01-01

Within a few sentences, listeners learn to understand severely degraded speech such as noise-vocoded speech. However, individuals vary in the amount of such perceptual learning and it is unclear what underlies these differences. The present study investigates whether perceptual learning in speech relates to statistical learning, as sensitivity to probabilistic information may aid identification of relevant cues in novel speech input. If statistical learning and perceptual learning (partly) draw on the same general mechanisms, then statistical learning in a non-auditory modality using non-linguistic sequences should predict adaptation to degraded speech. In the present study, 73 older adults (aged over 60 years) and 60 younger adults (aged between 18 and 30 years) performed a visual artificial grammar learning task and were presented with 60 meaningful noise-vocoded sentences in an auditory recall task. Within age groups, sentence recognition performance over exposure was analyzed as a function of statistical learning performance, and other variables that may predict learning (i.e., hearing, vocabulary, attention switching control, working memory, and processing speed). Younger and older adults showed similar amounts of perceptual learning, but only younger adults showed significant statistical learning. In older adults, improvement in understanding noise-vocoded speech was constrained by age. In younger adults, amount of adaptation was associated with lexical knowledge and with statistical learning ability. Thus, individual differences in general cognitive abilities explain listeners' variability in adapting to noise-vocoded speech. Results suggest that perceptual and statistical learning share mechanisms of implicit regularity detection, but that the ability to detect statistical regularities is impaired in older adults if visual sequences are presented quickly. PMID:25225475

Relationship between perceptual learning in speech and statistical learning in younger and older adults.

PubMed

Neger, Thordis M; Rietveld, Toni; Janse, Esther

2014-01-01

Within a few sentences, listeners learn to understand severely degraded speech such as noise-vocoded speech. However, individuals vary in the amount of such perceptual learning and it is unclear what underlies these differences. The present study investigates whether perceptual learning in speech relates to statistical learning, as sensitivity to probabilistic information may aid identification of relevant cues in novel speech input. If statistical learning and perceptual learning (partly) draw on the same general mechanisms, then statistical learning in a non-auditory modality using non-linguistic sequences should predict adaptation to degraded speech. In the present study, 73 older adults (aged over 60 years) and 60 younger adults (aged between 18 and 30 years) performed a visual artificial grammar learning task and were presented with 60 meaningful noise-vocoded sentences in an auditory recall task. Within age groups, sentence recognition performance over exposure was analyzed as a function of statistical learning performance, and other variables that may predict learning (i.e., hearing, vocabulary, attention switching control, working memory, and processing speed). Younger and older adults showed similar amounts of perceptual learning, but only younger adults showed significant statistical learning. In older adults, improvement in understanding noise-vocoded speech was constrained by age. In younger adults, amount of adaptation was associated with lexical knowledge and with statistical learning ability. Thus, individual differences in general cognitive abilities explain listeners' variability in adapting to noise-vocoded speech. Results suggest that perceptual and statistical learning share mechanisms of implicit regularity detection, but that the ability to detect statistical regularities is impaired in older adults if visual sequences are presented quickly.

Probabilistic Learning by Rodent Grid Cells

PubMed Central

Cheung, Allen

2016-01-01

Mounting evidence shows mammalian brains are probabilistic computers, but the specific cells involved remain elusive. Parallel research suggests that grid cells of the mammalian hippocampal formation are fundamental to spatial cognition but their diverse response properties still defy explanation. No plausible model exists which explains stable grids in darkness for twenty minutes or longer, despite being one of the first results ever published on grid cells. Similarly, no current explanation can tie together grid fragmentation and grid rescaling, which show very different forms of flexibility in grid responses when the environment is varied. Other properties such as attractor dynamics and grid anisotropy seem to be at odds with one another unless additional properties are assumed such as a varying velocity gain. Modelling efforts have largely ignored the breadth of response patterns, while also failing to account for the disastrous effects of sensory noise during spatial learning and recall, especially in darkness. Here, published electrophysiological evidence from a range of experiments are reinterpreted using a novel probabilistic learning model, which shows that grid cell responses are accurately predicted by a probabilistic learning process. Diverse response properties of probabilistic grid cells are statistically indistinguishable from rat grid cells across key manipulations. A simple coherent set of probabilistic computations explains stable grid fields in darkness, partial grid rescaling in resized arenas, low-dimensional attractor grid cell dynamics, and grid fragmentation in hairpin mazes. The same computations also reconcile oscillatory dynamics at the single cell level with attractor dynamics at the cell ensemble level. Additionally, a clear functional role for boundary cells is proposed for spatial learning. These findings provide a parsimonious and unified explanation of grid cell function, and implicate grid cells as an accessible neuronal population readout of a set of probabilistic spatial computations. PMID:27792723

The Gain-Loss Model: A Probabilistic Skill Multimap Model for Assessing Learning Processes

ERIC Educational Resources Information Center

Robusto, Egidio; Stefanutti, Luca; Anselmi, Pasquale

2010-01-01

Within the theoretical framework of knowledge space theory, a probabilistic skill multimap model for assessing learning processes is proposed. The learning process of a student is modeled as a function of the student's knowledge and of an educational intervention on the attainment of specific skills required to solve problems in a knowledge…

Hold it! The influence of lingering rewards on choice diversification and persistence.

PubMed

Schulze, Christin; van Ravenzwaaij, Don; Newell, Ben R

2017-11-01

Learning to choose adaptively when faced with uncertain and variable outcomes is a central challenge for decision makers. This study examines repeated choice in dynamic probability learning tasks in which outcome probabilities changed either as a function of the choices participants made or independently of those choices. This presence/absence of sequential choice-outcome dependencies was implemented by manipulating a single task aspect between conditions: the retention/withdrawal of reward across individual choice trials. The study addresses how people adapt to these learning environments and to what extent they engage in 2 choice strategies often contrasted as paradigmatic examples of striking violation of versus nominal adherence to rational choice: diversification and persistent probability maximizing, respectively. Results show that decisions approached adaptive choice diversification and persistence when sufficient feedback was provided on the dynamic rules of the probabilistic environments. The findings of divergent behavior in the 2 environments indicate that diversified choices represented a response to the reward retention manipulation rather than to the mere variability of outcome probabilities. Choice in both environments was well accounted for by the generalized matching law, and computational modeling-based strategy analyses indicated that adaptive choice arose mainly from reliance on reinforcement learning strategies. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

Bayesian deterministic decision making: a normative account of the operant matching law and heavy-tailed reward history dependency of choices.

PubMed

Saito, Hiroshi; Katahira, Kentaro; Okanoya, Kazuo; Okada, Masato

2014-01-01

The decision making behaviors of humans and animals adapt and then satisfy an "operant matching law" in certain type of tasks. This was first pointed out by Herrnstein in his foraging experiments on pigeons. The matching law has been one landmark for elucidating the underlying processes of decision making and its learning in the brain. An interesting question is whether decisions are made deterministically or probabilistically. Conventional learning models of the matching law are based on the latter idea; they assume that subjects learn choice probabilities of respective alternatives and decide stochastically with the probabilities. However, it is unknown whether the matching law can be accounted for by a deterministic strategy or not. To answer this question, we propose several deterministic Bayesian decision making models that have certain incorrect beliefs about an environment. We claim that a simple model produces behavior satisfying the matching law in static settings of a foraging task but not in dynamic settings. We found that the model that has a belief that the environment is volatile works well in the dynamic foraging task and exhibits undermatching, which is a slight deviation from the matching law observed in many experiments. This model also demonstrates the double-exponential reward history dependency of a choice and a heavier-tailed run-length distribution, as has recently been reported in experiments on monkeys.

Reward prediction error signal enhanced by striatum-amygdala interaction explains the acceleration of probabilistic reward learning by emotion.

PubMed

Watanabe, Noriya; Sakagami, Masamichi; Haruno, Masahiko

2013-03-06

Learning does not only depend on rationality, because real-life learning cannot be isolated from emotion or social factors. Therefore, it is intriguing to determine how emotion changes learning, and to identify which neural substrates underlie this interaction. Here, we show that the task-independent presentation of an emotional face before a reward-predicting cue increases the speed of cue-reward association learning in human subjects compared with trials in which a neutral face is presented. This phenomenon was attributable to an increase in the learning rate, which regulates reward prediction errors. Parallel to these behavioral findings, functional magnetic resonance imaging demonstrated that presentation of an emotional face enhanced reward prediction error (RPE) signal in the ventral striatum. In addition, we also found a functional link between this enhanced RPE signal and increased activity in the amygdala following presentation of an emotional face. Thus, this study revealed an acceleration of cue-reward association learning by emotion, and underscored a role of striatum-amygdala interactions in the modulation of the reward prediction errors by emotion.

Coupled dimensionality reduction and classification for supervised and semi-supervised multilabel learning

PubMed Central

Gönen, Mehmet

2014-01-01

Coupled training of dimensionality reduction and classification is proposed previously to improve the prediction performance for single-label problems. Following this line of research, in this paper, we first introduce a novel Bayesian method that combines linear dimensionality reduction with linear binary classification for supervised multilabel learning and present a deterministic variational approximation algorithm to learn the proposed probabilistic model. We then extend the proposed method to find intrinsic dimensionality of the projected subspace using automatic relevance determination and to handle semi-supervised learning using a low-density assumption. We perform supervised learning experiments on four benchmark multilabel learning data sets by comparing our method with baseline linear dimensionality reduction algorithms. These experiments show that the proposed approach achieves good performance values in terms of hamming loss, average AUC, macro F1, and micro F1 on held-out test data. The low-dimensional embeddings obtained by our method are also very useful for exploratory data analysis. We also show the effectiveness of our approach in finding intrinsic subspace dimensionality and semi-supervised learning tasks. PMID:24532862

Coupled dimensionality reduction and classification for supervised and semi-supervised multilabel learning.

PubMed

Gönen, Mehmet

2014-03-01

Coupled training of dimensionality reduction and classification is proposed previously to improve the prediction performance for single-label problems. Following this line of research, in this paper, we first introduce a novel Bayesian method that combines linear dimensionality reduction with linear binary classification for supervised multilabel learning and present a deterministic variational approximation algorithm to learn the proposed probabilistic model. We then extend the proposed method to find intrinsic dimensionality of the projected subspace using automatic relevance determination and to handle semi-supervised learning using a low-density assumption. We perform supervised learning experiments on four benchmark multilabel learning data sets by comparing our method with baseline linear dimensionality reduction algorithms. These experiments show that the proposed approach achieves good performance values in terms of hamming loss, average AUC, macro F 1 , and micro F 1 on held-out test data. The low-dimensional embeddings obtained by our method are also very useful for exploratory data analysis. We also show the effectiveness of our approach in finding intrinsic subspace dimensionality and semi-supervised learning tasks.

Punishment Learning in U.S. Veterans With Posttraumatic Stress Disorder.

PubMed

Sawyer, Alice T; Liverant, Gabrielle I; Jun, Janie J; Lee, Daniel J; Cohen, Andrew L; Dutra, Sunny J; Pizzagalli, Diego A; Sloan, Denise M

2016-08-01

Learning processes have been implicated in the development and course of posttraumatic stress disorder (PTSD); however, little is currently known about punishment-based learning in PTSD. The current study investigated impairments in punishment-based learning in U.S. veterans. We expected that veterans with PTSD would demonstrate greater punishment-based learning compared to a non-PTSD control group. We compared a PTSD group with and without co-occurring depression (n = 27) to a control group (with and without trauma exposure) without PTSD or depression (n = 29). Participants completed a computerized probabilistic punishment-based learning task. Compared to the non-PTSD control group, veterans with PTSD showed significantly greater punishment-based learning. Specifically, there was a significant Block × Group interaction, F(1, 54) = 4.12, p = .047, η(2) = .07. Veterans with PTSD demonstrated greater change in response bias for responding toward a less frequently punished stimulus across blocks. The observed hypersensitivity to punishment in individuals with PTSD may contribute to avoidant responses that are not specific to trauma cues. Copyright © 2016 International Society for Traumatic Stress Studies No claim to original US government works.

Fear of negative evaluation biases social evaluation inference: evidence from a probabilistic learning task.

PubMed

Button, Katherine S; Kounali, Daphne; Stapinski, Lexine; Rapee, Ronald M; Lewis, Glyn; Munafò, Marcus R

2015-01-01

Fear of negative evaluation (FNE) defines social anxiety yet the process of inferring social evaluation, and its potential role in maintaining social anxiety, is poorly understood. We developed an instrumental learning task to model social evaluation learning, predicting that FNE would specifically bias learning about the self but not others. During six test blocks (3 self-referential, 3 other-referential), participants (n = 100) met six personas and selected a word from a positive/negative pair to finish their social evaluation sentences "I think [you are / George is]…". Feedback contingencies corresponded to 3 rules, liked, neutral and disliked, with P[positive word correct] = 0.8, 0.5 and 0.2, respectively. As FNE increased participants selected fewer positive words (β = -0.4, 95% CI -0.7, -0.2, p = 0.001), which was strongest in the self-referential condition (FNE × condition 0.28, 95% CI 0.01, 0.54, p = 0.04), and the neutral and dislike rules (FNE × condition × rule, p = 0.07). At low FNE the proportion of positive words selected for self-neutral and self-disliked greatly exceeded the feedback contingency, indicating poor learning, which improved as FNE increased. FNE is associated with differences in processing social-evaluative information specifically about the self. At low FNE this manifests as insensitivity to learning negative self-referential evaluation. High FNE individuals are equally sensitive to learning positive or negative evaluation, which although objectively more accurate, may have detrimental effects on mental health.

Rasagiline in the Treatment of the Persistent Negative Symptoms of Schizophrenia.

PubMed

Buchanan, Robert W; Weiner, Elaine; Kelly, Deanna L; Gold, James M; Keller, William R; Waltz, James A; McMahon, Robert P; Gorelick, David A

2015-07-01

The current study examined the efficacy and safety of rasagiline, a selective MAO-B inhibitor, for the treatment of persistent negative symptoms. Sixty people with Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, schizophrenia or schizoaffective disorder, who met a priori criteria for persistent negative symptoms, were randomized to receive rasagiline, 1mg/d (n = 31) or placebo (n = 29) in a 12-week, double-blind, placebo-controlled clinical trial. The Scale for the Assessment of Negative Symptoms (SANS) total score was used to assess change in negative symptoms. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), N-Back test, a probabilistic learning task, and a delayed discounting task were used to assess cognition. In a mixed model analysis of covariance (MM-ANCOVA), with time as a continuous variable, there was a significant treatment × time effect for SANS total score (F = 5.61(df = 1,40.3), P = .023). The treatment × time interaction effect was also significant for the SANS avolition subscale score (F(1,40.2) = 10.41, P = .002). In a post hoc MM-ANCOVA analyses, with time as a categorical variable, group differences were significant at week 12 for SANS total score (t(37.3) = 2.15; P = .04; d = -0.41) and SANS avolition subscale score (t(49.0) = 3.06; P = .004; d = -0.46). There was a significant difference in number of participants with a ≥20% reduction in SANS avolition score (χ(2)(1) = 10.94; P = .0009), but not in SANS total score (χ(2)(1) = 1.11; P = .29). There were no significant group differences on the RBANS, N-Back, probabilistic learning, or delayed discounting tasks. Study results support future studies of the utility of rasagiline for the treatment of negative symptoms, including avolition (clinicaltrials.gov trial number: NCT00492336). © The Author 2014. Published by Oxford University Press on behalf of the Maryland Psychiatric Research Center. All rights reserved. For permissions, please email: journals.permissions@oup.com.

Rasagiline in the Treatment of the Persistent Negative Symptoms of Schizophrenia

PubMed Central

Buchanan, Robert W.; Weiner, Elaine; Kelly, Deanna L.; Gold, James M.; Keller, William R.; Waltz, James A.; McMahon, Robert P.; Gorelick, David A.

2015-01-01

Objective: The current study examined the efficacy and safety of rasagiline, a selective MAO-B inhibitor, for the treatment of persistent negative symptoms. Methods: Sixty people with Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, schizophrenia or schizoaffective disorder, who met a priori criteria for persistent negative symptoms, were randomized to receive rasagiline, 1mg/d (n = 31) or placebo (n = 29) in a 12-week, double-blind, placebo-controlled clinical trial. The Scale for the Assessment of Negative Symptoms (SANS) total score was used to assess change in negative symptoms. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), N-Back test, a probabilistic learning task, and a delayed discounting task were used to assess cognition. Results: In a mixed model analysis of covariance (MM-ANCOVA), with time as a continuous variable, there was a significant treatment × time effect for SANS total score (F = 5.61(df = 1,40.3), P = .023). The treatment × time interaction effect was also significant for the SANS avolition subscale score (F(1,40.2) = 10.41, P = .002). In a post hoc MM-ANCOVA analyses, with time as a categorical variable, group differences were significant at week 12 for SANS total score (t(37.3) = 2.15; P = .04; d = −0.41) and SANS avolition subscale score (t(49.0) = 3.06; P = .004; d = −0.46). There was a significant difference in number of participants with a ≥20% reduction in SANS avolition score (χ2(1) = 10.94; P = .0009), but not in SANS total score (χ2(1) = 1.11; P = .29). There were no significant group differences on the RBANS, N-Back, probabilistic learning, or delayed discounting tasks. Conclusions: Study results support future studies of the utility of rasagiline for the treatment of negative symptoms, including avolition (clinicaltrials.gov trial number: NCT00492336). PMID:25368372

Chronic pain impairs cognitive flexibility and engages novel learning strategies in rats.

PubMed

Cowen, Stephen L; Phelps, Caroline E; Navratilova, Edita; McKinzie, David L; Okun, Alec; Husain, Omar; Gleason, Scott D; Witkin, Jeffrey M; Porreca, Frank

2018-03-22

Cognitive flexibility, the ability to adapt behavior to changing outcomes, is critical for survival. The prefrontal cortex is a key site of cognitive control and chronic pain is known to lead to significant morphological changes to this brain region. Nevertheless, the effects of chronic pain on cognitive flexibility and learning remain uncertain. We used an instrumental paradigm to assess adaptive learning in an experimental model of chronic pain induced by tight ligation of the spinal nerves L5/6 (SNL model). Naïve, sham-operated, and SNL rats were trained to perform fixed-ratio, variable-ratio, and contingency-shift behaviors for food reward. Although all groups learned an initial lever-reward contingency, learning was slower in SNL animals in a subsequent choice task that reversed reinforcement contingencies. Temporal analysis of lever-press responses across sessions indicated no apparent deficits in memory consolidation or retrieval. However, analysis of learning within sessions revealed that the lever presses of SNL animals occurred in bursts followed by delays. Unexpectedly, the degree of bursting correlated positively with learning. Under a variable-ratio probabilistic task, SNL rats chose a less profitable behavioral strategy compared to naïve and sham-operated animals. Following extinction of behavior for learned preferences, SNL animals reverted to their initially preferred (i.e., less profitable) behavioral choice. Our data suggest, that in the face of uncertainty, chronic pain drives a preference for familiar associations, consistent with reduced cognitive flexibility. The observed burst-like responding may represent a novel learning strategy in animals with chronic pain.

Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis

PubMed Central

2013-01-01

Background Depression is characterised partly by blunted reactions to reward. However, tasks probing this deficiency have not distinguished insensitivity to reward from insensitivity to the prediction errors for reward that determine learning and are putatively reported by the phasic activity of dopamine neurons. We attempted to disentangle these factors with respect to anhedonia in the context of stress, Major Depressive Disorder (MDD), Bipolar Disorder (BPD) and a dopaminergic challenge. Methods Six behavioural datasets involving 392 experimental sessions were subjected to a model-based, Bayesian meta-analysis. Participants across all six studies performed a probabilistic reward task that used an asymmetric reinforcement schedule to assess reward learning. Healthy controls were tested under baseline conditions, stress or after receiving the dopamine D2 agonist pramipexole. In addition, participants with current or past MDD or BPD were evaluated. Reinforcement learning models isolated the contributions of variation in reward sensitivity and learning rate. Results MDD and anhedonia reduced reward sensitivity more than they affected the learning rate, while a low dose of the dopamine D2 agonist pramipexole showed the opposite pattern. Stress led to a pattern consistent with a mixed effect on reward sensitivity and learning rate. Conclusion Reward-related learning reflected at least two partially separable contributions. The first related to phasic prediction error signalling, and was preferentially modulated by a low dose of the dopamine agonist pramipexole. The second related directly to reward sensitivity, and was preferentially reduced in MDD and anhedonia. Stress altered both components. Collectively, these findings highlight the contribution of model-based reinforcement learning meta-analysis for dissecting anhedonic behavior. PMID:23782813

Unsupervised learning in probabilistic neural networks with multi-state metal-oxide memristive synapses

NASA Astrophysics Data System (ADS)

Serb, Alexander; Bill, Johannes; Khiat, Ali; Berdan, Radu; Legenstein, Robert; Prodromakis, Themis

2016-09-01

In an increasingly data-rich world the need for developing computing systems that cannot only process, but ideally also interpret big data is becoming continuously more pressing. Brain-inspired concepts have shown great promise towards addressing this need. Here we demonstrate unsupervised learning in a probabilistic neural network that utilizes metal-oxide memristive devices as multi-state synapses. Our approach can be exploited for processing unlabelled data and can adapt to time-varying clusters that underlie incoming data by supporting the capability of reversible unsupervised learning. The potential of this work is showcased through the demonstration of successful learning in the presence of corrupted input data and probabilistic neurons, thus paving the way towards robust big-data processors.

Performance on a probabilistic inference task in healthy subjects receiving ketamine compared with patients with schizophrenia

PubMed Central

Almahdi, Basil; Sultan, Pervez; Sohanpal, Imrat; Brandner, Brigitta; Collier, Tracey; Shergill, Sukhi S; Cregg, Roman; Averbeck, Bruno B

2012-01-01

Evidence suggests that some aspects of schizophrenia can be induced in healthy volunteers through acute administration of the non-competitive NMDA-receptor antagonist, ketamine. In probabilistic inference tasks, patients with schizophrenia have been shown to ‘jump to conclusions’ (JTC) when asked to make a decision. We aimed to test whether healthy participants receiving ketamine would adopt a JTC response pattern resembling that of patients. The paradigmatic task used to investigate JTC has been the ‘urn’ task, where participants are shown a sequence of beads drawn from one of two ‘urns’, each containing coloured beads in different proportions. Participants make a decision when they think they know the urn from which beads are being drawn. We compared performance on the urn task between controls receiving acute ketamine or placebo with that of patients with schizophrenia and another group of controls matched to the patient group. Patients were shown to exhibit a JTC response pattern relative to their matched controls, whereas JTC was not evident in controls receiving ketamine relative to placebo. Ketamine does not appear to promote JTC in healthy controls, suggesting that ketamine does not affect probabilistic inferences. PMID:22389244

A Re-Unification of Two Competing Models for Document Retrieval.

ERIC Educational Resources Information Center

Bodoff, David

1999-01-01

Examines query-oriented versus document-oriented information retrieval and feedback learning. Highlights include a reunification of the two approaches for probabilistic document retrieval and for vector space model (VSM) retrieval; learning in VSM and in probabilistic models; multi-dimensional scaling; and ongoing field studies. (LRW)

Language experience changes subsequent learning

PubMed Central

Onnis, Luca; Thiessen, Erik

2013-01-01

What are the effects of experience on subsequent learning? We explored the effects of language-specific word order knowledge on the acquisition of sequential conditional information. Korean and English adults were engaged in a sequence learning task involving three different sets of stimuli: auditory linguistic (nonsense syllables), visual non-linguistic (nonsense shapes), and auditory non-linguistic (pure tones). The forward and backward probabilities between adjacent elements generated two equally probable and orthogonal perceptual parses of the elements, such that any significant preference at test must be due to either general cognitive biases, or prior language-induced biases. We found that language modulated parsing preferences with the linguistic stimuli only. Intriguingly, these preferences are congruent with the dominant word order patterns of each language, as corroborated by corpus analyses, and are driven by probabilistic preferences. Furthermore, although the Korean individuals had received extensive formal explicit training in English and lived in an English-speaking environment, they exhibited statistical learning biases congruent with their native language. Our findings suggest that mechanisms of statistical sequential learning are implicated in language across the lifespan, and experience with language may affect cognitive processes and later learning. PMID:23200510

Does Feedback-Related Brain Response during Reinforcement Learning Predict Socio-motivational (In-)dependence in Adolescence?

PubMed Central

Raufelder, Diana; Boehme, Rebecca; Romund, Lydia; Golde, Sabrina; Lorenz, Robert C.; Gleich, Tobias; Beck, Anne

2016-01-01

This multi-methodological study applied functional magnetic resonance imaging to investigate neural activation in a group of adolescent students (N = 88) during a probabilistic reinforcement learning task. We related patterns of emerging brain activity and individual learning rates to socio-motivational (in-)dependence manifested in four different motivation types (MTs): (1) peer-dependent MT, (2) teacher-dependent MT, (3) peer-and-teacher-dependent MT, (4) peer-and-teacher-independent MT. A multinomial regression analysis revealed that the individual learning rate predicts students’ membership to the independent MT, or the peer-and-teacher-dependent MT. Additionally, the striatum, a brain region associated with behavioral adaptation and flexibility, showed increased learning-related activation in students with motivational independence. Moreover, the prefrontal cortex, which is involved in behavioral control, was more active in students of the peer-and-teacher-dependent MT. Overall, this study offers new insights into the interplay of motivation and learning with (1) a focus on inter-individual differences in the role of peers and teachers as source of students’ individual motivation and (2) its potential neurobiological basis. PMID:27199873

Impaired implicit learning and feedback processing after stroke.

PubMed

Lam, J M; Globas, C; Hosp, J A; Karnath, H-O; Wächter, T; Luft, A R

2016-02-09

The ability to learn is assumed to support successful recovery and rehabilitation therapy after stroke. Hence, learning impairments may reduce the recovery potential. Here, the hypothesis is tested that stroke survivors have deficits in feedback-driven implicit learning. Stroke survivors (n=30) and healthy age-matched control subjects (n=21) learned a probabilistic classification task with brain activation measured using functional magnetic resonance imaging in a subset of these individuals (17 stroke and 10 controls). Stroke subjects learned slower than controls to classify cues. After being rewarded with a smiley face, they were less likely to give the same response when the cue was repeated. Stroke subjects showed reduced brain activation in putamen, pallidum, thalamus, frontal and prefrontal cortices and cerebellum when compared with controls. Lesion analysis identified those stroke survivors as learning-impaired who had lesions in frontal areas, putamen, thalamus, caudate and insula. Lesion laterality had no effect on learning efficacy or brain activation. These findings suggest that stroke survivors have deficits in reinforcement learning that may be related to dysfunctional processing of feedback-based decision-making, reward signals and working memory. Copyright © 2015 IBRO. Published by Elsevier Ltd. All rights reserved.

Stress attenuates the flexible updating of aversive value

PubMed Central

Raio, Candace M.; Hartley, Catherine A.; Orederu, Temidayo A.; Li, Jian; Phelps, Elizabeth A.

2017-01-01

In a dynamic environment, sources of threat or safety can unexpectedly change, requiring the flexible updating of stimulus−outcome associations that promote adaptive behavior. However, aversive contexts in which we are required to update predictions of threat are often marked by stress. Acute stress is thought to reduce behavioral flexibility, yet its influence on the modulation of aversive value has not been well characterized. Given that stress exposure is a prominent risk factor for anxiety and trauma-related disorders marked by persistent, inflexible responses to threat, here we examined how acute stress affects the flexible updating of threat responses. Participants completed an aversive learning task, in which one stimulus was probabilistically associated with an electric shock, while the other stimulus signaled safety. A day later, participants underwent an acute stress or control manipulation before completing a reversal learning task during which the original stimulus−outcome contingencies switched. Skin conductance and neuroendocrine responses provided indices of sympathetic arousal and stress responses, respectively. Despite equivalent initial learning, stressed participants showed marked impairments in reversal learning relative to controls. Additionally, reversal learning deficits across participants were related to heightened levels of alpha-amylase, a marker of noradrenergic activity. Finally, fitting arousal data to a computational reinforcement learning model revealed that stress-induced reversal learning deficits emerged from stress-specific changes in the weight assigned to prediction error signals, disrupting the adaptive adjustment of learning rates. Our findings provide insight into how stress renders individuals less sensitive to changes in aversive reinforcement and have implications for understanding clinical conditions marked by stress-related psychopathology. PMID:28973957

Social inference and social anxiety: evidence of a fear-congruent self-referential learning bias.

PubMed

Button, Katherine S; Browning, Michael; Munafò, Marcus R; Lewis, Glyn

2012-12-01

Fears of negative evaluation characterise social anxiety, and preferential processing of fear-relevant information is implicated in maintaining symptoms. Little is known, however, about the relationship between social anxiety and the process of inferring negative evaluation. The ability to use social information to learn what others think about one, referred to here as self-referential learning, is fundamental for effective social interaction. The aim of this research was to examine whether social anxiety is associated with self-referential learning. 102 Females with either high (n = 52) or low (n = 50) self-reported social anxiety completed a novel probabilistic social learning task. Using trial and error, the task required participants to learn two self-referential rules, 'I am liked' and 'I am disliked'. Participants across the sample were better at learning the positive rule 'I am liked' than the negative rule 'I am disliked', β = -6.4, 95% CI [-8.0, -4.7], p < 0.001. This preference for learning positive self-referential information was strongest in the lowest socially anxious and was abolished in the most symptomatic participants. Relative to the low group, the high anxiety group were better at learning they were disliked and worse at learning they were liked, social anxiety by rule interaction β = 3.6; 95% CI [+0.3, +7.0], p = 0.03. The specificity of the results to self-referential processing requires further research. Healthy individuals show a robust preference for learning that they are liked relative to disliked. This positive self-referential bias is reduced in social anxiety in a way that would be expected to exacerbate anxiety symptoms. Copyright © 2012 Elsevier Ltd. All rights reserved.

Grammaticality, Acceptability, and Probability: A Probabilistic View of Linguistic Knowledge.

PubMed

Lau, Jey Han; Clark, Alexander; Lappin, Shalom

2017-07-01

The question of whether humans represent grammatical knowledge as a binary condition on membership in a set of well-formed sentences, or as a probabilistic property has been the subject of debate among linguists, psychologists, and cognitive scientists for many decades. Acceptability judgments present a serious problem for both classical binary and probabilistic theories of grammaticality. These judgements are gradient in nature, and so cannot be directly accommodated in a binary formal grammar. However, it is also not possible to simply reduce acceptability to probability. The acceptability of a sentence is not the same as the likelihood of its occurrence, which is, in part, determined by factors like sentence length and lexical frequency. In this paper, we present the results of a set of large-scale experiments using crowd-sourced acceptability judgments that demonstrate gradience to be a pervasive feature in acceptability judgments. We then show how one can predict acceptability judgments on the basis of probability by augmenting probabilistic language models with an acceptability measure. This is a function that normalizes probability values to eliminate the confounding factors of length and lexical frequency. We describe a sequence of modeling experiments with unsupervised language models drawn from state-of-the-art machine learning methods in natural language processing. Several of these models achieve very encouraging levels of accuracy in the acceptability prediction task, as measured by the correlation between the acceptability measure scores and mean human acceptability values. We consider the relevance of these results to the debate on the nature of grammatical competence, and we argue that they support the view that linguistic knowledge can be intrinsically probabilistic. Copyright © 2016 Cognitive Science Society, Inc.

Spatial working memory interferes with explicit, but not probabilistic cuing of spatial attention.

PubMed

Won, Bo-Yeong; Jiang, Yuhong V

2015-05-01

Recent empirical and theoretical work has depicted a close relationship between visual attention and visual working memory. For example, rehearsal in spatial working memory depends on spatial attention, whereas adding a secondary spatial working memory task impairs attentional deployment in visual search. These findings have led to the proposal that working memory is attention directed toward internal representations. Here, we show that the close relationship between these 2 constructs is limited to some but not all forms of spatial attention. In 5 experiments, participants held color arrays, dot locations, or a sequence of dots in working memory. During the memory retention interval, they performed a T-among-L visual search task. Crucially, the probable target location was cued either implicitly through location probability learning or explicitly with a central arrow or verbal instruction. Our results showed that whereas imposing a visual working memory load diminished the effectiveness of explicit cuing, it did not interfere with probability cuing. We conclude that spatial working memory shares similar mechanisms with explicit, goal-driven attention but is dissociated from implicitly learned attention. (c) 2015 APA, all rights reserved).

Spatial working memory interferes with explicit, but not probabilistic cuing of spatial attention

PubMed Central

Won, Bo-Yeong; Jiang, Yuhong V.

2014-01-01

Recent empirical and theoretical work has depicted a close relationship between visual attention and visual working memory. For example, rehearsal in spatial working memory depends on spatial attention, whereas adding a secondary spatial working memory task impairs attentional deployment in visual search. These findings have led to the proposal that working memory is attention directed toward internal representations. Here we show that the close relationship between these two constructs is limited to some but not all forms of spatial attention. In five experiments, participants held color arrays, dot locations, or a sequence of dots in working memory. During the memory retention interval they performed a T-among-L visual search task. Crucially, the probable target location was cued either implicitly through location probability learning, or explicitly with a central arrow or verbal instruction. Our results showed that whereas imposing a visual working memory load diminished the effectiveness of explicit cuing, it did not interfere with probability cuing. We conclude that spatial working memory shares similar mechanisms with explicit, goal-driven attention but is dissociated from implicitly learned attention. PMID:25401460

The Development of Categorization: Effects of Classification and Inference Training on Category Representation

PubMed Central

Deng, Wei (Sophia); Sloutsky, Vladimir M.

2015-01-01

Does category representation change in the course of development? And if so, how and why? The current study attempted to answer these questions by examining category learning and category representation. In Experiment 1, 4-year-olds, 6-year-olds, and adults were trained with either a classification task or an inference task and their categorization performance and memory for items were tested. Adults and 6-year-olds exhibited an important asymmetry: they relied on a single deterministic feature during classification training, but not during inference training. In contrast, regardless of the training condition, 4-year-olds relied on multiple probabilistic features. In Experiment 2, 4-year-olds were presented with classification training and their attention was explicitly directed to the deterministic feature. Under this condition, their categorization performance was similar to that of older participants in Experiment 1, yet their memory performance pointed to a similarity-based representation, which was similar to that of 4-year-olds in Experiment 1. These results are discussed in relation to theories of categorization and the role of selective attention in the development of category learning. PMID:25602938

The neural correlates of implicit sequence learning in schizophrenia.

PubMed

Marvel, Cherie L; Turner, Beth M; O'Leary, Daniel S; Johnson, Hans J; Pierson, Ronald K; Ponto, Laura L Boles; Andreasen, Nancy C

2007-11-01

Twenty-seven schizophrenia spectrum patients and 25 healthy controls performed a probabilistic version of the serial reaction time task (SRT) that included sequence trials embedded within random trials. Patients showed diminished, yet measurable, sequence learning. Postexperimental analyses revealed that a group of patients performed above chance when generating short spans of the sequence. This high-generation group showed SRT learning that was similar in magnitude to that of controls. Their learning was evident from the very 1st block; however, unlike controls, learning did not develop further with continued testing. A subset of 12 patients and 11 controls performed the SRT in conjunction with positron emission tomography. High-generation performance, which corresponded to SRT learning in patients, correlated to activity in the premotor cortex and parahippocampus. These areas have been associated with stimulus-driven visuospatial processing. Taken together, these results suggest that a subset of patients who showed moderate success on the SRT used an explicit stimulus-driven strategy to process the sequential stimuli. This adaptive strategy facilitated sequence learning but may have interfered with conventional implicit learning of the overall stimulus pattern. PsycINFO Database Record (c) 2007 APA, all rights reserved.

Independent tasks scheduling in cloud computing via improved estimation of distribution algorithm

NASA Astrophysics Data System (ADS)

Sun, Haisheng; Xu, Rui; Chen, Huaping

2018-04-01

To minimize makespan for scheduling independent tasks in cloud computing, an improved estimation of distribution algorithm (IEDA) is proposed to tackle the investigated problem in this paper. Considering that the problem is concerned with multi-dimensional discrete problems, an improved population-based incremental learning (PBIL) algorithm is applied, which the parameter for each component is independent with other components in PBIL. In order to improve the performance of PBIL, on the one hand, the integer encoding scheme is used and the method of probability calculation of PBIL is improved by using the task average processing time; on the other hand, an effective adaptive learning rate function that related to the number of iterations is constructed to trade off the exploration and exploitation of IEDA. In addition, both enhanced Max-Min and Min-Min algorithms are properly introduced to form two initial individuals. In the proposed IEDA, an improved genetic algorithm (IGA) is applied to generate partial initial population by evolving two initial individuals and the rest of initial individuals are generated at random. Finally, the sampling process is divided into two parts including sampling by probabilistic model and IGA respectively. The experiment results show that the proposed IEDA not only gets better solution, but also has faster convergence speed.

Problem Solving as Probabilistic Inference with Subgoaling: Explaining Human Successes and Pitfalls in the Tower of Hanoi

PubMed Central

Donnarumma, Francesco; Maisto, Domenico; Pezzulo, Giovanni

2016-01-01

How do humans and other animals face novel problems for which predefined solutions are not available? Human problem solving links to flexible reasoning and inference rather than to slow trial-and-error learning. It has received considerable attention since the early days of cognitive science, giving rise to well known cognitive architectures such as SOAR and ACT-R, but its computational and brain mechanisms remain incompletely known. Furthermore, it is still unclear whether problem solving is a “specialized” domain or module of cognition, in the sense that it requires computations that are fundamentally different from those supporting perception and action systems. Here we advance a novel view of human problem solving as probabilistic inference with subgoaling. In this perspective, key insights from cognitive architectures are retained such as the importance of using subgoals to split problems into subproblems. However, here the underlying computations use probabilistic inference methods analogous to those that are increasingly popular in the study of perception and action systems. To test our model we focus on the widely used Tower of Hanoi (ToH) task, and show that our proposed method can reproduce characteristic idiosyncrasies of human problem solvers: their sensitivity to the “community structure” of the ToH and their difficulties in executing so-called “counterintuitive” movements. Our analysis reveals that subgoals have two key roles in probabilistic inference and problem solving. First, prior beliefs on (likely) useful subgoals carve the problem space and define an implicit metric for the problem at hand—a metric to which humans are sensitive. Second, subgoals are used as waypoints in the probabilistic problem solving inference and permit to find effective solutions that, when unavailable, lead to problem solving deficits. Our study thus suggests that a probabilistic inference scheme enhanced with subgoals provides a comprehensive framework to study problem solving and its deficits. PMID:27074140

Problem Solving as Probabilistic Inference with Subgoaling: Explaining Human Successes and Pitfalls in the Tower of Hanoi.

PubMed

Donnarumma, Francesco; Maisto, Domenico; Pezzulo, Giovanni

2016-04-01

How do humans and other animals face novel problems for which predefined solutions are not available? Human problem solving links to flexible reasoning and inference rather than to slow trial-and-error learning. It has received considerable attention since the early days of cognitive science, giving rise to well known cognitive architectures such as SOAR and ACT-R, but its computational and brain mechanisms remain incompletely known. Furthermore, it is still unclear whether problem solving is a "specialized" domain or module of cognition, in the sense that it requires computations that are fundamentally different from those supporting perception and action systems. Here we advance a novel view of human problem solving as probabilistic inference with subgoaling. In this perspective, key insights from cognitive architectures are retained such as the importance of using subgoals to split problems into subproblems. However, here the underlying computations use probabilistic inference methods analogous to those that are increasingly popular in the study of perception and action systems. To test our model we focus on the widely used Tower of Hanoi (ToH) task, and show that our proposed method can reproduce characteristic idiosyncrasies of human problem solvers: their sensitivity to the "community structure" of the ToH and their difficulties in executing so-called "counterintuitive" movements. Our analysis reveals that subgoals have two key roles in probabilistic inference and problem solving. First, prior beliefs on (likely) useful subgoals carve the problem space and define an implicit metric for the problem at hand-a metric to which humans are sensitive. Second, subgoals are used as waypoints in the probabilistic problem solving inference and permit to find effective solutions that, when unavailable, lead to problem solving deficits. Our study thus suggests that a probabilistic inference scheme enhanced with subgoals provides a comprehensive framework to study problem solving and its deficits.

Life Span Differences in Electrophysiological Correlates of Monitoring Gains and Losses during Probabilistic Reinforcement Learning

ERIC Educational Resources Information Center

Hammerer, Dorothea; Li, Shu-Chen; Muller, Viktor; Lindenberger, Ulman

2011-01-01

By recording the feedback-related negativity (FRN) in response to gains and losses, we investigated the contribution of outcome monitoring mechanisms to age-associated differences in probabilistic reinforcement learning. Specifically, we assessed the difference of the monitoring reactions to gains and losses to investigate the monitoring of…

Learning to Look: Probabilistic Variation and Noise Guide Infants' Eye Movements

ERIC Educational Resources Information Center

Tummeltshammer, Kristen Swan; Kirkham, Natasha Z.

2013-01-01

Young infants have demonstrated a remarkable sensitivity to probabilistic relations among visual features (Fiser & Aslin, 2002; Kirkham et al., 2002). Previous research has raised important questions regarding the usefulness of statistical learning in an environment filled with variability and noise, such as an infant's natural world. In…

Fear of Negative Evaluation Biases Social Evaluation Inference: Evidence from a Probabilistic Learning Task

PubMed Central

Button, Katherine S.; Kounali, Daphne; Stapinski, Lexine; Rapee, Ronald M.; Lewis, Glyn; Munafò, Marcus R.

2015-01-01

Background Fear of negative evaluation (FNE) defines social anxiety yet the process of inferring social evaluation, and its potential role in maintaining social anxiety, is poorly understood. We developed an instrumental learning task to model social evaluation learning, predicting that FNE would specifically bias learning about the self but not others. Methods During six test blocks (3 self-referential, 3 other-referential), participants (n = 100) met six personas and selected a word from a positive/negative pair to finish their social evaluation sentences “I think [you are / George is]…”. Feedback contingencies corresponded to 3 rules, liked, neutral and disliked, with P[positive word correct] = 0.8, 0.5 and 0.2, respectively. Results As FNE increased participants selected fewer positive words (β = −0.4, 95% CI −0.7, −0.2, p = 0.001), which was strongest in the self-referential condition (FNE × condition 0.28, 95% CI 0.01, 0.54, p = 0.04), and the neutral and dislike rules (FNE × condition × rule, p = 0.07). At low FNE the proportion of positive words selected for self-neutral and self-disliked greatly exceeded the feedback contingency, indicating poor learning, which improved as FNE increased. Conclusions FNE is associated with differences in processing social-evaluative information specifically about the self. At low FNE this manifests as insensitivity to learning negative self-referential evaluation. High FNE individuals are equally sensitive to learning positive or negative evaluation, which although objectively more accurate, may have detrimental effects on mental health. PMID:25853835

Role of ionotropic glutamate receptors in delay and probability discounting in the rat.

PubMed

Yates, Justin R; Batten, Seth R; Bardo, Michael T; Beckmann, Joshua S

2015-04-01

Discounting of delayed and probabilistic reinforcement is linked to increased drug use and pathological gambling. Understanding the neurobiology of discounting is important for designing treatments for these disorders. Glutamate is considered to be involved in addiction-like behaviors; however, the role of ionotropic glutamate receptors (iGluRs) in discounting remains unclear. The current study examined the effects of N-methyl-D-aspartate (NMDA) and α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) glutamate receptor blockade on performance in delay and probability discounting tasks. Following training in either delay or probability discounting, rats (n = 12, each task) received pretreatments of the NMDA receptor antagonists MK-801 (0, 0.01, 0.03, 0.1, or 0.3 mg/kg, s.c.) or ketamine (0, 1.0, 5.0, or 10.0 mg/kg, i.p.), as well as the AMPA receptor antagonist CNQX (0, 1.0, 3.0, or 5.6 mg/kg, i.p.). Hyperbolic discounting functions were used to estimate sensitivity to delayed/probabilistic reinforcement and sensitivity to reinforcer amount. An intermediate dose of MK-801 (0.03 mg/kg) decreased sensitivity to both delayed and probabilistic reinforcement. In contrast, ketamine did not affect the rate of discounting in either task but decreased sensitivity to reinforcer amount. CNQX did not alter sensitivity to reinforcer amount or delayed/probabilistic reinforcement. These results show that blockade of NMDA receptors, but not AMPA receptors, decreases sensitivity to delayed/probabilistic reinforcement (MK-801) and sensitivity to reinforcer amount (ketamine). The differential effects of MK-801 and ketamine demonstrate that sensitivities to delayed/probabilistic reinforcement and reinforcer amount are pharmacologically dissociable.

Striatal dopamine release and impaired reinforcement learning in adults with 22q11.2 deletion syndrome.

PubMed

van Duin, Esther D A; Kasanova, Zuzana; Hernaus, Dennis; Ceccarini, Jenny; Heinzel, Alexander; Mottaghy, Felix; Mohammadkhani-Shali, Siamak; Winz, Oliver; Frank, Michael; Beck, Merrit C H; Booij, Jan; Myin-Germeys, Inez; van Amelsvoort, Thérèse

2018-06-01

22q11.2 deletion syndrome (22q11DS) is a genetic disorder caused by a microdeletion on chromosome 22q11.2 and associated with an increased risk for developing psychosis. The catechol-O-methyltransferase (COMT) gene is located in the deleted region and involved in dopamine (DA) breakdown. Impaired reinforcement learning (RL) is a recurrent feature in psychosis and thought to be related to abnormal striatal DA function. This study aims to examine RL and the potential association with striatal DA-ergic neuromodulation in 22q11DS. Twelve non-psychotic adults with 22q11DS and 16 healthy controls (HC) were included. A dopamine D 2/3 receptor [ 18 F]fallypride positron emission tomography (PET) scan was acquired while participants performed a modified version of the probabilistic stimulus selection task. RL-task performance was significantly worse in 22q11DS compared to HC. There were no group difference in striatal nondisplaceable binding potential (BP ND ) and task-induced DA release. In HC, striatal task-induced DA release was positively associated with task performance, but no such relation was found in 22q11DS subjects. Moreover, higher caudate nucleus task-induced DA release was found in COMT Met hemizygotes relative to Val hemizygotes. This study is the first to show impairments in RL in 22q11DS. It suggests that potentially motivational impairments are not only present in psychosis, but also in this genetic high risk group. These deficits may be underlain by abnormal striatal task-induced DA release, perhaps as a consequence of COMT haplo-insufficiency. Copyright © 2018 Elsevier B.V. and ECNP. All rights reserved.

Social and monetary reward learning engage overlapping neural substrates.

PubMed

Lin, Alice; Adolphs, Ralph; Rangel, Antonio

2012-03-01

Learning to make choices that yield rewarding outcomes requires the computation of three distinct signals: stimulus values that are used to guide choices at the time of decision making, experienced utility signals that are used to evaluate the outcomes of those decisions and prediction errors that are used to update the values assigned to stimuli during reward learning. Here we investigated whether monetary and social rewards involve overlapping neural substrates during these computations. Subjects engaged in two probabilistic reward learning tasks that were identical except that rewards were either social (pictures of smiling or angry people) or monetary (gaining or losing money). We found substantial overlap between the two types of rewards for all components of the learning process: a common area of ventromedial prefrontal cortex (vmPFC) correlated with stimulus value at the time of choice and another common area of vmPFC correlated with reward magnitude and common areas in the striatum correlated with prediction errors. Taken together, the findings support the hypothesis that shared anatomical substrates are involved in the computation of both monetary and social rewards. © The Author (2011). Published by Oxford University Press.

The effectiveness of learning material with Edmodo to enhance the level of student's probabilistic thinking

NASA Astrophysics Data System (ADS)

Sujadi, Imam; Kurniasih, Rini; Subanti, Sri

2017-05-01

In the era of 21st century learning, it needs to use technology as a learning media. Using Edmodo as a learning media is one of the options as the complement in learning process. However, this research focuses on the effectiveness of learning material using Edmodo. The aim of this research to determine whether the level of student's probabilistic thinking that use learning material with Edmodo is better than the existing learning materials (books) implemented to teach the subject of students grade 8th. This is quasi-experimental research using control group pretest and posttest. The population of this study was students grade 8 of SMPN 12 Surakarta and the sampling technique used random sampling. The analysis technique used to examine two independent sample using Kolmogorov-Smirnov test. The obtained value of test statistic is M=0.38, since 0.38 is the largest tabled critical one-tailed value M0.05=0.011. The result of the research is the learning materials with Edmodo more effectively to enhance the level of probabilistic thinking learners than the learning that use the existing learning materials (books). Therefore, learning material using Edmodo can be used in learning process. It can also be developed into another learning material through Edmodo.

The role of effective connectivity between the task-positive and task-negative network for evidence gathering [Evidence gathering and connectivity].

PubMed

Andreou, Christina; Steinmann, Saskia; Kolbeck, Katharina; Rauh, Jonas; Leicht, Gregor; Moritz, Steffen; Mulert, Christoph

2018-06-01

Reports linking a 'jumping-to-conclusions' bias to delusions have led to growing interest in the neurobiological correlates of probabilistic reasoning. Several brain areas have been implicated in probabilistic reasoning; however, findings are difficult to integrate into a coherent account. The present study aimed to provide additional evidence by investigating, for the first time, effective connectivity among brain areas involved in different stages of evidence gathering. We investigated evidence gathering in 25 healthy individuals using fMRI and a new paradigm (Box Task) designed such as to minimize the effects of cognitive effort and reward processing. Decisions to collect more evidence ('draws') were contrasted to decisions to reach a final choice ('conclusions') with respect to BOLD activity. Psychophysiological interaction analysis was used to investigate effective connectivity. Conclusion events were associated with extensive brain activations in widely distributed brain areas associated with the task-positive network. In contrast, draw events were characterized by higher activation in areas assumed to be part of the task-negative network. Effective connectivity between the two networks decreased during draws and increased during conclusion events. Our findings indicate that probabilistic reasoning may depend on the balance between the task-positive and task-negative network, and that shifts in connectivity between the two may be crucial for evidence gathering. Thus, abnormal connectivity between the two systems may significantly contribute to the jumping-to-conclusions bias. Copyright © 2018 Elsevier Inc. All rights reserved.

Learning rational temporal eye movement strategies.

PubMed

Hoppe, David; Rothkopf, Constantin A

2016-07-19

During active behavior humans redirect their gaze several times every second within the visual environment. Where we look within static images is highly efficient, as quantified by computational models of human gaze shifts in visual search and face recognition tasks. However, when we shift gaze is mostly unknown despite its fundamental importance for survival in a dynamic world. It has been suggested that during naturalistic visuomotor behavior gaze deployment is coordinated with task-relevant events, often predictive of future events, and studies in sportsmen suggest that timing of eye movements is learned. Here we establish that humans efficiently learn to adjust the timing of eye movements in response to environmental regularities when monitoring locations in the visual scene to detect probabilistically occurring events. To detect the events humans adopt strategies that can be understood through a computational model that includes perceptual and acting uncertainties, a minimal processing time, and, crucially, the intrinsic costs of gaze behavior. Thus, subjects traded off event detection rate with behavioral costs of carrying out eye movements. Remarkably, based on this rational bounded actor model the time course of learning the gaze strategies is fully explained by an optimal Bayesian learner with humans' characteristic uncertainty in time estimation, the well-known scalar law of biological timing. Taken together, these findings establish that the human visual system is highly efficient in learning temporal regularities in the environment and that it can use these regularities to control the timing of eye movements to detect behaviorally relevant events.

Exploring implicit and explicit aspects of sense of agency

PubMed Central

Moore, J.W.; Middleton, D.; Haggard, P.; Fletcher, P.C.

2012-01-01

Sense of agency refers to the sense of initiating and controlling actions in order to influence events in the outside world. Recently, a distinction between implicit and explicit aspects of sense of agency has been proposed, analogous to distinctions found in other areas of cognition, notably learning. However, there is yet no strong evidence supporting separable implicit and explicit components of sense of agency. The so-called ‘Perruchet paradigm’ offers one of the few convincing demonstrations of separable implicit and explicit learning systems. We adopted this approach to evaluate the implicit–explicit distinction in the context of a simple task in which outcomes were probabilistically caused by actions. In line with our initial predictions, we found evidence of a dissociation. We discuss the implications of this result for theories of sense of agency. PMID:23143153

Real-time value-driven diagnosis

NASA Technical Reports Server (NTRS)

Dambrosio, Bruce

1995-01-01

Diagnosis is often thought of as an isolated task in theoretical reasoning (reasoning with the goal of updating our beliefs about the world). We present a decision-theoretic interpretation of diagnosis as a task in practical reasoning (reasoning with the goal of acting in the world), and sketch components of our approach to this task. These components include an abstract problem description, a decision-theoretic model of the basic task, a set of inference methods suitable for evaluating the decision representation in real-time, and a control architecture to provide the needed continuing coordination between the agent and its environment. A principal contribution of this work is the representation and inference methods we have developed, which extend previously available probabilistic inference methods and narrow, somewhat, the gap between probabilistic and logical models of diagnosis.

Probability in reasoning: a developmental test on conditionals.

PubMed

Barrouillet, Pierre; Gauffroy, Caroline

2015-04-01

Probabilistic theories have been claimed to constitute a new paradigm for the psychology of reasoning. A key assumption of these theories is captured by what they call the Equation, the hypothesis that the meaning of the conditional is probabilistic in nature and that the probability of If p then q is the conditional probability, in such a way that P(if p then q)=P(q|p). Using the probabilistic truth-table task in which participants are required to evaluate the probability of If p then q sentences, the present study explored the pervasiveness of the Equation through ages (from early adolescence to adulthood), types of conditionals (basic, causal, and inducements) and contents. The results reveal that the Equation is a late developmental achievement only endorsed by a narrow majority of educated adults for certain types of conditionals depending on the content they involve. Age-related changes in evaluating the probability of all the conditionals studied closely mirror the development of truth-value judgements observed in previous studies with traditional truth-table tasks. We argue that our modified mental model theory can account for this development, and hence for the findings related with the probability task, which do not consequently support the probabilistic approach of human reasoning over alternative theories. Copyright © 2014 Elsevier B.V. All rights reserved.

A mediation model to explain decision making under conditions of risk among adolescents: the role of fluid intelligence and probabilistic reasoning.

PubMed

Donati, Maria Anna; Panno, Angelo; Chiesi, Francesca; Primi, Caterina

2014-01-01

This study tested the mediating role of probabilistic reasoning ability in the relationship between fluid intelligence and advantageous decision making among adolescents in explicit situations of risk--that is, in contexts in which information on the choice options (gains, losses, and probabilities) were explicitly presented at the beginning of the task. Participants were 282 adolescents attending high school (77% males, mean age = 17.3 years). We first measured fluid intelligence and probabilistic reasoning ability. Then, to measure decision making under explicit conditions of risk, participants performed the Game of Dice Task, in which they have to decide among different alternatives that are explicitly linked to a specific amount of gain or loss and have obvious winning probabilities that are stable over time. Analyses showed a significant positive indirect effect of fluid intelligence on advantageous decision making through probabilistic reasoning ability that acted as a mediator. Specifically, fluid intelligence may enhance ability to reason in probabilistic terms, which in turn increases the likelihood of advantageous choices when adolescents are confronted with an explicit decisional context. Findings show that in experimental paradigm settings, adolescents are able to make advantageous decisions using cognitive abilities when faced with decisions under explicit risky conditions. This study suggests that interventions designed to promote probabilistic reasoning, for example by incrementing the mathematical prerequisites necessary to reason in probabilistic terms, may have a positive effect on adolescents' decision-making abilities.

Role of Central Serotonin in Anticipation of Rewarding and Punishing Outcomes: Effects of Selective Amygdala or Orbitofrontal 5-HT Depletion

PubMed Central

Rygula, Rafal; Clarke, Hannah F.; Cardinal, Rudolf N.; Cockcroft, Gemma J.; Xia, Jing; Dalley, Jeff W.; Robbins, Trevor W.; Roberts, Angela C.

2015-01-01

Understanding the role of serotonin (or 5-hydroxytryptamine, 5-HT) in aversive processing has been hampered by the contradictory findings, across studies, of increased sensitivity to punishment in terms of subsequent response choice but decreased sensitivity to punishment-induced response suppression following gross depletion of central 5-HT. To address this apparent discrepancy, the present study determined whether both effects could be found in the same animals by performing localized 5-HT depletions in the amygdala or orbitofrontal cortex (OFC) of a New World monkey, the common marmoset. 5-HT depletion in the amygdala impaired response choice on a probabilistic visual discrimination task by increasing the effectiveness of misleading, or false, punishment and reward, and decreased response suppression in a variable interval test of punishment sensitivity that employed the same reward and punisher. 5-HT depletion in the OFC also disrupted probabilistic discrimination learning and decreased response suppression. Computational modeling of behavior on the discrimination task showed that the lesions reduced reinforcement sensitivity. A novel, unitary account of the findings in terms of the causal role of 5-HT in the anticipation of both negative and positive motivational outcomes is proposed and discussed in relation to current theories of 5-HT function and our understanding of mood and anxiety disorders. PMID:24879752

The Contribution of Primary Motor Cortex Is Essential for Probabilistic Implicit Sequence Learning: Evidence from Theta Burst Magnetic Stimulation

ERIC Educational Resources Information Center

Wilkinson, Leonora; Teo, James T.; Obeso, Ignacio; Rothwell, John C.; Jahanshahi, Marjan

2010-01-01

Theta burst transcranial magnetic stimulation (TBS) is considered to produce plastic changes in human motor cortex. Here, we examined the inhibitory and excitatory effects of TBS on implicit sequence learning using a probabilistic serial reaction time paradigm. We investigated the involvement of several cortical regions associated with implicit…

Neural Signatures of Cognitive Flexibility and Reward Sensitivity Following Nicotinic Receptor Stimulation in Dependent Smokers: A Randomized Trial.

PubMed

Lesage, Elise; Aronson, Sarah E; Sutherland, Matthew T; Ross, Thomas J; Salmeron, Betty Jo; Stein, Elliot A

2017-06-01

Withdrawal from nicotine is an important contributor to smoking relapse. Understanding how reward-based decision making is affected by abstinence and by pharmacotherapies such as nicotine replacement therapy and varenicline tartrate may aid cessation treatment. To independently assess the effects of nicotine dependence and stimulation of the nicotinic acetylcholine receptor on the ability to interpret valence information (reward sensitivity) and subsequently alter behavior as reward contingencies change (cognitive flexibility) in a probabilistic reversal learning task. Nicotine-dependent smokers and nonsmokers completed a probabilistic reversal learning task during acquisition of functional magnetic resonance imaging (fMRI) in a 2-drug, double-blind placebo-controlled crossover design conducted from January 21, 2009, to September 29, 2011. Smokers were abstinent from cigarette smoking for 12 hours for all sessions. In a fully Latin square fashion, participants in both groups underwent MRI twice while receiving varenicline and twice while receiving a placebo pill, wearing either a nicotine or a placebo patch. Imaging analysis was performed from June 15, 2015, to August 10, 2016. A well-established computational model captured effects of smoking status and administration of nicotine and varenicline on probabilistic reversal learning choice behavior. Neural effects of smoking status, nicotine, and varenicline were tested for on MRI contrasts that captured reward sensitivity and cognitive flexibility. The study included 24 nicotine-dependent smokers (12 women and 12 men; mean [SD] age, 35.8 [9.9] years) and 20 nonsmokers (10 women and 10 men; mean [SD] age, 30.4 [7.2] years). Computational modeling indicated that abstinent smokers were biased toward response shifting and that their decisions were less sensitive to the available evidence, suggesting increased impulsivity during withdrawal. These behavioral impairments were mitigated with nicotine and varenicline. Similarly, decreased mesocorticolimbic activity associated with cognitive flexibility in abstinent smokers was restored to the level of nonsmokers following stimulation of nicotinic acetylcholine receptors (familywise error-corrected P < .05). Conversely, neural signatures of decreased reward sensitivity in smokers (vs nonsmokers; familywise error-corrected P < .05) in the dorsal striatum and anterior cingulate cortex were not mitigated by nicotine or varenicline. There was a double dissociation between the effects of chronic nicotine dependence on neural representations of reward sensitivity and acute effects of stimulation of nicotinic acetylcholine receptors on behavioral and neural signatures of cognitive flexibility in smokers. These chronic and acute pharmacologic effects were observed in overlapping mesocorticolimbic regions, suggesting that available pharmacotherapies may alleviate deficits in the same circuitry for certain mental computations but not for others. clinicaltrials.gov Identifier: NCT00830739.

Statistical learning and probabilistic prediction in music cognition: mechanisms of stylistic enculturation.

PubMed

Pearce, Marcus T

2018-05-11

Music perception depends on internal psychological models derived through exposure to a musical culture. It is hypothesized that this musical enculturation depends on two cognitive processes: (1) statistical learning, in which listeners acquire internal cognitive models of statistical regularities present in the music to which they are exposed; and (2) probabilistic prediction based on these learned models that enables listeners to organize and process their mental representations of music. To corroborate these hypotheses, I review research that uses a computational model of probabilistic prediction based on statistical learning (the information dynamics of music (IDyOM) model) to simulate data from empirical studies of human listeners. The results show that a broad range of psychological processes involved in music perception-expectation, emotion, memory, similarity, segmentation, and meter-can be understood in terms of a single, underlying process of probabilistic prediction using learned statistical models. Furthermore, IDyOM simulations of listeners from different musical cultures demonstrate that statistical learning can plausibly predict causal effects of differential cultural exposure to musical styles, providing a quantitative model of cultural distance. Understanding the neural basis of musical enculturation will benefit from close coordination between empirical neuroimaging and computational modeling of underlying mechanisms, as outlined here. © 2018 The Authors. Annals of the New York Academy of Sciences published by Wiley Periodicals, Inc. on behalf of New York Academy of Sciences.

Translational Rodent Paradigms to Investigate Neuromechanisms Underlying Behaviors Relevant to Amotivation and Altered Reward Processing in Schizophrenia.

PubMed

Young, Jared W; Markou, Athina

2015-09-01

Amotivation and reward-processing deficits have long been described in patients with schizophrenia and considered large contributors to patients' inability to integrate well in society. No effective treatments exist for these symptoms, partly because the neuromechanisms mediating such symptoms are poorly understood. Here, we propose a translational neuroscientific approach that can be used to assess reward/motivational deficits related to the negative symptoms of schizophrenia using behavioral paradigms that can also be conducted in experimental animals. By designing and using objective laboratory behavioral tools that are parallel in their parameters in rodents and humans, the neuromechanisms underlying behaviors with relevance to these symptoms of schizophrenia can be investigated. We describe tasks that measure the motivation of rodents to expend physical and cognitive effort to gain rewards, as well as probabilistic learning tasks that assess both reward learning and feedback-based decision making. The latter tasks are relevant because of demonstrated links of performance deficits correlating with negative symptoms in patients with schizophrenia. These tasks utilize operant techniques in order to investigate neural circuits targeting a specific domain across species. These tasks therefore enable the development of insights into altered mechanisms leading to negative symptom-relevant behaviors in patients with schizophrenia. Such findings will then enable the development of targeted treatments for these altered neuromechanisms and behaviors seen in schizophrenia. © The Author 2015. Published by Oxford University Press on behalf of the Maryland Psychiatric Research Center. All rights reserved. For permissions, please email: journals.permissions@oup.com.

PySeqLab: an open source Python package for sequence labeling and segmentation.

PubMed

Allam, Ahmed; Krauthammer, Michael

2017-11-01

Text and genomic data are composed of sequential tokens, such as words and nucleotides that give rise to higher order syntactic constructs. In this work, we aim at providing a comprehensive Python library implementing conditional random fields (CRFs), a class of probabilistic graphical models, for robust prediction of these constructs from sequential data. Python Sequence Labeling (PySeqLab) is an open source package for performing supervised learning in structured prediction tasks. It implements CRFs models, that is discriminative models from (i) first-order to higher-order linear-chain CRFs, and from (ii) first-order to higher-order semi-Markov CRFs (semi-CRFs). Moreover, it provides multiple learning algorithms for estimating model parameters such as (i) stochastic gradient descent (SGD) and its multiple variations, (ii) structured perceptron with multiple averaging schemes supporting exact and inexact search using 'violation-fixing' framework, (iii) search-based probabilistic online learning algorithm (SAPO) and (iv) an interface for Broyden-Fletcher-Goldfarb-Shanno (BFGS) and the limited-memory BFGS algorithms. Viterbi and Viterbi A* are used for inference and decoding of sequences. Using PySeqLab, we built models (classifiers) and evaluated their performance in three different domains: (i) biomedical Natural language processing (NLP), (ii) predictive DNA sequence analysis and (iii) Human activity recognition (HAR). State-of-the-art performance comparable to machine-learning based systems was achieved in the three domains without feature engineering or the use of knowledge sources. PySeqLab is available through https://bitbucket.org/A_2/pyseqlab with tutorials and documentation. ahmed.allam@yale.edu or michael.krauthammer@yale.edu. Supplementary data are available at Bioinformatics online. © The Author 2017. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com

Combining MLC and SVM Classifiers for Learning Based Decision Making: Analysis and Evaluations

PubMed Central

Zhang, Yi; Ren, Jinchang; Jiang, Jianmin

2015-01-01

Maximum likelihood classifier (MLC) and support vector machines (SVM) are two commonly used approaches in machine learning. MLC is based on Bayesian theory in estimating parameters of a probabilistic model, whilst SVM is an optimization based nonparametric method in this context. Recently, it is found that SVM in some cases is equivalent to MLC in probabilistically modeling the learning process. In this paper, MLC and SVM are combined in learning and classification, which helps to yield probabilistic output for SVM and facilitate soft decision making. In total four groups of data are used for evaluations, covering sonar, vehicle, breast cancer, and DNA sequences. The data samples are characterized in terms of Gaussian/non-Gaussian distributed and balanced/unbalanced samples which are then further used for performance assessment in comparing the SVM and the combined SVM-MLC classifier. Interesting results are reported to indicate how the combined classifier may work under various conditions. PMID:26089862

Combining MLC and SVM Classifiers for Learning Based Decision Making: Analysis and Evaluations.

PubMed

Zhang, Yi; Ren, Jinchang; Jiang, Jianmin

2015-01-01

Maximum likelihood classifier (MLC) and support vector machines (SVM) are two commonly used approaches in machine learning. MLC is based on Bayesian theory in estimating parameters of a probabilistic model, whilst SVM is an optimization based nonparametric method in this context. Recently, it is found that SVM in some cases is equivalent to MLC in probabilistically modeling the learning process. In this paper, MLC and SVM are combined in learning and classification, which helps to yield probabilistic output for SVM and facilitate soft decision making. In total four groups of data are used for evaluations, covering sonar, vehicle, breast cancer, and DNA sequences. The data samples are characterized in terms of Gaussian/non-Gaussian distributed and balanced/unbalanced samples which are then further used for performance assessment in comparing the SVM and the combined SVM-MLC classifier. Interesting results are reported to indicate how the combined classifier may work under various conditions.

Active and observational reward learning in adults with autism spectrum disorder: relationship with empathy in an atypical sample.

PubMed

Bellebaum, Christian; Brodmann, Katja; Thoma, Patrizia

2014-01-01

Autism spectrum disorders (ASDs) are characterised by disturbances in social behaviour. A prevailing hypothesis suggests that these problems are related to deficits in assigning rewarding value to social stimuli. The present study aimed to examine monetary reward processing in adults with ASDs by means of event-related potentials (ERPs). Ten individuals with mild ASDs (Asperger's syndrome and high-functioning autism) and 12 healthy control subjects performed an active and an observational probabilistic reward-learning task. Both groups showed similar overall learning performance. With respect to reward processing, subjects with ASDs exhibited a general reduction in feedback-related negativity (FRN) amplitude, irrespective of feedback valence and type of learning (active or observational). Individuals with ASDs showed lower scores for cognitive empathy, while affective empathy did not differ between groups. Correlation analyses revealed that higher empathy (both cognitive and affective) negatively affected performance in observational learning in controls and in active learning in ASDs (only cognitive empathy). No relationships were seen between empathy and ERPs. Reduced FRN amplitudes are discussed in terms of a deficit in fast reward processing in ASDs, which may indicate altered reward system functioning.

Performance monitoring and empathy during active and observational learning in patients with major depression.

PubMed

Thoma, Patrizia; Norra, Christine; Juckel, Georg; Suchan, Boris; Bellebaum, Christian

2015-07-01

Previous literature established a link between major depressive disorder (MDD) and altered reward processing as well as between empathy and (observational) reward learning. The aim of the present study was to assess the effects of MDD on the electrophysiological correlates - the feedback-related negativity (FRN) and the P300 - of active and observational reward processing and to relate them to trait cognitive and affective empathy. Eighteen patients with MDD and 16 healthy controls performed an active and an observational probabilistic reward-learning task while event- related potentials were recorded. Also, participants were assessed with regard to self-reported cognitive and affective trait empathy. Relative to healthy controls, patients with MDD showed overall impaired learning and attenuated FRN amplitudes, irrespective of feedback valence and learning type (active vs. observational), but comparable P300 amplitudes. In the patient group, but not in controls, higher trait perspective taking scores were significantly correlated with reduced FRN amplitudes. The pattern of results suggests impaired prediction error processing and a negative effect of higher trait empathy on feedback-based learning in patients with MDD. Copyright © 2015 Elsevier B.V. All rights reserved.

Language experience changes subsequent learning.

PubMed

Onnis, Luca; Thiessen, Erik

2013-02-01

What are the effects of experience on subsequent learning? We explored the effects of language-specific word order knowledge on the acquisition of sequential conditional information. Korean and English adults were engaged in a sequence learning task involving three different sets of stimuli: auditory linguistic (nonsense syllables), visual non-linguistic (nonsense shapes), and auditory non-linguistic (pure tones). The forward and backward probabilities between adjacent elements generated two equally probable and orthogonal perceptual parses of the elements, such that any significant preference at test must be due to either general cognitive biases, or prior language-induced biases. We found that language modulated parsing preferences with the linguistic stimuli only. Intriguingly, these preferences are congruent with the dominant word order patterns of each language, as corroborated by corpus analyses, and are driven by probabilistic preferences. Furthermore, although the Korean individuals had received extensive formal explicit training in English and lived in an English-speaking environment, they exhibited statistical learning biases congruent with their native language. Our findings suggest that mechanisms of statistical sequential learning are implicated in language across the lifespan, and experience with language may affect cognitive processes and later learning. Copyright © 2012 Elsevier B.V. All rights reserved.

Jumping to conclusions and the continuum of delusional beliefs.

PubMed

Warman, Debbie M; Lysaker, Paul H; Martin, Joel M; Davis, Louanne; Haudenschield, Samantha L

2007-06-01

The present study examined the jumping to conclusions reasoning bias across the continuum of delusional ideation by investigating individuals with active delusions, delusion prone individuals, and non-delusion prone individuals. Neutral and highly self-referent probabilistic reasoning tasks were employed. Results indicated that individuals with delusions gathered significantly less information than delusion prone and non-delusion prone participants on both the neutral and self-referent tasks, (p<.001). Individuals with delusions made less accurate decisions than the delusion prone and non-delusion prone participants on both tasks (p<.001), yet were more confident about their decisions than were delusion prone and non-delusion prone participants on the self-referent task (p=.002). Those with delusions and those who were delusion prone reported higher confidence in their performance on the self-referent task than they did the neutral task (p=.02), indicating that high self-reference impacted information processing for individuals in both of these groups. The results are discussed in relation to previous research in the area of probabilistic reasoning and delusions.

Perception of Risk and Terrorism-Related Behavior Change: Dual Influences of Probabilistic Reasoning and Reality Testing.

PubMed

Denovan, Andrew; Dagnall, Neil; Drinkwater, Kenneth; Parker, Andrew; Clough, Peter

2017-01-01

The present study assessed the degree to which probabilistic reasoning performance and thinking style influenced perception of risk and self-reported levels of terrorism-related behavior change. A sample of 263 respondents, recruited via convenience sampling, completed a series of measures comprising probabilistic reasoning tasks (perception of randomness, base rate, probability, and conjunction fallacy), the Reality Testing subscale of the Inventory of Personality Organization (IPO-RT), the Domain-Specific Risk-Taking Scale, and a terrorism-related behavior change scale. Structural equation modeling examined three progressive models. Firstly, the Independence Model assumed that probabilistic reasoning, perception of risk and reality testing independently predicted terrorism-related behavior change. Secondly, the Mediation Model supposed that probabilistic reasoning and reality testing correlated, and indirectly predicted terrorism-related behavior change through perception of risk. Lastly, the Dual-Influence Model proposed that probabilistic reasoning indirectly predicted terrorism-related behavior change via perception of risk, independent of reality testing. Results indicated that performance on probabilistic reasoning tasks most strongly predicted perception of risk, and preference for an intuitive thinking style (measured by the IPO-RT) best explained terrorism-related behavior change. The combination of perception of risk with probabilistic reasoning ability in the Dual-Influence Model enhanced the predictive power of the analytical-rational route, with conjunction fallacy having a significant indirect effect on terrorism-related behavior change via perception of risk. The Dual-Influence Model possessed superior fit and reported similar predictive relations between intuitive-experiential and analytical-rational routes and terrorism-related behavior change. The discussion critically examines these findings in relation to dual-processing frameworks. This includes considering the limitations of current operationalisations and recommendations for future research that align outcomes and subsequent work more closely to specific dual-process models.

Perception of Risk and Terrorism-Related Behavior Change: Dual Influences of Probabilistic Reasoning and Reality Testing

PubMed Central

Denovan, Andrew; Dagnall, Neil; Drinkwater, Kenneth; Parker, Andrew; Clough, Peter

2017-01-01

The present study assessed the degree to which probabilistic reasoning performance and thinking style influenced perception of risk and self-reported levels of terrorism-related behavior change. A sample of 263 respondents, recruited via convenience sampling, completed a series of measures comprising probabilistic reasoning tasks (perception of randomness, base rate, probability, and conjunction fallacy), the Reality Testing subscale of the Inventory of Personality Organization (IPO-RT), the Domain-Specific Risk-Taking Scale, and a terrorism-related behavior change scale. Structural equation modeling examined three progressive models. Firstly, the Independence Model assumed that probabilistic reasoning, perception of risk and reality testing independently predicted terrorism-related behavior change. Secondly, the Mediation Model supposed that probabilistic reasoning and reality testing correlated, and indirectly predicted terrorism-related behavior change through perception of risk. Lastly, the Dual-Influence Model proposed that probabilistic reasoning indirectly predicted terrorism-related behavior change via perception of risk, independent of reality testing. Results indicated that performance on probabilistic reasoning tasks most strongly predicted perception of risk, and preference for an intuitive thinking style (measured by the IPO-RT) best explained terrorism-related behavior change. The combination of perception of risk with probabilistic reasoning ability in the Dual-Influence Model enhanced the predictive power of the analytical-rational route, with conjunction fallacy having a significant indirect effect on terrorism-related behavior change via perception of risk. The Dual-Influence Model possessed superior fit and reported similar predictive relations between intuitive-experiential and analytical-rational routes and terrorism-related behavior change. The discussion critically examines these findings in relation to dual-processing frameworks. This includes considering the limitations of current operationalisations and recommendations for future research that align outcomes and subsequent work more closely to specific dual-process models. PMID:29062288

How social information affects information search and choice in probabilistic inferences.

PubMed

Puskaric, Marin; von Helversen, Bettina; Rieskamp, Jörg

2018-01-01

When making decisions, people are often exposed to relevant information stemming from qualitatively different sources. For instance, when making a choice between two alternatives people can rely on the advice of other people (i.e., social information) or search for factual information about the alternatives (i.e., non-social information). Prior research in categorization has shown that social information is given special attention when both social and non-social information is available, even when the social information has no additional informational value. The goal of the current work is to investigate whether framing information as social or non-social also influences information search and choice in probabilistic inferences. In a first study, we found that framing cues (i.e., the information used to make a decision) with medium validity as social increased the probability that they were searched for compared to a task where the same cues were framed as non-social information, but did not change the strategy people relied on. A second and a third study showed that framing a cue with high validity as social information facilitated learning to rely on a non-compensatory decision strategy. Overall, the results suggest that social in comparison to non-social information is given more attention and is learned faster than non-social information. Copyright © 2017 Elsevier B.V. All rights reserved.

Punishment sensitivity modulates the processing of negative feedback but not error-induced learning.

PubMed

Unger, Kerstin; Heintz, Sonja; Kray, Jutta

2012-01-01

Accumulating evidence suggests that individual differences in punishment and reward sensitivity are associated with functional alterations in neural systems underlying error and feedback processing. In particular, individuals highly sensitive to punishment have been found to be characterized by larger mediofrontal error signals as reflected in the error negativity/error-related negativity (Ne/ERN) and the feedback-related negativity (FRN). By contrast, reward sensitivity has been shown to relate to the error positivity (Pe). Given that Ne/ERN, FRN, and Pe have been functionally linked to flexible behavioral adaptation, the aim of the present research was to examine how these electrophysiological reflections of error and feedback processing vary as a function of punishment and reward sensitivity during reinforcement learning. We applied a probabilistic learning task that involved three different conditions of feedback validity (100%, 80%, and 50%). In contrast to prior studies using response competition tasks, we did not find reliable correlations between punishment sensitivity and the Ne/ERN. Instead, higher punishment sensitivity predicted larger FRN amplitudes, irrespective of feedback validity. Moreover, higher reward sensitivity was associated with a larger Pe. However, only reward sensitivity was related to better overall learning performance and higher post-error accuracy, whereas highly punishment sensitive participants showed impaired learning performance, suggesting that larger negative feedback-related error signals were not beneficial for learning or even reflected maladaptive information processing in these individuals. Thus, although our findings indicate that individual differences in reward and punishment sensitivity are related to electrophysiological correlates of error and feedback processing, we found less evidence for influences of these personality characteristics on the relation between performance monitoring and feedback-based learning.

Probabilistic dual heuristic programming-based adaptive critic

NASA Astrophysics Data System (ADS)

Herzallah, Randa

2010-02-01

Adaptive critic (AC) methods have common roots as generalisations of dynamic programming for neural reinforcement learning approaches. Since they approximate the dynamic programming solutions, they are potentially suitable for learning in noisy, non-linear and non-stationary environments. In this study, a novel probabilistic dual heuristic programming (DHP)-based AC controller is proposed. Distinct to current approaches, the proposed probabilistic (DHP) AC method takes uncertainties of forward model and inverse controller into consideration. Therefore, it is suitable for deterministic and stochastic control problems characterised by functional uncertainty. Theoretical development of the proposed method is validated by analytically evaluating the correct value of the cost function which satisfies the Bellman equation in a linear quadratic control problem. The target value of the probabilistic critic network is then calculated and shown to be equal to the analytically derived correct value. Full derivation of the Riccati solution for this non-standard stochastic linear quadratic control problem is also provided. Moreover, the performance of the proposed probabilistic controller is demonstrated on linear and non-linear control examples.

Ongoing behavioral state information signaled in the lateral habenula guides choice flexibility in freely moving rats

PubMed Central

Baker, Phillip M.; Oh, Sujean E.; Kidder, Kevan S.; Mizumori, Sheri J. Y.

2015-01-01

The lateral habenula (LHb) plays a role in a wide variety of behaviors ranging from maternal care, to sleep, to various forms of cognition. One prominent theory with ample supporting evidence is that the LHb serves to relay basal ganglia and limbic signals about negative outcomes to midbrain monoaminergic systems. This makes it likely that the LHb is critically involved in behavioral flexibility as all of these systems have been shown to contribute when flexible behavior is required. Behavioral flexibility is commonly examined across species and is impaired in various neuropsychiatric conditions including autism, depression, addiction, and schizophrenia; conditions in which the LHb is thought to play a role. Therefore, a thorough examination of the role of the LHb in behavioral flexibility serves multiple functions including understanding possible connections with neuropsychiatric illnesses and additional insight into its role in cognition in general. Here, we assess the LHb’s role in behavioral flexibility through comparisons of the roles its afferent and efferent pathways are known to play. Additionally, we provide new evidence supporting the LHb contributions to behavioral flexibility through organization of specific goal directed actions under cognitively demanding conditions. Specifically, in the first experiment, a majority of neurons recorded from the LHb were found to correlate with velocity on a spatial navigation task and did not change significantly when reward outcomes were manipulated. Additionally, measurements of local field potential (LFP) in the theta band revealed significant changes in power relative to velocity and reward location. In a second set of experiments, inactivation of the LHb with the gamma-aminobutyric acid (GABA) agonists baclofen and muscimol led to an impairment in a spatial/response based repeated probabilistic reversal learning task. Control experiments revealed that this impairment was likely due to the demands of repeated switching behaviors as rats were unimpaired on initial discrimination acquisition or retention of probabilistic learning. Taken together, these novel findings compliment other work discussed supporting a role for the LHb in action selection when cognitive or emotional demands are increased. Finally, we discuss future mechanisms by which a superior understanding of the LHb can be obtained through additional examination of behavioral flexibility tasks. PMID:26582981

A quantitative model of optimal data selection in Wason's selection task.

PubMed

Hattori, Masasi

2002-10-01

The optimal data selection model proposed by Oaksford and Chater (1994) successfully formalized Wason's selection task (Wason, 1966). The model, however, involved some questionable assumptions and was also not sufficient as a model of the task because it could not provide quantitative predictions of the card selection frequencies. In this paper, the model was revised to provide quantitative fits to the data. The model can predict the selection frequencies of cards based on a selection tendency function (STF), or conversely, it enables the estimation of subjective probabilities from data. Past experimental data were first re-analysed based on the model. In Experiment 1, the superiority of the revised model was shown. However, when the relationship between antecedent and consequent was forced to deviate from the biconditional form, the model was not supported. In Experiment 2, it was shown that sufficient emphasis on probabilistic information can affect participants' performance. A detailed experimental method to sort participants by probabilistic strategies was introduced. Here, the model was supported by a subgroup of participants who used the probabilistic strategy. Finally, the results were discussed from the viewpoint of adaptive rationality.

Quantum Inference on Bayesian Networks

NASA Astrophysics Data System (ADS)

Yoder, Theodore; Low, Guang Hao; Chuang, Isaac

2014-03-01

Because quantum physics is naturally probabilistic, it seems reasonable to expect physical systems to describe probabilities and their evolution in a natural fashion. Here, we use quantum computation to speedup sampling from a graphical probability model, the Bayesian network. A specialization of this sampling problem is approximate Bayesian inference, where the distribution on query variables is sampled given the values e of evidence variables. Inference is a key part of modern machine learning and artificial intelligence tasks, but is known to be NP-hard. Classically, a single unbiased sample is obtained from a Bayesian network on n variables with at most m parents per node in time (nmP(e) - 1 / 2) , depending critically on P(e) , the probability the evidence might occur in the first place. However, by implementing a quantum version of rejection sampling, we obtain a square-root speedup, taking (n2m P(e) -1/2) time per sample. The speedup is the result of amplitude amplification, which is proving to be broadly applicable in sampling and machine learning tasks. In particular, we provide an explicit and efficient circuit construction that implements the algorithm without the need for oracle access.

The development of categorization: effects of classification and inference training on category representation.

PubMed

Deng, Wei Sophia; Sloutsky, Vladimir M

2015-03-01

Does category representation change in the course of development? And if so, how and why? The current study attempted to answer these questions by examining category learning and category representation. In Experiment 1, 4-year-olds, 6-year-olds, and adults were trained with either a classification task or an inference task and their categorization performance and memory for items were tested. Adults and 6-year-olds exhibited an important asymmetry: they relied on a single deterministic feature during classification training, but not during inference training. In contrast, regardless of the training condition, 4-year-olds relied on multiple probabilistic features. In Experiment 2, 4-year-olds were presented with classification training and their attention was explicitly directed to the deterministic feature. Under this condition, their categorization performance was similar to that of older participants in Experiment 1, yet their memory performance pointed to a similarity-based representation, which was similar to that of 4-year-olds in Experiment 1. These results are discussed in relation to theories of categorization and the role of selective attention in the development of category learning. (PsycINFO Database Record (c) 2015 APA, all rights reserved).

Displaying uncertainty: investigating the effects of display format and specificity.

PubMed

Bisantz, Ann M; Marsiglio, Stephanie Schinzing; Munch, Jessica

2005-01-01

We conducted four studies regarding the representation of probabilistic information. Experiments 1 through 3 compared performance on a simulated stock purchase task, in which information regarding stock profitability was probabilistic. Two variables were manipulated: display format for probabilistic information (blurred and colored icons, linguistic phrases, numeric expressions, and combinations) and specificity level (in which the number and size of discrete steps into which the probabilistic information was mapped differed). Results indicated few performance differences attributable to display format; however, performance did improve with greater specificity. Experiment 4, in which participants generated membership functions corresponding to three display formats, found a high degree of similarity in functions across formats and participants and a strong relationship between the shape of the membership function and the intended meaning of the representation. These results indicate that participants can successfully interpret nonnumeric representations of uncertainty and can use such representations in a manner similar to the way numeric expressions are used in a decision-making task. Actual or potential applications of this research include the use of graphical representations of uncertainty in systems such as command and control and situation displays.

Assessing and Increasing Staff Preference for Job Tasks Using Concurrent-Chains Schedules and Probabilistic Outcomes

ERIC Educational Resources Information Center

Reed, Derek D.; DiGennaro Reed, Florence D.; Campisano, Natalie; Lacourse, Kristen; Azulay, Richard L.

2012-01-01

The assessment and improvement of staff members' subjective valuation of nonpreferred work tasks may be one way to increase the quality of staff members' work life. The Task Enjoyment Motivation Protocol (Green, Reid, Passante, & Canipe, 2008) provides a process for supervisors to identify the aversive qualities of nonpreferred job tasks.…

Maritime Threat Detection Using Probabilistic Graphical Models

DTIC Science & Technology

2012-01-01

CRF, unlike an HMM, can represent local features, and does not require feature concatenation. MLNs For MLNs, we used Alchemy ( Alchemy 2011), an...open source statistical relational learning and probabilistic inferencing package. Alchemy supports generative and discriminative weight learning, and...that Alchemy creates a new formula for every possible combination of the values for a1 and a2 that fit the type specified in their predicate

Effects of feedback delay on learning from positive and negative feedback in patients with Parkinson's disease off medication.

PubMed

Weismüller, Benjamin; Ghio, Marta; Logmin, Kazimierz; Hartmann, Christian; Schnitzler, Alfons; Pollok, Bettina; Südmeyer, Martin; Bellebaum, Christian

2018-05-11

Phasic dopamine (DA) signals conveyed from the substantia nigra to the striatum and the prefrontal cortex crucially affect learning from feedback, with DA bursts facilitating learning from positive feedback and DA dips facilitating learning from negative feedback. Consequently, diminished nigro-striatal dopamine levels as in unmedicated patients suffering from Parkinson's Disease (PD) have been shown to lead to a negative learning bias. Recent studies suggested a diminished striatal contribution to feedback processing when the outcome of an action is temporally delayed. This study investigated whether the bias towards negative feedback learning induced by a lack of DA in PD patients OFF medication is modulated by feedback delay. To this end, PD patients OFF medication and healthy controls completed a probabilistic selection task, in which feedback was given immediately (after 800 ms) or delayed (after 6800 ms). PD patients were impaired in immediate but not delayed feedback learning. However, differences in the preference for positive/negative learning between patients and controls were seen for both learning from immediate and delayed feedback, with evidence of stronger negative learning in patients than controls. A Bayesian analysis of the data supports the conclusion that feedback timing did not affect the learning bias in the patients. These results hint at reduced, but still relevant nigro-striatal contribution to feedback learning, when feedback is delayed. Copyright © 2018 Elsevier Ltd. All rights reserved.

A further assessment of decision-making in anorexia nervosa.

PubMed

Adoue, C; Jaussent, I; Olié, E; Beziat, S; Van den Eynde, F; Courtet, P; Guillaume, S

2015-01-01

Anorexia nervosa (AN) may be associated with impaired decision-making. Cognitive processes underlying this impairment remain unclear, mainly because previous assessments of this complex cognitive function were completed with a single test. Furthermore, clinical features such as mood status may impact this association. We aim to further explore the hypothesis of altered decision-making in AN. Sixty-three adult women with AN and 49 female controls completed a clinical assessment and were assessed by three tasks related to decision-making [Iowa Gambling Task (IGT), Balloon Analogue Risk Task (BART), Probabilistic Reversal Learning Task (PRLT)]. People with AN had poorer performance on the IGT and made less risky choices on the BART, whereas performances were not different on PRLT. Notably, AN patients with a current major depressive disorder showed similar performance to those with no current major depressive disorder. These results tend to confirm an impaired decision making-process in people with AN and suggest that various cognitive processes such as inhibition to risk-taking or intolerance of uncertainty may underlie this condition Furthermore, these impairments seem unrelated to the potential co-occurent major depressive disorders. Copyright © 2014 Elsevier Masson SAS. All rights reserved.

A transdiagnostic investigation of 'theory of mind' and 'jumping to conclusions' in patients with persecutory delusions.

PubMed

Corcoran, R; Rowse, G; Moore, R; Blackwood, N; Kinderman, P; Howard, R; Cummins, S; Bentall, R P

2008-11-01

A tendency to make hasty decisions on probabilistic reasoning tasks and a difficulty attributing mental states to others are key cognitive features of persecutory delusions (PDs) in the context of schizophrenia. This study examines whether these same psychological anomalies characterize PDs when they present in the context of psychotic depression. Performance on measures of probabilistic reasoning and theory of mind (ToM) was examined in five subgroups differing in diagnostic category and current illness status. The tendency to draw hasty decisions in probabilistic settings and poor ToM tested using story format feature in PDs irrespective of diagnosis. Furthermore, performance on the ToM story task correlated with the degree of distress caused by and preoccupation with the current PDs in the currently deluded groups. By contrast, performance on the non-verbal ToM task appears to be more sensitive to diagnosis, as patients with schizophrenia spectrum disorders perform worse on this task than those with depression irrespective of the presence of PDs. The psychological anomalies associated with PDs examined here are transdiagnostic but different measures of ToM may be more or less sensitive to indices of severity of the PDs, diagnosis and trait- or state-related cognitive effects.

Stuttering Thoughts: Negative Self-Referent Thinking Is Less Sensitive to Aversive Outcomes in People with Higher Levels of Depressive Symptoms

PubMed Central

Iijima, Yudai; Takano, Keisuke; Boddez, Yannick; Raes, Filip; Tanno, Yoshihiko

2017-01-01

Learning theories of depression have proposed that depressive cognitions, such as negative thoughts with reference to oneself, can develop through a reinforcement learning mechanism. This negative self-reference is considered to be positively reinforced by rewarding experiences such as genuine support from others after negative self-disclosure, and negatively reinforced by avoidance of potential aversive situations. The learning account additionally predicts that negative self-reference would be maintained by an inability to adjust one’s behavior when negative self-reference no longer leads to such reward. To test this prediction, we designed an adapted version of the reversal-learning task. In this task, participants were reinforced to choose and engage in either negative or positive self-reference by probabilistic economic reward and punishment. Although participants were initially trained to choose negative self-reference, the stimulus-reward contingencies were reversed to prompt a shift toward positive self-reference (Study 1) and a further shift toward negative self-reference (Study 2). Model-based computational analyses showed that depressive symptoms were associated with a low learning rate of negative self-reference, indicating a high level of reward expectancy for negative self-reference even after the contingency reversal. Furthermore, the difficulty in updating outcome predictions of negative self-reference was significantly associated with the extent to which one possesses negative self-images. These results suggest that difficulty in adjusting action-outcome estimates for negative self-reference increases the chance to be faced with negative aspects of self, which may result in depressive symptoms. PMID:28824511

Stuttering Thoughts: Negative Self-Referent Thinking Is Less Sensitive to Aversive Outcomes in People with Higher Levels of Depressive Symptoms.

PubMed

Iijima, Yudai; Takano, Keisuke; Boddez, Yannick; Raes, Filip; Tanno, Yoshihiko

2017-01-01

Learning theories of depression have proposed that depressive cognitions, such as negative thoughts with reference to oneself, can develop through a reinforcement learning mechanism. This negative self-reference is considered to be positively reinforced by rewarding experiences such as genuine support from others after negative self-disclosure, and negatively reinforced by avoidance of potential aversive situations. The learning account additionally predicts that negative self-reference would be maintained by an inability to adjust one's behavior when negative self-reference no longer leads to such reward. To test this prediction, we designed an adapted version of the reversal-learning task. In this task, participants were reinforced to choose and engage in either negative or positive self-reference by probabilistic economic reward and punishment. Although participants were initially trained to choose negative self-reference, the stimulus-reward contingencies were reversed to prompt a shift toward positive self-reference (Study 1) and a further shift toward negative self-reference (Study 2). Model-based computational analyses showed that depressive symptoms were associated with a low learning rate of negative self-reference, indicating a high level of reward expectancy for negative self-reference even after the contingency reversal. Furthermore, the difficulty in updating outcome predictions of negative self-reference was significantly associated with the extent to which one possesses negative self-images. These results suggest that difficulty in adjusting action-outcome estimates for negative self-reference increases the chance to be faced with negative aspects of self, which may result in depressive symptoms.

Impairments in learning by monetary rewards and alcohol-associated rewards in detoxified alcoholic patients.

PubMed

Jokisch, Daniel; Roser, Patrik; Juckel, Georg; Daum, Irene; Bellebaum, Christian

2014-07-01

Excessive alcohol consumption has been linked to structural and functional brain changes associated with cognitive, emotional, and behavioral impairments. It has been suggested that neural processing in the reward system is also affected by alcoholism. The present study aimed at further investigating reward-based associative learning and reversal learning in detoxified alcohol-dependent patients. Twenty-one detoxified alcohol-dependent patients and 26 healthy control subjects participated in a probabilistic learning task using monetary and alcohol-associated rewards as feedback stimuli indicating correct responses. Performance during acquisition and reversal learning in the different feedback conditions was analyzed. Alcohol-dependent patients and healthy control subjects showed an increase in learning performance over learning blocks during acquisition, with learning performance being significantly lower in alcohol-dependent patients. After changing the contingencies, alcohol-dependent patients exhibited impaired reversal learning and showed, in contrast to healthy controls, different learning curves for different types of rewards with no increase in performance for high monetary and alcohol-associated feedback. The present findings provide evidence that dysfunctional processing in the reward system in alcohol-dependent patients leads to alterations in reward-based learning resulting in a generally reduced performance. In addition, the results suggest that alcohol-dependent patients are, in particular, more impaired in changing an established behavior originally reinforced by high rewards. Copyright © 2014 by the Research Society on Alcoholism.

Don't Fear Optimality: Sampling for Probabilistic-Logic Sequence Models

NASA Astrophysics Data System (ADS)

Thon, Ingo

One of the current challenges in artificial intelligence is modeling dynamic environments that change due to the actions or activities undertaken by people or agents. The task of inferring hidden states, e.g. the activities or intentions of people, based on observations is called filtering. Standard probabilistic models such as Dynamic Bayesian Networks are able to solve this task efficiently using approximative methods such as particle filters. However, these models do not support logical or relational representations. The key contribution of this paper is the upgrade of a particle filter algorithm for use with a probabilistic logical representation through the definition of a proposal distribution. The performance of the algorithm depends largely on how well this distribution fits the target distribution. We adopt the idea of logical compilation into Binary Decision Diagrams for sampling. This allows us to use the optimal proposal distribution which is normally prohibitively slow.

Spatiotemporal movement planning and rapid adaptation for manual interaction.

PubMed

Huber, Markus; Kupferberg, Aleksandra; Lenz, Claus; Knoll, Alois; Brandt, Thomas; Glasauer, Stefan

2013-01-01

Many everyday tasks require the ability of two or more individuals to coordinate their actions with others to increase efficiency. Such an increase in efficiency can often be observed even after only very few trials. Previous work suggests that such behavioral adaptation can be explained within a probabilistic framework that integrates sensory input and prior experience. Even though higher cognitive abilities such as intention recognition have been described as probabilistic estimation depending on an internal model of the other agent, it is not clear whether much simpler daily interaction is consistent with a probabilistic framework. Here, we investigate whether the mechanisms underlying efficient coordination during manual interactions can be understood as probabilistic optimization. For this purpose we studied in several experiments a simple manual handover task concentrating on the action of the receiver. We found that the duration until the receiver reacts to the handover decreases over trials, but strongly depends on the position of the handover. We then replaced the human deliverer by different types of robots to further investigate the influence of the delivering movement on the reaction of the receiver. Durations were found to depend on movement kinematics and the robot's joint configuration. Modeling the task was based on the assumption that the receiver's decision to act is based on the accumulated evidence for a specific handover position. The evidence for this handover position is collected from observing the hand movement of the deliverer over time and, if appropriate, by integrating this sensory likelihood with prior expectation that is updated over trials. The close match of model simulations and experimental results shows that the efficiency of handover coordination can be explained by an adaptive probabilistic fusion of a-priori expectation and online estimation.

Machine Learning

DOE Office of Scientific and Technical Information (OSTI.GOV)

Chikkagoudar, Satish; Chatterjee, Samrat; Thomas, Dennis G.

The absence of a robust and unified theory of cyber dynamics presents challenges and opportunities for using machine learning based data-driven approaches to further the understanding of the behavior of such complex systems. Analysts can also use machine learning approaches to gain operational insights. In order to be operationally beneficial, cybersecurity machine learning based models need to have the ability to: (1) represent a real-world system, (2) infer system properties, and (3) learn and adapt based on expert knowledge and observations. Probabilistic models and Probabilistic graphical models provide these necessary properties and are further explored in this chapter. Bayesian Networksmore » and Hidden Markov Models are introduced as an example of a widely used data driven classification/modeling strategy.« less

Development of probabilistic multimedia multipathway computer codes.

DOE Office of Scientific and Technical Information (OSTI.GOV)

Yu, C.; LePoire, D.; Gnanapragasam, E.

2002-01-01

The deterministic multimedia dose/risk assessment codes RESRAD and RESRAD-BUILD have been widely used for many years for evaluation of sites contaminated with residual radioactive materials. The RESRAD code applies to the cleanup of sites (soils) and the RESRAD-BUILD code applies to the cleanup of buildings and structures. This work describes the procedure used to enhance the deterministic RESRAD and RESRAD-BUILD codes for probabilistic dose analysis. A six-step procedure was used in developing default parameter distributions and the probabilistic analysis modules. These six steps include (1) listing and categorizing parameters; (2) ranking parameters; (3) developing parameter distributions; (4) testing parameter distributionsmore » for probabilistic analysis; (5) developing probabilistic software modules; and (6) testing probabilistic modules and integrated codes. The procedures used can be applied to the development of other multimedia probabilistic codes. The probabilistic versions of RESRAD and RESRAD-BUILD codes provide tools for studying the uncertainty in dose assessment caused by uncertain input parameters. The parameter distribution data collected in this work can also be applied to other multimedia assessment tasks and multimedia computer codes.« less

Assisting Movement Training and Execution With Visual and Haptic Feedback.

PubMed

Ewerton, Marco; Rother, David; Weimar, Jakob; Kollegger, Gerrit; Wiemeyer, Josef; Peters, Jan; Maeda, Guilherme

2018-01-01

In the practice of motor skills in general, errors in the execution of movements may go unnoticed when a human instructor is not available. In this case, a computer system or robotic device able to detect movement errors and propose corrections would be of great help. This paper addresses the problem of how to detect such execution errors and how to provide feedback to the human to correct his/her motor skill using a general, principled methodology based on imitation learning. The core idea is to compare the observed skill with a probabilistic model learned from expert demonstrations. The intensity of the feedback is regulated by the likelihood of the model given the observed skill. Based on demonstrations, our system can, for example, detect errors in the writing of characters with multiple strokes. Moreover, by using a haptic device, the Haption Virtuose 6D, we demonstrate a method to generate haptic feedback based on a distribution over trajectories, which could be used as an auxiliary means of communication between an instructor and an apprentice. Additionally, given a performance measurement, the haptic device can help the human discover and perform better movements to solve a given task. In this case, the human first tries a few times to solve the task without assistance. Our framework, in turn, uses a reinforcement learning algorithm to compute haptic feedback, which guides the human toward better solutions.

Biases in probabilistic category learning in relation to social anxiety

PubMed Central

Abraham, Anna; Hermann, Christiane

2015-01-01

Instrumental learning paradigms are rarely employed to investigate the mechanisms underlying acquired fear responses in social anxiety. Here, we adapted a probabilistic category learning paradigm to assess information processing biases as a function of the degree of social anxiety traits in a sample of healthy individuals without a diagnosis of social phobia. Participants were presented with three pairs of neutral faces with differing probabilistic accuracy contingencies (A/B: 80/20, C/D: 70/30, E/F: 60/40). Upon making their choice, negative and positive feedback was conveyed using angry and happy faces, respectively. The highly socially anxious group showed a strong tendency to be more accurate at learning the probability contingency associated with the most ambiguous stimulus pair (E/F: 60/40). Moreover, when pairing the most positively reinforced stimulus or the most negatively reinforced stimulus with all the other stimuli in a test phase, the highly socially anxious group avoided the most negatively reinforced stimulus significantly more than the control group. The results are discussed with reference to avoidance learning and hypersensitivity to negative socially evaluative information associated with social anxiety. PMID:26347685

Optimal Symmetric Multimodal Templates and Concatenated Random Forests for Supervised Brain Tumor Segmentation (Simplified) with ANTsR.

PubMed

Tustison, Nicholas J; Shrinidhi, K L; Wintermark, Max; Durst, Christopher R; Kandel, Benjamin M; Gee, James C; Grossman, Murray C; Avants, Brian B

2015-04-01

Segmenting and quantifying gliomas from MRI is an important task for diagnosis, planning intervention, and for tracking tumor changes over time. However, this task is complicated by the lack of prior knowledge concerning tumor location, spatial extent, shape, possible displacement of normal tissue, and intensity signature. To accommodate such complications, we introduce a framework for supervised segmentation based on multiple modality intensity, geometry, and asymmetry feature sets. These features drive a supervised whole-brain and tumor segmentation approach based on random forest-derived probabilities. The asymmetry-related features (based on optimal symmetric multimodal templates) demonstrate excellent discriminative properties within this framework. We also gain performance by generating probability maps from random forest models and using these maps for a refining Markov random field regularized probabilistic segmentation. This strategy allows us to interface the supervised learning capabilities of the random forest model with regularized probabilistic segmentation using the recently developed ANTsR package--a comprehensive statistical and visualization interface between the popular Advanced Normalization Tools (ANTs) and the R statistical project. The reported algorithmic framework was the top-performing entry in the MICCAI 2013 Multimodal Brain Tumor Segmentation challenge. The challenge data were widely varying consisting of both high-grade and low-grade glioma tumor four-modality MRI from five different institutions. Average Dice overlap measures for the final algorithmic assessment were 0.87, 0.78, and 0.74 for "complete", "core", and "enhanced" tumor components, respectively.

Automated Grading of Gliomas using Deep Learning in Digital Pathology Images: A modular approach with ensemble of convolutional neural networks.

PubMed

Ertosun, Mehmet Günhan; Rubin, Daniel L

2015-01-01

Brain glioma is the most common primary malignant brain tumors in adults with different pathologic subtypes: Lower Grade Glioma (LGG) Grade II, Lower Grade Glioma (LGG) Grade III, and Glioblastoma Multiforme (GBM) Grade IV. The survival and treatment options are highly dependent of this glioma grade. We propose a deep learning-based, modular classification pipeline for automated grading of gliomas using digital pathology images. Whole tissue digitized images of pathology slides obtained from The Cancer Genome Atlas (TCGA) were used to train our deep learning modules. Our modular pipeline provides diagnostic quality statistics, such as precision, sensitivity and specificity, of the individual deep learning modules, and (1) facilitates training given the limited data in this domain, (2) enables exploration of different deep learning structures for each module, (3) leads to developing less complex modules that are simpler to analyze, and (4) provides flexibility, permitting use of single modules within the framework or use of other modeling or machine learning applications, such as probabilistic graphical models or support vector machines. Our modular approach helps us meet the requirements of minimum accuracy levels that are demanded by the context of different decision points within a multi-class classification scheme. Convolutional Neural Networks are trained for each module for each sub-task with more than 90% classification accuracies on validation data set, and achieved classification accuracy of 96% for the task of GBM vs LGG classification, 71% for further identifying the grade of LGG into Grade II or Grade III on independent data set coming from new patients from the multi-institutional repository.

Automated Grading of Gliomas using Deep Learning in Digital Pathology Images: A modular approach with ensemble of convolutional neural networks

PubMed Central

Ertosun, Mehmet Günhan; Rubin, Daniel L.

2015-01-01

Brain glioma is the most common primary malignant brain tumors in adults with different pathologic subtypes: Lower Grade Glioma (LGG) Grade II, Lower Grade Glioma (LGG) Grade III, and Glioblastoma Multiforme (GBM) Grade IV. The survival and treatment options are highly dependent of this glioma grade. We propose a deep learning-based, modular classification pipeline for automated grading of gliomas using digital pathology images. Whole tissue digitized images of pathology slides obtained from The Cancer Genome Atlas (TCGA) were used to train our deep learning modules. Our modular pipeline provides diagnostic quality statistics, such as precision, sensitivity and specificity, of the individual deep learning modules, and (1) facilitates training given the limited data in this domain, (2) enables exploration of different deep learning structures for each module, (3) leads to developing less complex modules that are simpler to analyze, and (4) provides flexibility, permitting use of single modules within the framework or use of other modeling or machine learning applications, such as probabilistic graphical models or support vector machines. Our modular approach helps us meet the requirements of minimum accuracy levels that are demanded by the context of different decision points within a multi-class classification scheme. Convolutional Neural Networks are trained for each module for each sub-task with more than 90% classification accuracies on validation data set, and achieved classification accuracy of 96% for the task of GBM vs LGG classification, 71% for further identifying the grade of LGG into Grade II or Grade III on independent data set coming from new patients from the multi-institutional repository. PMID:26958289

Sampling Assumptions Affect Use of Indirect Negative Evidence in Language Learning.

PubMed

Hsu, Anne; Griffiths, Thomas L

2016-01-01

A classic debate in cognitive science revolves around understanding how children learn complex linguistic patterns, such as restrictions on verb alternations and contractions, without negative evidence. Recently, probabilistic models of language learning have been applied to this problem, framing it as a statistical inference from a random sample of sentences. These probabilistic models predict that learners should be sensitive to the way in which sentences are sampled. There are two main types of sampling assumptions that can operate in language learning: strong and weak sampling. Strong sampling, as assumed by probabilistic models, assumes the learning input is drawn from a distribution of grammatical samples from the underlying language and aims to learn this distribution. Thus, under strong sampling, the absence of a sentence construction from the input provides evidence that it has low or zero probability of grammaticality. Weak sampling does not make assumptions about the distribution from which the input is drawn, and thus the absence of a construction from the input as not used as evidence of its ungrammaticality. We demonstrate in a series of artificial language learning experiments that adults can produce behavior consistent with both sets of sampling assumptions, depending on how the learning problem is presented. These results suggest that people use information about the way in which linguistic input is sampled to guide their learning.

Sampling Assumptions Affect Use of Indirect Negative Evidence in Language Learning

PubMed Central

2016-01-01

A classic debate in cognitive science revolves around understanding how children learn complex linguistic patterns, such as restrictions on verb alternations and contractions, without negative evidence. Recently, probabilistic models of language learning have been applied to this problem, framing it as a statistical inference from a random sample of sentences. These probabilistic models predict that learners should be sensitive to the way in which sentences are sampled. There are two main types of sampling assumptions that can operate in language learning: strong and weak sampling. Strong sampling, as assumed by probabilistic models, assumes the learning input is drawn from a distribution of grammatical samples from the underlying language and aims to learn this distribution. Thus, under strong sampling, the absence of a sentence construction from the input provides evidence that it has low or zero probability of grammaticality. Weak sampling does not make assumptions about the distribution from which the input is drawn, and thus the absence of a construction from the input as not used as evidence of its ungrammaticality. We demonstrate in a series of artificial language learning experiments that adults can produce behavior consistent with both sets of sampling assumptions, depending on how the learning problem is presented. These results suggest that people use information about the way in which linguistic input is sampled to guide their learning. PMID:27310576

Evaluating bacterial gene-finding HMM structures as probabilistic logic programs.

PubMed

Mørk, Søren; Holmes, Ian

2012-03-01

Probabilistic logic programming offers a powerful way to describe and evaluate structured statistical models. To investigate the practicality of probabilistic logic programming for structure learning in bioinformatics, we undertook a simplified bacterial gene-finding benchmark in PRISM, a probabilistic dialect of Prolog. We evaluate Hidden Markov Model structures for bacterial protein-coding gene potential, including a simple null model structure, three structures based on existing bacterial gene finders and two novel model structures. We test standard versions as well as ADPH length modeling and three-state versions of the five model structures. The models are all represented as probabilistic logic programs and evaluated using the PRISM machine learning system in terms of statistical information criteria and gene-finding prediction accuracy, in two bacterial genomes. Neither of our implementations of the two currently most used model structures are best performing in terms of statistical information criteria or prediction performances, suggesting that better-fitting models might be achievable. The source code of all PRISM models, data and additional scripts are freely available for download at: http://github.com/somork/codonhmm. Supplementary data are available at Bioinformatics online.

Sex Differences in Risk Preference and c-Fos Expression in Paraventricular Thalamic Nucleus of Rats During Gambling Task

PubMed Central

Ishii, Hironori; Onodera, Mariko; Ohara, Shinya; Tsutsui, Ken-Ichiro; Iijima, Toshio

2018-01-01

Different biological requirements between males and females may cause sex differences in decision preference when choosing between taking a risk to get a higher gain or taking a lower but sure gain. Several studies have tested this assumption in rats, however the conclusion remains controversial because the previous real-world like gambling tasks contained a learning component to track a global payoff of probabilistic outcome in addition to risk preference. Therefore, we modified a simple gambling task allowing us to exclude such learning effect, and investigated the sex difference in risk preference of rats and its neural basis. The task required water deprived rats to choose between a risky option which provided four drops of water or no reward at a 50% random chance vs. a sure option which provided predictable amount x (x = 1, 2, 3, 4). The amount and the risk were explicitly instructed so that different choice conditions could be tested trial by trial without re-learning of reward contingency. Although both sexes correctly chose the sure option with the same level of accuracy when the sure option provided the best offer (x = 4), they exhibited different choice performances when two options had the same expected value (x = 2). Males and females both preferred to take risky choices than sure choices (risk seeking), but males were more risk seeking than females. Outcome-history analysis of their choice pattern revealed that females reduced their risk preference after losing risky choices, whereas males did not. Rather, as losses continued, reaction time for subsequent risky choices got shorter in males. Given that significant sex difference features mainly emerged after negative experiences, male and female rats may evaluate an unsuccessful outcome of their decision in different manners. Furthermore, c-Fos expression in the paraventricular nucleus of the thalamus (PV) was higher in the gambling task than for the control task in males while c-fos levels did not differ in females. The present study provides a clear evidence of sex differences in risk preference in rats and suggests that the PV is a candidate region contributing to sex differences in risky decision making. PMID:29692713

Probabilistic Inference: Task Dependency and Individual Differences of Probability Weighting Revealed by Hierarchical Bayesian Modeling

PubMed Central

Boos, Moritz; Seer, Caroline; Lange, Florian; Kopp, Bruno

2016-01-01

Cognitive determinants of probabilistic inference were examined using hierarchical Bayesian modeling techniques. A classic urn-ball paradigm served as experimental strategy, involving a factorial two (prior probabilities) by two (likelihoods) design. Five computational models of cognitive processes were compared with the observed behavior. Parameter-free Bayesian posterior probabilities and parameter-free base rate neglect provided inadequate models of probabilistic inference. The introduction of distorted subjective probabilities yielded more robust and generalizable results. A general class of (inverted) S-shaped probability weighting functions had been proposed; however, the possibility of large differences in probability distortions not only across experimental conditions, but also across individuals, seems critical for the model's success. It also seems advantageous to consider individual differences in parameters of probability weighting as being sampled from weakly informative prior distributions of individual parameter values. Thus, the results from hierarchical Bayesian modeling converge with previous results in revealing that probability weighting parameters show considerable task dependency and individual differences. Methodologically, this work exemplifies the usefulness of hierarchical Bayesian modeling techniques for cognitive psychology. Theoretically, human probabilistic inference might be best described as the application of individualized strategic policies for Bayesian belief revision. PMID:27303323

The dual reading of general conditionals: The influence of abstract versus concrete contexts.

PubMed

Wang, Moyun; Yao, Xinyun

2018-04-01

A current main issue on conditionals is whether the meaning of general conditionals (e.g., If a card is red, then it is round) is deterministic (exceptionless) or probabilistic (exception-tolerating). In order to resolve the issue, two experiments examined the influence of conditional contexts (with vs. without frequency information of truth table cases) on the reading of general conditionals. Experiment 1 examined the direct reading of general conditionals in the possibility judgment task. Experiment 2 examined the indirect reading of general conditionals in the truth judgment task. It was found that both the direct and indirect reading of general conditionals exhibited the duality: the predominant deterministic semantic reading of conditionals without frequency information, and the predominant probabilistic pragmatic reading of conditionals with frequency information. The context of general conditionals determined the predominant reading of general conditionals. There were obvious individual differences in reading general conditionals with frequency information. The meaning of general conditionals is relative, depending on conditional contexts. The reading of general conditionals is flexible and complex so that no simple deterministic and probabilistic accounts are able to explain it. The present findings are beyond the extant deterministic and probabilistic accounts of conditionals.

Effects of delay and probability combinations on discounting in humans

PubMed Central

Cox, David J.; Dallery, Jesse

2017-01-01

To determine discount rates, researchers typically adjust the amount of an immediate or certain option relative to a delayed or uncertain option. Because this adjusting amount method can be relatively time consuming, researchers have developed more efficient procedures. One such procedure is a 5-trial adjusting delay procedure, which measures the delay at which an amount of money loses half of its value (e.g., $1000 is valued at $500 with a 10-year delay to its receipt). Experiment 1 (n = 212) used 5-trial adjusting delay or probability tasks to measure delay discounting of losses, probabilistic gains, and probabilistic losses. Experiment 2 (n = 98) assessed combined probabilistic and delayed alternatives. In both experiments, we compared results from 5-trial adjusting delay or probability tasks to traditional adjusting amount procedures. Results suggest both procedures produced similar rates of probability and delay discounting in six out of seven comparisons. A magnitude effect consistent with previous research was observed for probabilistic gains and losses, but not for delayed losses. Results also suggest that delay and probability interact to determine the value of money. Five-trial methods may allow researchers to assess discounting more efficiently as well as study more complex choice scenarios. PMID:27498073

Beads task vs. box task: The specificity of the jumping to conclusions bias.

PubMed

Balzan, Ryan P; Ephraums, Rachel; Delfabbro, Paul; Andreou, Christina

2017-09-01

Previous research involving the probabilistic reasoning 'beads task' has consistently demonstrated a jumping-to-conclusions (JTC) bias, where individuals with delusions make decisions based on limited evidence. However, recent studies have suggested that miscomprehension may be confounding the beads task. The current study aimed to test the conventional beads task against a conceptually simpler probabilistic reasoning "box task" METHODS: One hundred non-clinical participants completed both the beads task and the box task, and the Peters et al. Delusions Inventory (PDI) to assess for delusion-proneness. The number of 'draws to decision' was assessed for both tasks. Additionally, the total amount of on-screen evidence was manipulated for the box task, and two new box task measures were assessed (i.e., 'proportion of evidence requested' and 'deviation from optimal solution'). Despite being conceptually similar, the two tasks did not correlate, and participants requested significantly less information on the beads task relative to the box task. High-delusion-prone participants did not demonstrate hastier decisions on either task; in fact, for box task, this group was observed to be significantly more conservative than low-delusion-prone group. Neither task was incentivized; results need replication with a clinical sample. Participants, and particularly those identified as high-delusion-prone, displayed a more conservative style of responding on the novel box task, relative to the beads task. The two tasks, whilst conceptually similar, appear to be tapping different cognitive processes. The implications of these results are discussed in relation to the JTC bias and the theoretical mechanisms thought to underlie it. Copyright © 2016 Elsevier Ltd. All rights reserved.

Information Processing at the Memoryful and Memoryless Channel Levels in Problem-Solving and Recall Tasks.

ERIC Educational Resources Information Center

Fazio, Frank; Moser, Gene W.

A probabilistic model (see SE 013 578) describing information processing during the cognitive tasks of recall and problem solving was tested, refined, and developed by testing graduate students on a number of tasks which combined oral, written, and overt "input" and "output" modes in several ways. In a verbal chain one subject…

Hierarchical pictorial structures for simultaneously localizing multiple organs in volumetric pre-scan CT

NASA Astrophysics Data System (ADS)

Montillo, Albert; Song, Qi; Das, Bipul; Yin, Zhye

2015-03-01

Parsing volumetric computed tomography (CT) into 10 or more salient organs simultaneously is a challenging task with many applications such as personalized scan planning and dose reporting. In the clinic, pre-scan data can come in the form of very low dose volumes acquired just prior to the primary scan or from an existing primary scan. To localize organs in such diverse data, we propose a new learning based framework that we call hierarchical pictorial structures (HPS) which builds multiple levels of models in a tree-like hierarchy that mirrors the natural decomposition of human anatomy from gross structures to finer structures. Each node of our hierarchical model learns (1) the local appearance and shape of structures, and (2) a generative global model that learns probabilistic, structural arrangement. Our main contribution is twofold. First we embed the pictorial structures approach in a hierarchical framework which reduces test time image interpretation and allows for the incorporation of additional geometric constraints that robustly guide model fitting in the presence of noise. Second we guide our HPS framework with the probabilistic cost maps extracted using random decision forests using volumetric 3D HOG features which makes our model fast to train and fast to apply to novel test data and posses a high degree of invariance to shape distortion and imaging artifacts. All steps require approximate 3 mins to compute and all organs are located with suitably high accuracy for our clinical applications such as personalized scan planning for radiation dose reduction. We assess our method using a database of volumetric CT scans from 81 subjects with widely varying age and pathology and with simulated ultra-low dose cadaver pre-scan data.

Relevance feedback for CBIR: a new approach based on probabilistic feature weighting with positive and negative examples.

PubMed

Kherfi, Mohammed Lamine; Ziou, Djemel

2006-04-01

In content-based image retrieval, understanding the user's needs is a challenging task that requires integrating him in the process of retrieval. Relevance feedback (RF) has proven to be an effective tool for taking the user's judgement into account. In this paper, we present a new RF framework based on a feature selection algorithm that nicely combines the advantages of a probabilistic formulation with those of using both the positive example (PE) and the negative example (NE). Through interaction with the user, our algorithm learns the importance he assigns to image features, and then applies the results obtained to define similarity measures that correspond better to his judgement. The use of the NE allows images undesired by the user to be discarded, thereby improving retrieval accuracy. As for the probabilistic formulation of the problem, it presents a multitude of advantages and opens the door to more modeling possibilities that achieve a good feature selection. It makes it possible to cluster the query data into classes, choose the probability law that best models each class, model missing data, and support queries with multiple PE and/or NE classes. The basic principle of our algorithm is to assign more importance to features with a high likelihood and those which distinguish well between PE classes and NE classes. The proposed algorithm was validated separately and in image retrieval context, and the experiments show that it performs a good feature selection and contributes to improving retrieval effectiveness.

Dissociating hippocampal and striatal contributions to sequential prediction learning

PubMed Central

Bornstein, Aaron M.; Daw, Nathaniel D.

2011-01-01

Behavior may be generated on the basis of many different kinds of learned contingencies. For instance, responses could be guided by the direct association between a stimulus and response, or by sequential stimulus-stimulus relationships (as in model-based reinforcement learning or goal-directed actions). However, the neural architecture underlying sequential predictive learning is not well-understood, in part because it is difficult to isolate its effect on choice behavior. To track such learning more directly, we examined reaction times (RTs) in a probabilistic sequential picture identification task. We used computational learning models to isolate trial-by-trial effects of two distinct learning processes in behavior, and used these as signatures to analyze the separate neural substrates of each process. RTs were best explained via the combination of two delta rule learning processes with different learning rates. To examine neural manifestations of these learning processes, we used functional magnetic resonance imaging to seek correlates of timeseries related to expectancy or surprise. We observed such correlates in two regions, hippocampus and striatum. By estimating the learning rates best explaining each signal, we verified that they were uniquely associated with one of the two distinct processes identified behaviorally. These differential correlates suggest that complementary anticipatory functions drive each region's effect on behavior. Our results provide novel insights as to the quantitative computational distinctions between medial temporal and basal ganglia learning networks and enable experiments that exploit trial-by-trial measurement of the unique contributions of both hippocampus and striatum to response behavior. PMID:22487032

Probabilistic and machine learning-based retrieval approaches for biomedical dataset retrieval

PubMed Central

Karisani, Payam; Qin, Zhaohui S; Agichtein, Eugene

2018-01-01

Abstract The bioCADDIE dataset retrieval challenge brought together different approaches to retrieval of biomedical datasets relevant to a user’s query, expressed as a text description of a needed dataset. We describe experiments in applying a data-driven, machine learning-based approach to biomedical dataset retrieval as part of this challenge. We report on a series of experiments carried out to evaluate the performance of both probabilistic and machine learning-driven techniques from information retrieval, as applied to this challenge. Our experiments with probabilistic information retrieval methods, such as query term weight optimization, automatic query expansion and simulated user relevance feedback, demonstrate that automatically boosting the weights of important keywords in a verbose query is more effective than other methods. We also show that although there is a rich space of potential representations and features available in this domain, machine learning-based re-ranking models are not able to improve on probabilistic information retrieval techniques with the currently available training data. The models and algorithms presented in this paper can serve as a viable implementation of a search engine to provide access to biomedical datasets. The retrieval performance is expected to be further improved by using additional training data that is created by expert annotation, or gathered through usage logs, clicks and other processes during natural operation of the system. Database URL: https://github.com/emory-irlab/biocaddie PMID:29688379

Conservative forgetful scholars: How people learn causal structure through sequences of interventions.

PubMed

Bramley, Neil R; Lagnado, David A; Speekenbrink, Maarten

2015-05-01

Interacting with a system is key to uncovering its causal structure. A computational framework for interventional causal learning has been developed over the last decade, but how real causal learners might achieve or approximate the computations entailed by this framework is still poorly understood. Here we describe an interactive computer task in which participants were incentivized to learn the structure of probabilistic causal systems through free selection of multiple interventions. We develop models of participants' intervention choices and online structure judgments, using expected utility gain, probability gain, and information gain and introducing plausible memory and processing constraints. We find that successful participants are best described by a model that acts to maximize information (rather than expected score or probability of being correct); that forgets much of the evidence received in earlier trials; but that mitigates this by being conservative, preferring structures consistent with earlier stated beliefs. We explore 2 heuristics that partly explain how participants might be approximating these models without explicitly representing or updating a hypothesis space. (c) 2015 APA, all rights reserved).

A Selective Role for Dopamine in Learning to Maximize Reward But Not to Minimize Effort: Evidence from Patients with Parkinson's Disease.

PubMed

Skvortsova, Vasilisa; Degos, Bertrand; Welter, Marie-Laure; Vidailhet, Marie; Pessiglione, Mathias

2017-06-21

Instrumental learning is a fundamental process through which agents optimize their choices, taking into account various dimensions of available options such as the possible reward or punishment outcomes and the costs associated with potential actions. Although the implication of dopamine in learning from choice outcomes is well established, less is known about its role in learning the action costs such as effort. Here, we tested the ability of patients with Parkinson's disease (PD) to maximize monetary rewards and minimize physical efforts in a probabilistic instrumental learning task. The implication of dopamine was assessed by comparing performance ON and OFF prodopaminergic medication. In a first sample of PD patients ( n = 15), we observed that reward learning, but not effort learning, was selectively impaired in the absence of treatment, with a significant interaction between learning condition (reward vs effort) and medication status (OFF vs ON). These results were replicated in a second, independent sample of PD patients ( n = 20) using a simplified version of the task. According to Bayesian model selection, the best account for medication effects in both studies was a specific amplification of reward magnitude in a Q-learning algorithm. These results suggest that learning to avoid physical effort is independent from dopaminergic circuits and strengthen the general idea that dopaminergic signaling amplifies the effects of reward expectation or obtainment on instrumental behavior. SIGNIFICANCE STATEMENT Theoretically, maximizing reward and minimizing effort could involve the same computations and therefore rely on the same brain circuits. Here, we tested whether dopamine, a key component of reward-related circuitry, is also implicated in effort learning. We found that patients suffering from dopamine depletion due to Parkinson's disease were selectively impaired in reward learning, but not effort learning. Moreover, anti-parkinsonian medication restored the ability to maximize reward, but had no effect on effort minimization. This dissociation suggests that the brain has evolved separate, domain-specific systems for instrumental learning. These results help to disambiguate the motivational role of prodopaminergic medications: they amplify the impact of reward without affecting the integration of effort cost. Copyright © 2017 the authors 0270-6474/17/376087-11$15.00/0.

Quantum-assisted learning of graphical models with arbitrary pairwise connectivity

NASA Astrophysics Data System (ADS)

Realpe-Gómez, John; Benedetti, Marcello; Biswas, Rupak; Perdomo-Ortiz, Alejandro

Mainstream machine learning techniques rely heavily on sampling from generally intractable probability distributions. There is increasing interest in the potential advantages of using quantum computing technologies as sampling engines to speedup these tasks. However, some pressing challenges in state-of-the-art quantum annealers have to be overcome before we can assess their actual performance. The sparse connectivity, resulting from the local interaction between quantum bits in physical hardware implementations, is considered the most severe limitation to the quality of constructing powerful machine learning models. Here we show how to surpass this `curse of limited connectivity' bottleneck and illustrate our findings by training probabilistic generative models with arbitrary pairwise connectivity on a real dataset of handwritten digits and two synthetic datasets in experiments with up to 940 quantum bits. Our model can be trained in quantum hardware without full knowledge of the effective parameters specifying the corresponding Boltzmann-like distribution. Therefore, the need to infer the effective temperature at each iteration is avoided, speeding up learning, and the effect of noise in the control parameters is mitigated, improving accuracy. This work was supported in part by NASA, AFRL, ODNI, and IARPA.

Enhanced negative feedback responses in remitted depression.

PubMed

Santesso, Diane L; Steele, Katherine T; Bogdan, Ryan; Holmes, Avram J; Deveney, Christen M; Meites, Tiffany M; Pizzagalli, Diego A

2008-07-02

Major depressive disorder (MDD) is characterized by hypersensitivity to negative feedback that might involve frontocingulate dysfunction. MDD patients exhibit enhanced electrophysiological responses to negative internal (errors) and external (feedback) cues. Whether this dysfunction extends to remitted depressed (RD) individuals with a history of MDD is currently unknown. To address this issue, we examined the feedback-related negativity in RD and control participants using a probabilistic punishment learning task. Despite equivalent behavioral performance, RD participants showed larger feedback-related negativities to negative feedback relative to controls; group differences remained after accounting for residual anxiety and depressive symptoms. The present findings suggest that abnormal responses to negative feedback extend to samples at increased risk for depressive episodes in the absence of current symptoms.

Learning Orthographic Structure With Sequential Generative Neural Networks.

PubMed

Testolin, Alberto; Stoianov, Ivilin; Sperduti, Alessandro; Zorzi, Marco

2016-04-01

Learning the structure of event sequences is a ubiquitous problem in cognition and particularly in language. One possible solution is to learn a probabilistic generative model of sequences that allows making predictions about upcoming events. Though appealing from a neurobiological standpoint, this approach is typically not pursued in connectionist modeling. Here, we investigated a sequential version of the restricted Boltzmann machine (RBM), a stochastic recurrent neural network that extracts high-order structure from sensory data through unsupervised generative learning and can encode contextual information in the form of internal, distributed representations. We assessed whether this type of network can extract the orthographic structure of English monosyllables by learning a generative model of the letter sequences forming a word training corpus. We show that the network learned an accurate probabilistic model of English graphotactics, which can be used to make predictions about the letter following a given context as well as to autonomously generate high-quality pseudowords. The model was compared to an extended version of simple recurrent networks, augmented with a stochastic process that allows autonomous generation of sequences, and to non-connectionist probabilistic models (n-grams and hidden Markov models). We conclude that sequential RBMs and stochastic simple recurrent networks are promising candidates for modeling cognition in the temporal domain. Copyright © 2015 Cognitive Science Society, Inc.

PREDICT: Privacy and Security Enhancing Dynamic Information Monitoring

DTIC Science & Technology

2015-08-03

consisting of global server-side probabilistic assignment by an untrusted server using cloaked locations, followed by feedback-loop guided local...12], consisting of global server-side probabilistic assignment by an untrusted server using cloaked locations, followed by feedback-loop guided...these methods achieve high sensing coverage with low cost using cloaked locations [3]. In follow-on work, the issue of mobility is addressed. Task

Perceptual Decision-Making as Probabilistic Inference by Neural Sampling.

PubMed

Haefner, Ralf M; Berkes, Pietro; Fiser, József

2016-05-04

We address two main challenges facing systems neuroscience today: understanding the nature and function of cortical feedback between sensory areas and of correlated variability. Starting from the old idea of perception as probabilistic inference, we show how to use knowledge of the psychophysical task to make testable predictions for the influence of feedback signals on early sensory representations. Applying our framework to a two-alternative forced choice task paradigm, we can explain multiple empirical findings that have been hard to account for by the traditional feedforward model of sensory processing, including the task dependence of neural response correlations and the diverging time courses of choice probabilities and psychophysical kernels. Our model makes new predictions and characterizes a component of correlated variability that represents task-related information rather than performance-degrading noise. It demonstrates a normative way to integrate sensory and cognitive components into physiologically testable models of perceptual decision-making. Copyright © 2016 Elsevier Inc. All rights reserved.

Increased anterior cingulate cortex response precedes behavioural adaptation in anorexia nervosa

PubMed Central

Geisler, Daniel; Ritschel, Franziska; King, Joseph A.; Bernardoni, Fabio; Seidel, Maria; Boehm, Ilka; Runge, Franziska; Goschke, Thomas; Roessner, Veit; Smolka, Michael N.; Ehrlich, Stefan

2017-01-01

Patients with anorexia nervosa (AN) are characterised by increased self-control, cognitive rigidity and impairments in set-shifting, but the underlying neural mechanisms are poorly understood. Here we used functional magnetic resonance imaging (fMRI) to elucidate the neural correlates of behavioural adaptation to changes in reward contingencies in young acutely ill AN patients. Thirty-six adolescent/young adult, non-chronic female AN patients and 36 age-matched healthy females completed a well-established probabilistic reversal learning task during fMRI. We analysed hemodynamic responses in empirically-defined regions of interest during positive feedback and negative feedback not followed/followed by behavioural adaptation and conducted functional connectivity analyses. Although overall task performance was comparable between groups, AN showed increased shifting after receiving negative feedback (lose-shift behaviour) and altered dorsal anterior cingulate cortex (dACC) responses as a function of feedback. Specifically, patients had increased dACC responses (which correlated with perfectionism) and task-related coupling with amygdala preceding behavioural adaption. Given the generally preserved task performance in young AN, elevated dACC responses specifically during behavioural adaption is suggestive of increased monitoring for the need to adjust performance strategies. Higher dACC-amygdala coupling and increased adaptation after negative feedback underlines this interpretation and could be related to intolerance of uncertainty which has been suggested for AN. PMID:28198813

Spectral feature extraction of EEG signals and pattern recognition during mental tasks of 2-D cursor movements for BCI using SVM and ANN.

PubMed

Bascil, M Serdar; Tesneli, Ahmet Y; Temurtas, Feyzullah

2016-09-01

Brain computer interface (BCI) is a new communication way between man and machine. It identifies mental task patterns stored in electroencephalogram (EEG). So, it extracts brain electrical activities recorded by EEG and transforms them machine control commands. The main goal of BCI is to make available assistive environmental devices for paralyzed people such as computers and makes their life easier. This study deals with feature extraction and mental task pattern recognition on 2-D cursor control from EEG as offline analysis approach. The hemispherical power density changes are computed and compared on alpha-beta frequency bands with only mental imagination of cursor movements. First of all, power spectral density (PSD) features of EEG signals are extracted and high dimensional data reduced by principle component analysis (PCA) and independent component analysis (ICA) which are statistical algorithms. In the last stage, all features are classified with two types of support vector machine (SVM) which are linear and least squares (LS-SVM) and three different artificial neural network (ANN) structures which are learning vector quantization (LVQ), multilayer neural network (MLNN) and probabilistic neural network (PNN) and mental task patterns are successfully identified via k-fold cross validation technique.

Automated Item Generation with Recurrent Neural Networks.

PubMed

von Davier, Matthias

2018-03-12

Utilizing technology for automated item generation is not a new idea. However, test items used in commercial testing programs or in research are still predominantly written by humans, in most cases by content experts or professional item writers. Human experts are a limited resource and testing agencies incur high costs in the process of continuous renewal of item banks to sustain testing programs. Using algorithms instead holds the promise of providing unlimited resources for this crucial part of assessment development. The approach presented here deviates in several ways from previous attempts to solve this problem. In the past, automatic item generation relied either on generating clones of narrowly defined item types such as those found in language free intelligence tests (e.g., Raven's progressive matrices) or on an extensive analysis of task components and derivation of schemata to produce items with pre-specified variability that are hoped to have predictable levels of difficulty. It is somewhat unlikely that researchers utilizing these previous approaches would look at the proposed approach with favor; however, recent applications of machine learning show success in solving tasks that seemed impossible for machines not too long ago. The proposed approach uses deep learning to implement probabilistic language models, not unlike what Google brain and Amazon Alexa use for language processing and generation.

Striatal dopaminergic modulation of reinforcement learning predicts reward-oriented behavior in daily life.

PubMed

Kasanova, Zuzana; Ceccarini, Jenny; Frank, Michael J; Amelsvoort, Thérèse van; Booij, Jan; Heinzel, Alexander; Mottaghy, Felix; Myin-Germeys, Inez

2017-07-01

Much human behavior is driven by rewards. Preclinical neurophysiological and clinical positron emission tomography (PET) studies have implicated striatal phasic dopamine (DA) release as a primary modulator of reward processing. However, the relationship between experimental reward-induced striatal DA release and responsiveness to naturalistic rewards, and therefore functional relevance of these findings, has been elusive. We therefore combined, for the first time, a DA D 2/3 receptor [ 18 F]fallypride PET during a probabilistic reinforcement learning (RL) task with a six day ecological momentary assessments (EMA) of reward-related behavior in the everyday life of 16 healthy volunteers. We detected significant reward-induced DA release in the bilateral putamen, caudate nucleus and ventral striatum, the extent of which was associated with better behavioral performance on the RL task across all regions. Furthermore, individual variability in the extent of reward-induced DA release in the right caudate nucleus and ventral striatum modulated the tendency to be actively engaged in a behavior if the active engagement was previously deemed enjoyable. This study suggests a link between striatal reward-related DA release and ecologically relevant reward-oriented behavior, suggesting an avenue for the inquiry into the DAergic basis of optimal and impaired motivational drive. Copyright © 2017 Elsevier B.V. All rights reserved.

Expectancy Learning from Probabilistic Input by Infants

PubMed Central

Romberg, Alexa R.; Saffran, Jenny R.

2013-01-01

Across the first few years of life, infants readily extract many kinds of regularities from their environment, and this ability is thought to be central to development in a number of domains. Numerous studies have documented infants’ ability to recognize deterministic sequential patterns. However, little is known about the processes infants use to build and update representations of structure in time, and how infants represent patterns that are not completely predictable. The present study investigated how infants’ expectations fora simple structure develope over time, and how infants update their representations with new information. We measured 12-month-old infants’ anticipatory eye movements to targets that appeared in one of two possible locations. During the initial phase of the experiment, infants either saw targets that appeared consistently in the same location (Deterministic condition) or probabilistically in either location, with one side more frequent than the other (Probabilistic condition). After this initial divergent experience, both groups saw the same sequence of trials for the rest of the experiment. The results show that infants readily learn from both deterministic and probabilistic input, with infants in both conditions reliably predicting the most likely target location by the end of the experiment. Local context had a large influence on behavior: infants adjusted their predictions to reflect changes in the target location on the previous trial. This flexibility was particularly evident in infants with more variable prior experience (the Probabilistic condition). The results provide some of the first data showing how infants learn in real time. PMID:23439947

Dynamic competitive probabilistic principal components analysis.

PubMed

López-Rubio, Ezequiel; Ortiz-DE-Lazcano-Lobato, Juan Miguel

2009-04-01

We present a new neural model which extends the classical competitive learning (CL) by performing a Probabilistic Principal Components Analysis (PPCA) at each neuron. The model also has the ability to learn the number of basis vectors required to represent the principal directions of each cluster, so it overcomes a drawback of most local PCA models, where the dimensionality of a cluster must be fixed a priori. Experimental results are presented to show the performance of the network with multispectral image data.

Learning Probabilistic Inference through Spike-Timing-Dependent Plasticity.

PubMed

Pecevski, Dejan; Maass, Wolfgang

2016-01-01

Numerous experimental data show that the brain is able to extract information from complex, uncertain, and often ambiguous experiences. Furthermore, it can use such learnt information for decision making through probabilistic inference. Several models have been proposed that aim at explaining how probabilistic inference could be performed by networks of neurons in the brain. We propose here a model that can also explain how such neural network could acquire the necessary information for that from examples. We show that spike-timing-dependent plasticity in combination with intrinsic plasticity generates in ensembles of pyramidal cells with lateral inhibition a fundamental building block for that: probabilistic associations between neurons that represent through their firing current values of random variables. Furthermore, by combining such adaptive network motifs in a recursive manner the resulting network is enabled to extract statistical information from complex input streams, and to build an internal model for the distribution p (*) that generates the examples it receives. This holds even if p (*) contains higher-order moments. The analysis of this learning process is supported by a rigorous theoretical foundation. Furthermore, we show that the network can use the learnt internal model immediately for prediction, decision making, and other types of probabilistic inference.

Learning Probabilistic Inference through Spike-Timing-Dependent Plasticity123

PubMed Central

Pecevski, Dejan

2016-01-01

Abstract Numerous experimental data show that the brain is able to extract information from complex, uncertain, and often ambiguous experiences. Furthermore, it can use such learnt information for decision making through probabilistic inference. Several models have been proposed that aim at explaining how probabilistic inference could be performed by networks of neurons in the brain. We propose here a model that can also explain how such neural network could acquire the necessary information for that from examples. We show that spike-timing-dependent plasticity in combination with intrinsic plasticity generates in ensembles of pyramidal cells with lateral inhibition a fundamental building block for that: probabilistic associations between neurons that represent through their firing current values of random variables. Furthermore, by combining such adaptive network motifs in a recursive manner the resulting network is enabled to extract statistical information from complex input streams, and to build an internal model for the distribution p* that generates the examples it receives. This holds even if p* contains higher-order moments. The analysis of this learning process is supported by a rigorous theoretical foundation. Furthermore, we show that the network can use the learnt internal model immediately for prediction, decision making, and other types of probabilistic inference. PMID:27419214

Continuous theta-burst stimulation (cTBS) over the lateral prefrontal cortex alters reinforcement learning bias.

PubMed

Ott, Derek V M; Ullsperger, Markus; Jocham, Gerhard; Neumann, Jane; Klein, Tilmann A

2011-07-15

The prefrontal cortex is known to play a key role in higher-order cognitive functions. Recently, we showed that this brain region is active in reinforcement learning, during which subjects constantly have to integrate trial outcomes in order to optimize performance. To further elucidate the role of the dorsolateral prefrontal cortex (DLPFC) in reinforcement learning, we applied continuous theta-burst stimulation (cTBS) either to the left or right DLPFC, or to the vertex as a control region, respectively, prior to the performance of a probabilistic learning task in an fMRI environment. While there was no influence of cTBS on learning performance per se, we observed a stimulation-dependent modulation of reward vs. punishment sensitivity: Left-hemispherical DLPFC stimulation led to a more reward-guided performance, while right-hemispherical cTBS induced a more avoidance-guided behavior. FMRI results showed enhanced prediction error coding in the ventral striatum in subjects stimulated over the left as compared to the right DLPFC. Both behavioral and imaging results are in line with recent findings that left, but not right-hemispherical stimulation can trigger a release of dopamine in the ventral striatum, which has been suggested to increase the relative impact of rewards rather than punishment on behavior. Copyright © 2011 Elsevier Inc. All rights reserved.

Learning to make things happen: Infants' observational learning of social and physical causal events.

PubMed

Waismeyer, Anna; Meltzoff, Andrew N

2017-10-01

Infants learn about cause and effect through hands-on experience; however, they also can learn about causality simply from observation. Such observational causal learning is a central mechanism by which infants learn from and about other people. Across three experiments, we tested infants' observational causal learning of both social and physical causal events. Experiment 1 assessed infants' learning of a physical event in the absence of visible spatial contact between the causes and effects. Experiment 2 developed a novel paradigm to assess whether infants could learn about a social causal event from third-party observation of a social interaction between two people. Experiment 3 compared learning of physical and social events when the outcomes occurred probabilistically (happening some, but not all, of the time). Infants demonstrated significant learning in all three experiments, although learning about probabilistic cause-effect relations was most difficult. These findings about infant observational causal learning have implications for children's rapid nonverbal learning about people, things, and their causal relations. Copyright © 2017 Elsevier Inc. All rights reserved.

Language acquisition and use: learning and applying probabilistic constraints.

PubMed

Seidenberg, M S

1997-03-14

What kinds of knowledge underlie the use of language and how is this knowledge acquired? Linguists equate knowing a language with knowing a grammar. Classic "poverty of the stimulus" arguments suggest that grammar identification is an intractable inductive problem and that acquisition is possible only because children possess innate knowledge of grammatical structure. An alternative view is emerging from studies of statistical and probabilistic aspects of language, connectionist models, and the learning capacities of infants. This approach emphasizes continuity between how language is acquired and how it is used. It retains the idea that innate capacities constrain language learning, but calls into question whether they include knowledge of grammatical structure.

The role of dopamine in positive and negative prediction error utilization during incidental learning - Insights from Positron Emission Tomography, Parkinson's disease and Huntington's disease.

PubMed

Mathar, David; Wilkinson, Leonora; Holl, Anna K; Neumann, Jane; Deserno, Lorenz; Villringer, Arno; Jahanshahi, Marjan; Horstmann, Annette

2017-05-01

Incidental learning of appropriate stimulus-response associations is crucial for optimal functioning within our complex environment. Positive and negative prediction errors (PEs) serve as neural teaching signals within distinct ('direct'/'indirect') dopaminergic pathways to update associations and optimize subsequent behavior. Using a computational reinforcement learning model, we assessed learning from positive and negative PEs on a probabilistic task (Weather Prediction Task - WPT) in three populations that allow different inferences on the role of dopamine (DA) signals: (1) Healthy volunteers that repeatedly underwent [ 11 C]raclopride Positron Emission Tomography (PET), allowing for assessment of striatal DA release during learning, (2) Parkinson's disease (PD) patients tested both on and off L-DOPA medication, (3) early Huntington's disease (HD) patients, a disease that is associated with hyper-activation of the 'direct' pathway. Our results show that learning from positive and negative feedback on the WPT is intimately linked to different aspects of dopaminergic transmission. In healthy individuals, the difference in [ 11 C]raclopride binding potential (BP) as a measure for striatal DA release was linearly associated with the positive learning rate. Further, asymmetry between baseline DA tone in the left and right ventral striatum was negatively associated with learning from positive PEs. Female patients with early HD exhibited exaggerated learning rates from positive feedback. In contrast, dopaminergic tone predicted learning from negative feedback, as indicated by an inverted u-shaped association observed with baseline [ 11 C]raclopride BP in healthy controls and the difference between PD patients' learning rate on and off dopaminergic medication. Thus, the ability to learn from positive and negative feedback is a sensitive marker for the integrity of dopaminergic signal transmission in the 'direct' and 'indirect' dopaminergic pathways. The present data are interesting beyond clinical context in that imbalances of dopaminergic signaling have not only been observed for neurological and psychiatric conditions but also been proposed for obesity and adolescence. Copyright © 2016 Elsevier Ltd. All rights reserved.

A Salient and Task-Irrelevant Collinear Structure Hurts Visual Search

PubMed Central

Tseng, Chia-huei; Jingling, Li

2015-01-01

Salient distractors draw our attention spontaneously, even when we intentionally want to ignore them. When this occurs, the real targets close to or overlapping with the distractors benefit from attention capture and thus are detected and discriminated more quickly. However, a puzzling opposite effect was observed in a search display with a column of vertical collinear bars presented as a task-irrelevant distractor [6]. In this case, it was harder to discriminate the targets overlapping with the salient distractor. Here we examined whether this effect originated from factors known to modulate attentional capture: (a) low probability—the probability occurrence of target location at the collinear column was much less (14%) than the rest of the display (86%), and observers might strategically direct their attention away from the collinear distractor; (b) attentional control setting—the distractor and target task interfered with each other because they shared the same continuity set in attentional task; and/or (c) lack of time to establish the optional strategy. We tested these hypotheses by (a) increasing to 60% the trials in which targets overlapped with the same collinear distractor columns, (b) replacing the target task to be connectivity-irrelevant (i.e., luminance discrimination), and (c) having our observers practice the same search task for 10 days. Our results speak against all these hypotheses and lead us to conclude that a collinear distractor impairs search at a level that is unaffected by probabilistic information, attentional setting, and learning. PMID:25909986

Model-based machine learning.

PubMed

Bishop, Christopher M

2013-02-13

Several decades of research in the field of machine learning have resulted in a multitude of different algorithms for solving a broad range of problems. To tackle a new application, a researcher typically tries to map their problem onto one of these existing methods, often influenced by their familiarity with specific algorithms and by the availability of corresponding software implementations. In this study, we describe an alternative methodology for applying machine learning, in which a bespoke solution is formulated for each new application. The solution is expressed through a compact modelling language, and the corresponding custom machine learning code is then generated automatically. This model-based approach offers several major advantages, including the opportunity to create highly tailored models for specific scenarios, as well as rapid prototyping and comparison of a range of alternative models. Furthermore, newcomers to the field of machine learning do not have to learn about the huge range of traditional methods, but instead can focus their attention on understanding a single modelling environment. In this study, we show how probabilistic graphical models, coupled with efficient inference algorithms, provide a very flexible foundation for model-based machine learning, and we outline a large-scale commercial application of this framework involving tens of millions of users. We also describe the concept of probabilistic programming as a powerful software environment for model-based machine learning, and we discuss a specific probabilistic programming language called Infer.NET, which has been widely used in practical applications.

Model-based machine learning

PubMed Central

Bishop, Christopher M.

2013-01-01

Several decades of research in the field of machine learning have resulted in a multitude of different algorithms for solving a broad range of problems. To tackle a new application, a researcher typically tries to map their problem onto one of these existing methods, often influenced by their familiarity with specific algorithms and by the availability of corresponding software implementations. In this study, we describe an alternative methodology for applying machine learning, in which a bespoke solution is formulated for each new application. The solution is expressed through a compact modelling language, and the corresponding custom machine learning code is then generated automatically. This model-based approach offers several major advantages, including the opportunity to create highly tailored models for specific scenarios, as well as rapid prototyping and comparison of a range of alternative models. Furthermore, newcomers to the field of machine learning do not have to learn about the huge range of traditional methods, but instead can focus their attention on understanding a single modelling environment. In this study, we show how probabilistic graphical models, coupled with efficient inference algorithms, provide a very flexible foundation for model-based machine learning, and we outline a large-scale commercial application of this framework involving tens of millions of users. We also describe the concept of probabilistic programming as a powerful software environment for model-based machine learning, and we discuss a specific probabilistic programming language called Infer.NET, which has been widely used in practical applications. PMID:23277612

Deductive and inductive reasoning in obsessive-compulsive disorder.

PubMed

Pélissier, Marie-Claude; O'Connor, Kieron P

2002-03-01

This study tested the hypothesis that people with obsessive-compulsive disorder (OCD) show an inductive reasoning style distinct from people with generalized anxiety disorder (GAD) and from participants in a non-anxious (NA) control group. The experimental procedure consisted of administering a range of six deductive and inductive tasks and a probabilistic task in order to compare reasoning processes between groups. Recruitment was in the Montreal area within a French-speaking population. The participants were 12 people with OCD, 12 NA controls and 10 people with GAD. Participants completed a series of written and oral reasoning tasks including the Wason Selection Task, a Bayesian probability task and other inductive tasks, designed by the authors. There were no differences between groups in deductive reasoning. On an inductive "bridging task", the participants with OCD always took longer than the NA control and GAD groups to infer a link between two statements and to elaborate on this possible link. The OCD group alone showed a significant decrease in their degree of conviction about an arbitrary statement after inductively generating reasons to support this statement. Differences in probabilistic reasoning replicated those of previous authors. The results pinpoint the importance of examining inference processes in people with OCD in order to further refine the clinical applications of behavioural-cognitive therapy for this disorder.

Students’ difficulties in probabilistic problem-solving

NASA Astrophysics Data System (ADS)

Arum, D. P.; Kusmayadi, T. A.; Pramudya, I.

2018-03-01

There are many errors can be identified when students solving mathematics problems, particularly in solving the probabilistic problem. This present study aims to investigate students’ difficulties in solving the probabilistic problem. It focuses on analyzing and describing students errors during solving the problem. This research used the qualitative method with case study strategy. The subjects in this research involve ten students of 9th grade that were selected by purposive sampling. Data in this research involve students’ probabilistic problem-solving result and recorded interview regarding students’ difficulties in solving the problem. Those data were analyzed descriptively using Miles and Huberman steps. The results show that students have difficulties in solving the probabilistic problem and can be divided into three categories. First difficulties relate to students’ difficulties in understanding the probabilistic problem. Second, students’ difficulties in choosing and using appropriate strategies for solving the problem. Third, students’ difficulties with the computational process in solving the problem. Based on the result seems that students still have difficulties in solving the probabilistic problem. It means that students have not able to use their knowledge and ability for responding probabilistic problem yet. Therefore, it is important for mathematics teachers to plan probabilistic learning which could optimize students probabilistic thinking ability.

Electrophysiological Correlates of Observational Learning in Children

ERIC Educational Resources Information Center

Rodriguez Buritica, Julia M.; Eppinger, Ben; Schuck, Nicolas W.; Heekeren, Hauke R.; Li, Shu-Chen

2016-01-01

Observational learning is an important mechanism for cognitive and social development. However, the neurophysiological mechanisms underlying observational learning in children are not well understood. In this study, we used a probabilistic reward-based observational learning paradigm to compare behavioral and electrophysiological markers of…

Development of Probabilistic Rigid Pavement Design Methodologies for Military Airfields.

DTIC Science & Technology

1983-12-01

4A161102AT22, Task AO, Work Unit 009, "Methodology for Considering Material Variability in Pavement Design." OCE Project Monitor was Mr. S. S. Gillespie. The...PREFACE. .. ............................. VOLUME 1: STATE OF THE ART VARIABILITY OF AIRFIELD PAVEMENT MATERIALS VOLUME 11: MATHEMATICAL FORMULATION OF...VOLUME IV: PROBABILISTIC ANALYSIS OF RIGID AIRFIELD DESIGN BY ELASTIC LAYERED THEORY VOLUME I STATE OF THE ART VARIABILITY OF AIRFIELD PAVEMENT MATERIALS

Towards an Artificial Space Object Taxonomy

DTIC Science & Technology

2013-09-01

demonstrate how to implement this taxonomy in Figaro, an open source probabilistic programming language. 2. INTRODUCTION Currently, US Space Command...Taxonomy 5a. CONTRACT NUMBER 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7...demonstrate how to implement this taxonomy in Figaro, an open source probabilistic programming language. 15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF

Effects of delay and probability combinations on discounting in humans.

PubMed

Cox, David J; Dallery, Jesse

2016-10-01

To determine discount rates, researchers typically adjust the amount of an immediate or certain option relative to a delayed or uncertain option. Because this adjusting amount method can be relatively time consuming, researchers have developed more efficient procedures. One such procedure is a 5-trial adjusting delay procedure, which measures the delay at which an amount of money loses half of its value (e.g., $1000 is valued at $500 with a 10-year delay to its receipt). Experiment 1 (n=212) used 5-trial adjusting delay or probability tasks to measure delay discounting of losses, probabilistic gains, and probabilistic losses. Experiment 2 (n=98) assessed combined probabilistic and delayed alternatives. In both experiments, we compared results from 5-trial adjusting delay or probability tasks to traditional adjusting amount procedures. Results suggest both procedures produced similar rates of probability and delay discounting in six out of seven comparisons. A magnitude effect consistent with previous research was observed for probabilistic gains and losses, but not for delayed losses. Results also suggest that delay and probability interact to determine the value of money. Five-trial methods may allow researchers to assess discounting more efficiently as well as study more complex choice scenarios. Copyright © 2016 Elsevier B.V. All rights reserved.

Probabilistic learning of nonlinear dynamical systems using sequential Monte Carlo

NASA Astrophysics Data System (ADS)

Schön, Thomas B.; Svensson, Andreas; Murray, Lawrence; Lindsten, Fredrik

2018-05-01

Probabilistic modeling provides the capability to represent and manipulate uncertainty in data, models, predictions and decisions. We are concerned with the problem of learning probabilistic models of dynamical systems from measured data. Specifically, we consider learning of probabilistic nonlinear state-space models. There is no closed-form solution available for this problem, implying that we are forced to use approximations. In this tutorial we will provide a self-contained introduction to one of the state-of-the-art methods-the particle Metropolis-Hastings algorithm-which has proven to offer a practical approximation. This is a Monte Carlo based method, where the particle filter is used to guide a Markov chain Monte Carlo method through the parameter space. One of the key merits of the particle Metropolis-Hastings algorithm is that it is guaranteed to converge to the "true solution" under mild assumptions, despite being based on a particle filter with only a finite number of particles. We will also provide a motivating numerical example illustrating the method using a modeling language tailored for sequential Monte Carlo methods. The intention of modeling languages of this kind is to open up the power of sophisticated Monte Carlo methods-including particle Metropolis-Hastings-to a large group of users without requiring them to know all the underlying mathematical details.

Probabilistic Open Set Recognition

NASA Astrophysics Data System (ADS)

Jain, Lalit Prithviraj

Real-world tasks in computer vision, pattern recognition and machine learning often touch upon the open set recognition problem: multi-class recognition with incomplete knowledge of the world and many unknown inputs. An obvious way to approach such problems is to develop a recognition system that thresholds probabilities to reject unknown classes. Traditional rejection techniques are not about the unknown; they are about the uncertain boundary and rejection around that boundary. Thus traditional techniques only represent the "known unknowns". However, a proper open set recognition algorithm is needed to reduce the risk from the "unknown unknowns". This dissertation examines this concept and finds existing probabilistic multi-class recognition approaches are ineffective for true open set recognition. We hypothesize the cause is due to weak adhoc assumptions combined with closed-world assumptions made by existing calibration techniques. Intuitively, if we could accurately model just the positive data for any known class without overfitting, we could reject the large set of unknown classes even under this assumption of incomplete class knowledge. For this, we formulate the problem as one of modeling positive training data by invoking statistical extreme value theory (EVT) near the decision boundary of positive data with respect to negative data. We provide a new algorithm called the PI-SVM for estimating the unnormalized posterior probability of class inclusion. This dissertation also introduces a new open set recognition model called Compact Abating Probability (CAP), where the probability of class membership decreases in value (abates) as points move from known data toward open space. We show that CAP models improve open set recognition for multiple algorithms. Leveraging the CAP formulation, we go on to describe the novel Weibull-calibrated SVM (W-SVM) algorithm, which combines the useful properties of statistical EVT for score calibration with one-class and binary support vector machines. Building from the success of statistical EVT based recognition methods such as PI-SVM and W-SVM on the open set problem, we present a new general supervised learning algorithm for multi-class classification and multi-class open set recognition called the Extreme Value Local Basis (EVLB). The design of this algorithm is motivated by the observation that extrema from known negative class distributions are the closest negative points to any positive sample during training, and thus should be used to define the parameters of a probabilistic decision model. In the EVLB, the kernel distribution for each positive training sample is estimated via an EVT distribution fit over the distances to the separating hyperplane between positive training sample and closest negative samples, with a subset of the overall positive training data retained to form a probabilistic decision boundary. Using this subset as a frame of reference, the probability of a sample at test time decreases as it moves away from the positive class. Possessing this property, the EVLB is well-suited to open set recognition problems where samples from unknown or novel classes are encountered at test. Our experimental evaluation shows that the EVLB provides a substantial improvement in scalability compared to standard radial basis function kernel machines, as well as P I-SVM and W-SVM, with improved accuracy in many cases. We evaluate our algorithm on open set variations of the standard visual learning benchmarks, as well as with an open subset of classes from Caltech 256 and ImageNet. Our experiments show that PI-SVM, WSVM and EVLB provide significant advances over the previous state-of-the-art solutions for the same tasks.

Quantum Tasks with Non-maximally Quantum Channels via Positive Operator-Valued Measurement

NASA Astrophysics Data System (ADS)

Peng, Jia-Yin; Luo, Ming-Xing; Mo, Zhi-Wen

2013-01-01

By using a proper positive operator-valued measure (POVM), we present two new schemes for probabilistic transmission with non-maximally four-particle cluster states. In the first scheme, we demonstrate that two non-maximally four-particle cluster states can be used to realize probabilistically sharing an unknown three-particle GHZ-type state within either distant agent's place. In the second protocol, we demonstrate that a non-maximally four-particle cluster state can be used to teleport an arbitrary unknown multi-particle state in a probabilistic manner with appropriate unitary operations and POVM. Moreover the total success probability of these two schemes are also worked out.

Ventromedial Frontal Cortex Is Critical for Guiding Attention to Reward-Predictive Visual Features in Humans.

PubMed

Vaidya, Avinash R; Fellows, Lesley K

2015-09-16

Adaptively interacting with our environment requires extracting information that will allow us to successfully predict reward. This can be a challenge, particularly when there are many candidate cues, and when rewards are probabilistic. Recent work has demonstrated that visual attention is allocated to stimulus features that have been associated with reward on previous trials. The ventromedial frontal lobe (VMF) has been implicated in learning in dynamic environments of this kind, but the mechanism by which this region influences this process is not clear. Here, we hypothesized that the VMF plays a critical role in guiding attention to reward-predictive stimulus features based on feedback. We tested the effects of VMF damage in human subjects on a visual search task in which subjects were primed to attend to task-irrelevant colors associated with different levels of reward, incidental to the search task. Consistent with previous work, we found that distractors had a greater influence on reaction time when they appeared in colors associated with high reward in the previous trial compared with colors associated with low reward in healthy control subjects and patients with prefrontal damage sparing the VMF. However, this reward modulation of attentional priming was absent in patients with VMF damage. Thus, an intact VMF is necessary for directing attention based on experience with cue-reward associations. We suggest that this region plays a role in selecting reward-predictive cues to facilitate future learning. There has been a swell of interest recently in the ventromedial frontal cortex (VMF), a brain region critical to associative learning. However, the underlying mechanism by which this region guides learning is not well understood. Here, we tested the effects of damage to this region in humans on a task in which rewards were linked incidentally to visual features, resulting in trial-by-trial attentional priming. Controls and subjects with prefrontal damage sparing the VMF showed normal reward priming, but VMF-damaged patients did not. This work sheds light on a potential mechanism through which this region influences behavior. We suggest that the VMF is necessary for directing attention to reward-predictive visual features based on feedback, facilitating future learning and decision-making. Copyright © 2015 the authors 0270-6474/15/3512813-11$15.00/0.

Saul: Towards Declarative Learning Based Programming

PubMed Central

Kordjamshidi, Parisa; Roth, Dan; Wu, Hao

2015-01-01

We present Saul, a new probabilistic programming language designed to address some of the shortcomings of programming languages that aim at advancing and simplifying the development of AI systems. Such languages need to interact with messy, naturally occurring data, to allow a programmer to specify what needs to be done at an appropriate level of abstraction rather than at the data level, to be developed on a solid theory that supports moving to and reasoning at this level of abstraction and, finally, to support flexible integration of these learning and inference models within an application program. Saul is an object-functional programming language written in Scala that facilitates these by (1) allowing a programmer to learn, name and manipulate named abstractions over relational data; (2) supporting seamless incorporation of trainable (probabilistic or discriminative) components into the program, and (3) providing a level of inference over trainable models to support composition and make decisions that respect domain and application constraints. Saul is developed over a declaratively defined relational data model, can use piecewise learned factor graphs with declaratively specified learning and inference objectives, and it supports inference over probabilistic models augmented with declarative knowledge-based constraints. We describe the key constructs of Saul and exemplify its use in developing applications that require relational feature engineering and structured output prediction. PMID:26635465

Saul: Towards Declarative Learning Based Programming.

PubMed

Kordjamshidi, Parisa; Roth, Dan; Wu, Hao

2015-07-01

We present Saul , a new probabilistic programming language designed to address some of the shortcomings of programming languages that aim at advancing and simplifying the development of AI systems. Such languages need to interact with messy, naturally occurring data, to allow a programmer to specify what needs to be done at an appropriate level of abstraction rather than at the data level, to be developed on a solid theory that supports moving to and reasoning at this level of abstraction and, finally, to support flexible integration of these learning and inference models within an application program. Saul is an object-functional programming language written in Scala that facilitates these by (1) allowing a programmer to learn, name and manipulate named abstractions over relational data; (2) supporting seamless incorporation of trainable (probabilistic or discriminative) components into the program, and (3) providing a level of inference over trainable models to support composition and make decisions that respect domain and application constraints. Saul is developed over a declaratively defined relational data model, can use piecewise learned factor graphs with declaratively specified learning and inference objectives, and it supports inference over probabilistic models augmented with declarative knowledge-based constraints. We describe the key constructs of Saul and exemplify its use in developing applications that require relational feature engineering and structured output prediction.

A Probabilistic Model of Social Working Memory for Information Retrieval in Social Interactions.

PubMed

Li, Liyuan; Xu, Qianli; Gan, Tian; Tan, Cheston; Lim, Joo-Hwee

2018-05-01

Social working memory (SWM) plays an important role in navigating social interactions. Inspired by studies in psychology, neuroscience, cognitive science, and machine learning, we propose a probabilistic model of SWM to mimic human social intelligence for personal information retrieval (IR) in social interactions. First, we establish a semantic hierarchy as social long-term memory to encode personal information. Next, we propose a semantic Bayesian network as the SWM, which integrates the cognitive functions of accessibility and self-regulation. One subgraphical model implements the accessibility function to learn the social consensus about IR-based on social information concept, clustering, social context, and similarity between persons. Beyond accessibility, one more layer is added to simulate the function of self-regulation to perform the personal adaptation to the consensus based on human personality. Two learning algorithms are proposed to train the probabilistic SWM model on a raw dataset of high uncertainty and incompleteness. One is an efficient learning algorithm of Newton's method, and the other is a genetic algorithm. Systematic evaluations show that the proposed SWM model is able to learn human social intelligence effectively and outperforms the baseline Bayesian cognitive model. Toward real-world applications, we implement our model on Google Glass as a wearable assistant for social interaction.

Assessing a learning process with functional ANOVA estimators of EEG power spectral densities.

PubMed

Gutiérrez, David; Ramírez-Moreno, Mauricio A

2016-04-01

We propose to assess the process of learning a task using electroencephalographic (EEG) measurements. In particular, we quantify changes in brain activity associated to the progression of the learning experience through the functional analysis-of-variances (FANOVA) estimators of the EEG power spectral density (PSD). Such functional estimators provide a sense of the effect of training in the EEG dynamics. For that purpose, we implemented an experiment to monitor the process of learning to type using the Colemak keyboard layout during a twelve-lessons training. Hence, our aim is to identify statistically significant changes in PSD of various EEG rhythms at different stages and difficulty levels of the learning process. Those changes are taken into account only when a probabilistic measure of the cognitive state ensures the high engagement of the volunteer to the training. Based on this, a series of statistical tests are performed in order to determine the personalized frequencies and sensors at which changes in PSD occur, then the FANOVA estimates are computed and analyzed. Our experimental results showed a significant decrease in the power of [Formula: see text] and [Formula: see text] rhythms for ten volunteers during the learning process, and such decrease happens regardless of the difficulty of the lesson. These results are in agreement with previous reports of changes in PSD being associated to feature binding and memory encoding.

Independent effects of age and levodopa on reversal learning in healthy volunteers.

PubMed

Vo, Andrew; Seergobin, Ken N; MacDonald, Penny A

2018-05-18

The dopamine overdose hypothesis has provided an important theoretical framework for understanding cognition in Parkinson's disease. It posits that effects of dopaminergic therapy on cognition in Parkinson's disease depend on baseline dopamine levels in brain regions that support different functions. Although functions performed by more severely dopamine-depleted brain regions improve with medication, those associated with less dopamine deficient areas are actually worsened. It is presumed that medication-related worsening of cognition owes to dopamine overdose. We investigated whether age-related changes in baseline dopamine levels would modulate effects of dopaminergic therapy on reward learning in healthy volunteers. In a double-blind, crossover design, healthy younger and older adults completed a probabilistic reversal learning task after treatment with 100/25 mg of levodopa/carbidopa versus placebo. Older adults learned more poorly than younger adults at baseline, being more likely to shift responses after misleading punishment. Levodopa worsened stimulus-reward learning relative to placebo to the same extent in both groups, irrespective of differences in baseline performance and expected dopamine levels. When order effects were eliminated, levodopa induced response shifts after reward more often than placebo. Our results reveal independent deleterious effects of age group and exogenous dopamine on reward learning, suggesting a more complex scenario than predicted by the dopamine overdose hypothesis. Copyright © 2018 Elsevier Inc. All rights reserved.

Cognitive and behavioural deficits associated with the orbitomedial prefrontal cortex in amyotrophic lateral sclerosis.

PubMed

Meier, Sandra L; Charleston, Alison J; Tippett, Lynette J

2010-11-01

Amyotrophic lateral sclerosis, a progressive disease affecting motor neurons, may variably affect cognition and behaviour. We tested the hypothesis that functions associated with orbitomedial prefrontal cortex are affected by evaluating the behavioural and cognitive performance of 18 participants with amyotrophic lateral sclerosis without dementia and 18 healthy, matched controls. We measured Theory of Mind (Faux Pas Task), emotional prosody recognition (Aprosodia Battery), reversal of behaviour in response to changes in reward (Probabilistic Reversal Learning Task), decision making without risk (Holiday Apartment Task) and aberrant behaviour (Neuropsychiatric Inventory). We also assessed dorsolateral prefrontal function, using verbal and written fluency and planning (One-touch Stockings of Cambridge), to determine whether impairments in tasks sensitive to these two prefrontal regions co-occur. The patient group was significantly impaired at identifying social faux pas, recognizing emotions and decision-making, indicating mild, but consistent impairment on most measures sensitive to orbitomedial prefrontal cortex. Significant levels of aberrant behaviour were present in 50% of patients. Patients were also impaired on verbal fluency and planning. Individual subject analyses involved computing classical dissociations between tasks sensitive to different prefrontal regions. These revealed heterogeneous patterns of impaired and spared cognitive abilities: 33% of participants had classical dissociations involving orbitomedial prefrontal tasks, 17% had classical dissociations involving dorsolateral prefrontal tasks, 22% had classical dissociations between tasks of both regions, and 28% had no classical dissociations. These data indicate subtle changes in behaviour, emotional processing, decision-making and altered social awareness, associated with orbitomedial prefrontal cortex, may be present in a significant proportion of individuals with amyotrophic lateral sclerosis without dementia, some with no signs of dysfunction in tasks sensitive to other regions of prefrontal cortex. This demonstration of variability in cognitive integrity supports previous research indicating amyotrophic lateral sclerosis is a heterogeneous disease.

Role of the site of synaptic competition and the balance of learning forces for Hebbian encoding of probabilistic Markov sequences

PubMed Central

Bouchard, Kristofer E.; Ganguli, Surya; Brainard, Michael S.

2015-01-01

The majority of distinct sensory and motor events occur as temporally ordered sequences with rich probabilistic structure. Sequences can be characterized by the probability of transitioning from the current state to upcoming states (forward probability), as well as the probability of having transitioned to the current state from previous states (backward probability). Despite the prevalence of probabilistic sequencing of both sensory and motor events, the Hebbian mechanisms that mold synapses to reflect the statistics of experienced probabilistic sequences are not well understood. Here, we show through analytic calculations and numerical simulations that Hebbian plasticity (correlation, covariance, and STDP) with pre-synaptic competition can develop synaptic weights equal to the conditional forward transition probabilities present in the input sequence. In contrast, post-synaptic competition can develop synaptic weights proportional to the conditional backward probabilities of the same input sequence. We demonstrate that to stably reflect the conditional probability of a neuron's inputs and outputs, local Hebbian plasticity requires balance between competitive learning forces that promote synaptic differentiation and homogenizing learning forces that promote synaptic stabilization. The balance between these forces dictates a prior over the distribution of learned synaptic weights, strongly influencing both the rate at which structure emerges and the entropy of the final distribution of synaptic weights. Together, these results demonstrate a simple correspondence between the biophysical organization of neurons, the site of synaptic competition, and the temporal flow of information encoded in synaptic weights by Hebbian plasticity while highlighting the utility of balancing learning forces to accurately encode probability distributions, and prior expectations over such probability distributions. PMID:26257637

Simple proof of the impossibility of bit commitment in generalized probabilistic theories using cone programming

NASA Astrophysics Data System (ADS)

Sikora, Jamie; Selby, John

2018-04-01

Bit commitment is a fundamental cryptographic task, in which Alice commits a bit to Bob such that she cannot later change the value of the bit, while, simultaneously, the bit is hidden from Bob. It is known that ideal bit commitment is impossible within quantum theory. In this work, we show that it is also impossible in generalized probabilistic theories (under a small set of assumptions) by presenting a quantitative trade-off between Alice's and Bob's cheating probabilities. Our proof relies crucially on a formulation of cheating strategies as cone programs, a natural generalization of semidefinite programs. In fact, using the generality of this technique, we prove that this result holds for the more general task of integer commitment.

Probability misjudgment, cognitive ability, and belief in the paranormal.

PubMed

Musch, Jochen; Ehrenberg, Katja

2002-05-01

According to the probability misjudgment account of paranormal belief (Blackmore & Troscianko, 1985), believers in the paranormal tend to wrongly attribute remarkable coincidences to paranormal causes rather than chance. Previous studies have shown that belief in the paranormal is indeed positively related to error rates in probabilistic reasoning. General cognitive ability could account for a relationship between these two variables without assuming a causal role of probabilistic reasoning in the forming of paranormal beliefs, however. To test this alternative explanation, a belief in the paranormal scale (BPS) and a battery of probabilistic reasoning tasks were administered to 123 university students. Confirming previous findings, a significant correlation between BPS scores and error rates in probabilistic reasoning was observed. This relationship disappeared, however, when cognitive ability as measured by final examination grades was controlled for. Lower cognitive ability correlated substantially with belief in the paranormal. This finding suggests that differences in general cognitive performance rather than specific probabilistic reasoning skills provide the basis for paranormal beliefs.

Statistical modelling of networked human-automation performance using working memory capacity.

PubMed

Ahmed, Nisar; de Visser, Ewart; Shaw, Tyler; Mohamed-Ameen, Amira; Campbell, Mark; Parasuraman, Raja

2014-01-01

This study examines the challenging problem of modelling the interaction between individual attentional limitations and decision-making performance in networked human-automation system tasks. Analysis of real experimental data from a task involving networked supervision of multiple unmanned aerial vehicles by human participants shows that both task load and network message quality affect performance, but that these effects are modulated by individual differences in working memory (WM) capacity. These insights were used to assess three statistical approaches for modelling and making predictions with real experimental networked supervisory performance data: classical linear regression, non-parametric Gaussian processes and probabilistic Bayesian networks. It is shown that each of these approaches can help designers of networked human-automated systems cope with various uncertainties in order to accommodate future users by linking expected operating conditions and performance from real experimental data to observable cognitive traits like WM capacity. Practitioner Summary: Working memory (WM) capacity helps account for inter-individual variability in operator performance in networked unmanned aerial vehicle supervisory tasks. This is useful for reliable performance prediction near experimental conditions via linear models; robust statistical prediction beyond experimental conditions via Gaussian process models and probabilistic inference about unknown task conditions/WM capacities via Bayesian network models.

Dropout Prediction in E-Learning Courses through the Combination of Machine Learning Techniques

ERIC Educational Resources Information Center

Lykourentzou, Ioanna; Giannoukos, Ioannis; Nikolopoulos, Vassilis; Mpardis, George; Loumos, Vassili

2009-01-01

In this paper, a dropout prediction method for e-learning courses, based on three popular machine learning techniques and detailed student data, is proposed. The machine learning techniques used are feed-forward neural networks, support vector machines and probabilistic ensemble simplified fuzzy ARTMAP. Since a single technique may fail to…

Bootstrapping language acquisition.

PubMed

Abend, Omri; Kwiatkowski, Tom; Smith, Nathaniel J; Goldwater, Sharon; Steedman, Mark

2017-07-01

The semantic bootstrapping hypothesis proposes that children acquire their native language through exposure to sentences of the language paired with structured representations of their meaning, whose component substructures can be associated with words and syntactic structures used to express these concepts. The child's task is then to learn a language-specific grammar and lexicon based on (probably contextually ambiguous, possibly somewhat noisy) pairs of sentences and their meaning representations (logical forms). Starting from these assumptions, we develop a Bayesian probabilistic account of semantically bootstrapped first-language acquisition in the child, based on techniques from computational parsing and interpretation of unrestricted text. Our learner jointly models (a) word learning: the mapping between components of the given sentential meaning and lexical words (or phrases) of the language, and (b) syntax learning: the projection of lexical elements onto sentences by universal construction-free syntactic rules. Using an incremental learning algorithm, we apply the model to a dataset of real syntactically complex child-directed utterances and (pseudo) logical forms, the latter including contextually plausible but irrelevant distractors. Taking the Eve section of the CHILDES corpus as input, the model simulates several well-documented phenomena from the developmental literature. In particular, the model exhibits syntactic bootstrapping effects (in which previously learned constructions facilitate the learning of novel words), sudden jumps in learning without explicit parameter setting, acceleration of word-learning (the "vocabulary spurt"), an initial bias favoring the learning of nouns over verbs, and one-shot learning of words and their meanings. The learner thus demonstrates how statistical learning over structured representations can provide a unified account for these seemingly disparate phenomena. Copyright © 2017 Elsevier B.V. All rights reserved.

Heuristic and Analytic Processing: Age Trends and Associations with Cognitive Ability and Cognitive Styles.

ERIC Educational Resources Information Center

Kokis, Judite V.; Macpherson, Robyn; Toplak, Maggie E.; West, Richard F.; Stanovich, Keith E.

2002-01-01

Examined developmental and individual differences in tendencies to favor analytic over heuristic responses in three tasks (inductive reasoning, deduction under belief bias conditions, probabilistic reasoning) in children varying in age and cognitive ability. Found significant increases in analytic responding with development on first two tasks.…

Promoting Probabilistic Programming System (PPS) Development in Probabilistic Programming for Advancing Machine Learning (PPAML)

DTIC Science & Technology

2018-03-01

MARCH 2018 FINAL TECHNICAL REPORT APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED STINFO COPY AIR FORCE RESEARCH LABORATORY INFORMATION...SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) Air Force Research Laboratory/RITA DARPA 525 Brooks Road 675 North Randolph Street Rome...1 3.0 METHODS , ASSUMPTIONS, AND PROCEDURES

Learning to Obtain Reward, but Not Avoid Punishment, Is Affected by Presence of PTSD Symptoms in Male Veterans: Empirical Data and Computational Model

PubMed Central

Myers, Catherine E.; Moustafa, Ahmed A.; Sheynin, Jony; VanMeenen, Kirsten M.; Gilbertson, Mark W.; Orr, Scott P.; Beck, Kevin D.; Pang, Kevin C. H.; Servatius, Richard J.

2013-01-01

Post-traumatic stress disorder (PTSD) symptoms include behavioral avoidance which is acquired and tends to increase with time. This avoidance may represent a general learning bias; indeed, individuals with PTSD are often faster than controls on acquiring conditioned responses based on physiologically-aversive feedback. However, it is not clear whether this learning bias extends to cognitive feedback, or to learning from both reward and punishment. Here, male veterans with self-reported current, severe PTSD symptoms (PTSS group) or with few or no PTSD symptoms (control group) completed a probabilistic classification task that included both reward-based and punishment-based trials, where feedback could take the form of reward, punishment, or an ambiguous “no-feedback” outcome that could signal either successful avoidance of punishment or failure to obtain reward. The PTSS group outperformed the control group in total points obtained; the PTSS group specifically performed better than the control group on reward-based trials, with no difference on punishment-based trials. To better understand possible mechanisms underlying observed performance, we used a reinforcement learning model of the task, and applied maximum likelihood estimation techniques to derive estimated parameters describing individual participants’ behavior. Estimations of the reinforcement value of the no-feedback outcome were significantly greater in the control group than the PTSS group, suggesting that the control group was more likely to value this outcome as positively reinforcing (i.e., signaling successful avoidance of punishment). This is consistent with the control group’s generally poorer performance on reward trials, where reward feedback was to be obtained in preference to the no-feedback outcome. Differences in the interpretation of ambiguous feedback may contribute to the facilitated reinforcement learning often observed in PTSD patients, and may in turn provide new insight into how pathological behaviors are acquired and maintained in PTSD. PMID:24015254

Glucocorticoid Regulation of Food-Choice Behavior in Humans: Evidence from Cushing's Syndrome.

PubMed

Moeller, Scott J; Couto, Lizette; Cohen, Vanessa; Lalazar, Yelena; Makotkine, Iouri; Williams, Nia; Yehuda, Rachel; Goldstein, Rita Z; Geer, Eliza B

2016-01-01

The mechanisms by which glucocorticoids regulate food intake and resulting body mass in humans are not well-understood. One potential mechanism could involve modulation of reward processing, but human stress models examining effects of glucocorticoids on behavior contain important confounds. Here, we studied individuals with Cushing's syndrome, a rare endocrine disorder characterized by chronic excess endogenous glucocorticoids. Twenty-three patients with Cushing's syndrome (13 with active disease; 10 with disease in remission) and 15 controls with a comparably high body mass index (BMI) completed two simulated food-choice tasks (one with "explicit" task contingencies and one with "probabilistic" task contingencies), during which they indicated their objective preference for viewing high calorie food images vs. standardized pleasant, unpleasant, and neutral images. All participants also completed measures of food craving, and approximately half of the participants provided 24-h urine samples for assessment of cortisol and cortisone concentrations. Results showed that on the explicit task (but not the probabilistic task), participants with active Cushing's syndrome made fewer food-related choices than participants with Cushing's syndrome in remission, who in turn made fewer food-related choices than overweight controls. Corroborating this group effect, higher urine cortisone was negatively correlated with food-related choice in the subsample of all participants for whom these data were available. On the probabilistic task, despite a lack of group differences, higher food-related choice correlated with higher state and trait food craving in active Cushing's patients. Taken together, relative to overweight controls, Cushing's patients, particularly those with active disease, displayed a reduced vigor of responding for food rewards that was presumably attributable to glucocorticoid abnormalities. Beyond Cushing's, these results may have relevance for elucidating glucocorticoid contributions to food-seeking behavior, enhancing mechanistic understanding of weight fluctuations associated with oral glucocorticoid therapy and/or chronic stress, and informing the neurobiology of neuropsychiatric conditions marked by abnormal cortisol dynamics (e.g., major depression, Alzheimer's disease).

Basal ganglia and Dopamine Contributions to Probabilistic Category Learning

PubMed Central

Shohamy, D.; Myers, C.E.; Kalanithi, J.; Gluck, M.A.

2009-01-01

Studies of the medial temporal lobe and basal ganglia memory systems have recently been extended towards understanding the neural systems contributing to category learning. The basal ganglia, in particular, have been linked to probabilistic category learning in humans. A separate parallel literature in systems neuroscience has emerged, indicating a role for the basal ganglia and related dopamine inputs in reward prediction and feedback processing. Here, we review behavioral, neuropsychological, functional neuroimaging, and computational studies of basal ganglia and dopamine contributions to learning in humans. Collectively, these studies implicate the basal ganglia in incremental, feedback-based learning that involves integrating information across multiple experiences. The medial temporal lobes, by contrast, contribute to rapid encoding of relations between stimuli and support flexible generalization of learning to novel contexts and stimuli. By breaking down our understanding of the cognitive and neural mechanisms contributing to different aspects of learning, recent studies are providing insight into how, and when, these different processes support learning, how they may interact with each other, and the consequence of different forms of learning for the representation of knowledge. PMID:18061261

The Study of the Relationship between Probabilistic Design and Axiomatic Design Methodology. Volume 1

NASA Technical Reports Server (NTRS)

Onwubiko, Chinyere; Onyebueke, Landon

1996-01-01

This program report is the final report covering all the work done on this project. The goal of this project is technology transfer of methodologies to improve design process. The specific objectives are: 1. To learn and understand the Probabilistic design analysis using NESSUS. 2. To assign Design Projects to either undergraduate or graduate students on the application of NESSUS. 3. To integrate the application of NESSUS into some selected senior level courses in Civil and Mechanical Engineering curricula. 4. To develop courseware in Probabilistic Design methodology to be included in a graduate level Design Methodology course. 5. To study the relationship between the Probabilistic design methodology and Axiomatic design methodology.

Social defeat disrupts reward learning and potentiates striatal nociceptin/orphanin FQ mRNA in rats.

PubMed

Der-Avakian, Andre; D'Souza, Manoranjan S; Potter, David N; Chartoff, Elena H; Carlezon, William A; Pizzagalli, Diego A; Markou, Athina

2017-05-01

Mood disorders can be triggered by stress and are characterized by deficits in reward processing, including disrupted reward learning (the ability to modulate behavior according to past rewards). Reward learning is regulated by the anterior cingulate cortex (ACC) and striatal circuits, both of which are implicated in the pathophysiology of mood disorders. Here, we assessed in rats the effects of a potent stressor (social defeat) on reward learning and gene expression in the ACC, ventral tegmental area (VTA), and striatum. Adult male Wistar rats were trained on an operant probabilistic reward task (PRT) and then exposed to 3 days of social defeat before assessment of reward learning. After testing, the ACC, VTA, and striatum were dissected, and expression of genes previously implicated in stress was assessed. Social defeat blunted reward learning (manifested as reduced response bias toward a more frequently rewarded stimulus) and was associated with increased nociceptin/orphanin FQ (N/OFQ) peptide mRNA levels in the striatum and decreased Fos mRNA levels in the VTA. Moreover, N/OFQ peptide and nociceptin receptor mRNA levels in the ACC, VTA and striatum were inversely related to reward learning. The behavioral findings parallel previous data in humans, suggesting that stress similarly disrupts reward learning in both species. Increased striatal N/OFQ mRNA in stressed rats characterized by impaired reward learning is consistent with accumulating evidence that antagonism of nociceptin receptors, which bind N/OFQ, has antidepressant-like effects. These results raise the possibility that nociceptin systems represent a molecular substrate through which stress produces reward learning deficits in mood disorders.

Probalistic Models for Solar Particle Events

NASA Technical Reports Server (NTRS)

Adams, James H., Jr.; Xapsos, Michael

2009-01-01

Probabilistic Models of Solar Particle Events (SPEs) are used in space mission design studies to describe the radiation environment that can be expected at a specified confidence level. The task of the designer is then to choose a design that will operate in the model radiation environment. Probabilistic models have already been developed for solar proton events that describe the peak flux, event-integrated fluence and missionintegrated fluence. In addition a probabilistic model has been developed that describes the mission-integrated fluence for the Z>2 elemental spectra. This talk will focus on completing this suite of models by developing models for peak flux and event-integrated fluence elemental spectra for the Z>2 element

Probabilistic Metrology Attains Macroscopic Cloning of Quantum Clocks

NASA Astrophysics Data System (ADS)

Gendra, B.; Calsamiglia, J.; Muñoz-Tapia, R.; Bagan, E.; Chiribella, G.

2014-12-01

It has recently been shown that probabilistic protocols based on postselection boost the performances of the replication of quantum clocks and phase estimation. Here we demonstrate that the improvements in these two tasks have to match exactly in the macroscopic limit where the number of clones grows to infinity, preserving the equivalence between asymptotic cloning and state estimation for arbitrary values of the success probability. Remarkably, the cloning fidelity depends critically on the number of rationally independent eigenvalues of the clock Hamiltonian. We also prove that probabilistic metrology can simulate cloning in the macroscopic limit for arbitrary sets of states when the performance of the simulation is measured by testing small groups of clones.

Probabilistic image modeling with an extended chain graph for human activity recognition and image segmentation.

PubMed

Zhang, Lei; Zeng, Zhi; Ji, Qiang

2011-09-01

Chain graph (CG) is a hybrid probabilistic graphical model (PGM) capable of modeling heterogeneous relationships among random variables. So far, however, its application in image and video analysis is very limited due to lack of principled learning and inference methods for a CG of general topology. To overcome this limitation, we introduce methods to extend the conventional chain-like CG model to CG model with more general topology and the associated methods for learning and inference in such a general CG model. Specifically, we propose techniques to systematically construct a generally structured CG, to parameterize this model, to derive its joint probability distribution, to perform joint parameter learning, and to perform probabilistic inference in this model. To demonstrate the utility of such an extended CG, we apply it to two challenging image and video analysis problems: human activity recognition and image segmentation. The experimental results show improved performance of the extended CG model over the conventional directed or undirected PGMs. This study demonstrates the promise of the extended CG for effective modeling and inference of complex real-world problems.

Increasing Dopamine Levels in the Brain Improves Feedback-Based Procedural Learning in Healthy Participants: An Artificial-Grammar-Learning Experiment

ERIC Educational Resources Information Center

de Vries, Meinou H.; Ulte, Catrin; Zwitserlood, Pienie; Szymanski, Barbara; Knecht, Stefan

2010-01-01

Recently, an increasing number of studies have suggested a role for the basal ganglia and related dopamine inputs in procedural learning, specifically when learning occurs through trial-by-trial feedback (Shohamy, Myers, Kalanithi, & Gluck. (2008). "Basal ganglia and dopamine contributions to probabilistic category learning." "Neuroscience and…

A Bayesian computational model for online character recognition and disability assessment during cursive eye writing.

PubMed

Diard, Julien; Rynik, Vincent; Lorenceau, Jean

2013-01-01

This research involves a novel apparatus, in which the user is presented with an illusion inducing visual stimulus. The user perceives illusory movement that can be followed by the eye, so that smooth pursuit eye movements can be sustained in arbitrary directions. Thus, free-flow trajectories of any shape can be traced. In other words, coupled with an eye-tracking device, this apparatus enables "eye writing," which appears to be an original object of study. We adapt a previous model of reading and writing to this context. We describe a probabilistic model called the Bayesian Action-Perception for Eye On-Line model (BAP-EOL). It encodes probabilistic knowledge about isolated letter trajectories, their size, high-frequency components of the produced trajectory, and pupil diameter. We show how Bayesian inference, in this single model, can be used to solve several tasks, like letter recognition and novelty detection (i.e., recognizing when a presented character is not part of the learned database). We are interested in the potential use of the eye writing apparatus by motor impaired patients: the final task we solve by Bayesian inference is disability assessment (i.e., measuring and tracking the evolution of motor characteristics of produced trajectories). Preliminary experimental results are presented, which illustrate the method, showing the feasibility of character recognition in the context of eye writing. We then show experimentally how a model of the unknown character can be used to detect trajectories that are likely to be new symbols, and how disability assessment can be performed by opportunistically observing characteristics of fine motor control, as letter are being traced. Experimental analyses also help identify specificities of eye writing, as compared to handwriting, and the resulting technical challenges.

A Bayesian computational model for online character recognition and disability assessment during cursive eye writing

PubMed Central

Diard, Julien; Rynik, Vincent; Lorenceau, Jean

2013-01-01

This research involves a novel apparatus, in which the user is presented with an illusion inducing visual stimulus. The user perceives illusory movement that can be followed by the eye, so that smooth pursuit eye movements can be sustained in arbitrary directions. Thus, free-flow trajectories of any shape can be traced. In other words, coupled with an eye-tracking device, this apparatus enables “eye writing,” which appears to be an original object of study. We adapt a previous model of reading and writing to this context. We describe a probabilistic model called the Bayesian Action-Perception for Eye On-Line model (BAP-EOL). It encodes probabilistic knowledge about isolated letter trajectories, their size, high-frequency components of the produced trajectory, and pupil diameter. We show how Bayesian inference, in this single model, can be used to solve several tasks, like letter recognition and novelty detection (i.e., recognizing when a presented character is not part of the learned database). We are interested in the potential use of the eye writing apparatus by motor impaired patients: the final task we solve by Bayesian inference is disability assessment (i.e., measuring and tracking the evolution of motor characteristics of produced trajectories). Preliminary experimental results are presented, which illustrate the method, showing the feasibility of character recognition in the context of eye writing. We then show experimentally how a model of the unknown character can be used to detect trajectories that are likely to be new symbols, and how disability assessment can be performed by opportunistically observing characteristics of fine motor control, as letter are being traced. Experimental analyses also help identify specificities of eye writing, as compared to handwriting, and the resulting technical challenges. PMID:24273525

Discounting of food, sex, and money.

PubMed

Holt, Daniel D; Newquist, Matthew H; Smits, Rochelle R; Tiry, Andrew M

2014-06-01

Discounting is a useful framework for understanding choice involving a range of delayed and probabilistic outcomes (e.g., money, food, drugs), but relatively few studies have examined how people discount other commodities (e.g., entertainment, sex). Using a novel discounting task, where the length of a line represented the value of an outcome and was adjusted using a staircase procedure, we replicated previous findings showing that individuals discount delayed and probabilistic outcomes in a manner well described by a hyperbola-like function. In addition, we found strong positive correlations between discounting rates of delayed, but not probabilistic, outcomes. This suggests that discounting of delayed outcomes may be relatively predictable across outcome types but that discounting of probabilistic outcomes may depend more on specific contexts. The generality of delay discounting and potential context dependence of probability discounting may provide important information regarding factors contributing to choice behavior.

Base-Rate Neglect as a Function of Base Rates in Probabilistic Contingency Learning

ERIC Educational Resources Information Center

Kutzner, Florian; Freytag, Peter; Vogel, Tobias; Fiedler, Klaus

2008-01-01

When humans predict criterion events based on probabilistic predictors, they often lend excessive weight to the predictor and insufficient weight to the base rate of the criterion event. In an operant analysis, using a matching-to-sample paradigm, Goodie and Fantino (1996) showed that humans exhibit base-rate neglect when predictors are associated…

Emergence of spontaneous anticipatory hand movements in a probabilistic environment

PubMed Central

Bruhn, Pernille

2013-01-01

In this article, we present a novel experimental approach to the study of anticipation in probabilistic cuing. We implemented a modified spatial cuing task in which participants made an anticipatory hand movement toward one of two probabilistic targets while the (x, y)-computer mouse coordinates of their hand movements were sampled. This approach allowed us to tap into anticipatory processes as they occurred, rather than just measuring their behavioral outcome through reaction time to the target. In different conditions, we varied the participants’ degree of certainty of the upcoming target position with probabilistic pre-cues. We found that participants initiated spontaneous anticipatory hand movements in all conditions, even when they had no information on the position of the upcoming target. However, participants’ hand position immediately before the target was affected by the degree of certainty concerning the target’s position. This modulation of anticipatory hand movements emerged rapidly in most participants as they encountered a constant probabilistic relation between a cue and an upcoming target position over the course of the experiment. Finally, we found individual differences in the way anticipatory behavior was modulated with an uncertain/neutral cue. Implications of these findings for probabilistic spatial cuing are discussed. PMID:23833694

Probabilistic drug connectivity mapping

PubMed Central

2014-01-01

Background The aim of connectivity mapping is to match drugs using drug-treatment gene expression profiles from multiple cell lines. This can be viewed as an information retrieval task, with the goal of finding the most relevant profiles for a given query drug. We infer the relevance for retrieval by data-driven probabilistic modeling of the drug responses, resulting in probabilistic connectivity mapping, and further consider the available cell lines as different data sources. We use a special type of probabilistic model to separate what is shared and specific between the sources, in contrast to earlier connectivity mapping methods that have intentionally aggregated all available data, neglecting information about the differences between the cell lines. Results We show that the probabilistic multi-source connectivity mapping method is superior to alternatives in finding functionally and chemically similar drugs from the Connectivity Map data set. We also demonstrate that an extension of the method is capable of retrieving combinations of drugs that match different relevant parts of the query drug response profile. Conclusions The probabilistic modeling-based connectivity mapping method provides a promising alternative to earlier methods. Principled integration of data from different cell lines helps to identify relevant responses for specific drug repositioning applications. PMID:24742351

Probabilistic numerics and uncertainty in computations

PubMed Central

Hennig, Philipp; Osborne, Michael A.; Girolami, Mark

2015-01-01

We deliver a call to arms for probabilistic numerical methods: algorithms for numerical tasks, including linear algebra, integration, optimization and solving differential equations, that return uncertainties in their calculations. Such uncertainties, arising from the loss of precision induced by numerical calculation with limited time or hardware, are important for much contemporary science and industry. Within applications such as climate science and astrophysics, the need to make decisions on the basis of computations with large and complex data have led to a renewed focus on the management of numerical uncertainty. We describe how several seminal classic numerical methods can be interpreted naturally as probabilistic inference. We then show that the probabilistic view suggests new algorithms that can flexibly be adapted to suit application specifics, while delivering improved empirical performance. We provide concrete illustrations of the benefits of probabilistic numeric algorithms on real scientific problems from astrometry and astronomical imaging, while highlighting open problems with these new algorithms. Finally, we describe how probabilistic numerical methods provide a coherent framework for identifying the uncertainty in calculations performed with a combination of numerical algorithms (e.g. both numerical optimizers and differential equation solvers), potentially allowing the diagnosis (and control) of error sources in computations. PMID:26346321

Probabilistic numerics and uncertainty in computations.

PubMed

Hennig, Philipp; Osborne, Michael A; Girolami, Mark

2015-07-08

We deliver a call to arms for probabilistic numerical methods : algorithms for numerical tasks, including linear algebra, integration, optimization and solving differential equations, that return uncertainties in their calculations. Such uncertainties, arising from the loss of precision induced by numerical calculation with limited time or hardware, are important for much contemporary science and industry. Within applications such as climate science and astrophysics, the need to make decisions on the basis of computations with large and complex data have led to a renewed focus on the management of numerical uncertainty. We describe how several seminal classic numerical methods can be interpreted naturally as probabilistic inference. We then show that the probabilistic view suggests new algorithms that can flexibly be adapted to suit application specifics, while delivering improved empirical performance. We provide concrete illustrations of the benefits of probabilistic numeric algorithms on real scientific problems from astrometry and astronomical imaging, while highlighting open problems with these new algorithms. Finally, we describe how probabilistic numerical methods provide a coherent framework for identifying the uncertainty in calculations performed with a combination of numerical algorithms (e.g. both numerical optimizers and differential equation solvers), potentially allowing the diagnosis (and control) of error sources in computations.

Energy and Power Aware Computing Through Management of Computational Entropy

DTIC Science & Technology

2008-01-01

18 2.4.1 ACIP living framework forum task...This research focused on two sub- tasks: (1) Assessing the need and planning for a potential “Living Framework Forum ” (LFF) software architecture...probabilistic switching with plausible device realizations to save energy in our patent application [35]. In [35], we showed an introverted switch in

Accelerated iTBS treatment in depressed patients differentially modulates reward system activity based on anhedonia.

PubMed

Duprat, Romain; Wu, Guo-Rong; De Raedt, Rudi; Baeken, Chris

2017-08-09

Accelerated intermittent theta-burst stimulation (aiTBS) anti-depressive working mechanisms are still unclear. Because aiTBS may work through modulating the reward system and the level of anhedonia may influence this modulation, we investigated the effect of aiTBS on reward responsiveness in high and low anhedonic MDD patients. In this registered RCT (NCT01832805), 50 MDD patients were randomised to a sham-controlled cross-over aiTBS treatment protocol over the left dorsolateral prefrontal cortex (DLPFC). Patients performed a probabilistic learning task in fMRI before and after each week of stimulation. Task performance analyses did not show any significant effects of aiTBS on reward responsiveness, nor differences between both groups of MDD patients. However, at baseline, low anhedonic patients displayed higher neural activity in the caudate and putamen. After the first week of aiTBS treatment, in low anhedonic patients we found a decreased neural activity within the reward system, in contrast to an increased activity observed in high anhedonic patients. No changes were observed in reward related neural regions after the first week of sham stimulation. Although both MDD groups showed no differences in task performance, our brain imaging findings suggest that left DLPFC aiTBS treatment modulates the reward system differently according to anhedonia severity.

Sentence Processing in an Artificial Language: Learning and Using Combinatorial Constraints

ERIC Educational Resources Information Center

Amato, Michael S.; MacDonald, Maryellen C.

2010-01-01

A study combining artificial grammar and sentence comprehension methods investigated the learning and online use of probabilistic, nonadjacent combinatorial constraints. Participants learned a small artificial language describing cartoon monsters acting on objects. Self-paced reading of sentences in the artificial language revealed comprehenders'…

Simulation Of Research And Development Projects

NASA Technical Reports Server (NTRS)

Miles, Ralph F.

1987-01-01

Measures of preference for alternative project plans calculated. Simulation of Research and Development Projects (SIMRAND) program aids in optimal allocation of research and development resources needed to achieve project goals. Models system subsets or project tasks as various network paths to final goal. Each path described in terms of such task variables as cost per hour, cost per unit, and availability of resources. Uncertainty incorporated by treating task variables as probabilistic random variables. Written in Microsoft FORTRAN 77.

Ethical sensitivity in obsessive-compulsive disorder and generalized anxiety disorder: the role of reversal learning.

PubMed

Szabó, Csilla; Németh, Attila; Kéri, Szabolcs

2013-12-01

In obsessive-compulsive disorder (OCD), amplified moral sensitivity may be related to the orbitofrontal-striatal circuit, which is also critical in reversal learning. This study examined three questions: (1) What aspects of ethical sensitivity is altered in OCD?; (2) What is the relationship between ethical sensitivity and reversal learning?; (3) Are potential alterations in ethical sensitivity and reversal learning present in generalized anxiety disorder (GAD)? Participants were 28 outpatients with OCD, 21 individuals with GAD, and 30 matched healthy controls. Participants received the ethical sensitivity scale questionnaire (ESSQ), rating scales for clinical symptoms, a reversal learning task, and the Wisconsin Card Sorting Test (WCST). We found higher ethical sensitivity scores in OCD compared with healthy controls in the case of generating interpretations and options and identifying the consequences of actions. Individuals with OCD displayed prolonged reaction times on probabilistic errors without shift and final reversal errors. Participants with GAD did not differ from healthy controls on the ESSQ, but they were slower on reversal learning relative to nonpatients. In OCD, reaction time on final reversal errors mediated the relationship between ethical sensitivity and compulsions. WCST performance was intact in OCD and GAD. Small sample size, limited neuropsychological assessment, self-rating scale for ethical sensitivity. Prolonged reaction time at switching reinforcement contingencies is related to increased ethical sensitivity in OCD. Slow affective switching may link ethical sensitivity and compulsions. Copyright © 2013 Elsevier Ltd. All rights reserved.

Fronto-temporal white matter connectivity predicts reversal learning errors

PubMed Central

Alm, Kylie H.; Rolheiser, Tyler; Mohamed, Feroze B.; Olson, Ingrid R.

2015-01-01

Each day, we make hundreds of decisions. In some instances, these decisions are guided by our innate needs; in other instances they are guided by memory. Probabilistic reversal learning tasks exemplify the close relationship between decision making and memory, as subjects are exposed to repeated pairings of a stimulus choice with a reward or punishment outcome. After stimulus–outcome associations have been learned, the associated reward contingencies are reversed, and participants are not immediately aware of this reversal. Individual differences in the tendency to choose the previously rewarded stimulus reveal differences in the tendency to make poorly considered, inflexible choices. Lesion studies have strongly linked reversal learning performance to the functioning of the orbitofrontal cortex, the hippocampus, and in some instances, the amygdala. Here, we asked whether individual differences in the microstructure of the uncinate fasciculus, a white matter tract that connects anterior and medial temporal lobe regions to the orbitofrontal cortex, predict reversal learning performance. Diffusion tensor imaging and behavioral paradigms were used to examine this relationship in 33 healthy young adults. The results of tractography revealed a significant negative relationship between reversal learning performance and uncinate axial diffusivity, but no such relationship was demonstrated in a control tract, the inferior longitudinal fasciculus. Our findings suggest that the uncinate might serve to integrate associations stored in the anterior and medial temporal lobes with expectations about expected value based on feedback history, computed in the orbitofrontal cortex. PMID:26150776

Medication Impairs Probabilistic Classification Learning in Parkinson's Disease

ERIC Educational Resources Information Center

Jahanshahi, Marjan; Wilkinson, Leonora; Gahir, Harpreet; Dharminda, Angeline; Lagnado, David A.

2010-01-01

In Parkinson's disease (PD), it is possible that tonic increase of dopamine associated with levodopa medication overshadows phasic release of dopamine, which is essential for learning. Thus while the motor symptoms of PD are improved with levodopa medication, learning would be disrupted. To test this hypothesis, we investigated the effect of…

Learning Orthographic Structure with Sequential Generative Neural Networks

ERIC Educational Resources Information Center

Testolin, Alberto; Stoianov, Ivilin; Sperduti, Alessandro; Zorzi, Marco

2016-01-01

Learning the structure of event sequences is a ubiquitous problem in cognition and particularly in language. One possible solution is to learn a probabilistic generative model of sequences that allows making predictions about upcoming events. Though appealing from a neurobiological standpoint, this approach is typically not pursued in…

Robust Arm and Hand Tracking by Unsupervised Context Learning

PubMed Central

Spruyt, Vincent; Ledda, Alessandro; Philips, Wilfried

2014-01-01

Hand tracking in video is an increasingly popular research field due to the rise of novel human-computer interaction methods. However, robust and real-time hand tracking in unconstrained environments remains a challenging task due to the high number of degrees of freedom and the non-rigid character of the human hand. In this paper, we propose an unsupervised method to automatically learn the context in which a hand is embedded. This context includes the arm and any other object that coherently moves along with the hand. We introduce two novel methods to incorporate this context information into a probabilistic tracking framework, and introduce a simple yet effective solution to estimate the position of the arm. Finally, we show that our method greatly increases robustness against occlusion and cluttered background, without degrading tracking performance if no contextual information is available. The proposed real-time algorithm is shown to outperform the current state-of-the-art by evaluating it on three publicly available video datasets. Furthermore, a novel dataset is created and made publicly available for the research community. PMID:25004155

An intelligent computer tutor to guide self-explanation while learning from examples

NASA Astrophysics Data System (ADS)

Conati, Cristina

1999-11-01

Many studies in cognitive science show that self-explanation---the process of clarifying and making more complete to oneself the solution of an example---improves learning, and that guiding self-explanation extends these benefits. This thesis presents an intelligent computer tutor that aims to improve learning from examples by supporting self-explanation. The tutor, known as the SE (self-explanation) Coach, is innovative in two ways. First, it represents the first attempt to develop a computer tutor that supports example studying instead of problem solving. Second, it explicitly guides a domain-general, meta-cognitive skill: self-explanation. The SE-Coach is part of the Andes tutoring system for college physics and is meant to be used in conjunction with the problem solving tasks that Andes supports. In order to maximize the system capability to trigger the same beneficial cognitive processes, every element of the SE-Coach embeds existing hypotheses about the features that make self-explanation effective for learning. Designing the SE-Coach involved finding solutions for three main challenges: (1) To design an interface that effectively monitors and supports self-explanation. (2) To devise a student model that allows the assessment of example understanding from reading and self-examination actions. (3) To effectively elicit further self-explanation that improves student's example understanding. In this work we present our solutions to these challenges: (1) An interface including principled, interactive tools to explore examples and build self-explanations under the SECoach's supervision. (2) A probabilistic student model based on a Bayesian network, which integrates a model of correct self-explanation and information on the student's knowledge and studying actions to generate a probabilistic assessment of the student's example understanding. (3) Tutorial interventions that rely on the student model to detect deficits in the student's example understanding and elicit self-explanations that overcome them. In this thesis we also present the results of a formal study with 56 college students to evaluate the effectiveness of the SE-Coach. We discuss some hypotheses to explain the obtained results, based on the analysis of the data collected during the experiment.

Probabilistic grammatical model for helix‐helix contact site classification

PubMed Central

2013-01-01

Background Hidden Markov Models power many state‐of‐the‐art tools in the field of protein bioinformatics. While excelling in their tasks, these methods of protein analysis do not convey directly information on medium‐ and long‐range residue‐residue interactions. This requires an expressive power of at least context‐free grammars. However, application of more powerful grammar formalisms to protein analysis has been surprisingly limited. Results In this work, we present a probabilistic grammatical framework for problem‐specific protein languages and apply it to classification of transmembrane helix‐helix pairs configurations. The core of the model consists of a probabilistic context‐free grammar, automatically inferred by a genetic algorithm from only a generic set of expert‐based rules and positive training samples. The model was applied to produce sequence based descriptors of four classes of transmembrane helix‐helix contact site configurations. The highest performance of the classifiers reached AUCROC of 0.70. The analysis of grammar parse trees revealed the ability of representing structural features of helix‐helix contact sites. Conclusions We demonstrated that our probabilistic context‐free framework for analysis of protein sequences outperforms the state of the art in the task of helix‐helix contact site classification. However, this is achieved without necessarily requiring modeling long range dependencies between interacting residues. A significant feature of our approach is that grammar rules and parse trees are human‐readable. Thus they could provide biologically meaningful information for molecular biologists. PMID:24350601

Learning of state-space models with highly informative observations: A tempered sequential Monte Carlo solution

NASA Astrophysics Data System (ADS)

Svensson, Andreas; Schön, Thomas B.; Lindsten, Fredrik

2018-05-01

Probabilistic (or Bayesian) modeling and learning offers interesting possibilities for systematic representation of uncertainty using probability theory. However, probabilistic learning often leads to computationally challenging problems. Some problems of this type that were previously intractable can now be solved on standard personal computers thanks to recent advances in Monte Carlo methods. In particular, for learning of unknown parameters in nonlinear state-space models, methods based on the particle filter (a Monte Carlo method) have proven very useful. A notoriously challenging problem, however, still occurs when the observations in the state-space model are highly informative, i.e. when there is very little or no measurement noise present, relative to the amount of process noise. The particle filter will then struggle in estimating one of the basic components for probabilistic learning, namely the likelihood p (data | parameters). To this end we suggest an algorithm which initially assumes that there is substantial amount of artificial measurement noise present. The variance of this noise is sequentially decreased in an adaptive fashion such that we, in the end, recover the original problem or possibly a very close approximation of it. The main component in our algorithm is a sequential Monte Carlo (SMC) sampler, which gives our proposed method a clear resemblance to the SMC2 method. Another natural link is also made to the ideas underlying the approximate Bayesian computation (ABC). We illustrate it with numerical examples, and in particular show promising results for a challenging Wiener-Hammerstein benchmark problem.

Modalities, Relations, and Learning

NASA Astrophysics Data System (ADS)

Müller, Martin Eric

While the popularity of statistical, probabilistic and exhaustive machine learning techniques still increases, relational and logic approaches are still a niche market in research. While the former approaches focus on predictive accuracy, the latter ones prove to be indispensable in knowledge discovery.

Early adversity and learning: implications for typical and atypical behavioral development.

PubMed

Hanson, Jamie L; van den Bos, Wouter; Roeber, Barbara J; Rudolph, Karen D; Davidson, Richard J; Pollak, Seth D

2017-07-01

Children who experience early adversity often develop emotion regulatory problems, but little is known about the mechanisms that mediate this relation. We tested whether general associative learning processes contribute to associations between adversity, in the form of child maltreatment, and negative behavioral outcomes. Eighty-one participants between 12 and 17 years of age were recruited for this study and completed a probabilistic learning Task. Forty-one of these participants had been exposed to physical abuse, a form of early adversity. Forty additional participants without any known history of maltreatment served as a comparison group. All participants (and their parents) also completed portions of the Youth Life Stress Interview to understand adolescent's behavior. We calculated measures of associative learning, and also constructed mathematical models of learning. We found that adolescents exposed to high levels of adversity early in their lives had lower levels of associative learning than comparison adolescents. In addition, we found that impaired associative learning partially explained the higher levels of behavioral problems among youth who suffered early adversity. Using mathematical models, we also found that two components of learning were specifically affected in children exposed to adversity: choice variability and biases in their beliefs about the likelihood of rewards in the environment. Participants who had been exposed to early adversity were less able than their peers to correctly learn which stimuli were likely to result in reward, even after repeated feedback. These individuals also used information about known rewards in their environments less often. In addition, individuals exposed to adversity made decisions early in the learning process as if rewards were less consistent and occurred more at random. These data suggest one mechanism through which early life experience shapes behavioral development. © 2017 Association for Child and Adolescent Mental Health.

Early adversity and learning: implications for typical and atypical behavioral development

PubMed Central

Hanson, Jamie L.; van den Bos, Wouter; Roeber, Barbara J.; Rudolph, Karen D.; Davidson, Richard J.; Pollak, Seth D.

2017-01-01

Background Children who experience early adversity often develop emotion regulatory problems, but little is known about the mechanisms that mediate this relation. We tested whether general associative learning processes contribute to associations between adversity, in the form of child maltreatment, and negative behavioral outcomes. Methods Eighty-one participants between 12 and 17 years of age were recruited for this study and completed a probabilistic learning Task. Forty-one of these participants had been exposed to physical abuse, a form of early adversity. Forty additional participants without any known history of maltreatment served as a comparison group. All participants (and their parents) also completed portions of the Youth Life Stress Interview to understand adolescent’s behavior. We calculated measures of associative learning, and also constructed mathematical models of learning. Results We found that adolescents exposed to high levels of adversity early in their lives had lower levels of associative learning than comparison adolescents. In addition, we found that impaired associative learning partially explained the higher levels of behavioral problems among youth who suffered early adversity. Using mathematical models, we also found that two components of learning were specifically affected in children exposed to adversity: choice variability and biases in their beliefs about the likelihood of rewards in the environment. Conclusions Participants who had been exposed to early adversity were less able than their peers to correctly learn which stimuli were likely to result in reward, even after repeated feedback. These individuals also used information about known rewards in their environments less often. In addition, individuals exposed to adversity made decisions early in the learning process as if rewards were less consistent and occurred more at random. These data suggest one mechanism through which early life experience shapes behavioral development. PMID:28158896

Learning and exploration in action-perception loops.

PubMed

Little, Daniel Y; Sommer, Friedrich T

2013-01-01

Discovering the structure underlying observed data is a recurring problem in machine learning with important applications in neuroscience. It is also a primary function of the brain. When data can be actively collected in the context of a closed action-perception loop, behavior becomes a critical determinant of learning efficiency. Psychologists studying exploration and curiosity in humans and animals have long argued that learning itself is a primary motivator of behavior. However, the theoretical basis of learning-driven behavior is not well understood. Previous computational studies of behavior have largely focused on the control problem of maximizing acquisition of rewards and have treated learning the structure of data as a secondary objective. Here, we study exploration in the absence of external reward feedback. Instead, we take the quality of an agent's learned internal model to be the primary objective. In a simple probabilistic framework, we derive a Bayesian estimate for the amount of information about the environment an agent can expect to receive by taking an action, a measure we term the predicted information gain (PIG). We develop exploration strategies that approximately maximize PIG. One strategy based on value-iteration consistently learns faster than previously developed reward-free exploration strategies across a diverse range of environments. Psychologists believe the evolutionary advantage of learning-driven exploration lies in the generalized utility of an accurate internal model. Consistent with this hypothesis, we demonstrate that agents which learn more efficiently during exploration are later better able to accomplish a range of goal-directed tasks. We will conclude by discussing how our work elucidates the explorative behaviors of animals and humans, its relationship to other computational models of behavior, and its potential application to experimental design, such as in closed-loop neurophysiology studies.

Design of Composite Structures for Reliability and Damage Tolerance

NASA Technical Reports Server (NTRS)

Rais-Rohani, Masoud

1999-01-01

A summary of research conducted during the first year is presented. The research objectives were sought by conducting two tasks: (1) investigation of probabilistic design techniques for reliability-based design of composite sandwich panels, and (2) examination of strain energy density failure criterion in conjunction with response surface methodology for global-local design of damage tolerant helicopter fuselage structures. This report primarily discusses the efforts surrounding the first task and provides a discussion of some preliminary work involving the second task.

Discovering Prerequisite Structure of Skills through Probabilistic Association Rules Mining

ERIC Educational Resources Information Center

Chen, Yang; Wuillemin, Pierre-Henr; Labat, Jean-Marc

2015-01-01

Estimating the prerequisite structure of skills is a crucial issue in domain modeling. Students usually learn skills in sequence since the preliminary skills need to be learned prior to the complex skills. The prerequisite relations between skills underlie the design of learning sequence and adaptation strategies for tutoring systems. The…

Reconstructing constructivism: causal models, Bayesian learning mechanisms, and the theory theory.

PubMed

Gopnik, Alison; Wellman, Henry M

2012-11-01

We propose a new version of the "theory theory" grounded in the computational framework of probabilistic causal models and Bayesian learning. Probabilistic models allow a constructivist but rigorous and detailed approach to cognitive development. They also explain the learning of both more specific causal hypotheses and more abstract framework theories. We outline the new theoretical ideas, explain the computational framework in an intuitive and nontechnical way, and review an extensive but relatively recent body of empirical results that supports these ideas. These include new studies of the mechanisms of learning. Children infer causal structure from statistical information, through their own actions on the world and through observations of the actions of others. Studies demonstrate these learning mechanisms in children from 16 months to 4 years old and include research on causal statistical learning, informal experimentation through play, and imitation and informal pedagogy. They also include studies of the variability and progressive character of intuitive theory change, particularly theory of mind. These studies investigate both the physical and the psychological and social domains. We conclude with suggestions for further collaborative projects between developmental and computational cognitive scientists.

Modeling semantic aspects for cross-media image indexing.

PubMed

Monay, Florent; Gatica-Perez, Daniel

2007-10-01

To go beyond the query-by-example paradigm in image retrieval, there is a need for semantic indexing of large image collections for intuitive text-based image search. Different models have been proposed to learn the dependencies between the visual content of an image set and the associated text captions, then allowing for the automatic creation of semantic indices for unannotated images. The task, however, remains unsolved. In this paper, we present three alternatives to learn a Probabilistic Latent Semantic Analysis model (PLSA) for annotated images, and evaluate their respective performance for automatic image indexing. Under the PLSA assumptions, an image is modeled as a mixture of latent aspects that generates both image features and text captions, and we investigate three ways to learn the mixture of aspects. We also propose a more discriminative image representation than the traditional Blob histogram, concatenating quantized local color information and quantized local texture descriptors. The first learning procedure of a PLSA model for annotated images is a standard EM algorithm, which implicitly assumes that the visual and the textual modalities can be treated equivalently. The other two models are based on an asymmetric PLSA learning, allowing to constrain the definition of the latent space on the visual or on the textual modality. We demonstrate that the textual modality is more appropriate to learn a semantically meaningful latent space, which translates into improved annotation performance. A comparison of our learning algorithms with respect to recent methods on a standard dataset is presented, and a detailed evaluation of the performance shows the validity of our framework.

The Effect of Motor Difficulty on the Acquisition of a Computer Task: A Comparison between Young and Older Adults

ERIC Educational Resources Information Center

Fezzani, K.; Albinet, C.; Thon, B.; Marquie, J. -C.

2010-01-01

The present study investigated the extent to which the impact of motor difficulty on the acquisition of a computer task varies as a function of age. Fourteen young and 14 older participants performed 352 sequences of 10 serial pointing movements with a wireless pen on a digitiser tablet. A conditional probabilistic structure governed the…

Temporal and probabilistic discounting of rewards in children and adolescents: effects of age and ADHD symptoms.

PubMed

Scheres, Anouk; Dijkstra, Marianne; Ainslie, Eleanor; Balkan, Jaclyn; Reynolds, Brady; Sonuga-Barke, Edmund; Castellanos, F Xavier

2006-01-01

This study investigated whether age and ADHD symptoms affected choice preferences in children and adolescents when they chose between (1) small immediate rewards and larger delayed rewards and (2) small certain rewards and larger probabilistic uncertain rewards. A temporal discounting (TD) task and a probabilistic discounting (PD) task were used to measure the degree to which the subjective value of a large reward decreased as one had to wait longer for it (TD), and as the probability of obtaining it decreased (PD). Rewards used were small amounts of money. In the TD task, the large reward (10 cents) was delayed by between 0 and 30s, and the immediate reward varied in magnitude (0-10 cents). In the PD task, receipt of the large reward (10 cents) varied in likelihood, with probabilities of 0, 0.25, 0.5, 0.75, and 1.0 used, and the certain reward varied in magnitude (0-10 cents). Age and diagnostic group did not affect the degree of PD of rewards: All participants made choices so that total gains were maximized. As predicted, young children, aged 6-11 years (n = 25) demonstrated steeper TD of rewards than adolescents, aged 12-17 years (n = 21). This effect remained significant even when choosing the immediate reward did not shorten overall task duration. This, together with the lack of interaction between TD task version and age, suggests that steeper discounting in young children is driven by reward immediacy and not by delay aversion. Contrary to our predictions, participants with ADHD (n = 22) did not demonstrate steeper TD of rewards than controls (n = 24). These results raise the possibility that strong preferences for small immediate rewards in ADHD, as found in previous research, depend on factors such as total maximum gain and the use of fixed versus varied delay durations. The decrease in TD as observed in adolescents compared to children may be related to developmental changes in the (dorsolateral) prefrontal cortex. Future research needs to investigate these possibilities.

Diffusion tensor MRI tractography reveals increased fractional anisotropy (FA) in arcuate fasciculus following music-cued motor training.

PubMed

Moore, Emma; Schaefer, Rebecca S; Bastin, Mark E; Roberts, Neil; Overy, Katie

2017-08-01

Auditory cues are frequently used to support movement learning and rehabilitation, but the neural basis of this behavioural effect is not yet clear. We investigated the microstructural neuroplasticity effects of adding musical cues to a motor learning task. We hypothesised that music-cued, left-handed motor training would increase fractional anisotropy (FA) in the contralateral arcuate fasciculus, a fibre tract connecting auditory, pre-motor and motor regions. Thirty right-handed participants were assigned to a motor learning condition either with (Music Group) or without (Control Group) musical cues. Participants completed 20minutes of training three times per week over four weeks. Diffusion tensor MRI and probabilistic neighbourhood tractography identified FA, axial (AD) and radial (RD) diffusivity before and after training. Results revealed that FA increased significantly in the right arcuate fasciculus of the Music group only, as hypothesised, with trends for AD to increase and RD to decrease, a pattern of results consistent with activity-dependent increases in myelination. No significant changes were found in the left ipsilateral arcuate fasciculus of either group. This is the first evidence that adding musical cues to movement learning can induce rapid microstructural change in white matter pathways in adults, with potential implications for therapeutic clinical practice. Copyright © 2017 The Authors. Published by Elsevier Inc. All rights reserved.

Causal learning and inference as a rational process: the new synthesis.

PubMed

Holyoak, Keith J; Cheng, Patricia W

2011-01-01

Over the past decade, an active line of research within the field of human causal learning and inference has converged on a general representational framework: causal models integrated with bayesian probabilistic inference. We describe this new synthesis, which views causal learning and inference as a fundamentally rational process, and review a sample of the empirical findings that support the causal framework over associative alternatives. Causal events, like all events in the distal world as opposed to our proximal perceptual input, are inherently unobservable. A central assumption of the causal approach is that humans (and potentially nonhuman animals) have been designed in such a way as to infer the most invariant causal relations for achieving their goals based on observed events. In contrast, the associative approach assumes that learners only acquire associations among important observed events, omitting the representation of the distal relations. By incorporating bayesian inference over distributions of causal strength and causal structures, along with noisy-logical (i.e., causal) functions for integrating the influences of multiple causes on a single effect, human judgments about causal strength and structure can be predicted accurately for relatively simple causal structures. Dynamic models of learning based on the causal framework can explain patterns of acquisition observed with serial presentation of contingency data and are consistent with available neuroimaging data. The approach has been extended to a diverse range of inductive tasks, including category-based and analogical inferences.

Probabilistic Inference in General Graphical Models through Sampling in Stochastic Networks of Spiking Neurons

PubMed Central

Pecevski, Dejan; Buesing, Lars; Maass, Wolfgang

2011-01-01

An important open problem of computational neuroscience is the generic organization of computations in networks of neurons in the brain. We show here through rigorous theoretical analysis that inherent stochastic features of spiking neurons, in combination with simple nonlinear computational operations in specific network motifs and dendritic arbors, enable networks of spiking neurons to carry out probabilistic inference through sampling in general graphical models. In particular, it enables them to carry out probabilistic inference in Bayesian networks with converging arrows (“explaining away”) and with undirected loops, that occur in many real-world tasks. Ubiquitous stochastic features of networks of spiking neurons, such as trial-to-trial variability and spontaneous activity, are necessary ingredients of the underlying computational organization. We demonstrate through computer simulations that this approach can be scaled up to neural emulations of probabilistic inference in fairly large graphical models, yielding some of the most complex computations that have been carried out so far in networks of spiking neurons. PMID:22219717

Psychics, aliens, or experience? Using the Anomalistic Belief Scale to examine the relationship between type of belief and probabilistic reasoning.

PubMed

Prike, Toby; Arnold, Michelle M; Williamson, Paul

2017-08-01

A growing body of research has shown people who hold anomalistic (e.g., paranormal) beliefs may differ from nonbelievers in their propensity to make probabilistic reasoning errors. The current study explored the relationship between these beliefs and performance through the development of a new measure of anomalistic belief, called the Anomalistic Belief Scale (ABS). One key feature of the ABS is that it includes a balance of both experiential and theoretical belief items. Another aim of the study was to use the ABS to investigate the relationship between belief and probabilistic reasoning errors on conjunction fallacy tasks. As expected, results showed there was a relationship between anomalistic belief and propensity to commit the conjunction fallacy. Importantly, regression analyses on the factors that make up the ABS showed that the relationship between anomalistic belief and probabilistic reasoning occurred only for beliefs about having experienced anomalistic phenomena, and not for theoretical anomalistic beliefs. Copyright © 2017 Elsevier Inc. All rights reserved.

Time-reversal and Bayesian inversion

NASA Astrophysics Data System (ADS)

Debski, Wojciech

2017-04-01

Probabilistic inversion technique is superior to the classical optimization-based approach in all but one aspects. It requires quite exhaustive computations which prohibit its use in huge size inverse problems like global seismic tomography or waveform inversion to name a few. The advantages of the approach are, however, so appealing that there is an ongoing continuous afford to make the large inverse task as mentioned above manageable with the probabilistic inverse approach. One of the perspective possibility to achieve this goal relays on exploring the internal symmetry of the seismological modeling problems in hand - a time reversal and reciprocity invariance. This two basic properties of the elastic wave equation when incorporating into the probabilistic inversion schemata open a new horizons for Bayesian inversion. In this presentation we discuss the time reversal symmetry property, its mathematical aspects and propose how to combine it with the probabilistic inverse theory into a compact, fast inversion algorithm. We illustrate the proposed idea with the newly developed location algorithm TRMLOC and discuss its efficiency when applied to mining induced seismic data.

The role of linguistic experience in the processing of probabilistic information in production.

PubMed

Gustafson, Erin; Goldrick, Matthew

2018-01-01

Speakers track the probability that a word will occur in a particular context and utilize this information during phonetic processing. For example, content words that have high probability within a discourse tend to be realized with reduced acoustic/articulatory properties. Such probabilistic information may influence L1 and L2 speech processing in distinct ways (reflecting differences in linguistic experience across groups and the overall difficulty of L2 speech processing). To examine this issue, L1 and L2 speakers performed a referential communication task, describing sequences of simple actions. The two groups of speakers showed similar effects of discourse-dependent probabilistic information on production, suggesting that L2 speakers can successfully track discourse-dependent probabilities and use such information to modulate phonetic processing.

Risk taking and adult attention deficit/hyperactivity disorder: A gap between real life behavior and experimental decision making.

PubMed

Pollak, Yehuda; Shalit, Reut; Aran, Adi

2018-01-01

Adults with attention deficit/hyperactivity disorder (ADHD) are prone to suboptimal decision making and risk taking. The aim of this study was to test performance on a theoretically-based probabilistic decision making task in well-characterized adults with and without ADHD, and examine the relation between experimental risk taking and history of real-life risk-taking behavior, defined as cigarette, alcohol, and street drug use. University students with and without ADHD completed a modified version of the Cambridge Gambling Test, in which they had to choose between alternatives varied by level of risk, and reported their history of substance use. Both groups showed similar patterns of risk taking on the experimental decision making task, suggesting that ADHD is not linked to low sensitivity to risk. Past and present substance use was more prevalent in adults with ADHD. These finding question the validity of experimental probabilistic decision making task as a valid model for ADHD-related risk-taking behavior. Copyright © 2017 Elsevier B.V. All rights reserved.

Agreement With Conjoined NPs Reflects Language Experience.

PubMed

Lorimor, Heidi; Adams, Nora C; Middleton, Erica L

2018-01-01

An important question within psycholinguistic research is whether grammatical features, such as number values on nouns, are probabilistic or discrete. Similarly, researchers have debated whether grammatical specifications are only set for individual lexical items, or whether certain types of noun phrases (NPs) also obtain number valuations at the phrasal level. Through a corpus analysis and an oral production task, we show that conjoined NPs can take both singular and plural verb agreement and that notional number (i.e., the numerosity of the referent of the subject noun phrase) plays an important role in agreement with conjoined NPs. In two written production tasks, we show that participants who are exposed to plural (versus singular or unmarked) agreement with conjoined NPs in a biasing story are more likely to produce plural agreement with conjoined NPs on a subsequent production task. This suggests that, in addition to their sensitivity to notional information, conjoined NPs have probabilistic grammatical specifications that reflect their distributional properties in language. These results provide important evidence that grammatical number reflects language experience, and that this language experience impacts agreement at the phrasal level, and not just the lexical level.

Agreement With Conjoined NPs Reflects Language Experience

PubMed Central

Lorimor, Heidi; Adams, Nora C.; Middleton, Erica L.

2018-01-01

An important question within psycholinguistic research is whether grammatical features, such as number values on nouns, are probabilistic or discrete. Similarly, researchers have debated whether grammatical specifications are only set for individual lexical items, or whether certain types of noun phrases (NPs) also obtain number valuations at the phrasal level. Through a corpus analysis and an oral production task, we show that conjoined NPs can take both singular and plural verb agreement and that notional number (i.e., the numerosity of the referent of the subject noun phrase) plays an important role in agreement with conjoined NPs. In two written production tasks, we show that participants who are exposed to plural (versus singular or unmarked) agreement with conjoined NPs in a biasing story are more likely to produce plural agreement with conjoined NPs on a subsequent production task. This suggests that, in addition to their sensitivity to notional information, conjoined NPs have probabilistic grammatical specifications that reflect their distributional properties in language. These results provide important evidence that grammatical number reflects language experience, and that this language experience impacts agreement at the phrasal level, and not just the lexical level. PMID:29725311

SVM2Motif—Reconstructing Overlapping DNA Sequence Motifs by Mimicking an SVM Predictor

PubMed Central

Vidovic, Marina M. -C.; Görnitz, Nico; Müller, Klaus-Robert; Rätsch, Gunnar; Kloft, Marius

2015-01-01

Identifying discriminative motifs underlying the functionality and evolution of organisms is a major challenge in computational biology. Machine learning approaches such as support vector machines (SVMs) achieve state-of-the-art performances in genomic discrimination tasks, but—due to its black-box character—motifs underlying its decision function are largely unknown. As a remedy, positional oligomer importance matrices (POIMs) allow us to visualize the significance of position-specific subsequences. Although being a major step towards the explanation of trained SVM models, they suffer from the fact that their size grows exponentially in the length of the motif, which renders their manual inspection feasible only for comparably small motif sizes, typically k ≤ 5. In this work, we extend the work on positional oligomer importance matrices, by presenting a new machine-learning methodology, entitled motifPOIM, to extract the truly relevant motifs—regardless of their length and complexity—underlying the predictions of a trained SVM model. Our framework thereby considers the motifs as free parameters in a probabilistic model, a task which can be phrased as a non-convex optimization problem. The exponential dependence of the POIM size on the oligomer length poses a major numerical challenge, which we address by an efficient optimization framework that allows us to find possibly overlapping motifs consisting of up to hundreds of nucleotides. We demonstrate the efficacy of our approach on a synthetic data set as well as a real-world human splice site data set. PMID:26690911

Effects of additional data on Bayesian clustering.

PubMed

Yamazaki, Keisuke

2017-10-01

Hierarchical probabilistic models, such as mixture models, are used for cluster analysis. These models have two types of variables: observable and latent. In cluster analysis, the latent variable is estimated, and it is expected that additional information will improve the accuracy of the estimation of the latent variable. Many proposed learning methods are able to use additional data; these include semi-supervised learning and transfer learning. However, from a statistical point of view, a complex probabilistic model that encompasses both the initial and additional data might be less accurate due to having a higher-dimensional parameter. The present paper presents a theoretical analysis of the accuracy of such a model and clarifies which factor has the greatest effect on its accuracy, the advantages of obtaining additional data, and the disadvantages of increasing the complexity. Copyright © 2017 Elsevier Ltd. All rights reserved.

Instructional control of reinforcement learning: A behavioral and neurocomputational investigation

PubMed Central

Doll, Bradley B.; Jacobs, W. Jake; Sanfey, Alan G.; Frank, Michael J.

2011-01-01

Humans learn how to behave directly through environmental experience and indirectly through rules and instructions. Behavior analytic research has shown that instructions can control behavior, even when such behavior leads to sub-optimal outcomes (Hayes, S. (Ed.). 1989. Rule-governed behavior: cognition, contingencies, and instructional control. Plenum Press.). Here we examine the control of behavior through instructions in a reinforcement learning task known to depend on striatal dopaminergic function. Participants selected between probabilistically reinforced stimuli, and were (incorrectly) told that a specific stimulus had the highest (or lowest) reinforcement probability. Despite experience to the contrary, instructions drove choice behavior. We present neural network simulations that capture the interactions between instruction-driven and reinforcement-driven behavior via two potential neural circuits: one in which the striatum is inaccurately trained by instruction representations coming from prefrontal cortex/hippocampus (PFC/HC), and another in which the striatum learns the environmentally based reinforcement contingencies, but is “overridden” at decision output. Both models capture the core behavioral phenomena but, because they differ fundamentally on what is learned, make distinct predictions for subsequent behavioral and neuroimaging experiments. Finally, we attempt to distinguish between the proposed computational mechanisms governing instructed behavior by fitting a series of abstract “Q-learning” and Bayesian models to subject data. The best-fitting model supports one of the neural models, suggesting the existence of a “confirmation bias” in which the PFC/HC system trains the reinforcement system by amplifying outcomes that are consistent with instructions while diminishing inconsistent outcomes. PMID:19595993

Online probabilistic learning with an ensemble of forecasts

NASA Astrophysics Data System (ADS)

Thorey, Jean; Mallet, Vivien; Chaussin, Christophe

2016-04-01

Our objective is to produce a calibrated weighted ensemble to forecast a univariate time series. In addition to a meteorological ensemble of forecasts, we rely on observations or analyses of the target variable. The celebrated Continuous Ranked Probability Score (CRPS) is used to evaluate the probabilistic forecasts. However applying the CRPS on weighted empirical distribution functions (deriving from the weighted ensemble) may introduce a bias because of which minimizing the CRPS does not produce the optimal weights. Thus we propose an unbiased version of the CRPS which relies on clusters of members and is strictly proper. We adapt online learning methods for the minimization of the CRPS. These methods generate the weights associated to the members in the forecasted empirical distribution function. The weights are updated before each forecast step using only past observations and forecasts. Our learning algorithms provide the theoretical guarantee that, in the long run, the CRPS of the weighted forecasts is at least as good as the CRPS of any weighted ensemble with weights constant in time. In particular, the performance of our forecast is better than that of any subset ensemble with uniform weights. A noteworthy advantage of our algorithm is that it does not require any assumption on the distributions of the observations and forecasts, both for the application and for the theoretical guarantee to hold. As application example on meteorological forecasts for photovoltaic production integration, we show that our algorithm generates a calibrated probabilistic forecast, with significant performance improvements on probabilistic diagnostic tools (the CRPS, the reliability diagram and the rank histogram).

Cardinal rules: Visual orientation perception reflects knowledge of environmental statistics

PubMed Central

Girshick, Ahna R.; Landy, Michael S.; Simoncelli, Eero P.

2011-01-01

Humans are remarkably good at performing visual tasks, but experimental measurements reveal substantial biases in the perception of basic visual attributes. An appealing hypothesis is that these biases arise through a process of statistical inference, in which information from noisy measurements is fused with a probabilistic model of the environment. But such inference is optimal only if the observer’s internal model matches the environment. Here, we provide evidence that this is the case. We measured performance in an orientation-estimation task, demonstrating the well-known fact that orientation judgements are more accurate at cardinal (horizontal and vertical) orientations, along with a new observation that judgements made under conditions of uncertainty are strongly biased toward cardinal orientations. We estimate observers’ internal models for orientation and find that they match the local orientation distribution measured in photographs. We also show how a neural population could embed probabilistic information responsible for such biases. PMID:21642976

Learning a Generative Probabilistic Grammar of Experience: A Process-Level Model of Language Acquisition

ERIC Educational Resources Information Center

Kolodny, Oren; Lotem, Arnon; Edelman, Shimon

2015-01-01

We introduce a set of biologically and computationally motivated design choices for modeling the learning of language, or of other types of sequential, hierarchically structured experience and behavior, and describe an implemented system that conforms to these choices and is capable of unsupervised learning from raw natural-language corpora. Given…

COMPOSER: A Probabilistic Solution to the Utility Problem in Speed-up Learning.

ERIC Educational Resources Information Center

Gratch, Jonathan; DeJong, Gerald

In machine learning there is considerable interest in techniques which improve planning ability. Initial investigations have identified a wide variety of techniques to address this issue. Progress has been hampered by the utility problem, a basic tradeoff between the benefit of learned knowledge and the cost to locate and apply relevant knowledge.…

Learning about Probability from Text and Tables: Do Color Coding and Labeling through an Interactive-User Interface Help?

ERIC Educational Resources Information Center

Clinton, Virginia; Morsanyi, Kinga; Alibali, Martha W.; Nathan, Mitchell J.

2016-01-01

Learning from visual representations is enhanced when learners appropriately integrate corresponding visual and verbal information. This study examined the effects of two methods of promoting integration, color coding and labeling, on learning about probabilistic reasoning from a table and text. Undergraduate students (N = 98) were randomly…

Acute stress selectively reduces reward sensitivity

PubMed Central

Berghorst, Lisa H.; Bogdan, Ryan; Frank, Michael J.; Pizzagalli, Diego A.

2013-01-01

Stress may promote the onset of psychopathology by disrupting reward processing. However, the extent to which stress impairs reward processing, rather than incentive processing more generally, is unclear. To evaluate the specificity of stress-induced reward processing disruption, 100 psychiatrically healthy females were administered a probabilistic stimulus selection task (PSST) that enabled comparison of sensitivity to reward-driven (Go) and punishment-driven (NoGo) learning under either “no stress” or “stress” (threat-of-shock) conditions. Cortisol samples and self-report measures were collected. Contrary to hypotheses, the groups did not differ significantly in task performance or cortisol reactivity. However, further analyses focusing only on individuals under “stress” who were high responders with regard to both cortisol reactivity and self-reported negative affect revealed reduced reward sensitivity relative to individuals tested in the “no stress” condition; importantly, these deficits were reward-specific. Overall, findings provide preliminary evidence that stress-reactive individuals show diminished sensitivity to reward, but not punishment, under stress. While such results highlight the possibility that stress-induced anhedonia might be an important mechanism linking stress to affective disorders, future studies are necessary to confirm this conjecture. PMID:23596406

Combining non selective gas sensors on a mobile robot for identification and mapping of multiple chemical compounds.

PubMed

Bennetts, Victor Hernandez; Schaffernicht, Erik; Pomareda, Victor; Lilienthal, Achim J; Marco, Santiago; Trincavelli, Marco

2014-09-17

In this paper, we address the task of gas distribution modeling in scenarios where multiple heterogeneous compounds are present. Gas distribution modeling is particularly useful in emission monitoring applications where spatial representations of the gaseous patches can be used to identify emission hot spots. In realistic environments, the presence of multiple chemicals is expected and therefore, gas discrimination has to be incorporated in the modeling process. The approach presented in this work addresses the task of gas distribution modeling by combining different non selective gas sensors. Gas discrimination is addressed with an open sampling system, composed by an array of metal oxide sensors and a probabilistic algorithm tailored to uncontrolled environments. For each of the identified compounds, the mapping algorithm generates a calibrated gas distribution model using the classification uncertainty and the concentration readings acquired with a photo ionization detector. The meta parameters of the proposed modeling algorithm are automatically learned from the data. The approach was validated with a gas sensitive robot patrolling outdoor and indoor scenarios, where two different chemicals were released simultaneously. The experimental results show that the generated multi compound maps can be used to accurately predict the location of emitting gas sources.

Improved detection of chemical substances from colorimetric sensor data using probabilistic machine learning

NASA Astrophysics Data System (ADS)

Mølgaard, Lasse L.; Buus, Ole T.; Larsen, Jan; Babamoradi, Hamid; Thygesen, Ida L.; Laustsen, Milan; Munk, Jens Kristian; Dossi, Eleftheria; O'Keeffe, Caroline; Lässig, Lina; Tatlow, Sol; Sandström, Lars; Jakobsen, Mogens H.

2017-05-01

We present a data-driven machine learning approach to detect drug- and explosives-precursors using colorimetric sensor technology for air-sampling. The sensing technology has been developed in the context of the CRIM-TRACK project. At present a fully- integrated portable prototype for air sampling with disposable sensing chips and automated data acquisition has been developed. The prototype allows for fast, user-friendly sampling, which has made it possible to produce large datasets of colorimetric data for different target analytes in laboratory and simulated real-world application scenarios. To make use of the highly multi-variate data produced from the colorimetric chip a number of machine learning techniques are employed to provide reliable classification of target analytes from confounders found in the air streams. We demonstrate that a data-driven machine learning method using dimensionality reduction in combination with a probabilistic classifier makes it possible to produce informative features and a high detection rate of analytes. Furthermore, the probabilistic machine learning approach provides a means of automatically identifying unreliable measurements that could produce false predictions. The robustness of the colorimetric sensor has been evaluated in a series of experiments focusing on the amphetamine pre-cursor phenylacetone as well as the improvised explosives pre-cursor hydrogen peroxide. The analysis demonstrates that the system is able to detect analytes in clean air and mixed with substances that occur naturally in real-world sampling scenarios. The technology under development in CRIM-TRACK has the potential as an effective tool to control trafficking of illegal drugs, explosive detection, or in other law enforcement applications.

Generalized Anxiety Disorder and Social Anxiety Disorder, but Not Panic Anxiety Disorder, Are Associated with Higher Sensitivity to Learning from Negative Feedback: Behavioral and Computational Investigation

PubMed Central

Khdour, Hussain Y.; Abushalbaq, Oday M.; Mughrabi, Ibrahim T.; Imam, Aya F.; Gluck, Mark A.; Herzallah, Mohammad M.; Moustafa, Ahmed A.

2016-01-01

Anxiety disorders, including generalized anxiety disorder (GAD), social anxiety disorder (SAD), and panic anxiety disorder (PAD), are a group of common psychiatric conditions. They are characterized by excessive worrying, uneasiness, and fear of future events, such that they affect social and occupational functioning. Anxiety disorders can alter behavior and cognition as well, yet little is known about the particular domains they affect. In this study, we tested the cognitive correlates of medication-free patients with GAD, SAD, and PAD, along with matched healthy participants using a probabilistic category-learning task that allows the dissociation between positive and negative feedback learning. We also fitted all participants' data to a Q-learning model and various actor-critic models that examine learning rate parameters from positive and negative feedback to investigate effects of valence vs. action on performance. SAD and GAD patients were more sensitive to negative feedback than either PAD patients or healthy participants. PAD, SAD, and GAD patients did not differ in positive-feedback learning compared to healthy participants. We found that Q-learning models provide the simplest fit of the data in comparison to other models. However, computational analysis revealed that groups did not differ in terms of learning rate or exploration values. These findings argue that (a) not all anxiety spectrum disorders share similar cognitive correlates, but are rather different in ways that do not link them to the hallmark of anxiety (higher sensitivity to negative feedback); and (b) perception of negative consequences is the core feature of GAD and SAD, but not PAD. Further research is needed to examine the similarities and differences between anxiety spectrum disorders in other cognitive domains and potential implementation of behavioral therapy to remediate cognitive deficits. PMID:27445719

Generalized Anxiety Disorder and Social Anxiety Disorder, but Not Panic Anxiety Disorder, Are Associated with Higher Sensitivity to Learning from Negative Feedback: Behavioral and Computational Investigation.

PubMed

Khdour, Hussain Y; Abushalbaq, Oday M; Mughrabi, Ibrahim T; Imam, Aya F; Gluck, Mark A; Herzallah, Mohammad M; Moustafa, Ahmed A

2016-01-01

Anxiety disorders, including generalized anxiety disorder (GAD), social anxiety disorder (SAD), and panic anxiety disorder (PAD), are a group of common psychiatric conditions. They are characterized by excessive worrying, uneasiness, and fear of future events, such that they affect social and occupational functioning. Anxiety disorders can alter behavior and cognition as well, yet little is known about the particular domains they affect. In this study, we tested the cognitive correlates of medication-free patients with GAD, SAD, and PAD, along with matched healthy participants using a probabilistic category-learning task that allows the dissociation between positive and negative feedback learning. We also fitted all participants' data to a Q-learning model and various actor-critic models that examine learning rate parameters from positive and negative feedback to investigate effects of valence vs. action on performance. SAD and GAD patients were more sensitive to negative feedback than either PAD patients or healthy participants. PAD, SAD, and GAD patients did not differ in positive-feedback learning compared to healthy participants. We found that Q-learning models provide the simplest fit of the data in comparison to other models. However, computational analysis revealed that groups did not differ in terms of learning rate or exploration values. These findings argue that (a) not all anxiety spectrum disorders share similar cognitive correlates, but are rather different in ways that do not link them to the hallmark of anxiety (higher sensitivity to negative feedback); and (b) perception of negative consequences is the core feature of GAD and SAD, but not PAD. Further research is needed to examine the similarities and differences between anxiety spectrum disorders in other cognitive domains and potential implementation of behavioral therapy to remediate cognitive deficits.

Decision-making impairments in the context of intact reward sensitivity in schizophrenia.

PubMed

Heerey, Erin A; Bell-Warren, Kimberly R; Gold, James M

2008-07-01

Deficits in motivated behavior and decision-making figure prominently in the behavioral syndrome that characterizes schizophrenia and are difficult both to treat and to understand. One explanation for these deficits is that schizophrenia decreases sensitivity to rewards in the environment. An alternate explanation is that sensitivity to rewards is intact but that poor integration of affective with cognitive information impairs the ability to use this information to guide behavior. We tested reward sensitivity with a modified version of an existing signal detection task with asymmetric reinforcement and decision-making with a probabilistic decision-making task in 40 participants with schizophrenia and 26 healthy participants. Results showed normal sensitivity to reward in participants with schizophrenia but differences in choice patterns on the decision-making task. A logistic regression model of the decision-making data showed that participants with schizophrenia differed from healthy participants in the ability to weigh potential outcomes, specifically potential losses, when choosing between competing response options. Deficits in working memory ability accounted for group differences in ability to use potential outcomes during decision-making. These results suggest that the implicit mechanisms that drive reward-based learning are surprisingly intact in schizophrenia but that poor ability to integrate cognitive and affective information when calculating the value of possible choices might hamper the ability to use such information during explicit decision-making.

Iowa Gambling Task (IGT): twenty years after – gambling disorder and IGT

PubMed Central

Brevers, Damien; Bechara, Antoine; Cleeremans, Axel; Noël, Xavier

2013-01-01

The Iowa Gambling Task (IGT) involves probabilistic learning via monetary rewards and punishments, where advantageous task performance requires subjects to forego potential large immediate rewards for small longer-term rewards to avoid larger losses. Pathological gamblers (PG) perform worse on the IGT compared to controls, relating to their persistent preference toward high, immediate, and uncertain rewards despite experiencing larger losses. In this contribution, we review studies that investigated processes associated with poor IGT performance in PG. Findings from these studies seem to fit with recent neurocognitive models of addiction, which argue that the diminished ability of addicted individuals to ponder short-term against long-term consequences of a choice may be the product of an hyperactive automatic attentional and memory system for signaling the presence of addiction-related cues (e.g., high uncertain rewards associated with disadvantageous decks selection during the IGT) and for attributing to such cues pleasure and excitement. This incentive-salience associated with gambling-related choice in PG may be so high that it could literally “hijack” resources [“hot” executive functions (EFs)] involved in emotional self-regulation and necessary to allow the enactment of further elaborate decontextualized problem-solving abilities (“cool” EFs). A framework for future research is also proposed, which highlights the need for studies examining how these processes contribute specifically to the aberrant choice profile displayed by PG on the IGT. PMID:24137138

Hot and cold cognition in unmedicated depressed subjects with bipolar disorder.

PubMed

Roiser, Jonathan P; Cannon, Dara M; Gandhi, Shilpa K; Taylor Tavares, Joana; Erickson, Kristine; Wood, Suzanne; Klaver, Jacqueline M; Clark, Luke; Zarate, Carlos A; Sahakian, Barbara J; Drevets, Wayne C

2009-03-01

Neuropsychological studies in subjects with bipolar disorder (BD) have reported deficits on a variety of cognitive measures. However, because the majority of subjects were medicated at the time of testing in previous studies, it is currently unclear whether the pattern of deficits reported is related to BD itself or to psychotropic medication. We addressed this issue by examining cognitive performance in a group of unmedicated, currently depressed subjects with BD. Forty-nine unmedicated subjects who met DSM-IV criteria for BD, depressed phase, and 55 control subjects participated in this study. Most patients were diagnosed with bipolar II disorder. Performance on emotion-dependent, or 'hot', and emotion-independent, or 'cold', cognitive tasks was assessed using tests from the Cambridge Neuropsychological Test Automated Battery. The groups were well matched with respect to general intelligence and demographic variables. Deficits in the unmedicated depressed BD group were apparent on tests tapping 'hot' cognitive processing, for example the Cambridge Gamble task and the Probabilistic Reversal Learning task. However, other than a deficit on the Spatial Span test in the depressed BD subjects, the groups performed equivalently on most measures of 'cold' cognitive processing, for example visual memory, attention, and working memory. These data suggest that deficits on tests involving reward processing, short-term spatial memory storage, and sensitivity to negative feedback in depressed BD subjects represent an effect of the illness itself and not mood-stabilizing medication.

The Role of Informative and Ambiguous Feedback in Avoidance Behavior: Empirical and Computational Findings

PubMed Central

Moustafa, Ahmed A.; Sheynin, Jony; Myers, Catherine E.

2015-01-01

Avoidance behavior is a critical component of many psychiatric disorders, and as such, it is important to understand how avoidance behavior arises, and whether it can be modified. In this study, we used empirical and computational methods to assess the role of informational feedback and ambiguous outcome in avoidance behavior. We adapted a computer-based probabilistic classification learning task, which includes positive, negative and no-feedback outcomes; the latter outcome is ambiguous as it might signal either a successful outcome (missed punishment) or a failure (missed reward). Prior work with this task suggested that most healthy subjects viewed the no-feedback outcome as strongly positive. Interestingly, in a later version of the classification task, when healthy subjects were allowed to opt out of (i.e. avoid) responding, some subjects (“avoiders”) reliably avoided trials where there was a risk of punishment, but other subjects (“non-avoiders”) never made any avoidance responses at all. One possible interpretation is that the “non-avoiders” valued the no-feedback outcome so positively on punishment-based trials that they had little incentive to avoid. Another possible interpretation is that the outcome of an avoided trial is unspecified and that lack of information is aversive, decreasing subjects’ tendency to avoid. To examine these ideas, we here tested healthy young adults on versions of the task where avoidance responses either did or did not generate informational feedback about the optimal response. Results showed that provision of informational feedback decreased avoidance responses and also decreased categorization performance, without significantly affecting the percentage of subjects classified as “avoiders.” To better understand these results, we used a modified Q-learning model to fit individual subject data. Simulation results suggest that subjects in the feedback condition adjusted their behavior faster following better-than-expected outcomes, compared to subjects in the no-feedback condition. Additionally, in both task conditions, “avoiders” adjusted their behavior faster following worse-than-expected outcomes, and treated the ambiguous no-feedback outcome as less rewarding, compared to non-avoiders. Together, results shed light on the important role of ambiguous and informative feedback in avoidance behavior. PMID:26630279

Imaging and machine learning techniques for diagnosis of Alzheimer's disease.

PubMed

Mirzaei, Golrokh; Adeli, Anahita; Adeli, Hojjat

2016-12-01

Alzheimer's disease (AD) is a common health problem in elderly people. There has been considerable research toward the diagnosis and early detection of this disease in the past decade. The sensitivity of biomarkers and the accuracy of the detection techniques have been defined to be the key to an accurate diagnosis. This paper presents a state-of-the-art review of the research performed on the diagnosis of AD based on imaging and machine learning techniques. Different segmentation and machine learning techniques used for the diagnosis of AD are reviewed including thresholding, supervised and unsupervised learning, probabilistic techniques, Atlas-based approaches, and fusion of different image modalities. More recent and powerful classification techniques such as the enhanced probabilistic neural network of Ahmadlou and Adeli should be investigated with the goal of improving the diagnosis accuracy. A combination of different image modalities can help improve the diagnosis accuracy rate. Research is needed on the combination of modalities to discover multi-modal biomarkers.

Individual differences in implicit motor learning: task specificity in sensorimotor adaptation and sequence learning

PubMed Central

Raza, Meher; Ivry, Richard B.

2016-01-01

In standard taxonomies, motor skills are typically treated as representative of implicit or procedural memory. We examined two emblematic tasks of implicit motor learning, sensorimotor adaptation and sequence learning, asking whether individual differences in learning are correlated between these tasks, as well as how individual differences within each task are related to different performance variables. As a prerequisite, it was essential to establish the reliability of learning measures for each task. Participants were tested twice on a visuomotor adaptation task and on a sequence learning task, either the serial reaction time task or the alternating reaction time task. Learning was evident in all tasks at the group level and reliable at the individual level in visuomotor adaptation and the alternating reaction time task but not in the serial reaction time task. Performance variability was predictive of learning in both domains, yet the relationship was in the opposite direction for adaptation and sequence learning. For the former, faster learning was associated with lower variability, consistent with models of sensorimotor adaptation in which learning rates are sensitive to noise. For the latter, greater learning was associated with higher variability and slower reaction times, factors that may facilitate the spread of activation required to form predictive, sequential associations. Interestingly, learning measures of the different tasks were not correlated. Together, these results oppose a shared process for implicit learning in sensorimotor adaptation and sequence learning and provide insight into the factors that account for individual differences in learning within each task domain. NEW & NOTEWORTHY We investigated individual differences in the ability to implicitly learn motor skills. As a prerequisite, we assessed whether individual differences were reliable across test sessions. We found that two commonly used tasks of implicit learning, visuomotor adaptation and the alternating serial reaction time task, exhibited good test-retest reliability in measures of learning and performance. However, the learning measures did not correlate between the two tasks, arguing against a shared process for implicit motor learning. PMID:27832611

Modeling the Evolution of Beliefs Using an Attentional Focus Mechanism

PubMed Central

Marković, Dimitrije; Gläscher, Jan; Bossaerts, Peter; O’Doherty, John; Kiebel, Stefan J.

2015-01-01

For making decisions in everyday life we often have first to infer the set of environmental features that are relevant for the current task. Here we investigated the computational mechanisms underlying the evolution of beliefs about the relevance of environmental features in a dynamical and noisy environment. For this purpose we designed a probabilistic Wisconsin card sorting task (WCST) with belief solicitation, in which subjects were presented with stimuli composed of multiple visual features. At each moment in time a particular feature was relevant for obtaining reward, and participants had to infer which feature was relevant and report their beliefs accordingly. To test the hypothesis that attentional focus modulates the belief update process, we derived and fitted several probabilistic and non-probabilistic behavioral models, which either incorporate a dynamical model of attentional focus, in the form of a hierarchical winner-take-all neuronal network, or a diffusive model, without attention-like features. We used Bayesian model selection to identify the most likely generative model of subjects’ behavior and found that attention-like features in the behavioral model are essential for explaining subjects’ responses. Furthermore, we demonstrate a method for integrating both connectionist and Bayesian models of decision making within a single framework that allowed us to infer hidden belief processes of human subjects. PMID:26495984

Functional mechanisms of probabilistic inference in feature- and space-based attentional systems.

PubMed

Dombert, Pascasie L; Kuhns, Anna; Mengotti, Paola; Fink, Gereon R; Vossel, Simone

2016-11-15

Humans flexibly attend to features or locations and these processes are influenced by the probability of sensory events. We combined computational modeling of response times with fMRI to compare the functional correlates of (re-)orienting, and the modulation by probabilistic inference in spatial and feature-based attention systems. Twenty-four volunteers performed two task versions with spatial or color cues. Percentage of cue validity changed unpredictably. A hierarchical Bayesian model was used to derive trial-wise estimates of probability-dependent attention, entering the fMRI analysis as parametric regressors. Attentional orienting activated a dorsal frontoparietal network in both tasks, without significant parametric modulation. Spatially invalid trials activated a bilateral frontoparietal network and the precuneus, while invalid feature trials activated the left intraparietal sulcus (IPS). Probability-dependent attention modulated activity in the precuneus, left posterior IPS, middle occipital gyrus, and right temporoparietal junction for spatial attention, and in the left anterior IPS for feature-based and spatial attention. These findings provide novel insights into the generality and specificity of the functional basis of attentional control. They suggest that probabilistic inference can distinctively affect each attentional subsystem, but that there is an overlap in the left IPS, which responds to both spatial and feature-based expectancy violations. Copyright © 2016 Elsevier Inc. All rights reserved.

When knowledge activated from memory intrudes on probabilistic inferences from description - the case of stereotypes.

PubMed

Dorrough, Angela R; Glöckner, Andreas; Betsch, Tilmann; Wille, Anika

2017-10-01

To make decisions in probabilistic inference tasks, individuals integrate relevant information partly in an automatic manner. Thereby, potentially irrelevant stimuli that are additionally presented can intrude on the decision process (e.g., Söllner, Bröder, Glöckner, & Betsch, 2014). We investigate whether such an intrusion effect can also be caused by potentially irrelevant or even misleading knowledge activated from memory. In four studies that combine a standard information board paradigm from decision research with a standard manipulation from social psychology, we investigate the case of stereotypes and demonstrate that stereotype knowledge can yield intrusion biases in probabilistic inferences from description. The magnitude of these biases increases with stereotype accessibility and decreases with a clarification of the rational solution. Copyright © 2017 Elsevier B.V. All rights reserved.

Abstract quantum computing machines and quantum computational logics

NASA Astrophysics Data System (ADS)

Chiara, Maria Luisa Dalla; Giuntini, Roberto; Sergioli, Giuseppe; Leporini, Roberto

2016-06-01

Classical and quantum parallelism are deeply different, although it is sometimes claimed that quantum Turing machines are nothing but special examples of classical probabilistic machines. We introduce the concepts of deterministic state machine, classical probabilistic state machine and quantum state machine. On this basis, we discuss the question: To what extent can quantum state machines be simulated by classical probabilistic state machines? Each state machine is devoted to a single task determined by its program. Real computers, however, behave differently, being able to solve different kinds of problems. This capacity can be modeled, in the quantum case, by the mathematical notion of abstract quantum computing machine, whose different programs determine different quantum state machines. The computations of abstract quantum computing machines can be linguistically described by the formulas of a particular form of quantum logic, termed quantum computational logic.

Probabilistic Classification Learning with Corrective Feedback Is Selectively Impaired in Early Huntington's Disease--Evidence for the Role of the Striatum in Learning with Feedback

ERIC Educational Resources Information Center

Holl, Anna K.; Wilkinson, Leonora; Tabrizi, Sarah J.; Painold, Annamaria; Jahanshahi, Marjan

2012-01-01

In general, declarative learning is associated with the activation of the medial temporal lobes (MTL), while the basal ganglia (BG) are considered the substrate for procedural learning. More recently it has been demonstrated the distinction of these systems may not be as absolute as previously thought and that not only the explicit or implicit…

Probabilistic graphs as a conceptual and computational tool in hydrology and water management

NASA Astrophysics Data System (ADS)

Schoups, Gerrit

2014-05-01

Originally developed in the fields of machine learning and artificial intelligence, probabilistic graphs constitute a general framework for modeling complex systems in the presence of uncertainty. The framework consists of three components: 1. Representation of the model as a graph (or network), with nodes depicting random variables in the model (e.g. parameters, states, etc), which are joined together by factors. Factors are local probabilistic or deterministic relations between subsets of variables, which, when multiplied together, yield the joint distribution over all variables. 2. Consistent use of probability theory for quantifying uncertainty, relying on basic rules of probability for assimilating data into the model and expressing unknown variables as a function of observations (via the posterior distribution). 3. Efficient, distributed approximation of the posterior distribution using general-purpose algorithms that exploit model structure encoded in the graph. These attributes make probabilistic graphs potentially useful as a conceptual and computational tool in hydrology and water management (and beyond). Conceptually, they can provide a common framework for existing and new probabilistic modeling approaches (e.g. by drawing inspiration from other fields of application), while computationally they can make probabilistic inference feasible in larger hydrological models. The presentation explores, via examples, some of these benefits.

Modeling Spanish Mood Choice in Belief Statements

ERIC Educational Resources Information Center

Robinson, Jason R.

2013-01-01

This work develops a computational methodology new to linguistics that empirically evaluates competing linguistic theories on Spanish verbal mood choice through the use of computational techniques to learn mood and other hidden linguistic features from Spanish belief statements found in corpora. The machine learned probabilistic linguistic models…

Learning redundant motor tasks with and without overlapping dimensions: facilitation and interference effects.

PubMed

Ranganathan, Rajiv; Wieser, Jon; Mosier, Kristine M; Mussa-Ivaldi, Ferdinando A; Scheidt, Robert A

2014-06-11

Prior learning of a motor skill creates motor memories that can facilitate or interfere with learning of new, but related, motor skills. One hypothesis of motor learning posits that for a sensorimotor task with redundant degrees of freedom, the nervous system learns the geometric structure of the task and improves performance by selectively operating within that task space. We tested this hypothesis by examining if transfer of learning between two tasks depends on shared dimensionality between their respective task spaces. Human participants wore a data glove and learned to manipulate a computer cursor by moving their fingers. Separate groups of participants learned two tasks: a prior task that was unique to each group and a criterion task that was common to all groups. We manipulated the mapping between finger motions and cursor positions in the prior task to define task spaces that either shared or did not share the task space dimensions (x-y axes) of the criterion task. We found that if the prior task shared task dimensions with the criterion task, there was an initial facilitation in criterion task performance. However, if the prior task did not share task dimensions with the criterion task, there was prolonged interference in learning the criterion task due to participants finding inefficient task solutions. These results show that the nervous system learns the task space through practice, and that the degree of shared task space dimensionality influences the extent to which prior experience transfers to subsequent learning of related motor skills. Copyright © 2014 the authors 0270-6474/14/348289-11$15.00/0.

Dopaminergic and Prefrontal Contributions to Reward-Based Learning and Outcome Monitoring during Child Development and Aging

ERIC Educational Resources Information Center

Hammerer, Dorothea; Eppinger, Ben

2012-01-01

In many instances, children and older adults show similar difficulties in reward-based learning and outcome monitoring. These impairments are most pronounced in situations in which reward is uncertain (e.g., probabilistic reward schedules) and if outcome information is ambiguous (e.g., the relative value of outcomes has to be learned).…

Reconstructing constructivism: Causal models, Bayesian learning mechanisms and the theory theory

PubMed Central

Gopnik, Alison; Wellman, Henry M.

2012-01-01

We propose a new version of the “theory theory” grounded in the computational framework of probabilistic causal models and Bayesian learning. Probabilistic models allow a constructivist but rigorous and detailed approach to cognitive development. They also explain the learning of both more specific causal hypotheses and more abstract framework theories. We outline the new theoretical ideas, explain the computational framework in an intuitive and non-technical way, and review an extensive but relatively recent body of empirical results that supports these ideas. These include new studies of the mechanisms of learning. Children infer causal structure from statistical information, through their own actions on the world and through observations of the actions of others. Studies demonstrate these learning mechanisms in children from 16 months to 4 years old and include research on causal statistical learning, informal experimentation through play, and imitation and informal pedagogy. They also include studies of the variability and progressive character of intuitive theory change, particularly theory of mind. These studies investigate both the physical and psychological and social domains. We conclude with suggestions for further collaborative projects between developmental and computational cognitive scientists. PMID:22582739

Quantification and Segmentation of Brain Tissues from MR Images: A Probabilistic Neural Network Approach

PubMed Central

Wang, Yue; Adalý, Tülay; Kung, Sun-Yuan; Szabo, Zsolt

2007-01-01

This paper presents a probabilistic neural network based technique for unsupervised quantification and segmentation of brain tissues from magnetic resonance images. It is shown that this problem can be solved by distribution learning and relaxation labeling, resulting in an efficient method that may be particularly useful in quantifying and segmenting abnormal brain tissues where the number of tissue types is unknown and the distributions of tissue types heavily overlap. The new technique uses suitable statistical models for both the pixel and context images and formulates the problem in terms of model-histogram fitting and global consistency labeling. The quantification is achieved by probabilistic self-organizing mixtures and the segmentation by a probabilistic constraint relaxation network. The experimental results show the efficient and robust performance of the new algorithm and that it outperforms the conventional classification based approaches. PMID:18172510

Individual differences in implicit motor learning: task specificity in sensorimotor adaptation and sequence learning.

PubMed

Stark-Inbar, Alit; Raza, Meher; Taylor, Jordan A; Ivry, Richard B

2017-01-01

In standard taxonomies, motor skills are typically treated as representative of implicit or procedural memory. We examined two emblematic tasks of implicit motor learning, sensorimotor adaptation and sequence learning, asking whether individual differences in learning are correlated between these tasks, as well as how individual differences within each task are related to different performance variables. As a prerequisite, it was essential to establish the reliability of learning measures for each task. Participants were tested twice on a visuomotor adaptation task and on a sequence learning task, either the serial reaction time task or the alternating reaction time task. Learning was evident in all tasks at the group level and reliable at the individual level in visuomotor adaptation and the alternating reaction time task but not in the serial reaction time task. Performance variability was predictive of learning in both domains, yet the relationship was in the opposite direction for adaptation and sequence learning. For the former, faster learning was associated with lower variability, consistent with models of sensorimotor adaptation in which learning rates are sensitive to noise. For the latter, greater learning was associated with higher variability and slower reaction times, factors that may facilitate the spread of activation required to form predictive, sequential associations. Interestingly, learning measures of the different tasks were not correlated. Together, these results oppose a shared process for implicit learning in sensorimotor adaptation and sequence learning and provide insight into the factors that account for individual differences in learning within each task domain. We investigated individual differences in the ability to implicitly learn motor skills. As a prerequisite, we assessed whether individual differences were reliable across test sessions. We found that two commonly used tasks of implicit learning, visuomotor adaptation and the alternating serial reaction time task, exhibited good test-retest reliability in measures of learning and performance. However, the learning measures did not correlate between the two tasks, arguing against a shared process for implicit motor learning. Copyright © 2017 the American Physiological Society.

Précis of bayesian rationality: The probabilistic approach to human reasoning.

PubMed

Oaksford, Mike; Chater, Nick

2009-02-01

According to Aristotle, humans are the rational animal. The borderline between rationality and irrationality is fundamental to many aspects of human life including the law, mental health, and language interpretation. But what is it to be rational? One answer, deeply embedded in the Western intellectual tradition since ancient Greece, is that rationality concerns reasoning according to the rules of logic--the formal theory that specifies the inferential connections that hold with certainty between propositions. Piaget viewed logical reasoning as defining the end-point of cognitive development; and contemporary psychology of reasoning has focussed on comparing human reasoning against logical standards. Bayesian Rationality argues that rationality is defined instead by the ability to reason about uncertainty. Although people are typically poor at numerical reasoning about probability, human thought is sensitive to subtle patterns of qualitative Bayesian, probabilistic reasoning. In Chapters 1-4 of Bayesian Rationality (Oaksford & Chater 2007), the case is made that cognition in general, and human everyday reasoning in particular, is best viewed as solving probabilistic, rather than logical, inference problems. In Chapters 5-7 the psychology of "deductive" reasoning is tackled head-on: It is argued that purportedly "logical" reasoning problems, revealing apparently irrational behaviour, are better understood from a probabilistic point of view. Data from conditional reasoning, Wason's selection task, and syllogistic inference are captured by recasting these problems probabilistically. The probabilistic approach makes a variety of novel predictions which have been experimentally confirmed. The book considers the implications of this work, and the wider "probabilistic turn" in cognitive science and artificial intelligence, for understanding human rationality.

Effective Learning of Probabilistic Models for Clinical Predictions from Longitudinal Data

ERIC Educational Resources Information Center

Yang, Shuo

2017-01-01

With the expeditious advancement of information technologies, health-related data presented unprecedented potentials for medical and health discoveries but at the same time significant challenges for machine learning techniques both in terms of size and complexity. Those challenges include: the structured data with various storage formats and…

Mathematical Models of Elementary Mathematics Learning and Performance. Final Report.

ERIC Educational Resources Information Center

Suppes, Patrick

This project was concerned with the development of mathematical models of elementary mathematics learning and performance. Probabilistic finite automata and register machines with a finite number of registers were developed as models and extensively tested with data arising from the elementary-mathematics strand curriculum developed by the…

Perspectives of Probabilistic Inferences: Reinforcement Learning and an Adaptive Network Compared

ERIC Educational Resources Information Center

Rieskamp, Jorg

2006-01-01

The assumption that people possess a strategy repertoire for inferences has been raised repeatedly. The strategy selection learning theory specifies how people select strategies from this repertoire. The theory assumes that individuals select strategies proportional to their subjective expectations of how well the strategies solve particular…

Comment Data Mining to Estimate Student Performance Considering Consecutive Lessons

ERIC Educational Resources Information Center

Sorour, Shaymaa E.; Goda, Kazumasa; Mine, Tsunenori

2017-01-01

The purpose of this study is to examine different formats of comment data to predict student performance. Having students write comment data after every lesson can reflect students' learning attitudes, tendencies and learning activities involved with the lesson. In this research, Latent Dirichlet Allocation (LDA) and Probabilistic Latent Semantic…

Real-time probabilistic covariance tracking with efficient model update.

PubMed

Wu, Yi; Cheng, Jian; Wang, Jinqiao; Lu, Hanqing; Wang, Jun; Ling, Haibin; Blasch, Erik; Bai, Li

2012-05-01

The recently proposed covariance region descriptor has been proven robust and versatile for a modest computational cost. The covariance matrix enables efficient fusion of different types of features, where the spatial and statistical properties, as well as their correlation, are characterized. The similarity between two covariance descriptors is measured on Riemannian manifolds. Based on the same metric but with a probabilistic framework, we propose a novel tracking approach on Riemannian manifolds with a novel incremental covariance tensor learning (ICTL). To address the appearance variations, ICTL incrementally learns a low-dimensional covariance tensor representation and efficiently adapts online to appearance changes of the target with only O(1) computational complexity, resulting in a real-time performance. The covariance-based representation and the ICTL are then combined with the particle filter framework to allow better handling of background clutter, as well as the temporary occlusions. We test the proposed probabilistic ICTL tracker on numerous benchmark sequences involving different types of challenges including occlusions and variations in illumination, scale, and pose. The proposed approach demonstrates excellent real-time performance, both qualitatively and quantitatively, in comparison with several previously proposed trackers.

Finding models to detect Alzheimer's disease by fusing structural and neuropsychological information

NASA Astrophysics Data System (ADS)

Giraldo, Diana L.; García-Arteaga, Juan D.; Velasco, Nelson; Romero, Eduardo

2015-12-01

Alzheimer's disease (AD) is a neurodegenerative disease that affects higher brain functions. Initial diagnosis of AD is based on the patient's clinical history and a battery of neuropsychological tests. The accuracy of the diagnosis is highly dependent on the examiner's skills and on the evolution of a variable clinical frame. This work presents an automatic strategy that learns probabilistic brain models for different stages of the disease, reducing the complexity, parameter adjustment and computational costs. The proposed method starts by setting a probabilistic class description using the information stored in the neuropsychological test, followed by constructing the different structural class models using membership values from the learned probabilistic functions. These models are then used as a reference frame for the classification problem: a new case is assigned to a particular class simply by projecting to the different models. The validation was performed using a leave-one-out cross-validation, two classes were used: Normal Control (NC) subjects and patients diagnosed with mild AD. In this experiment it is possible to achieve a sensibility and specificity of 80% and 79% respectively.

Probabilistic Approach to Enable Extreme-Scale Simulations under Uncertainty and System Faults. Final Technical Report

DOE Office of Scientific and Technical Information (OSTI.GOV)

Knio, Omar

2017-05-05

The current project develops a novel approach that uses a probabilistic description to capture the current state of knowledge about the computational solution. To effectively spread the computational effort over multiple nodes, the global computational domain is split into many subdomains. Computational uncertainty in the solution translates into uncertain boundary conditions for the equation system to be solved on those subdomains, and many independent, concurrent subdomain simulations are used to account for this bound- ary condition uncertainty. By relying on the fact that solutions on neighboring subdomains must agree with each other, a more accurate estimate for the global solutionmore » can be achieved. Statistical approaches in this update process make it possible to account for the effect of system faults in the probabilistic description of the computational solution, and the associated uncertainty is reduced through successive iterations. By combining all of these elements, the probabilistic reformulation allows splitting the computational work over very many independent tasks for good scalability, while being robust to system faults.« less

A real-time spiking cerebellum model for learning robot control.

PubMed

Carrillo, Richard R; Ros, Eduardo; Boucheny, Christian; Coenen, Olivier J-M D

2008-01-01

We describe a neural network model of the cerebellum based on integrate-and-fire spiking neurons with conductance-based synapses. The neuron characteristics are derived from our earlier detailed models of the different cerebellar neurons. We tested the cerebellum model in a real-time control application with a robotic platform. Delays were introduced in the different sensorimotor pathways according to the biological system. The main plasticity in the cerebellar model is a spike-timing dependent plasticity (STDP) at the parallel fiber to Purkinje cell connections. This STDP is driven by the inferior olive (IO) activity, which encodes an error signal using a novel probabilistic low frequency model. We demonstrate the cerebellar model in a robot control system using a target-reaching task. We test whether the system learns to reach different target positions in a non-destructive way, therefore abstracting a general dynamics model. To test the system's ability to self-adapt to different dynamical situations, we present results obtained after changing the dynamics of the robotic platform significantly (its friction and load). The experimental results show that the cerebellar-based system is able to adapt dynamically to different contexts.

Network reciprocity by coexisting learning and teaching strategies

NASA Astrophysics Data System (ADS)

Tanimoto, Jun; Brede, Markus; Yamauchi, Atsuo

2012-03-01

We propose a network reciprocity model in which an agent probabilistically adopts learning or teaching strategies. In the learning adaptation mechanism, an agent may copy a neighbor's strategy through Fermi pairwise comparison. The teaching adaptation mechanism involves an agent imposing its strategy on a neighbor. Our simulations reveal that the reciprocity is significantly affected by the frequency with which learning and teaching agents coexist in a network and by the structure of the network itself.

Studies on a Novel Neuro-dynamic Model for Prediction Learning of Fluctuated Data Streams: Beyond Dichotomy between Probabilistic and Deterministic Models

DTIC Science & Technology

2014-11-04

learning by robots as well as video image understanding by accumulated learning of the exemplars are discussed. 15. SUBJECT TERMS Cognitive ...learning to predict perceptual streams or encountering events by acquiring internal models is indispensable for intelligent or cognitive systems because...various cognitive functions are based on this compentency including goal-directed planning, mental simulation and recognition of the current situation

The Use of the Direct Optimized Probabilistic Calculation Method in Design of Bolt Reinforcement for Underground and Mining Workings

PubMed Central

Krejsa, Martin; Janas, Petr; Yilmaz, Işık; Marschalko, Marian; Bouchal, Tomas

2013-01-01

The load-carrying system of each construction should fulfill several conditions which represent reliable criteria in the assessment procedure. It is the theory of structural reliability which determines probability of keeping required properties of constructions. Using this theory, it is possible to apply probabilistic computations based on the probability theory and mathematic statistics. Development of those methods has become more and more popular; it is used, in particular, in designs of load-carrying structures with the required level or reliability when at least some input variables in the design are random. The objective of this paper is to indicate the current scope which might be covered by the new method—Direct Optimized Probabilistic Calculation (DOProC) in assessments of reliability of load-carrying structures. DOProC uses a purely numerical approach without any simulation techniques. This provides more accurate solutions to probabilistic tasks, and, in some cases, such approach results in considerably faster completion of computations. DOProC can be used to solve efficiently a number of probabilistic computations. A very good sphere of application for DOProC is the assessment of the bolt reinforcement in the underground and mining workings. For the purposes above, a special software application—“Anchor”—has been developed. PMID:23935412

Rats bred for high alcohol drinking are more sensitive to delayed and probabilistic outcomes.

PubMed

Wilhelm, C J; Mitchell, S H

2008-10-01

Alcoholics and heavy drinkers score higher on measures of impulsivity than nonalcoholics and light drinkers. This may be because of factors that predate drug exposure (e.g. genetics). This study examined the role of genetics by comparing impulsivity measures in ethanol-naive rats selectively bred based on their high [high alcohol drinking (HAD)] or low [low alcohol drinking (LAD)] consumption of ethanol. Replicates 1 and 2 of the HAD and LAD rats, developed by the University of Indiana Alcohol Research Center, completed two different discounting tasks. Delay discounting examines sensitivity to rewards that are delayed in time and is commonly used to assess 'choice' impulsivity. Probability discounting examines sensitivity to the uncertain delivery of rewards and has been used to assess risk taking and risk assessment. High alcohol drinking rats discounted delayed and probabilistic rewards more steeply than LAD rats. Discount rates associated with probabilistic and delayed rewards were weakly correlated, while bias was strongly correlated with discount rate in both delay and probability discounting. The results suggest that selective breeding for high alcohol consumption selects for animals that are more sensitive to delayed and probabilistic outcomes. Sensitivity to delayed or probabilistic outcomes may be predictive of future drinking in genetically predisposed individuals.

Mice plan decision strategies based on previously learned time intervals, locations, and probabilities

PubMed Central

Tosun, Tuğçe; Gür, Ezgi; Balcı, Fuat

2016-01-01

Animals can shape their timed behaviors based on experienced probabilistic relations in a nearly optimal fashion. On the other hand, it is not clear if they adopt these timed decisions by making computations based on previously learnt task parameters (time intervals, locations, and probabilities) or if they gradually develop their decisions based on trial and error. To address this question, we tested mice in the timed-switching task, which required them to anticipate when (after a short or long delay) and at which of the two delay locations a reward would be presented. The probability of short trials differed between test groups in two experiments. Critically, we first trained mice on relevant task parameters by signaling the active trial with a discriminative stimulus and delivered the corresponding reward after the associated delay without any response requirement (without inducing switching behavior). During the test phase, both options were presented simultaneously to characterize the emergence and temporal characteristics of the switching behavior. Mice exhibited timed-switching behavior starting from the first few test trials, and their performance remained stable throughout testing in the majority of the conditions. Furthermore, as the probability of the short trial increased, mice waited longer before switching from the short to long location (experiment 1). These behavioral adjustments were in directions predicted by reward maximization. These results suggest that rather than gradually adjusting their time-dependent choice behavior, mice abruptly adopted temporal decision strategies by directly integrating their previous knowledge of task parameters into their timed behavior, supporting the model-based representational account of temporal risk assessment. PMID:26733674

Location error uncertainties - an advanced using of probabilistic inverse theory

NASA Astrophysics Data System (ADS)

Debski, Wojciech

2016-04-01

The spatial location of sources of seismic waves is one of the first tasks when transient waves from natural (uncontrolled) sources are analyzed in many branches of physics, including seismology, oceanology, to name a few. Source activity and its spatial variability in time, the geometry of recording network, the complexity and heterogeneity of wave velocity distribution are all factors influencing the performance of location algorithms and accuracy of the achieved results. While estimating of the earthquake foci location is relatively simple a quantitative estimation of the location accuracy is really a challenging task even if the probabilistic inverse method is used because it requires knowledge of statistics of observational, modelling, and apriori uncertainties. In this presentation we addressed this task when statistics of observational and/or modeling errors are unknown. This common situation requires introduction of apriori constraints on the likelihood (misfit) function which significantly influence the estimated errors. Based on the results of an analysis of 120 seismic events from the Rudna copper mine operating in southwestern Poland we illustrate an approach based on an analysis of Shanon's entropy calculated for the aposteriori distribution. We show that this meta-characteristic of the aposteriori distribution carries some information on uncertainties of the solution found.

Common Randomness Principles of Secrecy

ERIC Educational Resources Information Center

Tyagi, Himanshu

2013-01-01

This dissertation concerns the secure processing of distributed data by multiple terminals, using interactive public communication among themselves, in order to accomplish a given computational task. In the setting of a probabilistic multiterminal source model in which several terminals observe correlated random signals, we analyze secure…

Frontal Theta Links Prediction Errors to Behavioral Adaptation in Reinforcement Learning

PubMed Central

Cavanagh, James F.; Frank, Michael J.; Klein, Theresa J.; Allen, John J.B.

2009-01-01

Investigations into action monitoring have consistently detailed a fronto-central voltage deflection in the Event-Related Potential (ERP) following the presentation of negatively valenced feedback, sometimes termed the Feedback Related Negativity (FRN). The FRN has been proposed to reflect a neural response to prediction errors during reinforcement learning, yet the single trial relationship between neural activity and the quanta of expectation violation remains untested. Although ERP methods are not well suited to single trial analyses, the FRN has been associated with theta band oscillatory perturbations in the medial prefrontal cortex. Medio-frontal theta oscillations have been previously associated with expectation violation and behavioral adaptation and are well suited to single trial analysis. Here, we recorded EEG activity during a probabilistic reinforcement learning task and fit the performance data to an abstract computational model (Q-learning) for calculation of single-trial reward prediction errors. Single-trial theta oscillatory activities following feedback were investigated within the context of expectation (prediction error) and adaptation (subsequent reaction time change). Results indicate that interactive medial and lateral frontal theta activities reflect the degree of negative and positive reward prediction error in the service of behavioral adaptation. These different brain areas use prediction error calculations for different behavioral adaptations: with medial frontal theta reflecting the utilization of prediction errors for reaction time slowing (specifically following errors), but lateral frontal theta reflecting prediction errors leading to working memory-related reaction time speeding for the correct choice. PMID:19969093

Studying Different Tasks of Implicit Learning across Multiple Test Sessions Conducted on the Web

PubMed Central

Sævland, Werner; Norman, Elisabeth

2016-01-01

Implicit learning is usually studied through individual performance on a single task, with the most common tasks being the Serial Reaction Time (SRT) task, the Dynamic System Control (DSC) task, and Artificial Grammar Learning (AGL). Few attempts have been made to compare performance across different implicit learning tasks within the same study. The current study was designed to explore the relationship between performance on the DSC Sugar factory task and the Alternating Serial Reaction Time (ASRT) task. We also addressed another limitation of traditional implicit learning experiments, namely that implicit learning is usually studied in laboratory settings over a restricted time span lasting for less than an hour. In everyday situations, implicit learning is assumed to involve a gradual accumulation of knowledge across several learning episodes over a longer time span. One way to increase the ecological validity of implicit learning experiments could be to present the learning material repeatedly across shorter test sessions. This can most easily be done by using a web-based setup in which participants can access the material from home. We therefore created an online web-based system for measuring implicit learning that could be administered in either single or multiple sessions. Participants (n = 66) were assigned to either a single session or a multiple session condition. Learning occurred on both tasks, and awareness measures suggested that acquired knowledge was not fully conscious on either of the tasks. Learning and the degree of conscious awareness of the learned regularities were compared across conditions and tasks. On the DSC task, performance was not affected by whether learning had taken place in one or over multiple sessions. On the ASRT task, RT improvement across blocks was larger in the multiple-session condition. Learning in the two tasks was not related. PMID:27375512

Guided SAR image despeckling with probabilistic non local weights

NASA Astrophysics Data System (ADS)

Gokul, Jithin; Nair, Madhu S.; Rajan, Jeny

2017-12-01

SAR images are generally corrupted by granular disturbances called speckle, which makes visual analysis and detail extraction a difficult task. Non Local despeckling techniques with probabilistic similarity has been a recent trend in SAR despeckling. To achieve effective speckle suppression without compromising detail preservation, we propose an improvement for the existing Generalized Guided Filter with Bayesian Non-Local Means (GGF-BNLM) method. The proposed method (Guided SAR Image Despeckling with Probabilistic Non Local Weights) replaces parametric constants based on heuristics in GGF-BNLM method with dynamically derived values based on the image statistics for weight computation. Proposed changes make GGF-BNLM method adaptive and as a result, significant improvement is achieved in terms of performance. Experimental analysis on SAR images shows excellent speckle reduction without compromising feature preservation when compared to GGF-BNLM method. Results are also compared with other state-of-the-art and classic SAR depseckling techniques to demonstrate the effectiveness of the proposed method.

Is there a basis for preferring characteristic earthquakes over a Gutenberg–Richter distribution in probabilistic earthquake forecasting?

USGS Publications Warehouse

Parsons, Thomas E.; Geist, Eric L.

2009-01-01

The idea that faults rupture in repeated, characteristic earthquakes is central to most probabilistic earthquake forecasts. The concept is elegant in its simplicity, and if the same event has repeated itself multiple times in the past, we might anticipate the next. In practice however, assembling a fault-segmented characteristic earthquake rupture model can grow into a complex task laden with unquantified uncertainty. We weigh the evidence that supports characteristic earthquakes against a potentially simpler model made from extrapolation of a Gutenberg–Richter magnitude-frequency law to individual fault zones. We find that the Gutenberg–Richter model satisfies key data constraints used for earthquake forecasting equally well as a characteristic model. Therefore, judicious use of instrumental and historical earthquake catalogs enables large-earthquake-rate calculations with quantifiable uncertainty that should get at least equal weighting in probabilistic forecasting.

Motivational orientation modulates the neural response to reward.

PubMed

Linke, Julia; Kirsch, Peter; King, Andrea V; Gass, Achim; Hennerici, Michael G; Bongers, André; Wessa, Michèle

2010-02-01

Motivational orientation defines the source of motivation for an individual to perform a particular action and can either originate from internal desires (e.g., interest) or external compensation (e.g., money). To this end, motivational orientation should influence the way positive or negative feedback is processed during learning situations and this might in turn have an impact on the learning process. In the present study, we thus investigated whether motivational orientation, i.e., extrinsic and intrinsic motivation modulates the neural response to reward and punishment as well as learning from reward and punishment in 33 healthy individuals. To assess neural responses to reward, punishment and learning of reward contingencies we employed a probabilistic reversal learning task during functional magnetic resonance imaging. Extrinsic and intrinsic motivation were assessed with a self-report questionnaire. Rewarding trials fostered activation in the medial orbitofrontal cortex and anterior cingulate gyrus (ACC) as well as the amygdala and nucleus accumbens, whereas for punishment an increased neural response was observed in the medial and inferior prefrontal cortex, the superior parietal cortex and the insula. High extrinsic motivation was positively correlated to increased neural responses to reward in the ACC, amygdala and putamen, whereas a negative relationship between intrinsic motivation and brain activation in these brain regions was observed. These findings show that motivational orientation indeed modulates the responsiveness to reward delivery in major components of the human reward system and therefore extends previous results showing a significant influence of individual differences in reward-related personality traits on the neural processing of reward. Copyright (c) 2009 Elsevier Inc. All rights reserved.

Event-related potentials reflect impaired temporal interval learning following haloperidol administration.

PubMed

Forster, Sarah E; Zirnheld, Patrick; Shekhar, Anantha; Steinhauer, Stuart R; O'Donnell, Brian F; Hetrick, William P

2017-09-01

Signals carried by the mesencephalic dopamine system and conveyed to anterior cingulate cortex are critically implicated in probabilistic reward learning and performance monitoring. A common evaluative mechanism purportedly subserves both functions, giving rise to homologous medial frontal negativities in feedback- and response-locked event-related brain potentials (the feedback-related negativity (FRN) and the error-related negativity (ERN), respectively), reflecting dopamine-dependent prediction error signals to unexpectedly negative events. Consistent with this model, the dopamine receptor antagonist, haloperidol, attenuates the ERN, but effects on FRN have not yet been evaluated. ERN and FRN were recorded during a temporal interval learning task (TILT) following randomized, double-blind administration of haloperidol (3 mg; n = 18), diphenhydramine (an active control for haloperidol; 25 mg; n = 20), or placebo (n = 21) to healthy controls. Centroparietal positivities, the Pe and feedback-locked P300, were also measured and correlations between ERP measures and behavioral indices of learning, overall accuracy, and post-error compensatory behavior were evaluated. We hypothesized that haloperidol would reduce ERN and FRN, but that ERN would uniquely track automatic, error-related performance adjustments, while FRN would be associated with learning and overall accuracy. As predicted, ERN was reduced by haloperidol and in those exhibiting less adaptive post-error performance; however, these effects were limited to ERNs following fast timing errors. In contrast, the FRN was not affected by drug condition, although increased FRN amplitude was associated with improved accuracy. Significant drug effects on centroparietal positivities were also absent. Our results support a functional and neurobiological dissociation between the ERN and FRN.

Reduced activation in the ventral striatum during probabilistic decision-making in patients in an at-risk mental state

PubMed Central

Rausch, Franziska; Mier, Daniela; Eifler, Sarah; Fenske, Sabrina; Schirmbeck, Frederike; Englisch, Susanne; Schilling, Claudia; Meyer-Lindenberg, Andreas; Kirsch, Peter; Zink, Mathias

2015-01-01

Background Patients with schizophrenia display metacognitive impairments, such as hasty decision-making during probabilistic reasoning — the “jumping to conclusion” bias (JTC). Our recent fMRI study revealed reduced activations in the right ventral striatum (VS) and the ventral tegmental area (VTA) to be associated with decision-making in patients with schizophrenia. It is unclear whether these functional alterations occur in the at-risk mental state (ARMS). Methods We administered the classical beads task and fMRI among ARMS patients and healthy controls matched for age, sex, education and premorbid verbal intelligence. None of the ARMS patients was treated with antipsychotics. Both tasks request probabilistic decisions after a variable amount of stimuli. We evaluated activation during decision-making under certainty versus uncertainty and the process of final decision-making. Results We included 24 AMRS patients and 24 controls in our study. Compared with controls, ARMS patients tended to draw fewer beads and showed significantly more JTC bias in the classical beads task, mirroring findings in patients with schizophrenia. During fMRI, ARMS patients did not demonstrate JTC bias on the behavioural level, but showed a significant hypoactivation in the right VS during the decision stage. Limitations Owing to the cross-sectional design of the study, results are constrained to a better insight into the neurobiology of risk constellations, but not pre-psychotic stages. Nine of the ARMS patients were treated with antidepressants and/or lorazepam. Conclusion As in patients with schizophrenia, a striatal hypoactivation was found in ARMS patients. Confounding effects of antipsychotic medication can be excluded. Our findings indicate that error prediction signalling and reward anticipation may be linked to striatal dysfunction during prodromal stages and should be examined for their utility in predicting transition risk. PMID:25622039

Reduced activation in the ventral striatum during probabilistic decision-making in patients in an at-risk mental state.

PubMed

Rausch, Franziska; Mier, Daniela; Eifler, Sarah; Fenske, Sabrina; Schirmbeck, Frederike; Englisch, Susanne; Schilling, Claudia; Meyer-Lindenberg, Andreas; Kirsch, Peter; Zink, Mathias

2015-05-01

Patients with schizophrenia display metacognitive impairments, such as hasty decision-making during probabilistic reasoning - the "jumping to conclusion" bias (JTC). Our recent fMRI study revealed reduced activations in the right ventral striatum (VS) and the ventral tegmental area (VTA) to be associated with decision-making in patients with schizophrenia. It is unclear whether these functional alterations occur in the at-risk mental state (ARMS). We administered the classical beads task and fMRI among ARMS patients and healthy controls matched for age, sex, education and premorbid verbal intelligence. None of the ARMS patients was treated with antipsychotics. Both tasks request probabilistic decisions after a variable amount of stimuli. We evaluated activation during decision-making under certainty versus uncertainty and the process of final decision-making. We included 24 AMRS patients and 24 controls in our study. Compared with controls, ARMS patients tended to draw fewer beads and showed significantly more JTC bias in the classical beads task, mirroring findings in patients with schizophrenia. During fMRI, ARMS patients did not demonstrate JTC bias on the behavioural level, but showed a significant hypoactivation in the right VS during the decision stage. Owing to the cross-sectional design of the study, results are constrained to a better insight into the neurobiology of risk constellations, but not prepsychotic stages. Nine of the ARMS patients were treated with antidepressants and/or lorazepam. As in patients with schizophrenia, a striatal hypoactivation was found in ARMS patients. Confounding effects of antipsychotic medication can be excluded. Our findings indicate that error prediction signalling and reward anticipation may be linked to striatal dysfunction during prodromal stages and should be examined for their utility in predicting transition risk.

The Effects of Study Tasks in a Computer-Based Chemistry Learning Environment

NASA Astrophysics Data System (ADS)

Urhahne, Detlef; Nick, Sabine; Poepping, Anna Christin; Schulz, Sarah Jayne

2013-12-01

The present study examines the effects of different study tasks on the acquisition of knowledge about acids and bases in a computer-based learning environment. Three different task formats were selected to create three treatment conditions: learning with gap-fill and matching tasks, learning with multiple-choice tasks, and learning only from text and figures without any additional tasks. Participants were 196 ninth-grade students who learned with a self-developed multimedia program in a pretest-posttest control group design. Research results reveal that gap-fill and matching tasks were most effective in promoting knowledge acquisition, followed by multiple-choice tasks, and no tasks at all. The findings are in line with previous research on this topic. The effects can possibly be explained by the generation-recognition model, which predicts that gap-fill and matching tasks trigger more encompassing learning processes than multiple-choice tasks. It is concluded that instructional designers should incorporate more challenging study tasks for enhancing the effectiveness of computer-based learning environments.

Nonmonetary Decision-Making Indices Discriminate Between Different Behavioral Components of Gambling.

PubMed

Navas, Juan F; Torres, Ana; Vilar, Raquel; Verdejo-García, Antonio; Catena, Andrés; Perales, José C

2015-12-01

Recent research has proposed that altered reward and punishment sensitivity, heightened impulsivity, and faulty dynamic decision-making are at the core of disordered gambling. However, each of these traits and cognitive aspects dimensionally vary in the normal population, such that the link between individual differences in these dimensions and gambling use can be ultimately informative to explain disordered gambling. The main aim of the present study was to investigate the contribution of such decision-making-related indices to gambling use parameters in a community sample of college students. Assessment included punishment and reward sensitivity (as measured by the shortened Sensitivity to Punishment and Sensitivity to Reward Questionnaire), impulsivity (as measured by the UPPS-P model and a motor inhibition Go/No-go task), and dynamic decision-making [as measured by the probabilistic reversal learning task (PRLT)]. A structured interview was conducted to explore quantitative aspects of the participants gambling habits (gambling presence, gambling frequency, and average amount of money spent in gambling per unit of time). Our results showed the existence of a decision-making profile of gambling, as it naturally occurs in college students, in which sensation seeking is directly and specifically related to gambling presence (gambling, or not gambling at all), punishment sensitivity is inversely related to gambling frequency, and inflexibility in the PRLT specifically predicts the losses accrued because of gambling. These results are compatible with the idea that sensation seeking and punishment insensitivity could increase exposure to gambling activities, whereas reversal learning inflexibility, in people who already gamble, could boost the risk to accumulate losses.

Multi-task feature learning by using trace norm regularization

NASA Astrophysics Data System (ADS)

Jiangmei, Zhang; Binfeng, Yu; Haibo, Ji; Wang, Kunpeng

2017-11-01

Multi-task learning can extract the correlation of multiple related machine learning problems to improve performance. This paper considers applying the multi-task learning method to learn a single task. We propose a new learning approach, which employs the mixture of expert model to divide a learning task into several related sub-tasks, and then uses the trace norm regularization to extract common feature representation of these sub-tasks. A nonlinear extension of this approach by using kernel is also provided. Experiments conducted on both simulated and real data sets demonstrate the advantage of the proposed approach.

SubCellProt: predicting protein subcellular localization using machine learning approaches.

PubMed

Garg, Prabha; Sharma, Virag; Chaudhari, Pradeep; Roy, Nilanjan

2009-01-01

High-throughput genome sequencing projects continue to churn out enormous amounts of raw sequence data. However, most of this raw sequence data is unannotated and, hence, not very useful. Among the various approaches to decipher the function of a protein, one is to determine its localization. Experimental approaches for proteome annotation including determination of a protein's subcellular localizations are very costly and labor intensive. Besides the available experimental methods, in silico methods present alternative approaches to accomplish this task. Here, we present two machine learning approaches for prediction of the subcellular localization of a protein from the primary sequence information. Two machine learning algorithms, k Nearest Neighbor (k-NN) and Probabilistic Neural Network (PNN) were used to classify an unknown protein into one of the 11 subcellular localizations. The final prediction is made on the basis of a consensus of the predictions made by two algorithms and a probability is assigned to it. The results indicate that the primary sequence derived features like amino acid composition, sequence order and physicochemical properties can be used to assign subcellular localization with a fair degree of accuracy. Moreover, with the enhanced accuracy of our approach and the definition of a prediction domain, this method can be used for proteome annotation in a high throughput manner. SubCellProt is available at www.databases.niper.ac.in/SubCellProt.

Probability Learning: Changes in Behavior across Time and Development

ERIC Educational Resources Information Center

Plate, Rista C.; Fulvio, Jacqueline M.; Shutts, Kristin; Green, C. Shawn; Pollak, Seth D.

2018-01-01

Individuals track probabilities, such as associations between events in their environments, but less is known about the degree to which experience--within a learning session and over development--influences people's use of incoming probabilistic information to guide behavior in real time. In two experiments, children (4-11 years) and adults…

Web-Assisted Education: From Evaluation to Learning

ERIC Educational Resources Information Center

Stefansson, Gunnar; Sigurdardottir, Asta Jenny

2011-01-01

Exams have traditionally been given to evaluate students but in recent years, with the appearance of freely accessible on-line tutoring, quizzes can also be used as a learning tool. In systems where students can request quiz items until a satisfactory grade is obtained, new probabilistic approaches are required for dealing items to students and…

Calibrating perceived understanding and competency in probability concepts: A diagnosis of learning difficulties based on Rasch probabilistic model

NASA Astrophysics Data System (ADS)

Mahmud, Zamalia; Porter, Anne; Salikin, Masniyati; Ghani, Nor Azura Md

2015-12-01

Students' understanding of probability concepts have been investigated from various different perspectives. Competency on the other hand is often measured separately in the form of test structure. This study was set out to show that perceived understanding and competency can be calibrated and assessed together using Rasch measurement tools. Forty-four students from the STAT131 Understanding Uncertainty and Variation course at the University of Wollongong, NSW have volunteered to participate in the study. Rasch measurement which is based on a probabilistic model is used to calibrate the responses from two survey instruments and investigate the interactions between them. Data were captured from the e-learning platform Moodle where students provided their responses through an online quiz. The study shows that majority of the students perceived little understanding about conditional and independent events prior to learning about it but tend to demonstrate a slightly higher competency level afterward. Based on the Rasch map, there is indication of some increase in learning and knowledge about some probability concepts at the end of the two weeks lessons on probability concepts.

Optimal quantum operations at zero energy cost

NASA Astrophysics Data System (ADS)

Chiribella, Giulio; Yang, Yuxiang

2017-08-01

Quantum technologies are developing powerful tools to generate and manipulate coherent superpositions of different energy levels. Envisaging a new generation of energy-efficient quantum devices, here we explore how coherence can be manipulated without exchanging energy with the surrounding environment. We start from the task of converting a coherent superposition of energy eigenstates into another. We identify the optimal energy-preserving operations, both in the deterministic and in the probabilistic scenario. We then design a recursive protocol, wherein a branching sequence of energy-preserving filters increases the probability of success while reaching maximum fidelity at each iteration. Building on the recursive protocol, we construct efficient approximations of the optimal fidelity-probability trade-off, by taking coherent superpositions of the different branches generated by probabilistic filtering. The benefits of this construction are illustrated in applications to quantum metrology, quantum cloning, coherent state amplification, and ancilla-driven computation. Finally, we extend our results to transitions where the input state is generally mixed and we apply our findings to the task of purifying quantum coherence.

Asking better questions: How presentation formats influence information search.

PubMed

Wu, Charley M; Meder, Björn; Filimon, Flavia; Nelson, Jonathan D

2017-08-01

While the influence of presentation formats have been widely studied in Bayesian reasoning tasks, we present the first systematic investigation of how presentation formats influence information search decisions. Four experiments were conducted across different probabilistic environments, where subjects (N = 2,858) chose between 2 possible search queries, each with binary probabilistic outcomes, with the goal of maximizing classification accuracy. We studied 14 different numerical and visual formats for presenting information about the search environment, constructed across 6 design features that have been prominently related to improvements in Bayesian reasoning accuracy (natural frequencies, posteriors, complement, spatial extent, countability, and part-to-whole information). The posterior variants of the icon array and bar graph formats led to the highest proportion of correct responses, and were substantially better than the standard probability format. Results suggest that presenting information in terms of posterior probabilities and visualizing natural frequencies using spatial extent (a perceptual feature) were especially helpful in guiding search decisions, although environments with a mixture of probabilistic and certain outcomes were challenging across all formats. Subjects who made more accurate probability judgments did not perform better on the search task, suggesting that simple decision heuristics may be used to make search decisions without explicitly applying Bayesian inference to compute probabilities. We propose a new take-the-difference (TTD) heuristic that identifies the accuracy-maximizing query without explicit computation of posterior probabilities. (PsycINFO Database Record (c) 2017 APA, all rights reserved).

Facilitating normative judgments of conditional probability: frequency or nested sets?

PubMed

Yamagishi, Kimihiko

2003-01-01

Recent probability judgment research contrasts two opposing views. Some theorists have emphasized the role of frequency representations in facilitating probabilistic correctness; opponents have noted that visualizing the probabilistic structure of the task sufficiently facilitates normative reasoning. In the current experiment, the following conditional probability task, an isomorph of the "Problem of Three Prisoners" was tested. "A factory manufactures artificial gemstones. Each gemstone has a 1/3 chance of being blurred, a 1/3 chance of being cracked, and a 1/3 chance of being clear. An inspection machine removes all cracked gemstones, and retains all clear gemstones. However, the machine removes 1/2 of the blurred gemstones. What is the chance that a gemstone is blurred after the inspection?" A 2 x 2 design was administered. The first variable was the use of frequency instruction. The second manipulation was the use of a roulette-wheel diagram that illustrated a "nested-sets" relationship between the prior and the posterior probabilities. Results from two experiments showed that frequency alone had modest effects, while the nested-sets instruction achieved a superior facilitation of normative reasoning. The third experiment compared the roulette-wheel diagram to tree diagrams that also showed the nested-sets relationship. The roulette-wheel diagram outperformed the tree diagrams in facilitation of probabilistic reasoning. Implications for understanding the nature of intuitive probability judgments are discussed.

Predictive coarse-graining

NASA Astrophysics Data System (ADS)

Schöberl, Markus; Zabaras, Nicholas; Koutsourelakis, Phaedon-Stelios

2017-03-01

We propose a data-driven, coarse-graining formulation in the context of equilibrium statistical mechanics. In contrast to existing techniques which are based on a fine-to-coarse map, we adopt the opposite strategy by prescribing a probabilistic coarse-to-fine map. This corresponds to a directed probabilistic model where the coarse variables play the role of latent generators of the fine scale (all-atom) data. From an information-theoretic perspective, the framework proposed provides an improvement upon the relative entropy method [1] and is capable of quantifying the uncertainty due to the information loss that unavoidably takes place during the coarse-graining process. Furthermore, it can be readily extended to a fully Bayesian model where various sources of uncertainties are reflected in the posterior of the model parameters. The latter can be used to produce not only point estimates of fine-scale reconstructions or macroscopic observables, but more importantly, predictive posterior distributions on these quantities. Predictive posterior distributions reflect the confidence of the model as a function of the amount of data and the level of coarse-graining. The issues of model complexity and model selection are seamlessly addressed by employing a hierarchical prior that favors the discovery of sparse solutions, revealing the most prominent features in the coarse-grained model. A flexible and parallelizable Monte Carlo - Expectation-Maximization (MC-EM) scheme is proposed for carrying out inference and learning tasks. A comparative assessment of the proposed methodology is presented for a lattice spin system and the SPC/E water model.

Predictive coarse-graining

DOE Office of Scientific and Technical Information (OSTI.GOV)

Schöberl, Markus, E-mail: m.schoeberl@tum.de; Zabaras, Nicholas; Department of Aerospace and Mechanical Engineering, University of Notre Dame, 365 Fitzpatrick Hall, Notre Dame, IN 46556

We propose a data-driven, coarse-graining formulation in the context of equilibrium statistical mechanics. In contrast to existing techniques which are based on a fine-to-coarse map, we adopt the opposite strategy by prescribing a probabilistic coarse-to-fine map. This corresponds to a directed probabilistic model where the coarse variables play the role of latent generators of the fine scale (all-atom) data. From an information-theoretic perspective, the framework proposed provides an improvement upon the relative entropy method and is capable of quantifying the uncertainty due to the information loss that unavoidably takes place during the coarse-graining process. Furthermore, it can be readily extendedmore » to a fully Bayesian model where various sources of uncertainties are reflected in the posterior of the model parameters. The latter can be used to produce not only point estimates of fine-scale reconstructions or macroscopic observables, but more importantly, predictive posterior distributions on these quantities. Predictive posterior distributions reflect the confidence of the model as a function of the amount of data and the level of coarse-graining. The issues of model complexity and model selection are seamlessly addressed by employing a hierarchical prior that favors the discovery of sparse solutions, revealing the most prominent features in the coarse-grained model. A flexible and parallelizable Monte Carlo – Expectation–Maximization (MC-EM) scheme is proposed for carrying out inference and learning tasks. A comparative assessment of the proposed methodology is presented for a lattice spin system and the SPC/E water model.« less

Fully automatized renal parenchyma volumetry using a support vector machine based recognition system for subject-specific probability map generation in native MR volume data

NASA Astrophysics Data System (ADS)

Gloger, Oliver; Tönnies, Klaus; Mensel, Birger; Völzke, Henry

2015-11-01

In epidemiological studies as well as in clinical practice the amount of produced medical image data strongly increased in the last decade. In this context organ segmentation in MR volume data gained increasing attention for medical applications. Especially in large-scale population-based studies organ volumetry is highly relevant requiring exact organ segmentation. Since manual segmentation is time-consuming and prone to reader variability, large-scale studies need automatized methods to perform organ segmentation. Fully automatic organ segmentation in native MR image data has proven to be a very challenging task. Imaging artifacts as well as inter- and intrasubject MR-intensity differences complicate the application of supervised learning strategies. Thus, we propose a modularized framework of a two-stepped probabilistic approach that generates subject-specific probability maps for renal parenchyma tissue, which are refined subsequently by using several, extended segmentation strategies. We present a three class-based support vector machine recognition system that incorporates Fourier descriptors as shape features to recognize and segment characteristic parenchyma parts. Probabilistic methods use the segmented characteristic parenchyma parts to generate high quality subject-specific parenchyma probability maps. Several refinement strategies including a final shape-based 3D level set segmentation technique are used in subsequent processing modules to segment renal parenchyma. Furthermore, our framework recognizes and excludes renal cysts from parenchymal volume, which is important to analyze renal functions. Volume errors and Dice coefficients show that our presented framework outperforms existing approaches.

Fully automatized renal parenchyma volumetry using a support vector machine based recognition system for subject-specific probability map generation in native MR volume data.

PubMed

Gloger, Oliver; Tönnies, Klaus; Mensel, Birger; Völzke, Henry

2015-11-21

In epidemiological studies as well as in clinical practice the amount of produced medical image data strongly increased in the last decade. In this context organ segmentation in MR volume data gained increasing attention for medical applications. Especially in large-scale population-based studies organ volumetry is highly relevant requiring exact organ segmentation. Since manual segmentation is time-consuming and prone to reader variability, large-scale studies need automatized methods to perform organ segmentation. Fully automatic organ segmentation in native MR image data has proven to be a very challenging task. Imaging artifacts as well as inter- and intrasubject MR-intensity differences complicate the application of supervised learning strategies. Thus, we propose a modularized framework of a two-stepped probabilistic approach that generates subject-specific probability maps for renal parenchyma tissue, which are refined subsequently by using several, extended segmentation strategies. We present a three class-based support vector machine recognition system that incorporates Fourier descriptors as shape features to recognize and segment characteristic parenchyma parts. Probabilistic methods use the segmented characteristic parenchyma parts to generate high quality subject-specific parenchyma probability maps. Several refinement strategies including a final shape-based 3D level set segmentation technique are used in subsequent processing modules to segment renal parenchyma. Furthermore, our framework recognizes and excludes renal cysts from parenchymal volume, which is important to analyze renal functions. Volume errors and Dice coefficients show that our presented framework outperforms existing approaches.

Blunted neural responses to reward in remitted major depression: A high-density event-related potential study

PubMed Central

Whitton, Alexis E.; Kakani, Pragya; Foti, Dan; Van’t Veer, Ashlee; Haile, Anja; Crowley, David J.; Pizzagalli, Diego A.

2015-01-01

Background Major depressive disorder (MDD) is a highly recurrent condition, and improving our understanding of the abnormalities that persist in remitted MDD (rMDD) may provide insight into mechanisms that contribute to relapse. MDD has been characterized by reward learning deficits linked to dysfunction in frontostriatal regions. Although initial behavioral evidence of reward learning deficits in rMDD has recently emerged, it is unclear whether these reflect impairments in neural reward processing that persist into remission. Methods We examined behavioral reward learning and 128-channel event-related potentials (ERP) during a well-validated probabilistic reward task in 26 rMDD individuals and 34 never-depressed controls. Temporo-spatial principal components analysis (PCA) was used to separate overlapping ERP components, and group differences in neural activity in a priori regions were examined using low-resolution electromagnetic tomography (LORETA). Results Individuals with rMDD displayed reduced behavioral reward learning, as well as blunted ERP amplitude to reward feedback. Importantly, the reduction in ERP amplitude occurred at a PCA factor that peaked during the time at which phasic reward feedback-related signaling – hypothesized to originate in the striatum and project to the anterior cingulate cortex (ACC) – are thought to modulate scalp-recorded activity. Consistent with this, LORETA analyses revealed reduced activity in the ACC in the rMDD group, and this blunting correlated with poorer reward learning. Conclusion These findings suggest that the reward learning impairment observed in acute MDD persists into full remission and that these impairments may be attributable to abnormalities in the neural processes that support reward feedback monitoring, particularly within the ACC. PMID:26858994

Probabilistic double guarantee kidnapping detection in SLAM.

PubMed

Tian, Yang; Ma, Shugen

2016-01-01

For determining whether kidnapping has happened and which type of kidnapping it is while a robot performs autonomous tasks in an unknown environment, a double guarantee kidnapping detection (DGKD) method has been proposed. The good performance of DGKD in a relative small environment is shown. However, a limitation of DGKD is found in a large-scale environment by our recent work. In order to increase the adaptability of DGKD in a large-scale environment, an improved method called probabilistic double guarantee kidnapping detection is proposed in this paper to combine probability of features' positions and the robot's posture. Simulation results demonstrate the validity and accuracy of the proposed method.

Human Factors Engineering. Student Supplement,

DTIC Science & Technology

1981-08-01

a job TASK TAXONOMY A classification scheme for the different levels of activities in a system, i.e., job - task - sub-task, etc. TASK-AN~ALYSIS...with the classification of learning objectives by learning category so as to identify learningPhas III guidelines necessary for optimum learning to...correct. .4... .the sequencing of all dependent tasks. .1.. .the classification of learning objectives by learning category and the Identification of

Resident Space Object Characterization and Behavior Understanding via Machine Learning and Ontology-based Bayesian Networks

NASA Astrophysics Data System (ADS)

Furfaro, R.; Linares, R.; Gaylor, D.; Jah, M.; Walls, R.

2016-09-01

In this paper, we present an end-to-end approach that employs machine learning techniques and Ontology-based Bayesian Networks (BN) to characterize the behavior of resident space objects. State-of-the-Art machine learning architectures (e.g. Extreme Learning Machines, Convolutional Deep Networks) are trained on physical models to learn the Resident Space Object (RSO) features in the vectorized energy and momentum states and parameters. The mapping from measurements to vectorized energy and momentum states and parameters enables behavior characterization via clustering in the features space and subsequent RSO classification. Additionally, Space Object Behavioral Ontologies (SOBO) are employed to define and capture the domain knowledge-base (KB) and BNs are constructed from the SOBO in a semi-automatic fashion to execute probabilistic reasoning over conclusions drawn from trained classifiers and/or directly from processed data. Such an approach enables integrating machine learning classifiers and probabilistic reasoning to support higher-level decision making for space domain awareness applications. The innovation here is to use these methods (which have enjoyed great success in other domains) in synergy so that it enables a "from data to discovery" paradigm by facilitating the linkage and fusion of large and disparate sources of information via a Big Data Science and Analytics framework.

Learning predictive statistics from temporal sequences: Dynamics and strategies

PubMed Central

Wang, Rui; Shen, Yuan; Tino, Peter; Welchman, Andrew E.; Kourtzi, Zoe

2017-01-01

Human behavior is guided by our expectations about the future. Often, we make predictions by monitoring how event sequences unfold, even though such sequences may appear incomprehensible. Event structures in the natural environment typically vary in complexity, from simple repetition to complex probabilistic combinations. How do we learn these structures? Here we investigate the dynamics of structure learning by tracking human responses to temporal sequences that change in structure unbeknownst to the participants. Participants were asked to predict the upcoming item following a probabilistic sequence of symbols. Using a Markov process, we created a family of sequences, from simple frequency statistics (e.g., some symbols are more probable than others) to context-based statistics (e.g., symbol probability is contingent on preceding symbols). We demonstrate the dynamics with which individuals adapt to changes in the environment's statistics—that is, they extract the behaviorally relevant structures to make predictions about upcoming events. Further, we show that this structure learning relates to individual decision strategy; faster learning of complex structures relates to selection of the most probable outcome in a given context (maximizing) rather than matching of the exact sequence statistics. Our findings provide evidence for alternate routes to learning of behaviorally relevant statistics that facilitate our ability to predict future events in variable environments. PMID:28973111

Learning predictive statistics from temporal sequences: Dynamics and strategies.

PubMed

Wang, Rui; Shen, Yuan; Tino, Peter; Welchman, Andrew E; Kourtzi, Zoe

2017-10-01

Human behavior is guided by our expectations about the future. Often, we make predictions by monitoring how event sequences unfold, even though such sequences may appear incomprehensible. Event structures in the natural environment typically vary in complexity, from simple repetition to complex probabilistic combinations. How do we learn these structures? Here we investigate the dynamics of structure learning by tracking human responses to temporal sequences that change in structure unbeknownst to the participants. Participants were asked to predict the upcoming item following a probabilistic sequence of symbols. Using a Markov process, we created a family of sequences, from simple frequency statistics (e.g., some symbols are more probable than others) to context-based statistics (e.g., symbol probability is contingent on preceding symbols). We demonstrate the dynamics with which individuals adapt to changes in the environment's statistics-that is, they extract the behaviorally relevant structures to make predictions about upcoming events. Further, we show that this structure learning relates to individual decision strategy; faster learning of complex structures relates to selection of the most probable outcome in a given context (maximizing) rather than matching of the exact sequence statistics. Our findings provide evidence for alternate routes to learning of behaviorally relevant statistics that facilitate our ability to predict future events in variable environments.

The Effects of Study Tasks in a Computer-Based Chemistry Learning Environment

ERIC Educational Resources Information Center

Urhahne, Detlef; Nick, Sabine; Poepping, Anna Christin; Schulz , Sarah Jayne

2013-01-01

The present study examines the effects of different study tasks on the acquisition of knowledge about acids and bases in a computer-based learning environment. Three different task formats were selected to create three treatment conditions: learning with gap-fill and matching tasks, learning with multiple-choice tasks, and learning only from text…

Measurement of functional task difficulty during motor learning: What level of difficulty corresponds to the optimal challenge point?

PubMed

Akizuki, Kazunori; Ohashi, Yukari

2015-10-01

The relationship between task difficulty and learning benefit was examined, as was the measurability of task difficulty. Participants were required to learn a postural control task on an unstable surface at one of four different task difficulty levels. Results from the retention test showed an inverted-U relationship between task difficulty during acquisition and motor learning. The second-highest level of task difficulty was the most effective for motor learning, while learning was delayed at the most and least difficult levels. Additionally, the results indicate that salivary α-amylase and the performance dimension of the National Aeronautics and Space Administration-Task Load Index (NASA-TLX) are useful indices of task difficulty. Our findings suggested that instructors may be able to adjust task difficulty based on salivary α-amylase and the performance dimension of the NASA-TLX to enhance learning. Copyright © 2015 Elsevier B.V. All rights reserved.

Brain networks for confidence weighting and hierarchical inference during probabilistic learning.

PubMed

Meyniel, Florent; Dehaene, Stanislas

2017-05-09

Learning is difficult when the world fluctuates randomly and ceaselessly. Classical learning algorithms, such as the delta rule with constant learning rate, are not optimal. Mathematically, the optimal learning rule requires weighting prior knowledge and incoming evidence according to their respective reliabilities. This "confidence weighting" implies the maintenance of an accurate estimate of the reliability of what has been learned. Here, using fMRI and an ideal-observer analysis, we demonstrate that the brain's learning algorithm relies on confidence weighting. While in the fMRI scanner, human adults attempted to learn the transition probabilities underlying an auditory or visual sequence, and reported their confidence in those estimates. They knew that these transition probabilities could change simultaneously at unpredicted moments, and therefore that the learning problem was inherently hierarchical. Subjective confidence reports tightly followed the predictions derived from the ideal observer. In particular, subjects managed to attach distinct levels of confidence to each learned transition probability, as required by Bayes-optimal inference. Distinct brain areas tracked the likelihood of new observations given current predictions, and the confidence in those predictions. Both signals were combined in the right inferior frontal gyrus, where they operated in agreement with the confidence-weighting model. This brain region also presented signatures of a hierarchical process that disentangles distinct sources of uncertainty. Together, our results provide evidence that the sense of confidence is an essential ingredient of probabilistic learning in the human brain, and that the right inferior frontal gyrus hosts a confidence-based statistical learning algorithm for auditory and visual sequences.

Brain networks for confidence weighting and hierarchical inference during probabilistic learning

PubMed Central

Meyniel, Florent; Dehaene, Stanislas

2017-01-01

Learning is difficult when the world fluctuates randomly and ceaselessly. Classical learning algorithms, such as the delta rule with constant learning rate, are not optimal. Mathematically, the optimal learning rule requires weighting prior knowledge and incoming evidence according to their respective reliabilities. This “confidence weighting” implies the maintenance of an accurate estimate of the reliability of what has been learned. Here, using fMRI and an ideal-observer analysis, we demonstrate that the brain’s learning algorithm relies on confidence weighting. While in the fMRI scanner, human adults attempted to learn the transition probabilities underlying an auditory or visual sequence, and reported their confidence in those estimates. They knew that these transition probabilities could change simultaneously at unpredicted moments, and therefore that the learning problem was inherently hierarchical. Subjective confidence reports tightly followed the predictions derived from the ideal observer. In particular, subjects managed to attach distinct levels of confidence to each learned transition probability, as required by Bayes-optimal inference. Distinct brain areas tracked the likelihood of new observations given current predictions, and the confidence in those predictions. Both signals were combined in the right inferior frontal gyrus, where they operated in agreement with the confidence-weighting model. This brain region also presented signatures of a hierarchical process that disentangles distinct sources of uncertainty. Together, our results provide evidence that the sense of confidence is an essential ingredient of probabilistic learning in the human brain, and that the right inferior frontal gyrus hosts a confidence-based statistical learning algorithm for auditory and visual sequences. PMID:28439014

The integration of probabilistic information during sensorimotor estimation is unimpaired in children with Cerebral Palsy

PubMed Central

Sokhey, Taegh; Gaebler-Spira, Deborah; Kording, Konrad P.

2017-01-01

Background It is important to understand the motor deficits of children with Cerebral Palsy (CP). Our understanding of this motor disorder can be enriched by computational models of motor control. One crucial stage in generating movement involves combining uncertain information from different sources, and deficits in this process could contribute to reduced motor function in children with CP. Healthy adults can integrate previously-learned information (prior) with incoming sensory information (likelihood) in a close-to-optimal way when estimating object location, consistent with the use of Bayesian statistics. However, there are few studies investigating how children with CP perform sensorimotor integration. We compare sensorimotor estimation in children with CP and age-matched controls using a model-based analysis to understand the process. Methods and findings We examined Bayesian sensorimotor integration in children with CP, aged between 5 and 12 years old, with Gross Motor Function Classification System (GMFCS) levels 1–3 and compared their estimation behavior with age-matched typically-developing (TD) children. We used a simple sensorimotor estimation task which requires participants to combine probabilistic information from different sources: a likelihood distribution (current sensory information) with a prior distribution (learned target information). In order to examine sensorimotor integration, we quantified how participants weighed statistical information from the two sources (prior and likelihood) and compared this to the statistical optimal weighting. We found that the weighing of statistical information in children with CP was as statistically efficient as that of TD children. Conclusions We conclude that Bayesian sensorimotor integration is not impaired in children with CP and therefore, does not contribute to their motor deficits. Future research has the potential to enrich our understanding of motor disorders by investigating the stages of motor processing set out by computational models. Therapeutic interventions should exploit the ability of children with CP to use statistical information. PMID:29186196

Intuitive Interference in Probabilistic Reasoning

ERIC Educational Resources Information Center

Babai, Reuven; Brecher, Tali; Stavy, Ruth; Tirosh, Dina

2006-01-01

One theoretical framework which addresses students' conceptions and reasoning processes in mathematics and science education is the intuitive rules theory. According to this theory, students' reasoning is affected by intuitive rules when they solve a wide variety of conceptually non-related mathematical and scientific tasks that share some common…

Asking Better Questions: How Presentation Formats Influence Information Search

ERIC Educational Resources Information Center

Wu, Charley M.; Meder, Björn; Filimon, Flavia; Nelson, Jonathan D.

2017-01-01

While the influence of presentation formats have been widely studied in Bayesian reasoning tasks, we present the first systematic investigation of how presentation formats influence information search decisions. Four experiments were conducted across different probabilistic environments, where subjects (N = 2,858) chose between 2 possible search…

Signal detection with criterion noise: applications to recognition memory.

PubMed

Benjamin, Aaron S; Diaz, Michael; Wee, Serena

2009-01-01

A tacit but fundamental assumption of the theory of signal detection is that criterion placement is a noise-free process. This article challenges that assumption on theoretical and empirical grounds and presents the noisy decision theory of signal detection (ND-TSD). Generalized equations for the isosensitivity function and for measures of discrimination incorporating criterion variability are derived, and the model's relationship with extant models of decision making in discrimination tasks is examined. An experiment evaluating recognition memory for ensembles of word stimuli revealed that criterion noise is not trivial in magnitude and contributes substantially to variance in the slope of the isosensitivity function. The authors discuss how ND-TSD can help explain a number of current and historical puzzles in recognition memory, including the inconsistent relationship between manipulations of learning and the isosensitivity function's slope, the lack of invariance of the slope with manipulations of bias or payoffs, the effects of aging on the decision-making process in recognition, and the nature of responding in remember-know decision tasks. ND-TSD poses novel, theoretically meaningful constraints on theories of recognition and decision making more generally, and provides a mechanism for rapprochement between theories of decision making that employ deterministic response rules and those that postulate probabilistic response rules.

Learning In networks

NASA Technical Reports Server (NTRS)

Buntine, Wray L.

1995-01-01

Intelligent systems require software incorporating probabilistic reasoning, and often times learning. Networks provide a framework and methodology for creating this kind of software. This paper introduces network models based on chain graphs with deterministic nodes. Chain graphs are defined as a hierarchical combination of Bayesian and Markov networks. To model learning, plates on chain graphs are introduced to model independent samples. The paper concludes by discussing various operations that can be performed on chain graphs with plates as a simplification process or to generate learning algorithms.

Self-reported strategies in decisions under risk: role of feedback, reasoning abilities, executive functions, short-term-memory, and working memory.

PubMed

Schiebener, Johannes; Brand, Matthias

2015-11-01

In decisions under objective risk conditions information about the decision options' possible outcomes and the rules for outcomes' occurrence are provided. Thus, deciders can base decision-making strategies on probabilistic laws. In many laboratory decision-making tasks, choosing the option with the highest winning probability in all trials (=maximization strategy) is probabilistically regarded the most rational behavior. However, individuals often behave less optimal, especially in case the individuals have lower cognitive functions or in case no feedback about consequences is provided in the situation. It is still unclear which cognitive functions particularly predispose individuals for using successful strategies and which strategies profit from feedback. We investigated 195 individuals with two decision-making paradigms, the Game of Dice Task (GDT) (with and without feedback), and the Card Guessing Game. Thereafter, participants reported which strategies they had applied. Interaction effects (feedback × strategy), effect sizes, and uncorrected single group comparisons suggest that feedback in the GDT tended to be more beneficial to individuals reporting exploratory strategies (e.g., use intuition). In both tasks, the self-reported use of more principled and more rational strategies was accompanied by better decision-making performance and better performances in reasoning and executive functioning tasks. The strategy groups did not significantly differ in most short-term and working-memory tasks. Thus, particularly individual differences in reasoning and executive functions seem to predispose individuals toward particular decision-making strategies. Feedback seems to be useful for individuals who rather explore the decision-making situation instead of following a certain plan.

Performance evaluation of 2D and 3D deep learning approaches for automatic segmentation of multiple organs on CT images

NASA Astrophysics Data System (ADS)

Zhou, Xiangrong; Yamada, Kazuma; Kojima, Takuya; Takayama, Ryosuke; Wang, Song; Zhou, Xinxin; Hara, Takeshi; Fujita, Hiroshi

2018-02-01

The purpose of this study is to evaluate and compare the performance of modern deep learning techniques for automatically recognizing and segmenting multiple organ regions on 3D CT images. CT image segmentation is one of the important task in medical image analysis and is still very challenging. Deep learning approaches have demonstrated the capability of scene recognition and semantic segmentation on nature images and have been used to address segmentation problems of medical images. Although several works showed promising results of CT image segmentation by using deep learning approaches, there is no comprehensive evaluation of segmentation performance of the deep learning on segmenting multiple organs on different portions of CT scans. In this paper, we evaluated and compared the segmentation performance of two different deep learning approaches that used 2D- and 3D deep convolutional neural networks (CNN) without- and with a pre-processing step. A conventional approach that presents the state-of-the-art performance of CT image segmentation without deep learning was also used for comparison. A dataset that includes 240 CT images scanned on different portions of human bodies was used for performance evaluation. The maximum number of 17 types of organ regions in each CT scan were segmented automatically and compared to the human annotations by using ratio of intersection over union (IU) as the criterion. The experimental results demonstrated the IUs of the segmentation results had a mean value of 79% and 67% by averaging 17 types of organs that segmented by a 3D- and 2D deep CNN, respectively. All the results of the deep learning approaches showed a better accuracy and robustness than the conventional segmentation method that used probabilistic atlas and graph-cut methods. The effectiveness and the usefulness of deep learning approaches were demonstrated for solving multiple organs segmentation problem on 3D CT images.

Optimization of Contrast Detection Power with Probabilistic Behavioral Information

PubMed Central

Cordes, Dietmar; Herzmann, Grit; Nandy, Rajesh; Curran, Tim

2012-01-01

Recent progress in the experimental design for event-related fMRI experiments made it possible to find the optimal stimulus sequence for maximum contrast detection power using a genetic algorithm. In this study, a novel algorithm is proposed for optimization of contrast detection power by including probabilistic behavioral information, based on pilot data, in the genetic algorithm. As a particular application, a recognition memory task is studied and the design matrix optimized for contrasts involving the familiarity of individual items (pictures of objects) and the recollection of qualitative information associated with the items (left/right orientation). Optimization of contrast efficiency is a complicated issue whenever subjects’ responses are not deterministic but probabilistic. Contrast efficiencies are not predictable unless behavioral responses are included in the design optimization. However, available software for design optimization does not include options for probabilistic behavioral constraints. If the anticipated behavioral responses are included in the optimization algorithm, the design is optimal for the assumed behavioral responses, and the resulting contrast efficiency is greater than what either a block design or a random design can achieve. Furthermore, improvements of contrast detection power depend strongly on the behavioral probabilities, the perceived randomness, and the contrast of interest. The present genetic algorithm can be applied to any case in which fMRI contrasts are dependent on probabilistic responses that can be estimated from pilot data. PMID:22326984

Climatological Observations for Maritime Prediction and Analysis Support Service (COMPASS)

NASA Astrophysics Data System (ADS)

OConnor, A.; Kirtman, B. P.; Harrison, S.; Gorman, J.

2016-02-01

Current US Navy forecasting systems cannot easily incorporate extended-range forecasts that can improve mission readiness and effectiveness; ensure safety; and reduce cost, labor, and resource requirements. If Navy operational planners had systems that incorporated these forecasts, they could plan missions using more reliable and longer-term weather and climate predictions. Further, using multi-model forecast ensembles instead of single forecasts would produce higher predictive performance. Extended-range multi-model forecast ensembles, such as those available in the North American Multi-Model Ensemble (NMME), are ideal for system integration because of their high skill predictions; however, even higher skill predictions can be produced if forecast model ensembles are combined correctly. While many methods for weighting models exist, the best method in a given environment requires expert knowledge of the models and combination methods.We present an innovative approach that uses machine learning to combine extended-range predictions from multi-model forecast ensembles and generate a probabilistic forecast for any region of the globe up to 12 months in advance. Our machine-learning approach uses 30 years of hindcast predictions to learn patterns of forecast model successes and failures. Each model is assigned a weight for each environmental condition, 100 km2 region, and day given any expected environmental information. These weights are then applied to the respective predictions for the region and time of interest to effectively stitch together a single, coherent probabilistic forecast. Our experimental results demonstrate the benefits of our approach to produce extended-range probabilistic forecasts for regions and time periods of interest that are superior, in terms of skill, to individual NMME forecast models and commonly weighted models. The probabilistic forecast leverages the strengths of three NMME forecast models to predict environmental conditions for an area spanning from San Diego, CA to Honolulu, HI, seven months in-advance. Key findings include: weighted combinations of models are strictly better than individual models; machine-learned combinations are especially better; and forecasts produced using our approach have the highest rank probability skill score most often.

A unified probabilistic framework for spontaneous facial action modeling and understanding.

PubMed

Tong, Yan; Chen, Jixu; Ji, Qiang

2010-02-01

Facial expression is a natural and powerful means of human communication. Recognizing spontaneous facial actions, however, is very challenging due to subtle facial deformation, frequent head movements, and ambiguous and uncertain facial motion measurements. Because of these challenges, current research in facial expression recognition is limited to posed expressions and often in frontal view. A spontaneous facial expression is characterized by rigid head movements and nonrigid facial muscular movements. More importantly, it is the coherent and consistent spatiotemporal interactions among rigid and nonrigid facial motions that produce a meaningful facial expression. Recognizing this fact, we introduce a unified probabilistic facial action model based on the Dynamic Bayesian network (DBN) to simultaneously and coherently represent rigid and nonrigid facial motions, their spatiotemporal dependencies, and their image measurements. Advanced machine learning methods are introduced to learn the model based on both training data and subjective prior knowledge. Given the model and the measurements of facial motions, facial action recognition is accomplished through probabilistic inference by systematically integrating visual measurements with the facial action model. Experiments show that compared to the state-of-the-art techniques, the proposed system yields significant improvements in recognizing both rigid and nonrigid facial motions, especially for spontaneous facial expressions.

Probability Modeling and Thinking: What Can We Learn from Practice?

ERIC Educational Resources Information Center

Pfannkuch, Maxine; Budgett, Stephanie; Fewster, Rachel; Fitch, Marie; Pattenwise, Simeon; Wild, Chris; Ziedins, Ilze

2016-01-01

Because new learning technologies are enabling students to build and explore probability models, we believe that there is a need to determine the big enduring ideas that underpin probabilistic thinking and modeling. By uncovering the elements of the thinking modes of expert users of probability models we aim to provide a base for the setting of…

Motor Task Variation Induces Structural Learning

PubMed Central

Braun, Daniel A.; Aertsen, Ad; Wolpert, Daniel M.; Mehring, Carsten

2009-01-01

Summary When we have learned a motor skill, such as cycling or ice-skating, we can rapidly generalize to novel tasks, such as motorcycling or rollerblading [1–8]. Such facilitation of learning could arise through two distinct mechanisms by which the motor system might adjust its control parameters. First, fast learning could simply be a consequence of the proximity of the original and final settings of the control parameters. Second, by structural learning [9–14], the motor system could constrain the parameter adjustments to conform to the control parameters' covariance structure. Thus, facilitation of learning would rely on the novel task parameters' lying on the structure of a lower-dimensional subspace that can be explored more efficiently. To test between these two hypotheses, we exposed subjects to randomly varying visuomotor tasks of fixed structure. Although such randomly varying tasks are thought to prevent learning, we show that when subsequently presented with novel tasks, subjects exhibit three key features of structural learning: facilitated learning of tasks with the same structure, strong reduction in interference normally observed when switching between tasks that require opposite control strategies, and preferential exploration along the learned structure. These results suggest that skill generalization relies on task variation and structural learning. PMID:19217296

Motor task variation induces structural learning.

PubMed

Braun, Daniel A; Aertsen, Ad; Wolpert, Daniel M; Mehring, Carsten

2009-02-24

When we have learned a motor skill, such as cycling or ice-skating, we can rapidly generalize to novel tasks, such as motorcycling or rollerblading [1-8]. Such facilitation of learning could arise through two distinct mechanisms by which the motor system might adjust its control parameters. First, fast learning could simply be a consequence of the proximity of the original and final settings of the control parameters. Second, by structural learning [9-14], the motor system could constrain the parameter adjustments to conform to the control parameters' covariance structure. Thus, facilitation of learning would rely on the novel task parameters' lying on the structure of a lower-dimensional subspace that can be explored more efficiently. To test between these two hypotheses, we exposed subjects to randomly varying visuomotor tasks of fixed structure. Although such randomly varying tasks are thought to prevent learning, we show that when subsequently presented with novel tasks, subjects exhibit three key features of structural learning: facilitated learning of tasks with the same structure, strong reduction in interference normally observed when switching between tasks that require opposite control strategies, and preferential exploration along the learned structure. These results suggest that skill generalization relies on task variation and structural learning.

Event-Based Media Enrichment Using an Adaptive Probabilistic Hypergraph Model.

PubMed

Liu, Xueliang; Wang, Meng; Yin, Bao-Cai; Huet, Benoit; Li, Xuelong

2015-11-01

Nowadays, with the continual development of digital capture technologies and social media services, a vast number of media documents are captured and shared online to help attendees record their experience during events. In this paper, we present a method combining semantic inference and multimodal analysis for automatically finding media content to illustrate events using an adaptive probabilistic hypergraph model. In this model, media items are taken as vertices in the weighted hypergraph and the task of enriching media to illustrate events is formulated as a ranking problem. In our method, each hyperedge is constructed using the K-nearest neighbors of a given media document. We also employ a probabilistic representation, which assigns each vertex to a hyperedge in a probabilistic way, to further exploit the correlation among media data. Furthermore, we optimize the hypergraph weights in a regularization framework, which is solved as a second-order cone problem. The approach is initiated by seed media and then used to rank the media documents using a transductive inference process. The results obtained from validating the approach on an event dataset collected from EventMedia demonstrate the effectiveness of the proposed approach.

Interleaved Practice in Multi-Dimensional Learning Tasks: Which Dimension Should We Interleave?

ERIC Educational Resources Information Center

Rau, Martina A.; Aleven, Vincent; Rummel, Nikol

2013-01-01

Research shows that multiple representations can enhance student learning. Many curricula use multiple representations across multiple task types. The temporal sequence of representations and task types is likely to impact student learning. Research on contextual interference shows that interleaving learning tasks leads to better learning results…

Jumping to conclusions and persecutory delusions.

PubMed

Startup, Helen; Freeman, Daniel; Garety, Philippa A

2008-09-01

It is unknown whether a 'jumping to conclusions' (JTC) data-gathering bias is apparent in specific delusion sub-types. A group with persecutory delusions is compared with a sample of non-clinical controls on a probabilistic reasoning task. Results suggest JTC is apparent in individuals with the persecutory sub-type of delusions.

Bayes and blickets: Effects of knowledge on causal induction in children and adults

PubMed Central

Griffiths, Thomas L.; Sobel, David M.; Tenenbaum, Joshua B.; Gopnik, Alison

2011-01-01

People are adept at inferring novel causal relations, even from only a few observations. Prior knowledge about the probability of encountering causal relations of various types and the nature of the mechanisms relating causes and effects plays a crucial role in these inferences. We test a formal account of how this knowledge can be used and acquired, based on analyzing causal induction as Bayesian inference. Five studies explored the predictions of this account with adults and 4-year-olds, using tasks in which participants learned about the causal properties of a set of objects. The studies varied the two factors that our Bayesian approach predicted should be relevant to causal induction: the prior probability with which causal relations exist, and the assumption of a deterministic or a probabilistic relation between cause and effect. Adults’ judgments (Experiments 1, 2, and 4) were in close correspondence with the quantitative predictions of the model, and children’s judgments (Experiments 3 and 5) agreed qualitatively with this account. PMID:21972897

Differential coding of reward and movement information in the dorsomedial striatal direct and indirect pathways.

PubMed

Shin, Jung Hwan; Kim, Dohoung; Jung, Min Whan

2018-01-26

The direct and indirect pathways of the basal ganglia have long been thought to mediate behavioral promotion and inhibition, respectively. However, this classic dichotomous model has been recently challenged. To better understand neural processes underlying reward-based learning and movement control, we recorded from direct (dSPNs) and indirect (iSPNs) pathway spiny projection neurons in the dorsomedial striatum of D1-Cre and D2-Cre mice performing a probabilistic Pavlovian conditioning task. dSPNs tend to increase activity while iSPNs decrease activity as a function of reward value, suggesting the striatum represents value in the relative activity levels of dSPNs versus iSPNs. Lick offset-related activity increase is largely dSPN selective, suggesting dSPN involvement in suppressing ongoing licking behavior. Rapid responses to negative outcome and previous reward-related responses are more frequent among iSPNs than dSPNs, suggesting stronger contributions of iSPNs to outcome-dependent behavioral adjustment. These findings provide new insights into striatal neural circuit operations.

Speech Motor Sequence Learning: Effect of Parkinson Disease and Normal Aging on Dual-Task Performance.

PubMed

Whitfield, Jason A; Goberman, Alexander M

2017-06-22

Everyday communication is carried out concurrently with other tasks. Therefore, determining how dual tasks interfere with newly learned speech motor skills can offer insight into the cognitive mechanisms underlying speech motor learning in Parkinson disease (PD). The current investigation examines a recently learned speech motor sequence under dual-task conditions. A previously learned sequence of 6 monosyllabic nonwords was examined using a dual-task paradigm. Participants repeated the sequence while concurrently performing a visuomotor task, and performance on both tasks was measured in single- and dual-task conditions. The younger adult group exhibited little to no dual-task interference on the accuracy and duration of the sequence. The older adult group exhibited variability in dual-task costs, with the group as a whole exhibiting an intermediate, though significant, amount of dual-task interference. The PD group exhibited the largest degree of bidirectional dual-task interference among all the groups. These data suggest that PD affects the later stages of speech motor learning, as the dual-task condition interfered with production of the recently learned sequence beyond the effect of normal aging. Because the basal ganglia is critical for the later stages of motor sequence learning, the observed deficits may result from the underlying neural dysfunction associated with PD.

Delay or probability discounting in a model of impulsive behavior: effect of alcohol.

PubMed Central

Richards, J B; Zhang, L; Mitchell, S H; de Wit, H

1999-01-01

Little is known about the acute effects of drugs of abuse on impulsivity and self-control. In this study, impulsivity was assessed in humans using a computer task that measured delay and probability discounting. Discounting describes how much the value of a reward (or punisher) is decreased when its occurrence is either delayed or uncertain. Twenty-four healthy adult volunteers ingested a moderate dose of ethanol (0.5 or 0.8 g/kg ethanol: n = 12 at each dose) or placebo before completing the discounting task. In the task the participants were given a series of choices between a small, immediate, certain amount of money and $10 that was either delayed (0, 2, 30, 180, or 365 days) or probabilistic (i.e., certainty of receipt was 1.0, .9, .75, .5, or .25). The point at which each individual was indifferent between the smaller immediate or certain reward and the $10 delayed or probabilistic reward was identified using an adjusting-amount procedure. The results indicated that (a) delay and probability discounting were well described by a hyperbolic function; (b) delay and probability discounting were positively correlated within subjects; (c) delay and probability discounting were moderately correlated with personality measures of impulsivity; and (d) alcohol had no effect on discounting. PMID:10220927

Context transfer in reinforcement learning using action-value functions.

PubMed

Mousavi, Amin; Nadjar Araabi, Babak; Nili Ahmadabadi, Majid

2014-01-01

This paper discusses the notion of context transfer in reinforcement learning tasks. Context transfer, as defined in this paper, implies knowledge transfer between source and target tasks that share the same environment dynamics and reward function but have different states or action spaces. In other words, the agents learn the same task while using different sensors and actuators. This requires the existence of an underlying common Markov decision process (MDP) to which all the agents' MDPs can be mapped. This is formulated in terms of the notion of MDP homomorphism. The learning framework is Q-learning. To transfer the knowledge between these tasks, the feature space is used as a translator and is expressed as a partial mapping between the state-action spaces of different tasks. The Q-values learned during the learning process of the source tasks are mapped to the sets of Q-values for the target task. These transferred Q-values are merged together and used to initialize the learning process of the target task. An interval-based approach is used to represent and merge the knowledge of the source tasks. Empirical results show that the transferred initialization can be beneficial to the learning process of the target task.

Context Transfer in Reinforcement Learning Using Action-Value Functions

PubMed Central

Mousavi, Amin; Nadjar Araabi, Babak; Nili Ahmadabadi, Majid

2014-01-01

This paper discusses the notion of context transfer in reinforcement learning tasks. Context transfer, as defined in this paper, implies knowledge transfer between source and target tasks that share the same environment dynamics and reward function but have different states or action spaces. In other words, the agents learn the same task while using different sensors and actuators. This requires the existence of an underlying common Markov decision process (MDP) to which all the agents' MDPs can be mapped. This is formulated in terms of the notion of MDP homomorphism. The learning framework is Q-learning. To transfer the knowledge between these tasks, the feature space is used as a translator and is expressed as a partial mapping between the state-action spaces of different tasks. The Q-values learned during the learning process of the source tasks are mapped to the sets of Q-values for the target task. These transferred Q-values are merged together and used to initialize the learning process of the target task. An interval-based approach is used to represent and merge the knowledge of the source tasks. Empirical results show that the transferred initialization can be beneficial to the learning process of the target task. PMID:25610457

Acute and chronic effects of cannabinoids on effort-related decision-making and reward learning: an evaluation of the cannabis 'amotivational' hypotheses.

PubMed

Lawn, Will; Freeman, Tom P; Pope, Rebecca A; Joye, Alyssa; Harvey, Lisa; Hindocha, Chandni; Mokrysz, Claire; Moss, Abigail; Wall, Matthew B; Bloomfield, Michael Ap; Das, Ravi K; Morgan, Celia Ja; Nutt, David J; Curran, H Valerie

2016-10-01

Anecdotally, both acute and chronic cannabis use have been associated with apathy, amotivation, and other reward processing deficits. To date, empirical support for these effects is limited, and no previous studies have assessed both acute effects of Δ-9-tetrahydrocannabinol (THC) and cannabidiol (CBD), as well as associations with cannabis dependence. The objectives of this study were (1) to examine acute effects of cannabis with CBD (Cann + CBD) and without CBD (Cann-CBD) on effort-related decision-making and (2) to examine associations between cannabis dependence, effort-related decision-making and reward learning. In study 1, 17 participants each received three acute vaporized treatments, namely Cann-CBD (8 mg THC), Cann + CBD (8 mg THC + 10 mg CBD) and matched placebo, followed by a 50 % dose top-up 1.5 h later, and completed the Effort Expenditure for Rewards Task (EEfRT). In study 2, 20 cannabis-dependent participants were compared with 20 non-dependent, drug-using control participants on the EEfRT and the Probabilistic Reward Task (PRT) in a non-intoxicated state. Cann-CBD reduced the likelihood of high-effort choices relative to placebo (p = 0.042) and increased sensitivity to expected value compared to both placebo (p = 0.014) and Cann + CBD (p = 0.006). The cannabis-dependent and control groups did not differ on the EEfRT. However, the cannabis-dependent group exhibited a weaker response bias than the control group on the PRT (p = 0.007). Cannabis acutely induced a transient amotivational state and CBD influenced the effects of THC on expected value. In contrast, cannabis dependence was associated with preserved motivation alongside impaired reward learning, although confounding factors, including depression, cannot be disregarded. This is the first well powered, fully controlled study to objectively demonstrate the acute amotivational effects of THC.

Distributed Bayesian Computation and Self-Organized Learning in Sheets of Spiking Neurons with Local Lateral Inhibition

PubMed Central

Bill, Johannes; Buesing, Lars; Habenschuss, Stefan; Nessler, Bernhard; Maass, Wolfgang; Legenstein, Robert

2015-01-01

During the last decade, Bayesian probability theory has emerged as a framework in cognitive science and neuroscience for describing perception, reasoning and learning of mammals. However, our understanding of how probabilistic computations could be organized in the brain, and how the observed connectivity structure of cortical microcircuits supports these calculations, is rudimentary at best. In this study, we investigate statistical inference and self-organized learning in a spatially extended spiking network model, that accommodates both local competitive and large-scale associative aspects of neural information processing, under a unified Bayesian account. Specifically, we show how the spiking dynamics of a recurrent network with lateral excitation and local inhibition in response to distributed spiking input, can be understood as sampling from a variational posterior distribution of a well-defined implicit probabilistic model. This interpretation further permits a rigorous analytical treatment of experience-dependent plasticity on the network level. Using machine learning theory, we derive update rules for neuron and synapse parameters which equate with Hebbian synaptic and homeostatic intrinsic plasticity rules in a neural implementation. In computer simulations, we demonstrate that the interplay of these plasticity rules leads to the emergence of probabilistic local experts that form distributed assemblies of similarly tuned cells communicating through lateral excitatory connections. The resulting sparse distributed spike code of a well-adapted network carries compressed information on salient input features combined with prior experience on correlations among them. Our theory predicts that the emergence of such efficient representations benefits from network architectures in which the range of local inhibition matches the spatial extent of pyramidal cells that share common afferent input. PMID:26284370

Association of Neural and Emotional Impacts of Reward Prediction Errors With Major Depression

PubMed Central

Moutoussis, Michael; Smittenaar, Peter; Zeidman, Peter; Taylor, Tanja; Hrynkiewicz, Louise; Lam, Jordan; Skandali, Nikolina; Siegel, Jenifer Z.; Ousdal, Olga T.; Prabhu, Gita; Dayan, Peter; Fonagy, Peter; Dolan, Raymond J.

2017-01-01

Importance Major depressive disorder (MDD) is associated with deficits in representing reward prediction errors (RPEs), which are the difference between experienced and predicted reward. Reward prediction errors underlie learning of values in reinforcement learning models, are represented by phasic dopamine release, and are known to affect momentary mood. Objective To combine functional neuroimaging, computational modeling, and smartphone-based large-scale data collection to test, in the absence of learning-related concerns, the hypothesis that depression attenuates the impact of RPEs. Design, Setting, and Participants Functional magnetic resonance imaging (fMRI) data were collected on 32 individuals with moderate MDD and 20 control participants who performed a probabilistic reward task. A risky decision task with repeated happiness ratings as a measure of momentary mood was also tested in the laboratory in 74 participants and with a smartphone-based platform in 1833 participants. The study was conducted from November 20, 2012, to February 17, 2015. Main Outcomes and Measures Blood oxygen level–dependent activity was measured in ventral striatum, a dopamine target area known to represent RPEs. Momentary mood was measured during risky decision making. Results Of the 52 fMRI participants (mean [SD] age, 34.0 [9.1] years), 30 (58%) were women and 32 had MDD. Of the 74 participants in the laboratory risky decision task (mean age, 34.2 [10.3] years), 44 (59%) were women and 54 had MDD. Of the smartphone group, 543 (30%) had a depression history and 1290 (70%) had no depression history; 918 (50%) were women, and 593 (32%) were younger than 30 years. Contrary to previous results in reinforcement learning tasks, individuals with moderate depression showed intact RPE signals in ventral striatum (z = 3.16; P = .002) that did not differ significantly from controls (z = 0.91; P = .36). Symptom severity correlated with baseline mood parameters in laboratory (ρ = −0.54; P < 1 × 10−6) and smartphone (ρ = −0.30; P < 1 × 10−39) data. However, participants with depression showed an intact association between RPEs and happiness in a computational model of momentary mood dynamics (z = 4.55; P < .001) that was not attenuated compared with controls (z = −0.42; P = .67). Conclusions and Relevance The neural and emotional impact of RPEs is intact in major depression. These results suggest that depression does not affect the expression of dopaminergic RPEs and that attenuated RPEs in previous reports may reflect downstream effects more closely related to aberrant behavior. The correlation between symptom severity and baseline mood parameters supports an association between depression and momentary mood fluctuations during cognitive tasks. These results demonstrate a potential for smartphones in large-scale computational phenotyping, which is a goal for computational psychiatry. PMID:28678984

Differences in perceptual learning transfer as a function of training task.

PubMed

Green, C Shawn; Kattner, Florian; Siegel, Max H; Kersten, Daniel; Schrater, Paul R

2015-01-01

A growing body of research--including results from behavioral psychology, human structural and functional imaging, single-cell recordings in nonhuman primates, and computational modeling--suggests that perceptual learning effects are best understood as a change in the ability of higher-level integration or association areas to read out sensory information in the service of particular decisions. Work in this vein has argued that, depending on the training experience, the "rules" for this read-out can either be applicable to new contexts (thus engendering learning generalization) or can apply only to the exact training context (thus resulting in learning specificity). Here we contrast learning tasks designed to promote either stimulus-specific or stimulus-general rules. Specifically, we compare learning transfer across visual orientation following training on three different tasks: an orientation categorization task (which permits an orientation-specific learning solution), an orientation estimation task (which requires an orientation-general learning solution), and an orientation categorization task in which the relevant category boundary shifts on every trial (which lies somewhere between the two tasks above). While the simple orientation-categorization training task resulted in orientation-specific learning, the estimation and moving categorization tasks resulted in significant orientation learning generalization. The general framework tested here--that task specificity or generality can be predicted via an examination of the optimal learning solution--may be useful in building future training paradigms with certain desired outcomes.

Selecting Learning Tasks: Effects of Adaptation and Shared Control on Learning Efficiency and Task Involvement

ERIC Educational Resources Information Center

Corbalan, Gemma; Kester, Liesbeth; van Merrienboer, Jeroen J. G.

2008-01-01

Complex skill acquisition by performing authentic learning tasks is constrained by limited working memory capacity [Baddeley, A. D. (1992). Working memory. "Science, 255", 556-559]. To prevent cognitive overload, task difficulty and support of each newly selected learning task can be adapted to the learner's competence level and perceived task…

Negative symptoms in schizophrenia result from a failure to represent the expected value of rewards: Behavioral and computational modeling evidence

PubMed Central

Gold, James M.; Waltz, James A.; Matveeva, Tatyana M.; Kasanova, Zuzana; Strauss, Gregory P.; Herbener, Ellen S.; Collins, Anne G.E.; Frank, Michael J.

2015-01-01

Context Negative symptoms are a core feature of schizophrenia, but their pathophysiology remains unclear. Objective Negative symptoms are defined by the absence of normal function. However, there must be a productive mechanism that leads to this absence. Here, we test a reinforcement learning account suggesting that negative symptoms result from a failure to represent the expected value of rewards coupled with preserved loss avoidance learning. Design Subjects performed a probabilistic reinforcement learning paradigm involving stimulus pairs in which choices resulted in either reward or avoidance of loss. Following training, subjects indicated their valuation of the stimuli in a transfer task. Computational modeling was used to distinguish between alternative accounts of the data. Setting A tertiary care research outpatient clinic. Patients A total of 47 clinically stable patients with a diagnosis of schizophrenia or schizoaffective disorder and 28 healthy volunteers participated. Patients were divided into high and low negative symptom groups. Main Outcome measures 1) The number of choices leading to reward or loss avoidance and 2) performance in the transfer phase. Quantitative fits from three different models were examined. Results High negative symptom patients demonstrated impaired learning from rewards but intact loss avoidance learning, and failed to distinguish rewarding stimuli from loss-avoiding stimuli in the transfer phase. Model fits revealed that high negative symptom patients were better characterized by an “actor-critic” model, learning stimulus-response associations, whereas controls and low negative symptom patients incorporated expected value of their actions (“Q-learning”) into the selection process. Conclusions Negative symptoms are associated with a specific reinforcement learning abnormality: High negative symptoms patients do not represent the expected value of rewards when making decisions but learn to avoid punishments through the use of prediction errors. This computational framework offers the potential to understand negative symptoms at a mechanistic level. PMID:22310503

Generalized lessons about sequence learning from the study of the serial reaction time task

PubMed Central

Schwarb, Hillary; Schumacher, Eric H.

2012-01-01

Over the last 20 years researchers have used the serial reaction time (SRT) task to investigate the nature of spatial sequence learning. They have used the task to identify the locus of spatial sequence learning, identify situations that enhance and those that impair learning, and identify the important cognitive processes that facilitate this type of learning. Although controversies remain, the SRT task has been integral in enhancing our understanding of implicit sequence learning. It is important, however, to ask what, if anything, the discoveries made using the SRT task tell us about implicit learning more generally. This review analyzes the state of the current spatial SRT sequence learning literature highlighting the stimulus-response rule hypothesis of sequence learning which we believe provides a unifying account of discrepant SRT data. It also challenges researchers to use the vast body of knowledge acquired with the SRT task to understand other implicit learning literatures too often ignored in the context of this particular task. This broad perspective will make it possible to identify congruences among data acquired using various different tasks that will allow us to generalize about the nature of implicit learning. PMID:22723815

Probabilistic analysis of the torsional effects on the tall building resistance due to earthquake even

NASA Astrophysics Data System (ADS)

Králik, Juraj; Králik, Juraj

2017-07-01

The paper presents the results from the deterministic and probabilistic analysis of the accidental torsional effect of reinforced concrete tall buildings due to earthquake even. The core-column structural system was considered with various configurations in plane. The methodology of the seismic analysis of the building structures in Eurocode 8 and JCSS 2000 is discussed. The possibilities of the utilization the LHS method to analyze the extensive and robust tasks in FEM is presented. The influence of the various input parameters (material, geometry, soil, masses and others) is considered. The deterministic and probability analysis of the seismic resistance of the structure was calculated in the ANSYS program.

DeepNAT: Deep convolutional neural network for segmenting neuroanatomy.

PubMed

Wachinger, Christian; Reuter, Martin; Klein, Tassilo

2018-04-15

We introduce DeepNAT, a 3D Deep convolutional neural network for the automatic segmentation of NeuroAnaTomy in T1-weighted magnetic resonance images. DeepNAT is an end-to-end learning-based approach to brain segmentation that jointly learns an abstract feature representation and a multi-class classification. We propose a 3D patch-based approach, where we do not only predict the center voxel of the patch but also neighbors, which is formulated as multi-task learning. To address a class imbalance problem, we arrange two networks hierarchically, where the first one separates foreground from background, and the second one identifies 25 brain structures on the foreground. Since patches lack spatial context, we augment them with coordinates. To this end, we introduce a novel intrinsic parameterization of the brain volume, formed by eigenfunctions of the Laplace-Beltrami operator. As network architecture, we use three convolutional layers with pooling, batch normalization, and non-linearities, followed by fully connected layers with dropout. The final segmentation is inferred from the probabilistic output of the network with a 3D fully connected conditional random field, which ensures label agreement between close voxels. The roughly 2.7million parameters in the network are learned with stochastic gradient descent. Our results show that DeepNAT compares favorably to state-of-the-art methods. Finally, the purely learning-based method may have a high potential for the adaptation to young, old, or diseased brains by fine-tuning the pre-trained network with a small training sample on the target application, where the availability of larger datasets with manual annotations may boost the overall segmentation accuracy in the future. Copyright © 2017 Elsevier Inc. All rights reserved.

Readmission prediction via deep contextual embedding of clinical concepts.

PubMed

Xiao, Cao; Ma, Tengfei; Dieng, Adji B; Blei, David M; Wang, Fei

2018-01-01

Hospital readmission costs a lot of money every year. Many hospital readmissions are avoidable, and excessive hospital readmissions could also be harmful to the patients. Accurate prediction of hospital readmission can effectively help reduce the readmission risk. However, the complex relationship between readmission and potential risk factors makes readmission prediction a difficult task. The main goal of this paper is to explore deep learning models to distill such complex relationships and make accurate predictions. We propose CONTENT, a deep model that predicts hospital readmissions via learning interpretable patient representations by capturing both local and global contexts from patient Electronic Health Records (EHR) through a hybrid Topic Recurrent Neural Network (TopicRNN) model. The experiment was conducted using the EHR of a real world Congestive Heart Failure (CHF) cohort of 5,393 patients. The proposed model outperforms state-of-the-art methods in readmission prediction (e.g. 0.6103 ± 0.0130 vs. second best 0.5998 ± 0.0124 in terms of ROC-AUC). The derived patient representations were further utilized for patient phenotyping. The learned phenotypes provide more precise understanding of readmission risks. Embedding both local and global context in patient representation not only improves prediction performance, but also brings interpretable insights of understanding readmission risks for heterogeneous chronic clinical conditions. This is the first of its kind model that integrates the power of both conventional deep neural network and the probabilistic generative models for highly interpretable deep patient representation learning. Experimental results and case studies demonstrate the improved performance and interpretability of the model.

Bayesian analogy with relational transformations.

PubMed

Lu, Hongjing; Chen, Dawn; Holyoak, Keith J

2012-07-01

How can humans acquire relational representations that enable analogical inference and other forms of high-level reasoning? Using comparative relations as a model domain, we explore the possibility that bottom-up learning mechanisms applied to objects coded as feature vectors can yield representations of relations sufficient to solve analogy problems. We introduce Bayesian analogy with relational transformations (BART) and apply the model to the task of learning first-order comparative relations (e.g., larger, smaller, fiercer, meeker) from a set of animal pairs. Inputs are coded by vectors of continuous-valued features, based either on human magnitude ratings, normed feature ratings (De Deyne et al., 2008), or outputs of the topics model (Griffiths, Steyvers, & Tenenbaum, 2007). Bootstrapping from empirical priors, the model is able to induce first-order relations represented as probabilistic weight distributions, even when given positive examples only. These learned representations allow classification of novel instantiations of the relations and yield a symbolic distance effect of the sort obtained with both humans and other primates. BART then transforms its learned weight distributions by importance-guided mapping, thereby placing distinct dimensions into correspondence. These transformed representations allow BART to reliably solve 4-term analogies (e.g., larger:smaller::fiercer:meeker), a type of reasoning that is arguably specific to humans. Our results provide a proof-of-concept that structured analogies can be solved with representations induced from unstructured feature vectors by mechanisms that operate in a largely bottom-up fashion. We discuss potential implications for algorithmic and neural models of relational thinking, as well as for the evolution of abstract thought. Copyright 2012 APA, all rights reserved.

Real-time Mainshock Forecast by Statistical Discrimination of Foreshock Clusters

NASA Astrophysics Data System (ADS)

Nomura, S.; Ogata, Y.

2016-12-01

Foreshock discremination is one of the most effective ways for short-time forecast of large main shocks. Though many large earthquakes accompany their foreshocks, discreminating them from enormous small earthquakes is difficult and only probabilistic evaluation from their spatio-temporal features and magnitude evolution may be available. Logistic regression is the statistical learning method best suited to such binary pattern recognition problems where estimates of a-posteriori probability of class membership are required. Statistical learning methods can keep learning discreminating features from updating catalog and give probabilistic recognition of forecast in real time. We estimated a non-linear function of foreshock proportion by smooth spline bases and evaluate the possibility of foreshocks by the logit function. In this study, we classified foreshocks from earthquake catalog by the Japan Meteorological Agency by single-link clustering methods and learned spatial and temporal features of foreshocks by the probability density ratio estimation. We use the epicentral locations, time spans and difference in magnitudes for learning and forecasting. Magnitudes of main shocks are also predicted our method by incorporating b-values into our method. We discuss the spatial pattern of foreshocks from the classifier composed by our model. We also implement a back test to validate predictive performance of the model by this catalog.

A simulation-based probabilistic design method for arctic sea transport systems

NASA Astrophysics Data System (ADS)

Martin, Bergström; Ove, Erikstad Stein; Sören, Ehlers

2016-12-01

When designing an arctic cargo ship, it is necessary to consider multiple stochastic factors. This paper evaluates the merits of a simulation-based probabilistic design method specifically developed to deal with this challenge. The outcome of the paper indicates that the incorporation of simulations and probabilistic design parameters into the design process enables more informed design decisions. For instance, it enables the assessment of the stochastic transport capacity of an arctic ship, as well as of its long-term ice exposure that can be used to determine an appropriate level of ice-strengthening. The outcome of the paper also indicates that significant gains in transport system cost-efficiency can be obtained by extending the boundaries of the design task beyond the individual vessel. In the case of industrial shipping, this allows for instance the consideration of port-based cargo storage facilities allowing for temporary shortages in transport capacity and thus a reduction in the required fleet size / ship capacity.